Book: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

by Cathy O'Neil

This book conducts a broad review of numerous areas in which the field of “big data” may be reinforcing societal inequalities. Discussed in the work are topics such as sentencing algorithms, teacher rating systems, credit rating practices, and resume screening applications.

Unfortunately, I feel that the book suffers due to its desire for mass-market appeal: the author displays a tendency to overlook salient issues in favor of emotion-grabbing oversimplifications. The book’s attempt to catalog numerous societal ills hinders its ability to generate a meaningful or actionable thesis.

Despite a data-focused subject matter, the author manages to avoid discussing any statistics or providing quantitative evidence to back up claims. And I don’t mean that she takes a Malcolm Gladwell approach, cherry-picking academic studies to back up a narrative. Her handful of citations point the reader towards newspaper articles lacking any scientific rigor.

There are a handful of valuable explanations: the author’s perspectives on candidate screening systems that reinforce racial biases, for example. But sadly, even there she relies on surface-level generalizations and fails to equip the reader with an understanding of why systems fail in those harmful ways.

Define Big Data However You Want

The author’s ideology continually overrides attempts to make convincing arguments about “big data”. Many examples, such as the way US News and World Report conducts college rankings or how some voters are much more influential in the electoral college system, read like a short Vox opinion piece. We know scoring systems can be unfair. We know that academic and political institutions have characteristics that can hinder progress. The author provides no evidence that these problems are symptomatic of larger structural changes in how society uses big data systems.

Overlooked Areas

Disturbingly, the author overlooks several of the most critical threats of today’s data ecosystem, in my opinion:

  • Government surveillance
  • Digital censorship
  • Data brokers and the market for customer information
  • Social media
  • Workforce automation
  • Cybersecurity and cyberwarfare

In a particularly striking omission, the author devotes a section to the use of “predatory ads” to target low-income individuals for payday loans and for-profit universities. That’s a problem, and parts of it seem illegal based on laws already on the books. But where is the discussion about the massive, multi-billion dollar ecosystem of data brokers (Experian, Acxiom, etc.) that openly brag about having a profile on every American? Or about the immense profit motives driving what has been termed “surveillance capitalism” in the absence of any meaningful privacy legislation?

2 out of 5 Stars

In conclusion, I can’t recommend this book. It’s not a critique of how data systems can fail in unanticipated ways. It’s not an informed plea for policy action or institutional reform. It’s a book intended to make you feel outraged, usually at software developers and capitalism. As a society, we desperately need to address the problems and blind spots associated with omnipresent data systems, but blaming individual greed and implying that algorithms are evil does a disservice to those trying to develop solutions.