In her book Weapons of Math Destruction , Cathy O'Neil (behind the blog mathbabe ) guides us through the unjust algorithms that have a measurable impact on various stages of our lives.
I've picked up this book because I wanted to learn more about inequality and how various algorithms end up increasing it. To that point, Cathy's book delivered, describing how you can end up being treated unjustily by a mathematical model you're not allowed to see.
Once you learn a thing or two about the models she likes to call Weapons of Math Destruction (or WMDs) and her journey towards writing this book in the first two chapters, you learn about how models affect your college application, the ads you see online, getting a job, keeping a job etc. Each of the chapters also offers a hypothetical or a real life victim of the models she discusses, which I found to be a nice touch.
What I didn't like is that chapters follow a predictable pattern: they start by introducing some form of inequality, mention an impact they had on the victim, and then move on other models that might impact you in the same sphere of life. They became too predictable, making the relatively short book somewhat bothersome to finish. Still, I've gained some valuable inisights from it that I wanted to share.
The first running theme is that each computational model has its own definition of success, which is usually slightly different than a definition of success that we would make as a collective. I've learned to ask "Who designed it?" and "What are they trying to achieve with it?" in order to reach a better decision about whether or not I should trust this model.
Another key insight that this book has provided me can be summed up in the following quote:
Big Data processes codify the past. They do not invent the future. Doing that requires moral imagination, and that's something only humans can provide.
O'Neil finishes the book with the conclusions section. In it, I found the arguments to be pretty weak. I find this to be understandable considering some of the topics she touched on are still at their infancy, but the book didn't reach a satisfying end.
She proposes something like a Hippocratic Oath for fresh data scientists as a way of self-regulation, but she does not bother with providing a proposed draft of such an oath. She mentions that she wants more involvement from the government, but the only concrete field in which she makes a proposal is to expand HIPAA act to include health data generated by various Internet of Things devices (like heart rate, amount of steps you take and similar).
Overall, this book served as an introduction to ways algorithms can treat people unfairly, and it definitely managed to plant some valuable insights in my head. Still, I found it to be a bit too introductory, not really capable of driving the point home, and slightly difficult to finish thanks to the predictability of how each individual chapter will develop. I am thankful for the few insights it has provided me with, but I will look elsewhere in order to get a better grip of the topic.
If you have thoughts on the book, I'd love to hear them .