This month has seen the publication of two high-profile books that pose fundamental questions about the big data credo. Yuval Noah Harari, bestselling author of Sapiens, returns with Homo Deus. While his former book examined the history of humanity, this follow-up peers into our future. Harari argues the growing belief in the power of big data analysis could spell the end of democracy and free will, as we humans cede ever more control over our decisions to computer-based algorithms. He calls this credo ‘Dataism’.
While many technologists and data scientists would dismiss this description as needlessly alarmist, in fact Harari’s tone is typically detached and observational. Rather than set out to demonize technological progress, in large part the book dispassionately tracks its probable trajectories. This, though, makes it all the more chilling and likely to strike a chord with the public.
Can Algorithms Be Prejudiced?
Cathy O’Neil’s Weapons of Math Destruction, on the other hand, offers more direct criticism of the big data credo. “The algorithms that power the data economy are based on choices made by fallible human beings. And, while some of them were made with good intentions, the algorithms encode human prejudice, misunderstanding and bias into automatic systems that increasingly manage our lives,” she writes.
Armed with plenty of examples, O’Neil shows how more and more organizations are deploying algorithms that have a direct (and often negative) effect on people, with little evidence of their validity or the unforeseen effects of their widespread deployment. This, she argues, has already had dire consequences for some. For example, she illustrates how the growing use of algorithms to automatically filter out job applicants results in perfectly capable people being routinely excluded from the job market without ever having their CV read by a human being.
“Those with the latest information learn what machines appreciate and what tangles them up, and tailor their applications accordingly. Those who don’t…may never know that they’re sending their resumes into a black hole. It’s one more example in which the wealthy and informed get the edge and the poor are more likely to lose out,” she wrote in a recent article previewing the book.
Towards A Code Of Ethics
With a growing number of studies, articles and books making similar criticisms of big data and how it’s used, proponents will find critics increasingly difficult to dismiss or ignore – and neither should they. Rather, they should welcome the fact that the wider society is beginning to question and debate the very real philosophical and ethical challenges that these technologies raise. And they should both acknowledge valid criticism and, more importantly, openly rise to the challenge of addressing it.
An industry code governing the ethical deployment of algorithms might be a good start – demanding a certain level of transparency and independent oversight of organizations’ use of decision-making algorithms, for example. Systems making decisions which affect people’s lives and livelihoods must be shown not to be inadequate or discriminatory, with appropriate feedback mechanisms, checks and balances to ensure that – rather than prompting a big data backlash – outside criticisms of the field fuel its ongoing improvement and public acceptance.