The Authority of Algorithms

I was recently watching a talk given by Yuval Noah Harari called Homo Deus: A Brief History of Tomorrow. His idea is that authority has shifted from divine truths or monarchist decrees to a liberal humanism focused upon individual feelings during the post-industrial revolution era and now finally to algorithms derived from big data analysis.

I disagree with this talk quite a lot, but it's a good exposition of a contemporary technocratic belief system. In the last part of the talk I was expecting him to say something like:

"But, you know, if you've been paying attention to anything I've said in the last twenty minutes you'll know that it's bullshit because..."

And then go on to explain the numerous problems with his earlier account. Except that he doesn't do that, and remains stuck on the technocratic track.

So here I'll summarize what the problems with "dataism" or algorithms as ultimate authority are.

The architecture of the internet is sufficiently dysfunctional that it is indeed possible for a small number of very large companies to gather massive amounts of data about the behavior and beliefs of large sections of the world's population. That's undoubtedly true, and it's also true that the computing power to analyse and derive results from these data points exists. Privacy issues aside, so far so good. There are some critical problems though:

  • These companies are not benevolent entities engaged in dispassionate contemplation of the human condition or the nurturing of human potentiality. They are for-profit enterprises with agendas which can be very different from that of the average person in the same manner that the agenda of the Soviet Politbureau differed from the Soviet worker.
  • Data - especially when taken "in the wild" from real situations - is often inherently ambiguous. Sometimes having more data helps to clarify the situation, but sometimes it only makes matters much worse.
  • Human societies are feedback systems, with bubble effects and so on. There is no sense in which you can "objectively analyse the data" within such a system of circular causality. The algorithm is feeding on data which may be influenced by the same algorithm and so on indefinitely.
  • The types of data gathered and the contexts from which it is obtained all constrain the range of possible conclusions. It's quite possible to design data gathering in such a way that certain favoured conclusions become inevitable (fitting a pre-defined solution to the available data). Even without conscious bias, subconscious biases often exist in data, especially when it's about the actions or beliefs of people.
  • Algorithms are not gods. They're made by a small fraction of mortals who usually have very explicit agendas in mind when creating them. Often the agenda is something like "I don't care what this does to the world so long as it increases my own personal profits". Also mortals can implement algorithms badly.

So unfortunately the belief that algorithms derived from massive data crunching by a few companies are, or shall become, the ultimate authority and decider of truth is at best a mirage and at worst a dodgy business model to get you to buy all sorts of tat that you don't actually need so that some executive can buy themselves another yacht.

Getting to The Truth about anything remains tough. If people cede their authority to algorithms then the results are likely to be no better than those obtained by emperors and presidents or other arbitrary authoritarians of the past. There is still no escaping the fact that individuals need to make their own decisions based upon their own context.