A giant supercomputer controls exactly what happens everywhere in the world. It could be the end of free will or the end of the world, pick your poison. This is the premise behind the (spoiler alert) third season of Westworld, the sci-fi show exploring the future of android tech and sticky philosophical quandaries.
The supercomputer, Rehoboam, stuck out to me. It was an interesting example of what could be the future of computing: centralized power given to one all-powerful machine, a god-mode to keep everything in order. Although in theory it sounded far fetched, I couldn't help but wonder how we may already be working towards that. The consolidation of data in the hands of a few large tech companies lays the seeds for future development of tech like Rehoboam.
Step 1: Surveillance for Data
The first step to developing tech like Rehoboam would be large-scale surveillance and data on nearly every individual alive. We already participate in this with our mobile devices. Companies like Facebook, Google, and Amazon can build comprehensive profiles on our activities across key areas: our relationships (social media), our purchases, and our work (search history). Nowadays with the integration of all these areas within singular applications (cough cough, Instagram), these profiles could be created from just one app. This is not to mention the iOT tech, video surveillance, and facial recognition that is now appearing in our physical spaces, also largely owned by the above 3 companies.
Step 2: Incentives for Exploitation
The next step to creating Rehoboam would be an incentive to utilize data for the purpose of controlling others. There is no clear reason for these companies to use this data to control us, yet. Right now, the services use our data to bring us more of what we like. They optimize for our attention and dollars by giving us exactly what we want to see.
This model works right now because our attention is lucrative. What happens when our attention isn't worth as much anymore? What if inciting instability becomes a money-maker? What if isolating us becomes more profitable? One can argue that the attention economy already does profit from all of those things. Due to the advantage of data & scale, the big companies are the only ones that reap the benefits. It wouldn't take more than a few bad actors near the top of the system for the goal for the use of this data to shift.
Behavior change could be (and already is) caused just through the manipulation of what individuals see on platforms like Facebook, Google search, Twitter, or Instagram. These platforms already profit off of our internet use, so the incentive is there to take it even further. The systems are also present. The lines between government, media, commerce, and entertainment are blurred, making it easier to exploit population actions at scale through those dominant platforms.
Step 3: Building the Technology
The final step to create Rehoboam is the actual tech needed to get there. Rehoboam is a form of superintelligence, and considering that AI can't even drive cars yet, we are far from there. Some experts predict, though, that we are not even 50 years away. More important than how the tech will get developed, though, is who will develop it. With the way tech development is happening now, it would be impossible for any entity without the data stores of Facebook, Google, or Amazon to get there. Data is the most important asset to devise effective algorithms. We place a lot of trust in the developers, then, in hoping that they will have our best interests in mind when they build.
The consolidation of power for these few companies does not bode well for the future of how tech will develop to play a part in our lives. We may not get Rehoboam, but something far more subtle and nefarious, and it's an interdisciplinary policy, technology, and ethics challenge to go about avoiding it.
