What Could Go Wrong?

In 1986 – when I was four years old – three researchers at the University of California, San Diego (UCSD) were working on an idea that would change the world of technology forever: they figured out an efficient way for smart computer systems they called “connectionist models” to learn from their own mistakes. Not by manually correcting each error individually but by using a procedural algorithm that tells every little part of the system how much it contributed to the error. This mechanism, which is now known as backpropagation, is the reason neural networks can learn patterns by adjusting the connections between “neurons” themselves. The only problem: it didn’t work. But not because they were wrong. But because to make it work really impressively well, back 1986, they would have needed about a million times more data and processing power.

Fast forward to 2025. A vast information network full of data is spanning the globe, transistors are now a million times smaller and more powerful than 40 years ago and one of those researchers is now a Professor Emeritus at the University of Toronto and known as the “Godfather of AI”. Geoffrey Hinton, who quit his job at Google to warn humanity of the dangers of AI, is calm, patient, humorous and very polite while he’s talking to a man most of us probably know from The Daily Show: Jon Stewart. Jon also has his own podcast called The Weekly Show, in which he sits down with his guests for in-depth conversations about culture and politics. And this week, Jon invited Geoffrey Hinton to talk about AI technology and its impact on society – and humanity as a whole.

And what can I say? It turned out to be a remarkable, eye-opening conversation. And that’s mainly because of the two very different parts of the conversation.

There’s the first half, where Jon Stewart basically takes on the role of the totally ignorant, but endlessly curious student in the first row, asking very basic and fundamental questions about how neural networks and large language models work. And, step by step, Geoffrey Hinton explains it using a lot of descriptive examples. I don’t think I’ve ever heard a clearer explanation of how neural networks are constructed and how they work and learn.

Part two of the interview is the “it will kill us all” part of the conversation. Here, Jon Stewart asks questions that cut straight to the heart of the different issues – not just about what the rapid progress of AI means for us individually, but what it means for societies, politics, and the world at large. They discuss how AI can cause disruption, the importance of regulating it, whether AI systems might have subjective experiences, and why there could actually be global collaboration to prevent AI from taking over. And did you know that, when they’re being tested, AI models already pretend to be dumber than they are?

Overall, it is a fascinating conversation, also because it made one thing clear to me: while we are rightfully debating the technology’s shortcomings, like where the training data comes from, or the electricity consumption and environmental impact of data centres, and whether we should therefore use AI in our work and whether AI, in its current form, is a failure – at its core, this technology represents such a radical shift in how software works that it will not go away. The people currently betting on AI and dominating the discourse might be motivated by money and power. And they might be creating a huge bubble at the moment. But this bubble is not a bubble because the technology behind products like ChatGPT, Claude, Gemini, or Midjourney is a hoax – unlike NFTs of apes, for example. This bubble is a market bubble driven by overhyped expectations and soaring valuations of AI companies. Many people are hoping to become very, very rich. Yet just as the Internet and the Web survived the dot-com crash, Large Language Models and AI will not disappear. The way these systems work at a fundamental level is superior to any previous computational approach. Although AI hasn’t surpassed human intelligence yet – and it may take much longer than some tech CEOs would have us believe – it is already a singularity in the sense that there is no going back.

The question is: what would good AI then look like?

~

Hinton:

Stewart:

Hinton:

📺 Watch the Interview on YouTube

❦

This is post 10 of Blogtober 2025.

~