Evil genius: what René Descartes can tell us about AI

Imagine that there exists a dominant force in the world that does everything it can to deceive you so that you can never be certain of your knowledge. How could we ever know the truth? This question lies at the heart of a thought experiment about the evil genius by René Descartes in his book Méditations Métaphysiques (1641). His question is more relevant than ever and provides an ideal starting point for thinking about how we should deal with AI.

For Descartes, this experiment was a crucial stage in a line of reasoning that would eventually lead him to the most famous one-liner in the history of philosophy: I think, therefore I am. Descartes reasoned that even if an all-powerful evil spirit existed that constantly deceived him, making him doubt whether his knowledge was correct, he himself would still have to exist in order to be deceived. The act of doubting proves the existence of the thinker. Hence his statement: I doubt, therefore I think, therefore I am.

What can this experiment teach us about generative AI? I will limit myself here to generative AI because this is where most confusion exists and because it is the form most people use.

Of all the impressive qualities that generative AI possesses, cultivating doubt is not high on the list. Generative AI does what we ask. Give me this. Do that. Look this up for me. Write a paper. Not only does generative AI answer all my questions. With every question I also receive the compliments of the jury for free: “excellent question” or “great insight.” Even my most foolish questions are met with applause.

That is pleasing for my vanity, but bad for my ability to think and create. None of us will read anything truly new from generative AI because it lives off the data that we have placed online. At most, new insights arise through new combinations of existing data. As a result, AI also repeats our mistakes and in that sense AI is always a little bit AS as well: artificial stupidity.

Of course we are all stupid in the sense that, despite our knowledge, we never know things completely correctly and therefore always lie a little unintentionally. That is why we must continue studying and try to reduce the margin of error, our stupidity, to a minimum. Is that not a beautiful mission for a university: forever a student? From that perspective the university remains an interesting space of freedom. Here we can exchange knowledge, point out to each other what we do not know, and identify the mistakes we make. Being a scientist, being a student, thrives on the recognition that the gap between what we know and what can be known can never be completely closed. The task, of course, is to keep that gap as small as possible.

The Argentine writer Jorge Luis Borges wrote a wonderful story about this titled The Aleph. In it someone discovers a mysterious point in a cellar in Buenos Aires called the Aleph, a point from which the entire universe can supposedly be seen at once. Whoever stands at that point can see every place, every moment and every detail simultaneously, without the passage of time. Borges described it as an “Archimedean” point, a place from which one could know the entire world from the outside. The metaphor refers to the work of Archimedes of Syracuse, the mathematical genius of ancient Greece.

Both Borges and Descartes confront us with an important question: how do we acquire knowledge? This question concerns all of us today. Throughout history the question was how to obtain information. Today the challenge is how to navigate through it. There is simply so much information that it has become almost inevitable to rely on technology.

That technology has changed spectacularly over the past few decades. Until the turn of the century you could consult large card catalogues in the entrance hall of our university library tower to find a source. After that you had to go to the desk before 3:40 p.m. with a completed request form for the book you wanted to read. Showing up later was no option because the gentlemen in brown lab coats still had to fetch the book from one of the many floors of the booktower and they did not want to risk working even one minute past 4 p.m. I am speaking about twenty-five years ago, not the late Renaissance.

Today things work differently, luckily. But exactly because everything has become so easy, it has also become more difficult. We all struggle with the same question: what is reliable knowledge and what is not? And also: what can technology do for us and what must we continue to do ourselves? For scientists this question is crucial. Science lives from doubt and uncertainty, from interpretation and provisional conclusions.

This brings us to the first lesson from Descartes: never be absolutely certain of your claim. Generative AI does not encourage reflection on that point. It is a kind of knowledge cloaca. Just as the artist Wim Delvoye once created a real cloaca, a machine capable of producing feces, generative AI suffers, if you will allow this scatological metaphor, from unstoppable knowledge diarrhea. That knowledge contains many mistakes and shortcomings, yet generative AI has not really learned to stop itself from time to time. In that sense generative AI is clearly a machine. It does not ask questions of itself. It lives from the questions we ask it and it does not care whether we continue to question ourselves afterwards.

This brings me to the second lesson from Descartes. AI answers our questions but it does not question itself. It is not a doubting being and it does not listen to what lies between or behind our question. We humans at least possess the potential to do so.

At the same time generative AI resembles humans in a remarkable way. It lies constantly. In addition to the unintentional stupidity from which both humans and AI suffer, AI is quite skilled at deliberate fabrication or hallucination. It invents things that do not exist. Even more than humans, who also lie quite often for many different reasons, this knowledge cloaca excels at inventing non-existent knowledge or connecting things that have no connection at all. That is partly because we demand definitive answers from generative AI.

Fabrication or hallucination is not unique to generative AI but it is very good at it and, above all, it suffers no discomfort from it. AI has all the characteristics of Descartes’ evil genius. When humans lie and have developed even a minimal sense of conscience, they usually feel somewhat uncomfortable. Generative AI has no sense of shame or guilt and no awareness of its own shortcomings. Humans, by contrast, are generally troubled by their imperfections.

This brings me back to science, the part of society that is very aware of its own shortcomings and rightly so. Scientists cannot claim anything they want. Everything they assert must be verifiable by others, both in terms of results and process. That is not easy, but it does limit falsehoods.

AI can help us reduce falsehoods. At the same time we should not grant AI a higher authority than the words of our fellow human beings. We must not switch off our doubt or return to what was common in medieval scholastic textbooks with the formula auctoritas est. By referring to a great name such as Aristotle, people were essentially saying: what follows is true because Aristotle said so.

Yet we often use generative AI in exactly that way. That, in my view, is the great trap we face today. We are critical of our fellow human beings, but we entrust our fate to a technology that is still developing. This leads me to the third and final lesson from Descartes. Certainty can only emerge after we have first doubted very deeply what we believe to be true. We must therefore not only think about the world but also about the position from which we do so. Asking a question about the world always also means asking a question about ourselves. Scientists in particular must keep doubt and uncertainty at the center of what they do. If things go wrong, we must remain responsible. If knowledge is incomplete, and will it ever be otherwise, we must remain in charge and continue the search.

In other words: because AI is so human, we should not trust it more than we trust ourselves.