Jürgen Schmidhuber, the man Alexa and Siri would call ‘dad’ if he wanted to talk to them
When he was 15 years old, Jürgen Schmidhuber dreamed of creating a robot smarter than him and retiring. He is 58, so it could be considered that the plan has been regular. “Well, I still have a few years left, give me room,” he asks in a telephone conversation. It will be necessary to give him a vote of confidence. If this German scientist has shown something over the years, it is constancy. And Schmidhuber is far from being a normal guy. He published his first work on artificial intelligence in 1987, with a thesis that proposed what was then insane: an algorithm capable of improving itself. More than 30 years later, his theory has been turned into practice and it’s in your pocket. The system Long Short Time Memory (LSTM), created by Schmidhuber and his student, Sepp Hochreiter, is used by 3 billion phones. Google, Apple, Facebook, Amazon and Microsoft include it in their products.
LSTM is a neural network architecture widely used in the deep learning. It mimics the functioning of the human body, with neurons that collect data (images, sounds, words), neurons that interpret them in the brain, and others that store them as memories. LSTM does the same, replacing neurons with algorithms. It sounds exotic and futuristic, but you use this technology several times a day. For example when requesting an automatic translation. Google Translate improved considerably in 2016 when it integrated the LSTM in its guts. You also use it if you are chatting with your voice assistant. If Alexa, Siri, or Ok Google had to address Schmidhuber, they would probably call him father. But they don’t. “I hardly use them. They are a way of collecting data to sell you tailored advertising and get your attention ”, the computer scientist says. To do this, they use a system that he helped create. When asked about this irony, he throws balls out. “Yes Yes. I know. Nor am I going to analyze the morality of its business model, it is simply that I do not want to be part of it, “he says.
Schmidhuber lives far from Silicon Valley. Both mentally and geographically. Answer this interview while taking a walk up a hill in the Alps outside of Lugano. There is the Institute of Artificial Intelligence of the University of Switzerland, of which he is scientific director. Also the company Nnaisense, which he founded together with four of his former students. Schmidhuber could work for one of the big tech companies, he could settle in Silicon Valley. But he chose Switzerland. Because of its proximity to his native Munich, but also because technology linked to science and research is understood there. “It is the country that spends the most money per capita on science, the place with the most Nobel prizes per capita,” he points out. And it also has beautiful hills to walk through.
From intelligence to artificial creativity
His system mimics the workings of the brain, but Schmidhuber does not speak so much of artificial intelligence, as of curiosity, of artificial creativity. But can an algorithm be taught to be creative? “Of course, I did it 30 years ago,” the computer scientist responds and goes on to explain the complex operation of antagonistic generative networks (GANs). “So we devised two neural networks that did not just obey orders,” he recalls. Without human supervision, they interact with each other, “as if they were two children trying new challenges with their toys.” One of the networks creates situations, invents problems. The other tries to solve them. “They are always trying to broaden their horizon of the known, they are like little scientists,” he proudly explains. GANs were invented in the 1990s, but have developed their potential many years later. Currently they are used to produce photorealistic images in the world of video games or for the creation of controversial deepfakes.
The creativity of the algorithms is advisable; that of the scientists who create them, essential. Schmidhuber has a lot, almost too much. That is why at the beginning of his career few believed in his theories and predictions. “I remember one of my first lectures where only one woman appeared. I said “what a shame, it looks like I’m going to have to give the talk just for you”, to which she replied: “Okay, but hurry up, my presentation starts right after yours.” The story (true or not) is told by Schmidhuber himself in a TED talk that touches half a million viewings. What he says remains the same, only now there are many more people listening.
Between one talk and another, Schmidhuber’s neural network did not change so much as the ability to train it. “Many of the basic ideas behind the revolution of the deep learning were published long ago, between 1990 and 1991. It was our Annus mirabilis”, Explains the expert. It was only on paper. His theories were good, but the computers were slow and the databases to train the algorithms scarce. They had to wait until 2010. Then huge bases appeared in the most unexpected places: video games and the internet. The two curious children Schmidhuber was referring to had new toys. And they were very powerful.
In search of recognition (own and others) and astronaut robots
The ten years were the decade of deep learning. It was then that the neural networks of Schmidhuber’s team began to draw the world’s attention. They could recognize road signs or Chinese spelling, even though none of their programmers knew Chinese. Later they learned to recognize faces, to converse with people or to park cars. The abstract research to which this engineer had dedicated his life was beginning to yield tangible results. The deep learning it began to be valued and its promoters became stars in their field. But not everyone received the same recognition.
In 2019 Geoffrey Hinton, Yann LeCun and Yoshua Bengio won the Turing Prize for their research in the field of deep learning. The first works for Google; the second, for Facebook; and the third for IBM and Microsoft. They had done several investigations together and some separately. They were quoted frequently. Those who did not cite were the theoretical pioneers in this field, such as Schmidhuber. He took it personally and since then has accused his “special friends”, as he calls them, of ignoring sources and taking credit for themselves much more diffuse and shared.
In technology, progress is always made on the shoulders of giants, and Schmidhuber has set out to name all giants. Point out those who stand out and highlight previous work, starting with yours. Their website he dedicates himself to this task with academic thoroughness. It is not the web one would expect from a computer engineer. It’s headed by a pixelated photo of Schmidhuber in a cowboy hat. It looks like the Myspace of a genius, or a particularly chaotic Wikipedia entry. The similarity is not only aesthetic, as it contains a huge amount of information about different technological advances. “You need an entire civilization to develop artificial intelligence,” explains the scientist. “What we are achieving now is only possible thanks to what has been done previously for centuries. And you have to claim it ”. Explain that it is not a question of ego, that it is of justice. And trust that time will prove him right. “The truth is like the sun, you can hide it for a while, but it won’t go away,” he says, quoting Elvis.
Jürgen Schmidhuber looked to the human body to develop his LSTM, but believes that current technology is very anthropocentric. “Everything is designed to lengthen our lives or to suck our attention,” he says. He believes that artificial intelligence is something more important, that transcends biology, and even Humanity, because it will be the robots who will conquer space. “This is more than another industrial revolution,” he warns. His assertions may sound a bit exaggerated, but the same thought his colleagues in the eighties, when he talked about concepts such as deep learning. Then he dreamed of creating a robot much smarter than him and retiring. He did it in a carefree and nihilistic way, a computer joke. Today he still has the same dream, but its implications are more epic and transcend his own retirement. And that of all of us.