In this series of interviews. I ask scientists, engineers, and ethicists how technology might change our future. We had these conversations during the research for my book, Welcome to the Future (Quarto, 2021).
Interview 8 – Melanie Mitchell, AI expert
Dr. Melanie Mitchell is a computer scientist and the Davis Professor of Complexity at the Santa Fe Institute, where she studies artificial intelligence and cognitive science. Her latest book is Artificial Intelligence: A Guide for Thinking Humans (Farrar, Straus, and Giroux, 2019). It’s a wonderful book and I highly recommend it.
Most fascinating for me, though, is the fact that her dissertation advisor was Douglas Hofstadter, author of many excellent books about AI, cognitive science, and philosophy. His books influenced me greatly and helped inspire me to become a science writer. We spoke in February 2020.
In your book you write about AI “crashing the barrier of meaning.” What is the barrier of meaning, and what would it mean to crash through that?
When we talk to each other, we understand what the other person is saying. What that means is we have a mental image. You tell me you are writing a book, and I say that I understand that, because I understand what a book is. I can answer some questions about what a book is. I understand what it means to write a book or to read a book, and I can picture myself doing it. I can predict some things about what you’re going to do next after you finish talking to me on the phone.
If you’re in a conversation with a computer, like a chat bot, and you tell the chat bot, “I’m writing a book,” it might answer something like, “What is your book about?” But it actually doesn’t have any understanding in the way we do of what a book is, what writing is, it’s learned some patterns of language that it can use, but it doesn’t have that deeper understanding or structured thinking that we humans have. It can’t answer a lot of questions we can answer or predict things or imagine things about that concept. That’s what I referring to with the barrier of meaning. Humans have this thing we call understanding, and machines don’t have it yet.
How do we know machines aren’t there yet?
They fail in so many ways that a human would never fail. Once you start probing them, you find that they are unable to behave in the way a human would behave with the same kind of input. That really shows that they don’t have this understanding that we have.
Can you give a specific example of a failure that might surprise someone?
Here’s one example that was in my book. There was a computer that learned how to play Atari video games like Breakout, where you move a paddle and the paddle hits the ball and the ball takes out these bricks. It did better than humans on this game. But somebody tried it where they took the paddle and moved it up by a few pixels on the screen. The machine could no longer play the game. So it didn’t really understand what a paddle was. Even though it had learned to play the game with one configuration, the fact that it fails when we change the game a slight amount, in a way that wouldn’t affect a human, shows that it hasn’t learned the same concepts that we humans use to play the game.
People get very excited about deep learning and the possibility that it might give computers human-like understanding. How are people taking that too far?
Deep learning systems are very good on certain kinds of tasks. Usually what “very good” means is that they are tested on a particular set of examples and they do really well. Then we assume because they are so good on the examples, they will be good at the task in general. One example was in answering questions about pictures. There are some machines that are very good at answering questions about pictures, like “is there an animal in this picture?” Or “how many animals are there in this picture?” But this was just one set of examples it had been tested on. If you tested it on a much broader set of examples, it would do poorly. The media tends to report, “this machine is as good at humans at answering questions about pictures,” which is completely false. The media tends to extrapolate to an entire task from one particular set of examples.
In general, the more data and the better data a deep learning model has to work with, the more accurate it can become. Could something like the internet of things or virtual reality provide more complete and accurate data that could get us closer to human-like understanding?
It’s possible. I don’t think we really know yet. In the book I talk about self-driving cars. The question was, given more and more data, can they learn enough about the world to deal with [unexpected] situations that a human driver can deal with. I’m skeptical that they can. Some people think they can. But if they don’t have the vast knowledge of the way the world works that we humans have, I’m not sure they can get that from data. There’s this concept of the long tail. There are so many possible situations that could happen, most of them quite unlikely, but in a world full of self-driving cars, some self-driving car somewhere is going to face an unlikely situation.
Right, and deep learning is a statistical machine. If it has seen a situation before, it knows what to do. But if it hasn’t seen it before, it can’t make an analogy as a human would. It can’t apply something similar to a situation that’s different.
Yes, that’s exactly right. People talk about this long tail problem. It’s debatable to me whether that can be overcome just with more data.
What might you be able to overcome it with?
I think we have to understand better how people use what we call common sense to deal with these situations. That involves being able to abstract concepts and to make analogies. And that’s something we don’t know how to do yet with computers very well. It’s an open problem.
I know that you’ve worked with Douglas Hofstadter on these ideas. I actually got to interview him a few years ago for an article I wrote about his “letter spirit” project. That was a program that was making analogies.
Yeah, exactly. But to be able to go from idealized analogies between letters to the real world is still a challenge. One thing I find fascinating is we have these machines that can do really difficult things, like play GO or chess better than any human in the world, but it’s still a grand challenge in AI to get a machine that has the common sense of an 18 month old baby. That’s something that DARPA is funding. That’s an interesting paradox.
Something I used to think was likely is that you could make a machine that would act like a baby. A computer program or robot could follow some sort of learning path, and through random interactions with the world, gain intelligence like that 18-month-old human. I’m starting to realize maybe that’s not as straightforward as it used to seem to me.
Well, right, because there’s so much we don’t know about how babies learn. There’s also this big debate about what babies are born with. Are they blank slates or do they have a lot of innate abilities? I don’t think anybody knows the answer to that.
So we can’t tell the machine to do the same thing until we know how babies are doing it?
Yeah, that’s one possibility, yes.
Do you think if we could find a way to put common sense as well as concept and analogy-making into machines, could they reach intelligence similar to what humans have?
I think they could in principle. But as I quoted somebody saying, that may be 100 Nobel Prizes away. We’re far from being able to do that now. There are a lot of big discoveries that have to be made before that can happen. I think there’s nothing in principle that is preventing us from creating machines that have human intelligence. It’s just a very hard problem.
Then we have to think about whether we should do it or not.
What do you think about that?
I have very mixed feelings. It could actually be extremely beneficial for society, but there’s also a lot of risk of harm. We already have machines that are possibly harming society in various ways. You can imagine that as machines get smarter, it might get more and more potential for harm. I don’t think the main harm is going to be machines taking over and enslaving us, or any of those science fiction scenarios. More that humans misuse them.
You were talking about artificial general intelligence (AGI) being 100 Nobel Prizes away. On the other side you have people like Ray Kurzweil saying that in the next few decades, we’ll have some sort of intelligence explosion that leads to a singularity, a point beyond which we’re irreversibly changed thanks to AI. What do you think kids should know about that?
Kurzweil’s ideas – very few people in the AI community adopt that extreme position. If you’re working in the trenches you can see how far we are. Kurzweil’s idea is that there’s this exponential curve of progress, which I think is quite debatable. We’ve had it with hardware, but certainly haven’t had it with software. And we certainly haven’t had it with understanding our own intelligence. So, I can’t really engage with his predictions because I don’t believe the premise. That being said, I do think AI is going to change society and change our lives, but maybe not to the extreme or on the time scale that Kurzweil is predicting.
One thing I found fascinating in your book is the idea that if you combine the best aspects of human intelligence into a computer program, you might not be able to have all the things we want from computer intelligence. In other words, superintelligence may not even be possible in the way people like Kurzweil are imagining it! Could you talk about that idea?
Sure. There are a lot of limitations we humans have in our thinking. We have these cognitive biases. We are not really rational. We’re distractable, we get bored. All of these things have to do with having the kinds of brains and bodies that we have. There’s this “fantasy” we’ll call it, that we could have the best of humans, our general intelligence, without these limitations. Something that doesn’t get bored, doesn’t need to sleep, and all that stuff. I don’t even buy that, because I don’t think it’s obvious at all that we could have general intelligence of the kind we humans have evolved without the kinds of limitations we have. That may be because to have our kind of intelligence, we need to have our kinds of bodies.
People might say, well, we don’t need computers to have our kind of intelligence, they could have their own kind of intelligence. Which is true, but then it’s very difficult to get them to do the tasks we want them to do. I don’t even know what that would mean – “their” intelligence. So I’m not a fan of the “superintelligence” argument that we could have a computer that was generally intelligent on par with a human, but would not understand humans enough to know that for instance, in order to solve climate change, killing all the humans is a bad solution. That just doesn’t make any sense to me.
If it’s going to be smart enough to take on a challenge like that, it has to have gained some of the human types of understanding of what is good or bad, or desirable or not desirable?
Exactly. I think so.
Hopefully! When you think about AI in the future, what are you most hopeful for?
I’m hopeful that we’l lbe able to understand our own intelligence better. And that we’ll be able to take the technologies we’re creating and use them for benefit rather than for harm. I think there are huge potential benefits, even with self-driving cars, if we can trust them. There are also benefits for human lives, for the quality of our lives, through AI in medicine. There’s so much possible benefit there. But, again, I hope that we can regulate these technologies so that they are not being used to harm people. I definitely worry about that.
Is there a project you’re working on right now that you’re especially excited about that might get us closer to some sort of understanding in computers?
I’m thinking quite a bit about analogy and about how we form concepts. One project that I’m really excited about is trying to bring together people from AI with people from fields where people think about these kinds of things in human and animals, to better understand what intelligence is. I’m working on trying to build systems that can make analogies, but also trying to better understand what that process is in humans.
Any last words?
One thing AI has taught us is how much we underestimate our own intelligence. We think that certain things about our intelligence should be easy to mimic in computers. But it turns out it’s much harder than we thought because so much of our own intelligence is invisible to us. How much we know, how we’re able to use our knowledge reflexively. Consciously, we don’t even notice it. That’s something that AI has really brought out.