Stuart Russell on the Future of A.I.

QUANTA MAGAZINE: What concerns you about the possibility of human-level AI?

STUART RUSSELL: In the first [1994] edition of my book there’s a section called, “What if we do succeed?” Because it seemed to me that people in AI weren’t really thinking about that very much. Probably it was just too far away. But it’s pretty clear that success would be an enormous thing. “The biggest event in human history” might be a good way to describe it. And if that’s true, then we need to put a lot more thought than we are doing into what the precise shape of that event might be.

The basic idea of the intelligence explosion is that once machines reach a certain level of intelligence, they’ll be able to work on AI just like we do and improve their own capabilities — redesign their own hardware and so on — and their intelligence will zoom off the charts. Over the last few years, the community has gradually refined its arguments as to why there might be a problem. The most convincing argument has to do with value alignment: You build a system that’s extremely good at optimizing some utility function, but the utility function isn’t quite right. In [Oxford philosopher] Nick Bostrom’s book [Superintelligence], he has this example of paperclips. You say, “Make some paperclips.” And it turns the entire planet into a vast junkyard of paperclips. You build a super-optimizer; what utility function do you give it? Because it’s going to do it.

— Excerpt from a Quanta Magazine interview by Natalie Wolchover at the March 2015 meeting of the American Physical Society in San Antonio. The full interview is available at Quantamagazine.org

[An excellent presentation of the issues of risk surrounding A.I. development was recently given by Russell at the Centre for the Study of Existential Risk (15 May 2015, at Cambridge University). It is about an hour and twenty minutes, but well worth the time; he is quite intelligent and well-spoken. [see]