The difficulty of predicting difficulty

In his 1979 Pulitzer Prize winning book “Gödel, Escher, Bach,” Indiana University computer scientist Douglas Hofstadter surmised about what it might take for an AI to beat the best humans at Chess. He predicted that the AI would have to possess such a high-level of general intelligence, it might decline your suggestion to play chess, and suggest talking about poetry instead. 

In 1997, less than 20 years after Hofstadter’s prediction, IBM’s Deep Blue computer defeated Chess world champion Garry Kasparov. Deep Blue achieved this feat using a fairly basic algorithmic technique, called alpha-beta pruning, to conduct brute force search through millions of Chess positions per second. Not only did Deep Blue lack an appreciation for poetry, it actually could do nothing other than play Chess.

This is a cautionary tale about the difficulty of predicting the evolution of machine intelligence. It is notoriously difficult to assess what intelligence capabilities are required to achieve a given feat. A seemingly simple task, like recognizing traffic signs under different weather conditions, may turn out to be much harder than initially believed, while a difficult task, like defeating the reigning Chess world champion, turns out to be much easier. We would do well to keep this lesson in mind as we make predictions about the evolution of AI technology.

References

• Hofstadter, D. R. & Others. Gödel, Escher, Bach: an eternal golden braid. vol. 13 (Basic books New York, 1979)

• Hsu, F.-H. Behind Deep Blue: Building the Computer that Defeated the World Chess Champion. (Princeton University Press, 2002)