Beyond the Turing Test

2 March 2011


There is a remarkable quote from Alfred North Whitehead, revealing of the little-recognized subversive side of Whitehead’s thought, that came to mind recently:

“Discussions on the method of science wander off onto the topic of experiment. But experiment is nothing else than a mode of cooking the facts for the sake of exemplifying the law. Unfortunately the facts of history, even those of private individual history, are on too large a scale. They surge forward beyond control.”

Alfred North Whitehead, Adventures of Ideas, 1967, Chapter VI, “Foresight,” p.88

As a co-author of Principia Mathematica, along with Bertrand Russell, Whitehead is particular appropriate in the present context, having participated directly in the systematization of logic that ultimately made computers possible — with that possibility understood both in terms of the theoretical precedent to actual hardware as well as laying the foundations for what was to become computer science. In short, although not a household name, Alfred North Whitehead is a figure to be reckoned with in the history of ideas that have transformed human life.

Although this Whitehead quote occurs in the context of suggesting a scientific approach to history, Whitehead clearly had empirical science in mind when he wrote this, but we could make the same claim, or a similar claim, for thought experiments as well: a thought experiment is nothing else than a mode of cooking the facts for the sake of exemplifying the law.

One of the most famous thought experiments, known commonly even outside the provinces of philosophy, is the Turing test, proposed by Alan Turing and now a staple in almost all discussions of artificial intelligence and machine consciousness. Turing begins his paper “Computing Machinery and Intelligence” with the question, “Can machines think?” He suggests that the question is ambiguous and can be replaced by another formulation, which he called the imitation game. Turing describes the imitation game as follows:

It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart front the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either “X is A and Y is B” or “X is B and Y is A.” The interrogator is allowed to put questions to A and B thus:

C: Will X please tell me the length of his or her hair?

Now suppose X is actually A, then A must answer. It is A’s object in the game to try and cause C to make the wrong identification. His answer might therefore be:

“My hair is shingled, and the longest strands are about nine inches long.”

In order that tones of voice may not help the interrogator the answers should be written, or better still, typewritten. The ideal arrangement is to have a teleprinter communicating between the two rooms. Alternatively the question and answers can be repeated by an intermediary. The object of the game for the third player (B) is to help the interrogator. The best strategy for her is probably to give truthful answers. She can add such things as “I am the woman, don’t listen to him!” to her answers, but it will avail nothing as the man can make similar remarks.

We now ask the question, “What will happen when a machine takes the part of A in this game?” Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, “Can machines think?”

Turing, A.M. (1950). Computing machinery and intelligence. Mind, 59, 433-460.

Turing’s classic paper is better than anything that has been written in the past sixty years that has followed his original formulations. And while the imitation game is presented as a thought experiment, Turing himself immediately saw that it would not remain such, and he ventured this prediction in the same article:

I believe that in about fifty years’ time it will be possible, to programme computers, with a storage capacity of about 109, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning. The original question, “Can machines think?” I believe to be too meaningless to deserve discussion. Nevertheless I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted. I believe further that no useful purpose is served by concealing these beliefs. The popular view that scientists proceed inexorably from well-established fact to well-established fact, never being influenced by any improved conjecture, is quite mistaken. Provided it is made clear which are proved facts and which are conjectures, no harm can result. Conjectures are of great importance since they suggest useful lines of research.

There have been many attempts to create more or less authentic imitation games such as Turing described, but we can generically refer to all those tests that would have us attempt to distinguish between man and machine “Turing Tests.” I will use the term even more broadly and include man-machine contests in which the contestants are known and it is only a question of who wins, rather than than relative degree of indiscernibility of man and machine.

One very famous Turing Test, broadly construed, was the Deep Blue versus Garry Kasparov chess match. In 1996 Kasparov won the six-game human-computer match, and then in 1997 Deep Blue won the rematch. The contest was widely — and histrionically — reported at the time.

Quite recently, another specialized IBM computer, Watson, appeared on a special edition of the television game show Jeopardy! against two champions, soundly trouncing them. The New York Times reported this as Computer Wins on ‘Jeopardy!’: Trivial, It’s Not. Again, much histrionic reportage and commentary followed.

While we continue to converge upon the human/computer indistinguishability originally imagined and forecast by Turing, though the timeframe has slipped a little since Turing’s prediction, and Deep Blue and Watson have become emblematic of these efforts, something important has been lost in the way these events have been reported.

The Turing Tests I have mentioned (again, broadly construed) have been perfect examples of Whitehead’s claim that, “experiment is nothing else than a mode of cooking the facts for the sake of exemplifying the law.” Much in these Turing scenarios has been “cooked” and it highly artificial. Because of the developers of advanced computer systems and their software want to show up what they’ve been up to, they arrange stunts like the chess match and the Jeopardy! game. The fact that these are stunts occurring under controlled conditions are important — like a professional driver on a closed course demonstrating a car in a way about which most people can only fantasize.

Human beings long ago lost out in the contest of brute strength with the machines they have created. Everyone knows that human muscle cannot complete with mechanical engineering. However, there are still physical tasks that human beings can do that machines cannot do. This is why, although many routine tasks have been automated, at least as many have not been automated. Now the man/machine contest has moved to the mind, and the same kind of brute force approach (what Kurzweil calls, “simple methods combined with heavy doses of computation”) that demonstrated machine “superiority” in rote labor is being employed in AI, and the computer scientists pursuing this research program are looking for venues in which this approach can be shown in its best light. Hence the chess and Jeopardy! stunts.

We will know that the balance has shifted, and shifted decisively, in favor of the machines when there are contests transparently constructed with the opposite bias in mind: trying to find a particular approach to mental ability at which human beings excel but which machines are not good at. And it is to be expected that, whatever the gains in AI and expert systems and machine consciousness, there will tasks at which machines will greatly outstrip human efforts (any handheld calculator can do this already), but there will also be tasks at which human beings will continue to hold the edge. We can cook the facts either to show machine superiority or human superiority, as we desire.

Experiments take place under controlled conditions. That is to say, there are constraints placed upon what occurs in an experiment. The chess and Jeopardy! stunts artfully employ constraints in order to demonstrate what computers can do well, and perhaps do better than any human being. We could, however, just as well artfully employ constraints in order to demonstrate what human beings can do well, and perhaps do better than any machine. Anyone with any degree of perception can immediately see how patronizing this is, and our metaphysical pride stays our hand for the moment from any such demonstrations.

When the day comes that human pride must be sustained by patronizing stunts formulated for the purposes of demonstrating our abilities, then we will know that that pride is already a casualty, for the demonstration will be proof of that.

When the tables are well and truly turned it will not be human beings in the audience.

. . . . .


. . . . .


One Response to “Beyond the Turing Test”

  1. […] Beyond the Turing Test ( […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: