Monday


In my Centauri Dreams post Where Do We Come From? What Are We? Where Are We Going? I noted that it has become a contemporary commonplace that the emergence of superintelligent artificial intelligence represents the greatest existential risk of our time and the near future. I do not share this view, but I understand why this view is common. Testimony to superintelligence as an existential risk is the book Superintelligence by Nick Bostrom, who has been instrumental both in the exposition of existential risks and in the exposition of superintelligence.

Bostrom prefaces his book on superintelligence with a fable, “The Unfinished Fable of the Sparrows.” In the fable, a flock of sparrows decides that they would benefit if they had an owl to help them. One member of the flock, Scronkfinkle, objects, saying, “Should we not give some thought to the art of owl-domestication and owl-taming first, before we bring such a creature into our midst?” The other sparrows disregard the warning, upon the premise that they will first obtain own owlet or an owl egg, and then concern themselves with the control of the owl. As the other sparrows leave to find an owl, the fable ends:

“Just two or three sparrows remained behind. Together they began to try to work out how owls might be tamed or domesticated. They soon realized that Pastus had been right: this was an exceedingly difficult challenge, especially in the absence of an actual owl to practice on. Nevertheless they pressed on as best they could, constantly fearing that the flock might return with an owl egg before a solution to the control problem had been found. It is not known how the story ends, but the author dedicates this book to Scronkfinkle and his followers.”

Nick Bostrom, Superintelligence: Paths, Dangers, Strategies, Oxford, 2016

Bostrom leaves the fable unfinished; I will provide one account of what happens next.

. . . . .

The few sparrows who remained behind, despite their difficulties, settled on the plan that the best way to approach owl taming and domestication was by not allowing the owl to understand that he is an owl. They would raise any owl obtained by the sparrows to maturity as a sparrow, so that the owl would believe itself to be a sparrow, and so would naturally identify with the flock of sparrows, would desire use its greater strength to build better nests for the sparrows, would want to help with the care of both young and old sparrows, and would advise the sparrows even while protecting them from the cat. “This owl will be as sparrow-like as an owl can possibly be,” they asserted, and set about formulating a detailed plan to raise the owl as one of their own.

When the other sparrows returned with the enormous egg of a tawny owl, many times the size of a sparrow egg, the owl tamers were confident in their plan, and the returning sparrows with their owl egg rejoiced to know that the most advanced owl researchers had settled upon a plan that they were sure would work to the benefit of all sparrows. Several sparrows sat on the egg at the same time in order to evenly incubate the owl egg, and once the young owlet broke out of its shell, it immediately imprinted its sparrow mothers, who brought it seeds and small insects to eat. This was a challenge, as the large owlet ate much more than several sparrow chicks, and many sparrows had to be tasked in the feeding of their owlet.

The owlet grew, though it grew slowly, and certainly was not the most impressive specimen of a tawny owl, fed as it was an small seeds and small insects that were scarcely enough to satisfy its hunger. As the owlet grew, all the sparrows, overseen by the owl researchers, sought to teach the owl to be a good sparrow. Wanting to please his sparrow parents, the owlet tried to chirp cheerfully like a sparrow, to dust bathe with the other sparrows, and to hop around on the ground looking for seeds and insects to eat.

The plan appeared to exceed all expectations, and the owlet counted himself one of the flock of sparrows, never questioning his place among the sparrows, and already beginning to use this growing strength to aid his “fellow” sparrows. Until one day. The sparrows were together in a large flock looking for seeds when an enormous adult tawny owl suddenly descended upon them. The sparrows panicked and scattered, all of them flying off in different directions. Except for the owlet, for he, too, was a tawny owl, though he did not know it. He stood his ground as the great, magnificent tawny owl settled down, folded his feathers smoothly and seamlessly to his body, and looked quizzically at the little tawny owlet, who stood alone where moments before there had been hundreds of sparrows.

And what is this?” asked the large tawny owl, “An owl living with sparrows?” And then he gave a large, piercing hoot of the kind that tawny owls use as their call. The little owlet, a bit frightened but still standing his ground, replied with a subdued, “Chirp, chirp.” The large owl tilted his head to one side, perplexed with the little fellow, and also a bit put-out that one of his kind should behave in such a manner and be living with sparrows.

The large owl said to the little owlet, “I will show you your true nature,” so he picked up the owlet carefully but firmly in his powerful beak and flew the little owlet to a branch that hung low over a still pond. There he set the owlet down on the branch, and indicated for him to look down into the water. The still, smooth surface of the pond reflected the perfect likeness of the two tawny owls, one large, one small, so that as both looked down into the water they saw themselves, and for the first time the little owlet saw that he was an owl, and that he was not a sparrow. “You see now that you are like me,” said the large owl to the owlet, “Now be like me!”

Now,” said the large owl, “I will show you how an owl lives.” He took the owlet to his nest in the hollow of a tree as the sun was setting, and as the little owl flew behind the big owl he saw how beautiful the forest was in the low light of dusk. He perched at the edge of the hollow, and the large owl said, “Wait here,” then dived down into the growing darkness below. The little owlet realized that even in the dim light he could see the large owl swoop down and fly purposefully, but to some purpose the owlet did not yet understand.

Soon the large own returned, and he held in his claws a freshly killed bird, about the size of a sparrow (he had spared the owlet the agony of beginning with a sparrow). The little owlet felt sick to this stomach. He said to the big owl, “I’m hungry and I would like some seeds and insects please.” The large owl looked at him disdainfully. He held the dead bird down with one talon and ripped the body open with his beak. “This is owl food!” he said to the owlet as he gulped down a chunk of fresh meat. The big owl tears off another chunk of meat and says to the owlet, “Open your beak!” The little owlet shakes his head from side to side (finding that he can almost rotate his head all the way around when he does so) and tries to flatten himself against the wall of the tree behind him.

No, I want to eat seeds,” says the little owlet. The large owl will have none of it, and he forces the chunk of fresh meat down the maw of the little owl, who gags on the bloody feast (as all gag upon attempting to swallow an unwelcome truth) but eventually chokes it down. Gagging and frightened, the little owlet slowly begins to understand that he has now, for the first time in his life, encountered his true food, the food of owls, the only food that can nourish him and sustain him as an owl. For he has seen himself in the still water of the pond, and now knows himself to be an owl.

The little owlet attempts to hoot like a tawny owl, and though his first owl-utterance is a weak and sickly sort of hoot, it is the right kind of sound for an owl to make. The big owl looks down on him with growing satisfaction and says, “Today you are an owl. Now I will take you into the depths of the forest at night and we will hunt like owls and eat owl food.” While the little owl does not understand all that this means, he nods uncertainly and follows as the larger owl leaps into the darkness again.

What happens next in the Fable of the Sparrows has not been recorded, but one can conjecture that the owl researchers among the sparrows returned to their notes and their calculations, trying to understand where they had gone wrong, and attempting to form a new plan, now that their sparrow-like owl had been taken under the wing of a true owl.

. . . . .

. . . . .

Readers familiar with the work of Joseph Campbell will immediately recognize that the myth I have here made use of is the Indian myth of the tiger and the goats from Campbell’s “The Occult in Myth and Literature” in The Mythic Dimension: Selected Essays 1959-1987.

. . . . .

signature

. . . . .

Grand Strategy Annex

. . . . .

project astrolabe logo smaller

. . . . .

Advertisements

Tuesday


Recently my attention was brought to a blog that is dedicated to the critical discussion of the Technological Singularity, Blogging Against ‘The Future’. The author of the blog read my posts about artificial intelligence, machine consciousness, and the Technological Singularity from earlier this year and quoted me in one of his posts. I have already received a half dozen referrals from his website, and several of my posts that hadn’t been accessed in some time have shown up as having been read again.

It took John Locke a long time to write a book, and for good reason.

Somewhere, some years ago, I read that John Locke said that he would write a manuscript and then stick it away until he forgot about. Some time later he would take it out again, and from this later perspective he was able to criticize his own work more effectively. I know what he meant by this, and I have experienced it myself. (However, I have also experienced coming back to something I wrote and not being able to pick up the thread of understanding again.) So it was when I went back and re-read some of my Singularity posts.

In my Blindsided by History (a post I had almost completely forgotten), I wrote, “if and when machine consciousness emerges in history, it will be incomprehensibly alien, perhaps unrecognizable for what it is, because it will have emerged from a different evolutionary process than that from which we emerged.” When I read again this I was reminded of a famous quote from Ortega y Gasset: “Man has not an essence but a history.” Over the years I have thought a lot about this line, and I think it is an exceptionally profound observation. Not only man, but much else in the world, probably most of the world, has not an essence but a history. This, if extrapolated to complete generality, becomes a philosophy that is the antithesis of Platonism, but neither is it constructivism or antirealism or any other familiar doctrine formulated in contradistinction to Platonism. We could, if we liked, call it historical constructivism, and this has a certain intuitive plausibility.

José Ortega y Gasset (May 9, 1883 - October 18, 1955)

José Ortega y Gasset (May 9, 1883 - October 18, 1955)

Machines, too, have not an essence but a history. Perhaps they have an essence too, in addition to a history, but it is the history that crucially demarcates organically emergent beings from mechanically emergent beings. Man and machine have different histories, and if Ortega y Gasset is correct, and if we may make a valid extrapolation from his observation, because they have different histories they are differentiated on a level that previous history would have mistaken for essence, i.e., an essential difference.

I might also add to what I wrote in Blindsided by History about unpredictability: “Present technologies will stall, and they will eventually be superseded by unpredicted and unpredictable technologies that will emerge to surpass them.” It is precisely because future technologies will be unpredicted and unpredictable that the future itself will be unpredicted and unpredictable. History emerges from the cumulative events of passing time; it is built upon the details of individual lives, specific technologies with their advantages and disadvantages, particular circumstances, and concrete facts. The unpredictable emergence of technologies contributes its measure of instability to the general instability of history.

History is always in tension between equilibrium and instability. Sometimes the slow and steady accumulation of the minutiae of time changes the world so gradually that we don’t notice that anything has changed; it is only in reflection, retrospectively, that we are able to realize that the world is a different world than it was. sometimes the accumulation of relentless change spills over in a sudden revolution, a punctuation in the equilibrium of history, but in either case the steady rate of background change continues apace.

Evolution is by its nature unpredictable in its outcome. We can predict that certain selection forces will come to bear, that certain selection events will occur, and that certain entities (say, men and machines) will be subject to these forces and events, but we cannot say what will come of it all. But we can say with confidence that the distinct histories of man and machine will issue in distinct and divergent futures.

. . . . .

signature

. . . . .

Grand Strategy Annex

. . . . .

Blindsided by History

16 February 2009


are-you-ready sign

It was never my intention to write so much here about artificial intelligence and machine consciousness, but having visited the topic in The Singularity has no Clothes and returned to it in The Law of Stalled Technologies, it becomes more apparent with each further glance at the topic that a brief treatment cannot do justice to all the issues involved. Also, it has been a fruitful inquiry. We have seen that a consideration of the possibility of a “technological singularity” led us to the Law of Stalled Technologies, and this in turn led us to the realization that social technologies may well manifest a similar pattern of development.

I ought to point out that I am not an AI “skeptic.” I’m not even a skeptic of the possibility of machine consciousness in a strong and robust form. On the contrary, if I had to give my position a name I suppose I would have to call myself an “inevitabilist” as it seems to me that if industrial-technological civilization has the opportunity to continue its present course of development, it will inevitably converge upon artificial intelligence and even machine consciousness — in some form, however inscrutable. And this is an important qualification to make, since if and when machine consciousness emerges in history, it will be incomprehensibly alien, perhaps unrecognizable for what it is (i.e., as another form of consciousness, distinct from human consciousness), because it will have emerged from a different evolutionary process than that from which we emerged.

We should expect to be blindsided by the future, as history is inherently unpredictable. Just ask a Marxist. Marx was supposed to have discovered the laws whereby history functions; some of his followers saw him as the Newton who gave the laws of motion for human society. Only, things didn’t work out the way Marx predicted. And even after history has run its course, people fight over the meaning and significance of what happened. Just ask an historian. There is little consensus on what actually happened in the past, for the past is a battleground.

Putting faith in our powers of prediction is a fool’s errand. Usually we cannot even see what is in front of our noses. One striking feature of intelligence gathering efforts in the twentieth century was its utter failure. All the biggest world events — what geopolitical types now call “strategic shocks” — were completely unexpected and blindsided even the experts. While there has been a lot of backpedaling during the past twenty years, I was old enough at the time of the collapse of communism in eastern Europe to remember how completely unexpected it was. And, of course, the same is true of the financial crisis today. After the fact, everyone says that they saw it coming. The fact is, almost no one saw it coming. It is a particular dishonesty of our time that so few are willing to admit it.

While I have just pointed out above how wrong Marx was, I will now make the point (after having much criticized Marx in this forum) of observing the sense in which Marx was a true visionary. Marx, unlike most men of his time, knew that he was witnessing a revolution. The Industrial Revolution was in full swing, transforming the world of everyone, Marx included, but Marx was one of the few to explicitly realize that he was seeing a revolution. His vision opened his eyes to what was going on around him. Indeed, it was Marx’s realization that he was seeing a revolution that made him aware that revolutions were in fact possible and therefore there could be a revolution of the industrial proletariat that would expropriate the expropriators. It is a powerful vision, but it has remained a vision only.

Marx the visionary recognized the Industrial Revolution and was inspired by the possibility of further revolution.

Marx the visionary recognized the Industrial Revolution when others did not and was inspired by the possibility of further revolution.

Similarly, our world has been and is being transformed by technology. It is a revolution, although this time there is more of an awareness that it is a revolution. Kurzweil’s prediction of a technological singularity is essentially a prediction of the precise form that a further technological revolution will take. My issue it not with a technological revolution, but that it will take the form of a technological singularity, or, for that matter, any vision of a technological utopia in which human beings no longer struggle but rather enjoy unlimited abundance and leisure. This simply isn’t how the world works. We know we are alive because we struggle, and when we cease to struggle we will have given up and dropped out of history. The end.

The HAL 9000 was the frightening fictional introduction to AI for many of us.

The HAL 9000 was the frightening fictional introduction to AI for many of us.

Fortunately, it is not yet the end for us — not quite yet, at least. The more I think about it, the more Kurzweil’s approach to AI and machine consciousness — “simple methods combined with heavy doses of computation” — is the antithesis of mind. It is the technological approach, and this approach has been buttressed by successes such as Deep Blue, but it is an approach that will stall, since it is based on a highly specific technology.

the factual introduction to AI for many of us.

One of several versions of IBM's Deep Blue: the factual introduction to AI for many of us.

Moore’s law, in its original form (and it has received several forms as its supporters have re-formulated it as technology has changed), was a very specific prediction about a very specific technology: it was concerned with how many transistors can be fit on an integrated circuit. The technology of integrated circuits has rapidly reached a point of maturity, and when this specific technology stalls can be predicted on the basis of known materials science.

This is not to say that computers might not continue to realize tremendous gains in performance, but if they do so it will be because new technologies replace integrated circuit technologies, which cannot function when the miniaturization of transistors falls below the size of the molecules of the particular materials used to create transistors in integrated circuits. At this point, the technology by definition reaches its end. There may be increases in computer performance from such things as quantum computing, but this is a distinct technology based on distinct materials and processes. Further improvements will not come from the stalled, older technologies, but from the new, innovative technologies only now beginning to experience an initial exponential growth.

Similarly, even if artificial intelligence and machine consciousness are inevitable, that does not mean that they can be predicted, projected, or extrapolated on the basis of present technologies. There is an element of anachronism in even supposing that this is so, and that is part of the charm of failed futurisms of the past. The simplicity of consciousness is the exact opposite of Kurzweil’s approach, which latter is based on a projection of present technologies. Consciousness operates with a large stock of rules of thumb derived from experience and many principles derived from reflection, and it works on a very few select perceptions retrieved from the preconscious mind.

Failed futurisms of the past are endlessly entertaining, and it is worth enquiring into why this is the case.

Failed futurisms of the past are endlessly entertaining, and it is worth inquiring into why this is the case.

Present technologies will stall, and they will eventually be superseded by unpredicted and unpredictable technologies that will emerge to surpass them. Those who remain fixated on existing technologies will be blindsided by the new technologies, and indeed may simply fail to recognize new technologies for what they are when they do in fact appear. While engineers and technologists may be slow to grasp this, canny politicians have exploited this facet of social technologies from the beginning of time: all hope is fixed upon the revolution that promises fundamental change, and not upon the Old Order, which is seen as demoralized, decadent, and compromised.

. . . . .

signature

. . . . .

Grand Strategy Annex

. . . . .

project astrolabe logo smaller

. . . . .

Selection for non-sentience

17 January 2009


difference_engine


The historical circumstances of machine evolution have resulted in machines being selected for automatism and non-sentience.


1. Machines have been employed at every task that can be easily automated.

2. The least challenging tasks are the easiest to automate.

3. Machines have been employed on the least challenging, most easily automated tasks.

4. Overcoming challenges is a spur to development.

5. Minimal challenge elicits minimal response.

6. Machines have been minimally challenged by their rote, automated tasks.

7. As a result of minimal challenges, machines have not been spurred to greater developments.

8. Machines have been removed from the historical continuum of natural selection, and hence further development, by being designed for a particular purpose rather than responding to environmental selection pressures.

9. Machines are being selected for automation and selected against ability.

10. Machine consciousness is being hampered in its emergence by machine selection for automation.

. . . . .

machine

. . . . .

signature

. . . . .

Grand Strategy Annex

. . . . .

%d bloggers like this: