Tuesday, March 4, 2014

Excerpt 13 The Literary Engine, Turing's Love Letter, Eliza

The Literary Engine

In Gulliver’s Travels, Jonathan Swift made light of devices that create language automatically and randomly. The engine was meant to parody the Royal Society, who were interested in codes and ciphers as well as the study of nature. Like the infinite shelves of books described in Borges’ story “The Library of Babel,” the device he describes contained all possible sentences, both sensible and nonsensical:
 Six Hours a-day the young Students were employed in this Labour, and the Professor shewed me several Volumes in large Folio already collected, of broken Sentences, which he intended to piece together, and out of those rich Materials to give the World a compleat Body of all Arts and Sciences; which however might be still improved, and much expedited, if the Publick would raise a Fund for making and employing five hundred such Frames in Lagado, and oblige the Managers to contribute in common their several Collections.Every one knew how laborious the usual Method is of attaining to Arts and Sciences; whereas by his Contrivance, the most ignorant Person at a reasonable Charge, and with a little bodily Labour, may write Books in Philosophy, Poetry, Politicks, Law, Mathematicks and Theology, without the least Assistance from Genius or Study. He then led me to the Frame, about the Sides whereof all his Pupils stood in Ranks. It was twenty Foot Square, placed in the middle of the Room. The Superficies was composed of several bits of Wood, about the bigness of a Dye, but some larger than others. They were all linked together by slender Wires. These bits of Wood were covered on every Square with Paper pasted on them, and on these Papers were written all the Words of their Language, in their several Moods, Tenses, and Declensions, but without any Order. The Professor then desired me to observe, for he was going to set his Engine at Work. The Pupils at his Command took each of them hold of an Iron Handle, whereof there were fourty fixed round the Edges of the Frame, and giving them a sudden turn, the whole Disposition of the Words was entirely changed. He then commanded six and thirty of the Lads to read the several Lines softly as they appeared upon the Frame; and where they found three or four Words together that might make part of a Sentence, they dictated to the four remaining Boys who were Scribes. This Work was repeated three or four Times, and at every turn the Engine was so contrived that the Words shifted into new Places, as the Square bits of Wood moved upside down.
 He assured me, that this Invention had employed all his Thoughts from his Youth, that he had emptyed the whole Vocabulary into his Frame, and made the strictest Computation of the general Proportion there is in Books between the Numbers of Particles, Nouns, and Verbs, and other Parts of Speech.[1]

Turing’s Love Letter

Perhaps the first person to use an electronic computer to compose original pieces of writing was Alan Turing. He wrote a love poem generator as a joke in 1947. It used random numbers to choose nouns and adjectives to fill some of the parts of a canned love letter[2]:

Darling Sweetheart,

You are my avid fellow feeling. My affection curiously clings to your passionate wish. My liking yearns to your heart. You are my wistful sympathy: my tender liking.

Yours beautifully


Compared to the love letters of, say, Robert and Elizabeth Browning, it comes up a little short. It is similar in structure to how the utterances of fate were determined in divination rituals. The design is the kaleidoscopic pattern:
  • ·        A system of generating unpredictable output (in this case, a pseudorandom number generator).
  • ·        Elements to be recombined which themselves have some aesthetic appeal. (The words such as sweetheart, wistful, or tender which are randomly chosen.)
  • ·        A structure that imposes certain constraints on the results so that they will be interpreted as a sensible arrangement. (The fixed sentence grammar.)

Artists will protest that by “creative” we mean something more than this. What more could be added to a machine besides a larger number of elements and ways to be rearranged, and a more versatile and subtle structure to put them in? I think that the answer lies in the ability to recognize beauty or quality, in order to reject generated art that doesn’t meet this standard, rather than simply constraining the possible creations. Research into how to accomplish this is still in its infancy.


The first program designed for natural language conversation was a simulation of a psychiatrist, known as Eliza. Eliza’s conversation used the patient’s words in a superficial attempt to keep the conversation going on a topic without contributing much of its own. For example:

Well, my boyfriend made me come here.


He says I'm depressed much of the time.


It's true. I am unhappy


I need some help, that much seems certain.

Eliza is an attempt to automate (or perhaps parody) nondirective Rogerian psychotherapy, a method developed in the 1940’s and 50’s by psychologist Carl Rogers. Rogers recommended a technique he called “reflection,” in which a therapist restates the patient’s concern to show empathy and understanding, and to help the patient to find a way to solve his or her own problem. The key to this method is that the doctor is not providing solutions to the problems; the patient is providing both the problems and the solutions.
This helps us understand how meaning can be created by divination or artistic machines. When a fortune teller sits down with a client, very little actually comes from the system of manipulating signs. Instead, the fortune teller provides a way for the subconscious mind of the client to interpret a signal out of noise.
Similarly, our response to generated art is like finding shapes in the clouds or Rorschach’s inkblots. The beauty and meaning come from our attempt to find something we recognize in randomness. The machine itself does not have actual experiences to draw on. But what it can do is form an effective mirror where the viewer both provides, and is affected by, the meaning.
The author of the Eliza program, Joseph Weizenbaum, stated in an interview:
You can see Eliza using one basic method or, you could even say, trick: Eliza relies on the fact that the human being interprets the signals he perceives. He interprets these signals according to his needs and his interests. He projects his own image of his partner, whether this is a living human, conversing, or whether this is a living human being and a doll interacting or whatever.
He doubted the possibility of a programmed machine ever being able to actually mean the things it was saying:
No, that's impossible. The human being becomes a human, because he is understood and treated as a human by other humans. And that's where the deepest truths come from which nourish the human being - for example trust: to trust another human. There are things, like for example a hand on your shoulder: language is closely related to this and is learnt and developed by being based on such experiences. The computer can't have these experiences.
So we return to the question of meaning.[3] Do the words in a book have meaning, after the writer has written them and before the reader has picked it up? Imagine finding a book in the library, and being moved by what is written there. If the words that one finds meaningful got there by some other process than being written by an author (say, by monkeys pounding on typewriters who got really lucky), can we say that the meaning is somehow false, not meaning at all because it wasn’t meant? That doesn’t seem right. But Weizenbaum’s point too, seems like common sense. Meaning can’t just pop up, like so many crocuses in the spring. For there to be meaning, it seems like there has to be a mind.
It is simple to create a system that contains the fact “PARIS is the CAPITAL of FRANCE.” The same system could also contain the phrase “PARIS is the CAPTAL of THE MOON,” without protest. This is because it doesn’t understand the words PARIS, CAPITAL, FRANCE, or THE MOON. But systems are being developed[4] that will contain the fact that capitals have to be of countries, that all countries are on the earth, that one can’t have a city where no one lives, that Paris is a city, that the moon is uninhabitable, and so on—millions upon millions of facts and the logical means to connect them and derive new facts from them. Such a system would balk at being told that Paris is the capital of the moon, because it is inconsistent with the large body of facts it already contains. In this limited sense, it can be said to “understand” what the sentence means.
However, there is another sense of the word “understand” that will be discussed in Chapter VI, where we “understand” when someone mentions a particular sensory experience they have had, such as listening to music. This kind of understanding cannot be communicated through a network of relationships and definitions. Whenever humans understand something, it is at the lowest level grounded in this kind of direct understanding, direct experiencing, that can’t be broken down further. Part of what I mean by THE MOON is what it feels like to be gazing up at it on a cold October evening. I can only point to that experience, and if you’ve had one similar, you can understand. If not, no amount of explaining is going to communicate it to you.
For practical purposes this doesn’t make much difference, and most AI researchers are mainly concerned about practical purposes. For artists, though, it seems to matter a great deal. For some reason, we do care whether an artist is being authentic. The idea of receiving love letters written by someone who is not actually in love, but is incapable of feeling at all and is only “going through the motions” is distasteful even if we’re sure they’ll keep up the pretence.
In review, then, machines made to generate text all followed a similar pattern: strict rules to guarantee grammatical correctness with a few random elements. To the extent that the text they generated was meaningful, the meaning originated in the creator of the machine or in the reader.
For all their flaws, though, these machines did generate new text every time they were run. The machines in the next chapter, while imitating many fascinating abilities, don’t rise to that standard. However, they do illustrate the growing capability of machines to imitate other human faculties needed for creative expression.

[1] Swift, Gulliver’s Travels III:V
[2]Lavington, 1975 from Boden, Mind as Machine p. 674. It would be interesting to discover who Turing intended these letters for, given what is now common knowledge about his sexual orientation.
[3] The study of meaning in language is called semantics. The phrase “semantick philosophy” was used in the 1600’s and 1700s to refer to the study of divination systems. For example, in The British Apollo (Vol. III, 1708), the anonymous author writes “Bacon proposes this and several other sorts of divination as parts of rational and useful knowledge. Whatever this Semantick Philosophy was in former times…”
A better known reference is from John Spencer, A Discourse Concerning Prodigies, 1665.
[4] Doug Lenat’s CYC or the Commonsense Computing Initiative at MIT are two prominent examples, though they will probably be surpassed by other efforts soon. The Semantic Web is a related effort advocated by Tim Berners-Lee, who invented the web. “Semantic” refers to meaning; the Semantic Web is an effort to develop tools and practices that will allow this kind of automated reasoning to take place across the internet.