Tolkien and Darwin Among the Machines

In The Letters of J.R.R. Tolkien, I found the following passage. He is writing to his son Christopher, who was training to be an air force pilot in Africa during World War II:

I wonder how you are getting on with your flying since you first went solo-- the last news we had of this. I especially noted your observations on the skimming martins. That touches to the heart of things, doesn't it? There is the tragedy and despair of all machinery laid bare. Unlike art which is content to create a new secondary world in the mind, it attempts to actualize desire, and so to create power in this World; and that cannot really be done with any real satisfaction. Labour-saving machinery only creates endless and worse labour. And in addition to this fundamental disability of a creature, is added the Fall, which makes our devices not only fail of their desire but turn to new and horrible evil. So we come inevitably from Daedalus and Icarus to the Giant Bomber. It is not an advance in wisdom! This terrible truth, glimpsed long ago by Sam Butler, sticks out so plainly and is so horrifyingly exhibited in our time, with its even worse menace for the future, that it seems almost a world wide mental disease that only a tiny minority perceive it. Even if people have heard the legends (which is getting rarer) they have no inkling of their portent...Well, I have got over 2 thousand words onto this flimsy little airletter; and I will forgive the Mordor-gadgets some of their sins, if they will bring it quickly to you... (7 July 1944, letter 75)

Christopher seems to have pointed out in a letter some contrast between the natural flying of the martins (swallows that fly with their mouths open to skim insects) and the laborious difficulty of flying and taking care of his own air force planes. His father was reminded of the main theme of The Lord of The Rings, which he was deep in the middle of writing at that point (and in fact he considered Christopher his main audience for the work). The idea is that our attempt to improve the world through creating technology is doomed to bring about our destruction in the end. This apocalyptic vision of the future of automation he felt had been captured in the writings of Samuel Butler-- referring, surely, to Darwin Among the Machines. Tolkien is, as far as I am aware, the only reader before the 1950s who ever took Butler's story as a serious warning about a possible future, rather than just a silly parody. I would love to be able to read Tolkien's own vision of an apocalyptic dystopian future where the machines have risen.
Later, in January 1945, upon learning that the Russians were 60 miles from Berlin, he writes about the tragedy of the destruction of Germany ("be it 100 times merited") and then writes,

Well, the first War of the Machines seems to be drawing to its final inconclusive chapter -- leaving, alas, everyone the poorer, many bereaved or maimed and millions dead, and only one thing triumphant: the Machines. As the servants of the Machine are becoming a privileged class, the Machines are going to be enormously more powerful. What's their next move? (January 30, 1945, Letter 96)

He writes upon learning of the atomic bomb (again in a letter to Christopher):

The news today about 'Atomic bombs' is so horrifying one is stunned. The utter folly of these lunatic physicists to do such work for war purposes: calmly plotting the destruction of the world! Such explosives in men's hands, while their moral and intellectual status is declining, is about as useful as giving out firearms to all inmates of a gaol and then saying that you hope 'this will ensure peace.' But one good thing may arise out of it, I suppose, if the write-ups are not overheated: Japan ought to cave in. Well, we're all in God's hands. But he does not look kindly on Babel-builders. (9 August 1945, letter  102)

Later he writes to his publisher Unwin:

You can make the Ring an allegory of our own time, if you like: an allegory of the inevitable fate that awaits all attempts to defeat evil power by power. But that is only because all power magical or mechanical does always so work. (31 July 1947, letter 109)

(C.S. Lewis shared many of these concerns, by the way. There is the 'Deplorable Word' that destroyed the world of Charn in The Magician's Nephew and one of the best warnings about the danger of messing around with sentients' utility functions I have ever read in The Abolition of Man.)

Excerpt 5

The problem becomes, then, how to make a machine that grows in ability over time that is not limited by the initial choices made by the inventor of the machine.

Brewster conceived of the kaleidoscope as a labor saving device for artists, an automation of part of the creative process:

"When we consider, that in this busy island thousands of individuals are wholly occupied with the composition of symmetrical designs, and that there is scarcely any profession into which these designs do not enter as a necessary part, so as to employ a portion of the time of every artist, we shall not hesitate in admitting, that an instrument must have no small degree of utility which abridges the labour of so many individuals. If we reflect further on the nature of the designs which are thus composed, and on the methods which must be employed in their composition, the Kaleidoscope will assume the character of the highest class of machinery, which improves at the same time that it abridges the exertions of individuals. There are few machines, indeed, which rise higher above the operations of human skill. It will create, in a single hour, what a thousand artists could not invent in the course of a year; and while it works with such unexampled rapidity, it works also with a corresponding beauty and precision.[1]"

The Reverend Leigh Richmond wrote on a similar theme:

"I took up my kaleidoscope; and, as I viewed with delight the extraordinary succession of beautiful images which it presented to my sight, I was struck…with the singular phenomenon of perfect order being invariably and constantly produced out of perfect disorder—so that, as by magical influence, confusion and irregularity seemed to become the prolific parents of symmetry and beauty.

 It occurred to me, that the universality of its adoption would imperceptibly lead to the cultivation of the principles of taste, elegance, and beauty, through the whole of the present and following generations, and that from the philosopher and artist down to the poorest child in the community…

 I saw a vast accession to the sources of invention, in its application to the elegant arts and manufactures, and the consequent growth of a more polished and highly cultivated state of habits, manners, and refinement, in both… I was struck with the idea of infinite variety more strikingly demonstrated to the eye than by any former experiment. Here the sublime mingles with the beautiful.

 I perceived a kind of visible music. The combination of form and colour produced harmony—their succession melody: thus, what an organ or pianoforte is to the ear, the kaleidoscope is to the eye. I was delighted with this analogy between the senses, as exercised in this interesting experiment.…[2]"

Things did not turn out quite as Sir Brewster and Rev. Richmond expected. Our use of machines to automate work previously done by artists has modified our concept of beauty. What used to take a great deal of skill and time could be done immediately and without effort by a mechanical process. This led to society valuing designs of rigid perfect symmetry less. A similar effect occurred with the invention of photography. Because of the ease of obtaining a perfectly accurate likeness, abstract and nonrepresentational art became more highly valued by the art world.

Brewster was at the same time able to see his invention as a toy, and as an important advance of science.[3] He saw it as a kind of proof of principle, which later engineers would use for practical purposes, in much the same way that components of automata found their way into practical machinery:

"The passion for automatic exhibitions which characterized the eighteenth century gave rise to the most ingenious mechanical devices, and introduced among the higher orders of artists habits of nice and accurate execution in the formation of the most delicate pieces of machinery. The same combination of the mechanical powers which made the spider crawl, or which waved the tiny rod of the magician, contributed in future years to purposes of higher import. Those wheels and pinions, which almost eluded our senses by their minuteness, reappeared in the stupendous mechanism of our spinning-machines and our steam engines. The elements of the tumbling puppet were revived in the chronometer, which now conducts our navy through the ocean; and the shapeless wheel which directed the hand of the drawing automaton has served in the present age to guide the movements of the tambouring engine. Those mechanical wonders which in one century enriched only the conjurer who used them, contributed in another to augment the wealth of the nation; and those automatic toys which once amused the vulgar, are now employed in extending the power and promoting the civilization of our species. In whatever way, indeed, the power of genius may invent or combine, and to whatever low or even ludicrous purposes that invention or combination may be originally applied, society receives a gift which it can never lose; and though the value of the seed may not be at once recognized, and though it may lie long unproductive in the ungenial till of human knowledge, it will some time or other evolve its germ, and yield to mankind its natural and abundant harvest.[4]"

---

[1] Ibid.

[2] Rev. Richmond, Hogg's Weekly Instructor, Volume 6 1851

[3] At the Media Research Laboratory at NYU where I studied in 2004 and 2005, a form of kaleidoscope was actually used to study the way isotropic materials, like satin, appear and respond to light at various angles. It was a realization of Brewster’s hope that the kaleidoscope would find a scientific use.

[4] Brewster, Letters on Natural Magic, 1868, p.336

Excerpt 4

Sir David Brewster

Sir David Brewster, like many people of the time with an inclination to research and reading, studied theology to become a teacher and a licensed preacher. His interest in optics led to many significant discoveries about diffraction, refraction, and the use of lenses. Along with Fresnel, he was responsible for getting Fresnel lenses installed in lighthouses, and he invented the lenticular stereoscope (which uses a prism rather than mirrors to combine the stereo images). Brewster was conducting experiments on light polarization and happened on the design of the kaleidoscope. When Brewster showed his prototype kaleidoscope to manufacturers of optical instruments, pirate copies began cropping up all over the London and soon spread around the world:

You can form no conception of the effect which the instrument excited in London; all that you have heard falls infinitely short of the reality. No book and no instrument in the memory of man ever produced such a singular effect. They are exhibited publicly on the streets for a penny, and I had the pleasure of paying this sum yesterday; these are about two feet long and a foot wide. Infants are seen carrying them in their hands, the coachmen on their boxes are busy using them, and thousands of poor people make their bread by making and selling them. (Letter from Brewster to his wife, May 1818)

The kaleidoscope allowed the viewer to enter into a virtual world, filled with bright colors and concealed symmetries. If it was a scientific instrument (as the name implied), it was an instrument of some faerie science, a science of beauty. It partook of the potential of mirrors to create other worlds, to open up new infinite spaces. The forms were reminiscent of magical mandalas, and viewers often compared the hypnotic effect of looking through a shifting kaleidoscope to that of listening to music.¹

Brewster was an early proponent of the idea that magic and beauty could be found in technology. He wrote a series of letters to Sir Walter Scott on the topic of natural magic. For Brewster, understanding how magic tricks and automata worked only increased their appeal. This was in stark contrast to his contemporaries, romantic artists like John Keats, who felt that science killed beauty:

…Do not all charms fly

At the mere touch of cold philosophy?

There was an awful rainbow once in heaven:

We know her woof, her texture; she is given

In the dull catalogue of common things.

Philosophy will clip an Angel’s wings,

Conquer all mysteries by rule and line,

Empty the haunted air, and gnomed mine -

Unweave a rainbow, as it erewhile made

The tender-person’d Lamia melt into a shade. ²

Brewster’s conception of beauty, on the other hand, was grounded in neoclassicism. Symmetry and geometric order were key ideas in this. Beyond that, he assumed that a science of beauty was possible, that universal principles of beauty could be discovered:

If we examine the various objects of art which have exercised the skill and ingenuity of man, we shall find that they derive all their beauty from the symmetry of their form, and that one work of art excels another in proportion as it exhibits a more perfect development of this principle of beauty. Even the forms of animal, vegetable, and mineral bodies, derive their beauty from the same source...³

In The Kaleidoscope (a book on the optical theory behind the construction of his invention) he gives a theory of color harmony and repeatedly emphasizes the importance of carefully constructed devices that don’t allow the slightest imperfection in symmetry.

Yet both neoclassical and romantic concepts are evident in the kaleidoscope: the hand selected elements are romantically beautiful, beautiful in how they present themselves to the senses. The formal constraints, the mirrors, are classically beautiful in how they appeal to the intellect.

It is comparatively simple to set up a system of rules and generate new images. It is much more difficult to choose a set of rules that will produce images that are aesthetically pleasing. In order to do the latter, we need to have some theory of beauty or interest. The attempt to mechanize requires that we understand; but the attempt to understand beauty transforms it. There is essentially a paradox here: creativity must continually be pushing the boundaries of what is new. Simply being new is not enough, however; to be considered creative it must be both new and beautiful. Any static conception of beauty must quickly become inadequate.

1 The idea of an analogy between color and music dated at least to 1590, when the artist Arcimboldo invented a system for composing color-music. In 1725, the Jesuit monk Louis Bertrand Castel invented an “ocular harpsichord,” which opened a curtain concealing a bit of colored glass whenever a note was played. Isaac Newton was the first to realize that there may be a deeper connection in that both colors and sounds have characteristic frequencies. Despite thousands of related efforts over the years, including the light bars on an equalizer, Disney’s Fantasia, and MTV, visual music that is able to give the same kind of effect through the eyes that music gives through the ears is still elusive.

2 John Keats, Lamia, Part II, 1819

3 David Brewster, The Kaleidoscope, chapter 20

Excerpt 3

Beauty from the Symmetry of Their Form:

The Invention of the Kaleidoscope

In 1817, Sir David Brewster patented the kaleidoscope. Others had noticed the effect of two mirrors meeting at an angle before, as recounted in this selection from an 1818 article in The Edinburgh Magazine and Literary Miscellany:

The repetition and reversion of images in a glass is noticed in the Masfiti Naturalis of Baptista Porta, a Neapolitan nobleman, who flourished about the latter part of the sixteenth century, and was distinguished for his zeal in promoting philosophical pursuits…

In the Ars Magna Lucís et Umbra of Kircher, printed in 1646, we have an account of the same circumstance, and also of the repetition of the sectors round the centre of the circle:

“A wonderful property,” says he, “and one which has not, as far as I know, been observed by any one, is exhibited with two specula, so constructed as to open and shut like a book; and placed on any plane in which you have described a semicircle divided into its degrees. For, if the point in which the specula meet be placed in the centre of the semicircle, so that the side of each speculum shall stand upon the diameter, the image of an object will only be seen once, and two objects will appear, one without the specula, the true one,—and one within, the image. But if the sides be placed at an angle of 120°, you will see the image of the object within the specula twice, that is, along with the real image, three objects But if the specula intercept an angle of 90°, you will see the circle divided into four parts, and four objects; in the same manner, at an angle of 60°, you will see a hexagon with six objects.’¹

He then applies the principle to some curious contrivances which, by his own account, filled his spectators with astonishment. With one candle he shows how to make a complete chandelier. “With angles of 120°, 72°, and 45°, you will see,” says he, “with no less delight than admiration, a chandelier with three, with five, and with eight branches.”

1 Athanasius Kircher, Ars Magna Lucís et Umbra, 1646

Excerpt 2

Illusions

A few of the machines covered in this book were intended for practical purposes, but most of them were for built for entertainment, magic, or both. These machines play on many illusions that are built into the human way of seeing the world. The early scientists drew little distinction between experiment and demonstration, and many instruments designed to illustrate a principle or entertain an audience were later used to further scientific knowledge. Showing how these illusions are built into our understanding of creative machines and processes is a second theme of the book. In the earliest case, divination machines, the devices were treated as magical by true believers in magic. Through the 1800s, automata were part of magic shows, presented as if they were magical, but with the audience aware that the magic was a carefully contrived illusion. More serious efforts at artificial intelligence beginning with invention of electronic computers inadvertently followed many of the same techniques, and had the same effect of fooling audiences into seeing the illusion of a mind, but their inventors often neglected to acknowledge the underlying illusions.

A History of Creative Devices

The title of this book, Machinamenta, is a Latin word that means “machines.” It has only the oldest connotations—machines as siege engines, as tools of stagecraft, as ingenious contraptions. It was also used to mean clever schemes—devices in the other sense, or machinations. A primary meaning of machina in the middle ages was the cranes used by architects for building. So there is a sense of “creation” in this early definition. It was used as a metaphor in the phrase machina mentis, machines of the mind, to describe how the tools of memory could be used as a tool for innovation. The 17^th century scholar Athanasius Kircher used machinamenta to describe some marvelous devices, including the self-playing Aeolian harp. So it seemed appropriate to gather under this term this diverse collection of artistic devices.

The world of computers changes incredibly quickly. Papers from a decade ago in my own field, computer vision and graphics, are almost certain to have been surpassed by more recent research that has built on them. Many of the pioneers involved with the first digital computers are still alive today. A drawback of this is that as a field, we have a very short memory. We forget that other people have been struggling with the same questions for many, many years. The problems faced in trying to build intelligent and creative machines are not merely technical, but philosophical. What is the difference between creative and derivative? What is the nature of beauty? What makes something interesting? How does the mind work?

The history of the field of computer science usually only goes back as far as World War II, with perhaps a mention of Babbage. Predictions of the future of the field, however, have never been in short supply. The field of artificial intelligence has more than its share of prophets, playing on the same hopes and fears that have been associated with machines that can speak to us since prehistoric times. Only by examining the project of AI in terms of its deep philosophical, mechanical, and spiritual roots can we make proper judgments about the nature of these machines now and in the future.

Excerpt 1

Introduction

Are machines capable of being creative? The concept at first appears to be an oxymoron—“mechanical” is an antonym of “creative.” Yet for centuries, people have imagined the potential of automated thought and creativity. While they made some naïve mistakes, they often had a broader perspective on how this problem fits into the larger context of philosophy and society.

The history this book presents is a miscellaneous one. It draws from the history of the arts, magic, religion, toys, games, staged entertainment, philosophy and language. All of these threads are part of the same story: the invention of machines to automate the creation of new designs or new ideas.

The Kaleidoscope Pattern

For all their diversity, there is a pattern common to many of these devices, one that will show up again and again throughout the book. The pattern can be seen most easily in the construction of a kaleidoscope. A kaleidoscope is a very simple machine consisting of three parts:

•         Colored bits of glass.
•         Two mirrors that impose a regularity, or formal structure.
•         A means of randomizing the arrangement.

These three parts show up in virtually every attempt to make creative machines (of which the kaleidoscope is one example). They embody a theory of creativity that is centuries old: that the random rearrangement of interesting ideas or images along with an enforced logical structure is the way that our minds are able to invent new objects.

Such machines are wonderful and fascinating in their own right. One of the main purposes of this book is simply to gather many examples of these machines and exhibit them together as examples of early efforts at automating creativity. Because they fall in the cracks between art and science, many of them have been nearly forgotten by both artists and scientists, and this is a tragedy.

For all their beauty and intricacy, however, these machines are ultimately unable to deliver on their promise of true, sustained creativity. Eventually, the new images created by a kaleidoscope no longer have the power to delight and intrigue. We come to see the theme behind the variations, and each individual work no longer brings anything new to our understanding of that theme. A machine that was truly creative would be able to find ways to keep being new, and to be new in new ways.

No one has built such a machine. Despite the promise of evolutionary algorithms and machine learning techniques, every attempt so far has eventually petered out. After some initial surprises, all of these programs in the long run end up coming up only with new variations on the same themes. No programs have been run for year after year without human interference, coming up with new creations that are enthusiastically exhibited and admired.

Probably the most famous program to create visual art is Harold Cohen’s AARON. AARON’s work has been exhibited in major museums around the world, and its output has been described as creative by both artists and computer scientists. Each new painting created by AARON is original, and can be surprising even to Mr. Cohen. Yet he feels that despite AARON’s success in the art world, it is still not creative. The reason he feels this is that AARON is essentially a kaleidoscope at heart. It has a model of a human figure, something like a paper marionette. The pose of the model, the proportions of the limbs, the placement on the page, the colors of the body segments, are all chosen randomly. The random values have constraints that keep them within reasonable bounds. A similar process generates plants and backgrounds for the scenes. Once all of this has been placed, a separate routine traces the outline of these shapes in a semi-random fashion. How is it, then, that so many people take AARON to be creative?  The answer is that the program is taking advantage of certain very powerful illusions.

Article in Steampunk Magazine #9

I published an article concentrating on Victorian creative machines in Steampunk Magazine #9. It''s available for free as a pdf. 

Statistical measures of visual interest

This salad captures many aspects of what makes a Pinterest image popular, according to researchers. The features they mention, however (lack of faces, proportions, color) are features it is easy to measure with current image processing algorithms. They don't take into account any of the meaning of what is in the picture. For example, it includes food, which is a popular topic on Pinterest. Any method of trying to understand what people like that ignores the semantics of the image is going to be very limited.