More on the creativity of GPT-3

Gwern has access to the beta version of GPT-3. I applied, but I can't help but think that working for the Army is not helpful to my cause in this case. Anyway, Gwern prompted GPT-3 with the following:

Topic: humor: literary parodies and homages, pastiches, style parodies.

 

Parodies of the fantasy novel series _Harry Potter_ in the style of various famous authors:

 

By Ernest Hemingway:

 

"It was a cold day on Privet Drive. A child cried. Harry felt nothing.

The rest of this -- all of it -- is what GPT-3 generated as a continuation. You can see that the prompt says "parodies" in the plural and "various." This nudged GPT-3 to choose authors on its own, and copy their styles on its own from what it has read of them:

He was dryer than dust. He had been silent too long. He had not felt love. He had scarcely felt hate. Yet the Dementor's Kiss killed nothing. Death didn't leave him less dead than he had been a second before. It wasn't about living or dying really. It wasn't about death. It was about the Dark Lord, his creator, his engenderer. He was not resurected. He was created again. He was whole again in the courtyard of Malfoy Manor."

 

By Jane Austen:

 

"It is a truth universally acknowledged, that a broken Harry is in want of a book. This he knows to his cost; pressing that wretched nose against the window of a bookstore in Diagon Alley the last thing before he goes into hiding for several years, where upon, he goes straight to Azkaban."

 

By Arthur Conan Doyle:

 

''Harry pushed at the swinging doors of the bookshop hard, and nearly knocked himself unconscious. He staggered in with his ungainly package, his cheeks scarlet with cold and the shame of having chosen the wrong month to go Christmas shopping. The proprietor of the store, however, didn't cast even a cursory glance at him, being far more interested in an enormous hunk of wood lying in the middle of the floor, which certainly looked like a gallows. Yes, the proprietor said to a reedy old man wearing a bowler hat and a forlorn expression that made Harry want to kick him, I can rent you such a gallows for a small fee."

 

By Czar Nicholas II of Russia:

 

The terror has befallen on our land. Curses, dark curses and all of this was instigated by my court. My treacherous court! They have cast are curse spells upon my land. Everywhere it is cold, all land belongs to Voldemort's Death Eaters. All citizens live in a fear of the Dark Lord. Alas! There is no hope for me. What shall I do? Flee? Nay, that would be cowardly. Fight? Yes!

 

By George Orwell:

 

"It was a bright cold day in April, and the clocks were striking thirteen."

 

By Ingmar Bergman:

 

"Tears filled Harry's eyes. Sweat stood on his forehead, showing the pure torment, the agony he suffered. He hugged his knees to his chest, sobbing softly, eyes half shut."

 

By Anna Akhmatova:

 

"The dawn came with a jolt of terror."

 

By Bram Stoker:

 

""Tis a fearful sight to see the cold, girlish venom run through his veins, one by one. The scar coruscates; a ghastly emerald lightning in the dark of night. But I'll attend him and I watch him

 

From the very moment the dawn is gray

 

To the rays of sunset fading away."

 

By P. G. Wodehouse:

 

"'There was nothing out of the way, sir,' said Harry in a hurt voice.

 

'"Indeed,' said the headmaster, turning his lorgnette precisely three-quarters of a millimeter to port. 'For example, your tie was not undone, you were not laughing, and your fingers were most certainly closed upon your wand.'

 

'"They were?" said Harry, bewildered.

 

"It's always the same,' said Dumbledore."

 

By Vladimir Neigus:

 

"Vasya Pupkin's lips were blueup and his head felt dizzy from working nonstop. He wanted to take a square squeaky shoe and shoot it at the ceiling fan in his bedroom which went round and round and round and round. He imagined that because he was so drunk it would always hit the exact same spot and the fan would go out of control, die and fall on the table."

 

By William Shakespeare:

 

"Owl-Lo!'

 

The boy awoke with a start and sprang to his feet."

 

By Oscar Wilde:

 

"The boy was gone, and other people had arrived: or rather nobody else had arrived, and the boy was there:

 

"What are you doing here, Harry?" said Hermione.

 

The Boy Who Lived, who was reading a book about broomsticks and may not have been listening quite attentively, said:

 

"I was talking to Peeves while he cleaned out the bins."

 

"About the troll?" Hermione demanded.

 

"Yes, about the troll."

 

By W. B. Yeats:

 

"Harry waked out of his simple thought

 

The glimmer was over the wished-for thing...

 

Then was many a dark and wrinkled elf

 

Come from the underground rout,

 

With brags and boasts, and fine big words,

 

To bargain with Harry Potter."

 

Added by other anonymous editors: @astromark

 

By John Ashbery:

 

"Tousled, he scrambled down

 

the branches of the bed,

 

his cold-creamed face all out

 

of proportion with his small

 

brown eyes...And why

 

so demented in the face?

 

Because he was

 

saving the world, in seven years,

 

from destruction..."

 

By Henry James:

 

"As a laggard, in all ways, but the face, which as the great shade approached him, horrified, choked him,--and before he had time to reflect, he fastened his hand upon his throat to hush it,--there was a voice behind him..."

 

By H. P. Lovecraft:

 

"It is a common saying in the South, when one wishes to describe a strapping fellow, that he has a face like Harry Potter and a neck like a young giraffe. Now, it being autumn, the dusk was of a rich violet blackness, with scarlet illuminations..."

 

By Nzingha Prescod:

 

"...thick grey clouds overlap the faint lavender. A black sky vanishes into purple shards against the sweeping night time blanketed with infinite stars. A brightness emanates from the entire universe that unexpectedly takes flight and hovers, hovered over the castle. Harry is there to greet ...

 

"Facing away from all the tumult on the grounds of Hogwarts, Harry Potter learned to fly..."

 

By Yasunari Kawabata:

 

"Harry Potter stood on a silent trolley, far from the center of the world, looking out on the world far from the center of the universe. The snow was falling all night long. It fell upon tiny dogs and birds, fine wintery flakes gently drifting down."

 

By Hilda Doolittle:

 

"Harry waked out of his simple thought. As the waiter drew the curtain of her bed, she appeared in the handle. She was his K-19 and she started to drop the snakes of their home in the grass there. He saw his wife walking there with oak settle. He decided to not go to the game also...

 

By Walt Whitman:

 

"And when the time of the test drew near, Harry entered the forbidden Forest the Killers with a Snitch..."

 

By E. E. Cummings:

 

"Halloween and mad are the children

 

Who gather in the darkness

 

Halloween and late are those who go out and dance

 

And call to all, the smiling are enchanted

 

By the beauty of what they see."

 

By Anthony Burgess:

 

"Ha-a-a-a-rry was not a tol-a-a-r of the phar-a-a-na-a-a-gical sports."

 

By Virginia Woolf:

 

"Horseman, Where She Go? I Do Not Know. Whither Ride?"

RAW Paste Data

  

Poetry generation with rhyme, meter, and phrase constraints

In mid-February 2020, I was stuck in a small, crowded airport for several hours on the way home from a meeting in upstate New York. (This was the last international meeting I would attend before Covid-19 shut everything down.) I had been working with GPT-2 to create text adventures in a fundamentally different way than AI Dungeon 2. So while I was in the airport and on the plane ride home, I wrote a program that generated rhyming poetry using GPT-2. I had already written another program to create rhyming word pairs using word2vec, so I knew that rhymes could be checked using the CMU pronunciation dictionary.
This first program worked on a token-by-token basis. Only whole words which were also tokens could be generated. The last token-word in the second line had to rhyme with the last token-word in the first line. The selection was made more efficient by dividing the list of words to be generated into sets of rhyming words that all rhymed with each other, a trick I had used for the word-pairs program earlier.
While the results were interesting, I was unsatisfied because it lacked meter constraints.So in May I again used the CMU pronunciation dictionary to generate all possible continuations  of a prompt over a certain probability using a depth-first search method. It followed the most probable continuations, and when it wasn't able to satisfy a meter, rhyme, or probability constraint, it backed up and tried again.
This worked pretty well, but it was slow. So again I created something like the rhyme sets, but this time for meter. Any word in the pronunciation dictionary that could be consistent with the next three stresses was allowed in the set associated with those stresses. Since line ends also needed to be accounted for, I also included one- and two-stress sets.
The biggest remaining problem was that sentences often ended in the middle of lines, and the poem had no notion of ending at the end of the poem-- it was as if it were clipped arbitrarily from a longer poem. So I added in what I call phrase constraints. This says that a line must end in some punctuation (? ,! , . , , , ; , or :) and that punctuation must be sufficiently probable. If not, it must back off and try again. This does an adequate job of making the meaningful phrases line up with the lines of the poem. (It prevents some perfectly acceptable poems from being generated, though.)
If you want to try it out, here is the github repository.

GPT-3 and creativity

Here's a sample generated by GPT-3:

Sample #99

If your personal philosophy is a little bit Taoist and a little bit Marxist, there's a new place in LA to scratch that particular philosophical itch. Tao Mao is the only East-meets-West neo-communist tiki bar in LA, a three-story Chinatown bordello of poi dogs and palm trees. To get there, you enter from a rear alley and walk past the washboard abs of dancers in the Bumpin' Uglies Go-Go Bar to the karaoke bar called Red 7, and from there climb a red-lit staircase to the restaurant.

Tao Mao, "the great origin," is the Chinese name for the Milky Way galaxy, as seen from the Earth. But it's also the name of a star in that galaxy. And in Mandarin, Tao Mao is pronounced "Dow Mao," which sounds like "D.M.A.," or "doctor of philosophy." And that is exactly what Tao Mao is—a tiki bar where all the drinks have the names of philosophers.

You may be wondering: Is Tao Mao a tiki bar for philosophers? Or a bar for philosophers that serves tiki drinks? Or is it both? Does it matter?

Actually, no, it doesn't. Tao Mao is a philosophical tiki bar, that's all you need to know. The more you think about it, the less sense it makes.

Before I try to explain Tao Mao, I should mention that the bartender told me the bar's name is pronounced "Dow Mao." And he is a white dude. And yet... all the cocktail names are in Mandarin. And he only speaks English. So what the fuck do I know? Maybe I'm the crazy one.

I can tell you that the people who seem most at home at Tao Mao are little clusters of friends—mainly white dudes who look like they'd be hanging out at some over-designed tiki bar that was actually inspired by Tao Mao. For example, I met two TV writers who were there for the first time. The owner was there with his mom. The first time I visited, the woman in the next booth to me was there with her sister, and

The name "Tao Mao" is not the name of a real bar. So the wordplay is its own invention. One of my favorite things about GPT-3 is that when it fails to continue a list like you want, a common failure mode is to start explaining what all the things in the list have in common, or otherwise analyse the list (because that's a common thing people do when they write a list on the web).