Sunday, March 17, 2019


GPT-2, a text generator by OpenAI, has been making waves the last few weeks. The Turing Test is basically a kind of constrained text generation, so this area is particularly important in artificially creative machines that are meant to demonstrate intelligence. It seems to have motivated OpenAI to change their business model completely.

The post where they show off the kind of results they are getting with the model trained on a large dataset

You can try it yourself here, trained on a smaller dataset
If you haven't used a colaboratory before, it takes a little poking around to get how it works. What you need to do is run the first cell, get everything initialized, run the second cell, and then enter the text you want it to riff off at the prompt.
Some of the things I tried entering at the prompt, and getting it to generate:
computer code (it closes parentheses)
SCP artifacts (one of my favorites)
Bible (it creates sequential verses)
Star wars fanfiction
love letters

Here is material from a non-existent role playing game manual (unedited, except for line breaks):

Amulet of the Planeswalker 
Wondrous item, uncommon (requires attunement)
 You learn the art of andether and control the ever losing hearts of planeswalkers. In this celestial dance, you perceive your planes at will through the concepts of psionics. 

Amulet of the Heart 
Wondrous item, rare (requires attunement) 
This amulet increases mobility and allies' movement speeds. You gain an extra Strength score, as early as 1 level. 

Amulet of the Rathbone 
Wondrous item, rare (requires attunement) 
This amulet increases movement speeds and increases attack bonus against flying creatures by 10 percent. Known by creatures that receive different skills from you or others fanged, you are hit by a flurry of powerful and piercing attacks while studying this amulet. 

Amulet of the Spellcasters 
Wondrous item, uncommon (requires attunement) 
The amulet of death abides by a vague spell that negates the effects of spells. Spells you cast may damage your mortal senses by a plus-d3 to any target of the spell, even if the surrounding allies perceive you as their own. On an attack roll of +1, you may stun the target for 30 seconds. On all scores, the target may now remain stunned or staggered. Further, when you cast a spell other than Seismic Prayer, you may cause the target's save DC and the caster's Cha modifier to equal your spell level before you take the spell. Use of the Ward cost 1d8 damage and Anticipation cost 4d8 damage. 

Amulet of Shadows 
Wondrous item, rare (requires attunement) 
Hand-to-Hand Combat changes the appearance of your vision to resemble the appearance of any other rogue. 
Style: While wielding a staff of silver, you sense the motions of the luminous, maddening movement of the 23-foot-radius (88 cubits) green dragon and can pass through on every turn. While the wearer is dead, enemy spells or creatures that are within 30 feet gain the illusion of you. 10 If 1 want the same effect, roll a d6 to determine the desired effect. 

Amulet in the Mystic Alchemical 
You can re-energize this amulet with the follow enchantment. It will easily hold 300 magicka in it's holder. Fiendish Stone-Sands Replaceting this item causes the Inquisitor to hydrate the ground around Daisuki in his decaying room. It cures 2000 ft of fur poisoning. 

If you want to see various options for the next word, and how probable it thinks they are, you can try it at the AllenAI institute site. Try always clicking the top word, versus always clicking the second word or the tenth word to get a taste of how you could introduce variety into the generated sentences.

You can get better results by fine-tuning the model on a dataset that contains only the kind of thing you are trying to generate. Gwern has been playing with poetry generation. Scott Alexander had some commentary on this.
Mario Klingemann has been generating wise quotes.

Thursday, March 14, 2019

Style Transfer for Jewelry Design

I've been playing a lot with style transfer this year, and one of my favorite results has been from copying jewelry styles. I've done a little cleanup on these-- I only generate one half and mirror it, make sure it has a clean background, and I use the "healing brush" tool here and there. Here are a few examples (click to see larger):

People are always interested to see the style images I used, so here are a few:

Friday, December 14, 2018

Progress on music generation and style transfer

The Google Brain team has out a new post with samples from their Music Transformer software. I feel like the most successful piece is improvisation based on the chord progression from Hotel California (under score conditioning) but I dislike jazz piano in general so the additional structure helps it feel more like music and less like randomness to me.

Sunday, October 7, 2018

Steampunk city-sized computers

"When the vast extent of a machine sufficiently large to include all words and sequences is considered, we observe at once the absolute impossibility of forming one for practical purposes, in as much as it would cover an area exceeding probably all London, and the very attempt to move its respective parts upon each other, would inevitably cause its own destruction."

From The process of thought adapted to words and language, by Alfred Smee, 1851 

Monday, October 1, 2018

Logic Machines and Diagrams

I just got from Amazon a little known book by Martin Gardner called Logic Machines and Diagrams. The cover shows Ramon Llull's diagram, which was the cover to the prototype copy of Machinamenta. Here's the first paragraph of the foreword: 
If the various branches of discovery were to be measured by their relative antiquities, then of all scientific pursuits the mechanization of thought must be the most respectable. The ancient Babylonians had mechanical aids to reckoning, and in Plato's time geometers were already building machines to support formal derivations. 
I think this is going to be my kind of book!  

Friday, January 5, 2018

Semantic Primes

I've been thinking about how to build up a dictionary how you would if you were trapped on a desert island with someone who didn't speak your language and you wanted to teach it to them from scratch. I got a very interesting book called Semantics: Primes and Universals by Anna Wierzbicka that talks about the very first words you would use to start such a project. The "semantic primes" are around 40 words that she takes as undefinable but from which a much larger number of words can be defined. She says that these words are special because every human language has a word (or a sense of a word) that is a direct translation of each of these 40 words, and that they are among the first concepts that children learn to express. She also talks a little about a simplified grammar that lets you combine these words. is an online dictionary/lesson plan that builds up English in this way. It starts with

61 semantic primes, defined mainly by pictures and examples. Using only these 61 words, it defines

300 "semantic molecules." Using only these semantic molecules, it defines

2000 words used in the Longman Defining Vocabulary. These words are used to define

230,000 words in the Longman Dictionary of Contemporary English.

Now imagine that you found a way to program the meaning of these 61 words into a robot, and programmed in the ability to read a sentence using these 61 words and derive the meaning of a new word from that. You could build up to the meaning of all the words in the dictionary this way. Programming those first 61 words and the grammar would be challenging, but I don't think impossible.

I don't think this would be sufficient to understand everything about those concepts. Suppose I gave the definition "an arc-shaped fruit, around 6-12 inches long, with soft white flesh and a skin that is green when unripe, yellow when ripe, and soft and brown when overripe, and grows in bunches." This would be enough to pick out a banana from any other food in the supermarket, but it wouldn't tell you much about what a banana really looks like. You wouldn't be able to recognize a banana split from such a definition. A good definition generally tells you just enough to distinguish the item from any potential confusers. But it would be an excellent start that you could begin to flesh out with other capabilities.   

Wednesday, December 6, 2017

Some of my recent experiments with style transfer

These combine style transfer with some manual editing. I'm using the deep learning output as a tool to get images I like to see, and I like to see what I can get it to do. Some of the tricks I've learned:

  • If you want to add details to a real item, make sure that the scale and lighting are consistent between the source and style images.
  • Monochrome is generally more successful than depending on it to get color right.
  • Bas-relief style details, and other shallow sculptural details work really well.
  • If two images are very similar, you can use the details of the high-resolution one to enhance the low resolution one. But this requires a very close correspondence.
  • You can make fractals by doing the low-res to high-res trick repeatedly with the same image, zooming in on small parts of it.
  • You can force the output to have symmetry by making sure both the source and style image are symmetric. This works especially well when the style image is symmetric with some shallow depth but lit from one side. Then the output will be, too.
  • You can use frequency decomposition to add details to an image without affecting the overall composition. In Photoshop, this is done by the following process:

    • detail image: high pass at radius n, linear light blending mode, 50 % opacity, on top of:
    • background image: gaussian blur at radius n