Wednesday, June 5, 2019

Randomization as an advantage in divination

Sometimes being random can be an advantage. When you are playing a game against an opponent who is trying to guess your strategy, having the ability to use randomization to be unguessable is critical. In hunting, the caribou are trying to guess the strategy of the hunters to elude them. From The Secret of Our Success by anthropologist Joseph Heinreich:

"Traditionally, Naskapi hunters decided where to go to hunt using divination and believed that the shoulder bones of caribou could point the way to success. To start the ritual, the shoulder blade was heated over hot coals in a way that caused patterns of cracks and burnt spots to form. This patterning was then read as a kind of map, which was held in a pre-specified orientation. The cracking patterns were (probably) essentially random from the point of view of hunting locations, since the outcomes depended on myriad details about the bone, fire, ambient temperature, and heating process. Thus, these divination rituals may have provided a crude randomizing device that helped hunters avoid their own decision-making biases."

(Thanks for the book review, Scott Alexander)

Friday, April 26, 2019

OpenAI's Musenet

Musenet uses the same 'transformer' architecture as GPT-2-- it is a general method for continuing any sequence. You can even feed it an image as a scan-line and it will finish up the image in a fairly realistic way. You can find some nice fugues here:

Sunday, March 17, 2019


GPT-2, a text generator by OpenAI, has been making waves the last few weeks. The Turing Test is basically a kind of constrained text generation, so this area is particularly important in artificially creative machines that are meant to demonstrate intelligence. It seems to have motivated OpenAI to change their business model completely.

The post where they show off the kind of results they are getting with the model trained on a large dataset

You can try it yourself here, trained on a smaller dataset
If you haven't used a colaboratory before, it takes a little poking around to get how it works. What you need to do is run the first cell, get everything initialized, run the second cell, and then enter the text you want it to riff off at the prompt.
Some of the things I tried entering at the prompt, and getting it to generate:
computer code (it closes parentheses)
SCP artifacts (one of my favorites)
Bible (it creates sequential verses)
Star wars fanfiction
love letters

Here is material from a non-existent role playing game manual (unedited, except for line breaks):

Amulet of the Planeswalker 
Wondrous item, uncommon (requires attunement)
 You learn the art of andether and control the ever losing hearts of planeswalkers. In this celestial dance, you perceive your planes at will through the concepts of psionics. 

Amulet of the Heart 
Wondrous item, rare (requires attunement) 
This amulet increases mobility and allies' movement speeds. You gain an extra Strength score, as early as 1 level. 

Amulet of the Rathbone 
Wondrous item, rare (requires attunement) 
This amulet increases movement speeds and increases attack bonus against flying creatures by 10 percent. Known by creatures that receive different skills from you or others fanged, you are hit by a flurry of powerful and piercing attacks while studying this amulet. 

Amulet of the Spellcasters 
Wondrous item, uncommon (requires attunement) 
The amulet of death abides by a vague spell that negates the effects of spells. Spells you cast may damage your mortal senses by a plus-d3 to any target of the spell, even if the surrounding allies perceive you as their own. On an attack roll of +1, you may stun the target for 30 seconds. On all scores, the target may now remain stunned or staggered. Further, when you cast a spell other than Seismic Prayer, you may cause the target's save DC and the caster's Cha modifier to equal your spell level before you take the spell. Use of the Ward cost 1d8 damage and Anticipation cost 4d8 damage. 

Amulet of Shadows 
Wondrous item, rare (requires attunement) 
Hand-to-Hand Combat changes the appearance of your vision to resemble the appearance of any other rogue. 
Style: While wielding a staff of silver, you sense the motions of the luminous, maddening movement of the 23-foot-radius (88 cubits) green dragon and can pass through on every turn. While the wearer is dead, enemy spells or creatures that are within 30 feet gain the illusion of you. 10 If 1 want the same effect, roll a d6 to determine the desired effect. 

Amulet in the Mystic Alchemical 
You can re-energize this amulet with the follow enchantment. It will easily hold 300 magicka in it's holder. Fiendish Stone-Sands Replaceting this item causes the Inquisitor to hydrate the ground around Daisuki in his decaying room. It cures 2000 ft of fur poisoning. 

If you want to see various options for the next word, and how probable it thinks they are, you can try it at the AllenAI institute site. Try always clicking the top word, versus always clicking the second word or the tenth word to get a taste of how you could introduce variety into the generated sentences.

You can get better results by fine-tuning the model on a dataset that contains only the kind of thing you are trying to generate. Gwern has been playing with poetry generation. Scott Alexander had some commentary on this.
Mario Klingemann has been generating wise quotes.

Thursday, March 14, 2019

Style Transfer for Jewelry Design

I've been playing a lot with style transfer this year, and one of my favorite results has been from copying jewelry styles. I've done a little cleanup on these-- I only generate one half and mirror it, make sure it has a clean background, and I use the "healing brush" tool here and there. Here are a few examples (click to see larger):

People are always interested to see the style images I used, so here are a few:

Friday, December 14, 2018

Progress on music generation and style transfer

The Google Brain team has out a new post with samples from their Music Transformer software. I feel like the most successful piece is improvisation based on the chord progression from Hotel California (under score conditioning) but I dislike jazz piano in general so the additional structure helps it feel more like music and less like randomness to me.

Sunday, October 7, 2018

Steampunk city-sized computers

"When the vast extent of a machine sufficiently large to include all words and sequences is considered, we observe at once the absolute impossibility of forming one for practical purposes, in as much as it would cover an area exceeding probably all London, and the very attempt to move its respective parts upon each other, would inevitably cause its own destruction."

From The process of thought adapted to words and language, by Alfred Smee, 1851