Friday, July 24, 2020

Regarding GPT-3's faculties


What mental faculties GPT-3 lacks:


  • It doesn't express "beliefs" as such. Depending on how it is prompted, it will say wildly different things. If it is in a context where factual answers are expected, then many of its answers will be true or partially true. But it will also confabulate with no indication that it is doing so.
  • It doesn't contain any information on where its knowledge came from. When we remember a fact, we tend to also remember where we learned that fact, the provenance of the information. This is completely lacking in GPT-3. Sometimes it can guess at where information came from, but it is just an educated guess.
  • It doesn't have senses. When it talks about the world, it is only talking about what it has "read", not what it has directly experienced.
  • It has no qualia. That is, you can't talk about what it would be like to be GPT-3, any more than you can talk about what it would be like to be a waterfall. It can't experience pain or pleasure. Well, it's impossible to say for sure (since we don't know what causes qualia in general), but I don't think it does. Certainly, when it produces text saying "I am seeing a red square" it is not very different from it producing text saying anything else, and is untrue, since it doesn't have any eyes or cameras.
  • It has no inherent conscience or guilt. However, it knows what a conscience is, what good and evil acts are, how a good person would behave in a new situation, and so on. So with the right prompt, it is able to make use of this knowledge as a moral compass.
  • It has difficulty staying on task for more than about two or three thousand words. If a plot takes longer than that to resolve, or a point takes longer than that to make, it probably won't get around to resolving the plot or concluding its argument.


What faculties it has:


  • It does contain correct and incorrect knowledge. It would be impossible to answer trivia questions as well as it does without something that should rightly be called "knowledge." (I would say that Wikipedia also contains knowledge in this sense.)
  • It does have a capability that I would argue is understanding or comprehension of some topics. When it uses a word, it can answer questions about what the word means, it can restate what it is saying in other words, it can provide more details, and it can summarize. It is not just moving around meaningless symbols, like earlier conversation bots such as ELIZA. Probing the limits of its understanding can be tricky at times, because of its tendency to confabulate. But I think it is misleading to say it has no understanding at all. (I would say that Google also contains some limited understanding, though less than GPT-3.)
  • It has something I would call "concepts." A concept, to my thinking, is a representation of meaning that has extensive, subtle connections with many related ideas. These concepts are stored as patterns in its network weights, and can be manipulated in many of the same ways humans make use of our concepts.
  • It is creative, if that word is to ever have meaning when applied to machines. It can write original fiction that, if it were produced by a human, we would call very creative. It can combine ideas and contexts to create new ideas.
  • It is good at analogies and metaphors. It can create original extended metaphors, and explain what role every related term plays in the metaphor. It can solve four-term (SAT style) analogies and explain why a particular analogy works.
  • It has a strong notion of context. It is able to correctly respond to the ways that the meaning of words, phrases, and sentences changes with context.
  • It has a limited theory of mind. It can figure out how a person would reasonably react to many kinds of situations.
  • It has one "goal" or "objective function": to produce text that is a plausible continuation of the prompt. With clever prompt design, this goal can be made to behave like many other goals. But the underlying, root goal is unchangeable. 


Where it's complicated:


  • Its network weights can be thought of as a kind of permanent memory, containing many facts about the world and about writing. Its prompt can be thought of as a kind of short-term memory. But it has no memory of any previous interactions not recorded in the prompt.
  • It can handle some kinds of humor well, while others are completely baffling to it. It can do a fair imitation of a humorous author. It can generate satire, exaggeration for effect, and dramatic irony. It cannot produce original puns very well. If it produces one-liners, they are typically either quoted or nonsensical. It's not good at creating original jokes as such.
  • It has limited spatial reasoning capability. It can correctly reason about prepositions like "in", "over", and "on." But if you describe a spatial arrangement of multiple objects, it can't reliably answer questions about how the objects are arranged relative to each other.
  • It has limited ability to perform deductive reasoning. When it is writing fiction, it can usually correctly deduce the consequences of actions and situations, in subtle ways that seem to require multiple steps of deduction. However, when given tests of deductive ability, it only does a little better than chance.
  • It isn't great at abstraction. When a scene is placed in a rich context, it is much better at figuring out what will happen than when it is reduced to its essence for a test.
  • It has only limited ability to work with images. It is aware of the meanings of many UNICODE emojis, and can use them fairly accurately. It can remember in-line ASCII art, and produce it where appropriate, but can't creatively come up with new ASCII art except in fairly trivial ways. It does a great job at describing realistic visual scenes, though. Also, the Transformer architecture has been shown to be able to realistically extend pixel images as well as It extends text.
  • It is not great at math. With the right formatting, it can do multi-digit addition, subtraction, and multiplication. It has memorized some math facts, but it doen't apply them. The Transformer architecture has been shown to support solving advanced calculus problems, though, when trained in the right way.
  • It has no self-awareness. If you use the prompt to tell GPT-3 about itself, however, it can be said to have some self-awareness, in a strictly functional sense.
  • It can easily be put in a state where its reactions are similar to humans experiencing emotions. This seems to me more like acting as if it has emotions than actually having them. If I say that, though, how is that different than saying it "acts as if it has knowledge"? It is an internal state that affects its behavior in ways similar to the way human emotional states affect human behavior. Similarly, it can simulate having desires and appetites.
  • Its ability to solve problems is difficult to characterize. It has certainly memorized the solution to many problems, and can sometimes apply those solutions to slightly different situations. It can brainstorm solutions to a problem, but its solutions can sometimes be impossible for one reason or another. It is difficult to give it enough knowledge of the situation to allow it to solve a problem, without essentially giving away the solution in the prompt.
  • Regarding willpower: I'm not sure what exactly that is, but there are a couple of ways you can get better results out of it that could be characterized as "trying harder": 1. When you give it a kind of internal dialogue, it can talk its way around to solving a problem it wouldn't solve without. 2. Prompting it to write an "award-winning" story or similar adjectives seems to be able to improve the quality of the results. 

No comments:

Post a Comment