Jaron Lanier writes about the Myth of AI, and how unquestioning acceptance of it distorts work in the field. He's primarily interested in economic ramifications which I haven't given much thought to, but I'm glad that he also sees that there is an element of P.T. Barnum to the whole enterprise:
"If you ask: is a recommendation engine like Amazon more manipulative, or more of a legitimate measurement device? There's no way to know. At this point there's no way to know, because it's too universal. The same thing can be said for any other big data system that recommends courses of action to people, whether it's the Google ad business, or social networks like Facebook deciding what you see, or any of the myriad of dating apps. All of these things, there's no baseline, so we don't know to what degree they're measurement versus manipulation.
"If you ask: is a recommendation engine like Amazon more manipulative, or more of a legitimate measurement device? There's no way to know. At this point there's no way to know, because it's too universal. The same thing can be said for any other big data system that recommends courses of action to people, whether it's the Google ad business, or social networks like Facebook deciding what you see, or any of the myriad of dating apps. All of these things, there's no baseline, so we don't know to what degree they're measurement versus manipulation.
Dating always has an element of manipulation; shopping always has an element of manipulation; in a sense, a lot of the things that people use these things for have always been a little manipulative. There's always been a little bit of nonsense. And that's not necessarily a terrible thing, or the end of the world.
But it's important to understand it if this is becoming the basis of the whole economy and the whole civilization. If people are deciding what books to read based on a momentum within the recommendation engine that isn't going back to a virgin population, that hasn't been manipulated, then the whole thing is spun out of control and doesn't mean anything anymore. It's not so much a rise of evil as a rise of nonsense. It's a mass incompetence, as opposed to Skynet from the Terminator movies. That's what this type of AI turns into..."
