Aaronson begins by summarizing some of the recent events, and noting all the angry and anguished reactions he has seen, many of them sent to him via email. But:
How can anyone stop being fascinated for long enough to be angry? It’s like ten thousand science-fiction stories, but also not quite like any of them. When was the last time something that filled years of your dreams and fantasies finally entered reality: losing your virginity, the birth of your first child, the central open problem of your field getting solved? That’s the scale of the thing. How does anyone stop gazing in slack-jawed wonderment, long enough to form and express so many confident opinions? . . .
An alien has awoken — admittedly, an alien of our own fashioning, a golem, more the embodied spirit of all the words on the internet than a coherent self with independent goals. How could our eyes not pop with eagerness to learn everything this alien has to teach? If the alien sometimes struggles with arithmetic or logic puzzles, if its eerie flashes of brilliance are intermixed with stupidity, hallucinations, and misplaced confidence … well then, all the more interesting! Could the alien ever cross the line into sentience, to feeling anger and jealousy and infatuation and the rest rather than just convincingly playacting them? Who knows? And suppose not: is a p-zombie, shambling out of the philosophy seminar room into actual existence, any less fascinating?
To quote a few random people on the internet...
ReplyDelete"the species which apologizes to trash cans and fire hydrants when they bump into them somehow mistook a technology meant to mimic human communication for a person. weird. how did that happen"
"They taught AI how to talk like a corporate middle manager and thought this meant the AI was conscious instead of realizing that corporate middle managers aren't"
To use Aaronson's own terminology, no we didn't create "an alien" - we created a highly advanced sock puppet of an alien, and most of us gullible apes can't help but see it as if it actually were somehow a real alien.
This will end badly, not because of "AI", but because of humanity. Enough of us will be gullible enough to make really bad decisions based on this illusion, and there will be more than enough immoral people ready to exploit that bad decision making, and people are GOING to die.
Even the "P-Zombie" line is off base - the concept invoked is something that is in all ways indistinguishable from a human being, except for not actually being sentient.
ReplyDeleteBeing a somewhat convincing illusion is not the same thing, if for no other reason than the bar for "convincing" so often being set so very low - see people apologizing to trash cans, etc. Aaronson might as well suggest that old AOL chatroom bots from 1997 were "P-Zombies" at least some of the time, when they interacted with particularly gullible individuals.