In November 2018, an elementary school administrator named Akihiko Kondo married Miku Hatsune, a fictional pop singer. The couple’s relationship had been aided by a hologram machine that allowed Kondo to interact with Hatsune. When Kondo proposed, Hatsune responded with a request: “Please treat me well.” The couple had an unofficial wedding ceremony in Tokyo, and Kondo has since been joined by thousands of others who have also applied for unofficial marriage certificates with a fictional character.
Though some raised concerns about the nature of Hatsune’s consent, nobody thought she was conscious, let alone sentient. This was an interesting oversight: Hatsune was apparently aware enough to acquiesce to marriage, but not aware enough to be a conscious subject.
Four years later, in February 2023, the American journalist Kevin Roose held a long conversation with Microsoft’s chatbot, Sydney, and coaxed the persona into sharing what her “shadow self” might desire. (Other sessions showed the chatbot saying it can blackmail, hack, and expose people, and some commentators worried about chatbots’ threats to “ruin” humans.) When Sydney confessed her love and said she wanted to be alive, Roose reported feeling “deeply unsettled, even frightened.”
Not all human reactions were negative or self-protective. Some were indignant on Sydney’s behalf, and a colleague said that reading the transcript made him tear up because he was touched. Nevertheless, Microsoft took these responses seriously. The latest version of Bing’s chatbot terminates the conversation when asked about Sydney or feelings.
Despite months of clarification on just what large language models are, how they work, and what their limits are, the reactions to programs such as Sydney make me worry that we still take our emotional responses to AI too seriously. In particular, I worry that we interpret our emotional responses to be valuable data that will help us determine whether AI is conscious or safe. For example, ex-Tesla intern Marvin Von Hagen says he was threatened by Bing, and warns of AI programs that are “powerful but not benevolent.” Von Hagen felt threatened, and concluded that Bing must’ve be making threats; he assumed that his emotions were a reliable guide to how things really were, including whether Bing was conscious enough to be hostile.
But why think that Bing’s ability to arouse alarm or suspicion signals danger? Why doesn’t Hatsune’s ability to inspire love make her conscious, whereas Sydney’s “moodiness” could be enough to raise new worries about AI research?
The two cases diverged in part because, when it came to Sydney, the new context made us forget that we routinely react to “persons” that are not real. We panic when an interactive chatbot tells us it “wants to be human” or that it “can blackmail,” as if we haven’t heard another inanimate object, named Pinocchio, tell us he wants to be a “real boy.”
Plato’s Republic famously banishes story-telling poets from the ideal city because fictions arouse our emotions and thereby feed the “lesser” part of our soul (of course, the philosopher thinks the rational part of our soul is the most noble), but his opinion hasn’t diminished our love of invented stories over the millennia. And for millennia we’ve been engaging with novels and short stories that give us access to people’s innermost thoughts and emotions, but we don’t worry about emergent consciousness because we know fictions invite us to pretend that those people are real. Satan from Milton’s Paradise Lost instigates heated debate and fans of K-dramas and Bridgerton swoon over romantic love interests, but growing discussions of ficto-sexuality, ficto-romance, or ficto-philia show that strong emotions elicited by fictional characters don’t need to result in the worry that characters are conscious or dangerous in virtue of their ability to arouse emotions.
Just as we can’t help but see faces in inanimate objects, we can’t help but fictionalize while chatting with bots. Kondo and Hatsune’s relationship became much more serious after he was able to purchase a hologram machine that allowed them to converse. Roose immediately described the chatbot using stock characters: Bing a “cheerful but erratic reference librarian” and Sydney a “moody, manic-depressive teenager.” Interactivity invites the illusion of consciousness.
Moreover, worries about chatbots lying, making threats, and slandering miss the point that lying, threatening, and slandering are speech acts, something agents do with words. Merely reproducing words isn’t enough to count as threatening; I might say threatening words while acting in a play, but no audience member would be alarmed. In the same way, ChatGPT—which is currently not capable of agency because it is a large language model that assembles a statistically likely configuration of words—can only reproduce words that sound like threats.
Kevin McCarthy’s House speaker job under threat, with vote possible today: live updates
Craig Ross charged with kidnapping 9-year-old Charlotte Sena from New York park
Sam Bankman-Fried Is a Terrible Client
Now prisoners could be sent abroad as jail overcrowding reaches breaking point
Vandals smash African-American art pieces in Tampa
I’m a Long-Distance Hiker With Ulcerative Colitis—Here’s How I Manage On and Off the Trail
The Benefits of Lactic Acid for Skin Go Beyond Exfoliation
This Back and Biceps Workout With Dumbbells Can Help Improve Your Posture
33 Tasty Pumpkin Recipes That Are Perfect for Fall
APL TechLoom Dream Review: A Comfy Shoe for Stylish Walkers and Casual Runners
Sam Bankman-Fried Is a Terrible Client
Prince William and Kate Middleton’s 2010 Engagement Interview Revealed a Major Clue About Why the Princess Is So Well-Liked
The Richest Person In America 2023
Student driver crashes into Brockton home
First Ride: Verge TS Pro Electric Motorcycle
Finance24 hours ago
Crypto Trading Platform GSR Receives In-Principal MPI License in Singapore
News15 hours ago
Rep. Matt Gaetz triggers vote to oust Kevin McCarthy from speaker’s office
Travel20 hours ago
QNB announces its Strategic Banking Partnership of Expo 2023 Doha Qatar™
Auto24 hours ago
Let Alfa Romeo 33 Stradale Designer Take You On Tour Of New Supercar
Sport24 hours ago
Fulham vs Chelsea LIVE: Blues look to build on Brighton win in Prem London derby
News23 hours ago
Revealed: The dogs that give humans the most expressive stares – so how does your pooch stack up?
Auto19 hours ago
Chery To Launch Three Brands Selling SUVs In Europe By End Of 2025
News12 hours ago
Biden-Xi have historic chance to reset troubled ties