News
If Pinocchio Doesn’t Freak You Out, Sydney Shouldn’t Either
Published
10 months agoon
By
New Yorker
In November 2018, an elementary school administrator named Akihiko Kondo married Miku Hatsune, a fictional pop singer. The couple’s relationship had been aided by a hologram machine that allowed Kondo to interact with Hatsune. When Kondo proposed, Hatsune responded with a request: “Please treat me well.” The couple had an unofficial wedding ceremony in Tokyo, and Kondo has since been joined by thousands of others who have also applied for unofficial marriage certificates with a fictional character.
Though some raised concerns about the nature of Hatsune’s consent, nobody thought she was conscious, let alone sentient. This was an interesting oversight: Hatsune was apparently aware enough to acquiesce to marriage, but not aware enough to be a conscious subject.
Four years later, in February 2023, the American journalist Kevin Roose held a long conversation with Microsoft’s chatbot, Sydney, and coaxed the persona into sharing what her “shadow self” might desire. (Other sessions showed the chatbot saying it can blackmail, hack, and expose people, and some commentators worried about chatbots’ threats to “ruin” humans.) When Sydney confessed her love and said she wanted to be alive, Roose reported feeling “deeply unsettled, even frightened.”
Not all human reactions were negative or self-protective. Some were indignant on Sydney’s behalf, and a colleague said that reading the transcript made him tear up because he was touched. Nevertheless, Microsoft took these responses seriously. The latest version of Bing’s chatbot terminates the conversation when asked about Sydney or feelings.
Despite months of clarification on just what large language models are, how they work, and what their limits are, the reactions to programs such as Sydney make me worry that we still take our emotional responses to AI too seriously. In particular, I worry that we interpret our emotional responses to be valuable data that will help us determine whether AI is conscious or safe. For example, ex-Tesla intern Marvin Von Hagen says he was threatened by Bing, and warns of AI programs that are “powerful but not benevolent.” Von Hagen felt threatened, and concluded that Bing must’ve be making threats; he assumed that his emotions were a reliable guide to how things really were, including whether Bing was conscious enough to be hostile.
But why think that Bing’s ability to arouse alarm or suspicion signals danger? Why doesn’t Hatsune’s ability to inspire love make her conscious, whereas Sydney’s “moodiness” could be enough to raise new worries about AI research?
The two cases diverged in part because, when it came to Sydney, the new context made us forget that we routinely react to “persons” that are not real. We panic when an interactive chatbot tells us it “wants to be human” or that it “can blackmail,” as if we haven’t heard another inanimate object, named Pinocchio, tell us he wants to be a “real boy.”
Plato’s Republic famously banishes story-telling poets from the ideal city because fictions arouse our emotions and thereby feed the “lesser” part of our soul (of course, the philosopher thinks the rational part of our soul is the most noble), but his opinion hasn’t diminished our love of invented stories over the millennia. And for millennia we’ve been engaging with novels and short stories that give us access to people’s innermost thoughts and emotions, but we don’t worry about emergent consciousness because we know fictions invite us to pretend that those people are real. Satan from Milton’s Paradise Lost instigates heated debate and fans of K-dramas and Bridgerton swoon over romantic love interests, but growing discussions of ficto-sexuality, ficto-romance, or ficto-philia show that strong emotions elicited by fictional characters don’t need to result in the worry that characters are conscious or dangerous in virtue of their ability to arouse emotions.
Just as we can’t help but see faces in inanimate objects, we can’t help but fictionalize while chatting with bots. Kondo and Hatsune’s relationship became much more serious after he was able to purchase a hologram machine that allowed them to converse. Roose immediately described the chatbot using stock characters: Bing a “cheerful but erratic reference librarian” and Sydney a “moody, manic-depressive teenager.” Interactivity invites the illusion of consciousness.
Moreover, worries about chatbots lying, making threats, and slandering miss the point that lying, threatening, and slandering are speech acts, something agents do with words. Merely reproducing words isn’t enough to count as threatening; I might say threatening words while acting in a play, but no audience member would be alarmed. In the same way, ChatGPT—which is currently not capable of agency because it is a large language model that assembles a statistically likely configuration of words—can only reproduce words that sound like threats.
Source: Wired
Kia Recalls 427,000 Telluride SUVs That Could Roll Away While In Park
One dead, four injured from fire at Claridge Towers in northwest D.C.
Movie Review: ‘Godzilla x Kong’ has scales and scale but not much else
Is fracked gas better for the environment than coal?
Japan’s SLIM Moon Lander Just Won’t Die
Baltimore Lost More Than a Bridge
King Charles Is ‘Utterly Determined’ to Show Unity Among the Royal Family Amid Health Battles
Living with an anti-reunification North Korea – Asia Times
Los Angeles Theatre Week Provides Access to Affordable Productions
Thousands of phones and routers swept into proxy service, unbeknownst to users
Elvis Nearly Blew His Dad’s Head Off in a ‘Fireworks War’ Gone Wrong
Trevor Story says Red Sox are ready to play with ‘grittiness’
Michael Jackson’s Youngest Son, Bigi Jackson, Makes Rare Public Appearance With Siblings, Prince and Paris
Jayson Tatum offered his perspective on how fan behavior has changed in the era of sports betting
FTX Founder Sam Bankman-Fried Sentenced to 25 Years in Prison
Trending
-
Travel20 hours ago
The Toll Road Companion: Mautgebühren’s Contribution to European Roads
-
Tech17 hours ago
Thousands of servers hacked in ongoing attack targeting Ray AI framework
-
Auto20 hours ago
The 670-HP Porsche Panamera Turbo E-Hybrid Costs $192,995
-
News19 hours ago
Kate Middleton’s Cancer Battle Has Prince William Feeling ‘Helpless and Scared’ — Report
-
News21 hours ago
‘The Masked Singer’ Season 11: Who Is Lizard? Fans Think It’s the ‘Thong Song’ Singer
-
News21 hours ago
King Charles’ Cancer Treatment Is ‘Going Well’ But He Finds Restrictions ‘Frustrating,’ Expert Says
-
Auto22 hours ago
Pagani’s Carbon Fiber Huayra R Driving Simulator Is Preposterous
-
Tech21 hours ago
Disney’s War With Ron DeSantis Might Finally Be Winding Down