{"id":827080,"date":"2025-02-15T01:11:59","date_gmt":"2025-02-15T07:11:59","guid":{"rendered":"https:\/\/newsycanuse.com\/index.php\/2025\/02\/15\/a-woman-made-her-ai-voice-clone-say-arse-then-she-got-banned\/"},"modified":"2025-02-15T01:11:59","modified_gmt":"2025-02-15T07:11:59","slug":"a-woman-made-her-ai-voice-clone-say-arse-then-she-got-banned","status":"publish","type":"post","link":"https:\/\/newsycanuse.com\/index.php\/2025\/02\/15\/a-woman-made-her-ai-voice-clone-say-arse-then-she-got-banned\/","title":{"rendered":"A woman made her AI voice clone say \u201carse.\u201d Then she got banned."},"content":{"rendered":"
\n
\n

This article first appeared in The Checkup,\u00a0<\/em>MIT Technology Review\u2019s\u00a0weekly biotech newsletter. To receive it in your inbox every Thursday, and read articles like this first,\u00a0<\/em>sign up here<\/em><\/a>.<\/em><\/p>\n

Over the past couple of weeks, I\u2019ve been speaking to people who have lost their voices. Both Joyce Esser, who lives in the UK, and Jules Rodriguez, who lives in Miami, Florida, have forms of motor neuron disease\u2014a class of progressive disorders that result in the gradual loss of the ability to move and control muscles.<\/p>\n<\/p><\/div>\n

\n

It\u2019s a crushing diagnosis for everyone involved.<\/strong> Jules\u2019s wife, Maria, told me that once it was official, she and Jules left the doctor\u2019s office gripping each other in floods of tears. Their lives were turned upside down. Four and a half years later, Jules cannot move his limbs, and a tracheostomy has left him unable to speak.<\/p>\n

\u201cTo say this diagnosis has been devastating is an understatement,\u201d says Joyce, who has bulbar MND\u2014she can still move her limbs but struggles to speak and swallow. \u201cLosing my voice has been a massive deal for me because it\u2019s such a big part of who I am.\u201d<\/p>\n

AI is bringing back those lost voices. <\/strong>Both Jules and Joyce have fed an AI tool built by ElevenLabs recordings of their old voices to re-create them. Today, they can \u201cspeak\u201d in their old voices by typing sentences into devices, selecting letters by hand or eye gaze. It\u2019s been a remarkable and extremely emotional experience for them\u2014both thought they\u2019d lost their voices for good.<\/p>\n

But speaking through a device has limitations. It\u2019s slow, and it doesn\u2019t sound completely natural. And, strangely, users might be limited in what they\u2019re allowed to say.<\/p>\n

Joyce doesn\u2019t use her voice clone all that often.<\/strong> She finds it impractical for everyday conversations. But she does like to hear her old voice and will use it on occasion. One such occasion was when she was waiting for her husband, Paul, to get ready to go out.<\/p>\n

Joyce typed a message for her voice clone to read out: \u201cCome on, Hunnie, get your arse in gear!!\u201d She then added: \u201cI\u2019d better get my knickers on too!!!\u201d<\/p>\n

\u201cThe next day I got a warning from ElevenLabs that I was using inappropriate language and not to do it again!!!\u201d Joyce told me via email <\/strong>(we communicated with a combination of email, speech, text-to-voice tools, and a writing board). She wasn\u2019t sure what had been inappropriate, exactly. It\u2019s not as though she\u2019d used any especially vile language\u2014just, as she puts it, \u201cnormal British banter between a couple getting ready to go out.\u201d<\/p>\n

Joyce assumed that one of the words she\u2019d used had been automatically flagged up by \u201cthe prudish American computer,\u201d and that once someone from the ElevenLabs team had assessed the warning, it would be dismissed.<\/p>\n<\/p><\/div>\n

\n

\u201cWell, apparently not, because the next day a human banned me!!!!\u201d says Joyce.<\/strong> She says she felt mortified. \u201cI\u2019d just got my voice back and now they\u2019d taken it away from me \u2026 and only two days after I\u2019d done a presentation to my local MND group telling them how amazing ElevenLabs were.\u201d<\/p>\n

Joyce contacted ElevenLabs, who apologized and reinstated her account. But it\u2019s still not clear why she was banned in the first place. When I first asked Sophia Noel, a company representative, about the incident, she directed me to the company\u2019s prohibited use policy<\/a>.<\/p>\n

There are rules against threatening child safety, engaging in illegal behavior, providing medical advice, impersonating others, interfering with elections, and more.<\/strong> But there\u2019s nothing specifically about inappropriate language. I asked Noel about this, and she said that Joyce\u2019s remark was most likely interpreted as a threat.<\/p>\n

ElevenLabs\u2019 terms of use<\/a> state that the company does not have any obligation to screen, edit, or monitor content but add that it may \u201cterminate or suspend\u201d access to its services when content is \u201creasonably likely, in our sole determination, to violate applicable law or [the user] Terms.\u201d ElevenLabs has a moderation tool that \u201cscreens content to ensure it aligns with our Terms of Service,\u201d says Dustin Blank, head of partnerships at the company.<\/p>\n

The question is: Should companies be screening the language of people with motor neuron disease?<\/strong><\/p>\n

After all, that\u2019s not how other communication devices for people with this condition work. People with MND are usually advised to \u201cbank\u201d their voices as soon as they can\u2014to record set phrases that can be used to create a synthetic voice that sounds a bit like them, albeit a somewhat robotic-sounding version. (Jules recently joked that his sounded like \u201ca Daft Punk song at quarter speed.\u201d)<\/p>\n

Banked voices aren\u2019t subject to the same scrutiny, says Joyce\u2019s husband, Paul.<\/strong> \u201cJoyce was told \u2026 you can put whatever [language] you want in there,\u201d he says. Voice banking wasn\u2019t an option for Joyce, whose speech had already deteriorated by the time she was diagnosed with MND. Jules did bank his voice but doesn\u2019t tend to use it, because the voice clone sounds so much better.\u00a0<\/p>\n

Joyce doesn\u2019t hold a grudge\u2014and her experience is far from universal. Jules uses the same technology, but he hasn\u2019t received any warnings about his language\u2014even though a comedy routine he performs using his voice clone contains plenty of curse words, says his wife, Maria. He opened a recent set<\/a> by yelling \u201cFuck you guys!\u201d at the audience\u2014his way of ensuring they don\u2019t give him any pity laughs, he joked. That comedy set is even promoted on the ElevenLabs website.<\/p>\n<\/p><\/div>\n

\n

Blank says language like that used by Joyce is no longer restricted. <\/strong>\u201cThere is no specific swear ban that I know of,\u201d says Noel. That\u2019s just as well.\u00a0<\/p>\n

\u201cPeople living with MND should be able to say whatever is on their mind, even swearing,\u201d says Richard Cave of the MND Association in the UK, who helps people with MND set up their voice clones. \u201cThere\u2019s plenty to swear about.\u201d<\/p>\n

Now read the rest of The Checkup<\/h2>\n

Read more from MIT Technology Review<\/em>\u2019s archive<\/h3>\n

You can read more about how voice clones are re-creating the voices of people with motor neuron disease in this story<\/a>.<\/strong><\/p>\n

Researchers are working to create realistic avatars of people with strokes and amyotrophic lateral sclerosis that can be controlled via a brain implant.<\/strong> Last year, two such individuals were able to use these devices to speak at a rate of around 60 to 70 words per minute<\/a>\u2014half the rate of typical speech, but more than four times faster than had previously been achieved using a similar approach.\u00a0<\/p>\n

Other people with ALS who are locked in\u2014completely paralyzed but cognitively able\u2014have used brain implants to communicate, too. <\/strong>A few years ago, a man in Germany used such a device to ask for massages and beer, and to tell his son he loved him<\/a>.\u00a0<\/p>\n

Several companies are working on creating hyperrealistic avatars.<\/strong> Don\u2019t call them deepfakes\u2014 they prefer to think of them as \u201csynthetic media,\u201d writes my former colleague Melissa Heikkil\u00e4, who created her own avatar<\/a> with the company Synthesia.<\/p>\n

ElevenLabs\u2019 tool can be used to create \u201chumanlike speech\u201d in 32 languages.<\/strong> Meta is building a model that can translate over 100 languages into 36 other languages<\/a>.<\/p>\n

From around the web<\/h3>\n

Covid-19 conspiracy theorists\u2014some of whom believe the virus is an intentionally engineered bioweapon\u2014will soon be heading US agencies.<\/strong> Some federal workers are worried they may be out for revenge against current and former employees. (Wired<\/a>)<\/p>\n<\/p><\/div>\n

\n

Cats might have spread bird flu to humans\u2014and vice versa. <\/strong>That\u2019s according to data from the US Centers for Disease Control and Prevention, which published the finding but then abruptly removed it. (The New York Times<\/a>)<\/p>\n

And a dairy worker is confirmed to have been infected with a second strain of bird flu that more recently spilled over from birds to cows.<\/strong> The person\u2019s only symptom was conjunctivitis. (Ars Technica<\/a>)<\/p>\n

Health officials in states with abortion bans are claiming that either few or zero abortions are taking place. <\/strong>The claims are \u201cludicrous,\u201d according to doctors in those states. (KFF Health News<\/a>)<\/p>\n

A judge in the UK has warned women against accepting sperm donations from a man who claims to have fathered more than 180 children in several countries.<\/strong> Robert Charles Albon, who calls himself Joe Donor, has subjected a female couple to a \u201cnightmare\u201d of controlling behavior, the judge said. (The Guardian<\/a>) <\/p>\n<\/div>\n<\/div>\n

Read More<\/a>
\n Jessica Hamzelou<\/p>\n","protected":false},"excerpt":{"rendered":"

This article first appeared in The Checkup,\u00a0MIT Technology Review\u2019s\u00a0weekly biotech newsletter. To receive it in your inbox every Thursday, and read articles like this first,\u00a0 sign up here . Over the past couple of weeks, I\u2019ve been speaking to people who have lost their voices. Both Joyce Esser, who lives in the UK, and Jules<\/p>\n","protected":false},"author":1,"featured_media":827081,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[46,950,262],"tags":[],"class_list":{"0":"post-827080","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-technology","8":"category-voice","9":"category-woman"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/827080","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/comments?post=827080"}],"version-history":[{"count":0,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/827080\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media\/827081"}],"wp:attachment":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media?parent=827080"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/categories?post=827080"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/tags?post=827080"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}