Nature Scientific Reports [full text] – People are poorly equipped to detect AI-powered voice clones. Sarah Barrington, Emily A. Cooper & Hany Farid. “As generative artificial intelligence (AI) continues its ballistic trajectory, everything from text to audio, image, and video generation continues to improve at mimicking human-generated content. Through a series of perceptual studies, we report on the realism of AI-generated voices in terms of identity matching and naturalness. We find human participants cannot consistently identify recordings of AI-generated voices. Specifically, participants perceived the identity of an AI-generated voice to be the same as its real counterpart approximately 80% of the time, and correctly identified a voice as AI generated only about 60% of the time. In January 2024, in the lead up to the November United States presidential election, an estimated tens of thousands of Democratic party voters received a robocall in the voice of President Biden instructing them not to vote in the upcoming New Hampshire primaries. The voice was AI-generated. The perpetrators of this attempted election interference were Steven Kramer (a political consultant), Paul Carpenter (a magician and hypnotist who was paid $150 to create the fake audio), and a telecommunications company called Lingo Telecom1,2. Carpenter used ElevenLabs, a platform offering instant voice cloning for as little as $5 a month. Kramer was fined $6 million and subsequently charged with two dozen crimes including impersonating a candidate and voter suppression, while the telecommunications company, Lingo Telecom, received a $1 million fine for transmitting the calls. This is just one of many examples of how the rise of generative AI is being weaponized, from election interference, to disinformation campaigns3, to small-4 and large-scale financial fraud.

AI is being weaponized, from election interference, to disinformation campaigns, to small- and large-scale– financial fraud. There is large literature on technologies that can automatically determine whether media – such as audio, video, and images – has been manipulated either by humans or generative AI6. These techniques, however, largely operate asynchronously and not as an audio or video call is unfolding in real time. The synchronous detection of fraudulent media, such as the phone calls that attempted to suppress voter turnout in New Hampshire, still poses significant technological challenges. Until technology can monitor every landline, mobile device, and video call (which itself would raise additional privacy concerns), people are largely left to their own defenses to sort out the real from the fake. The question then naturally arises: how well-equipped are people for the perceptual challenge of distinguishing real from AI-generated content? The answer, of course, depends on both the quality of the fake and the modality of the media. For example, studies focusing on visual perception of images of people have concluded that participants are at chance at distinguishing real and AI-generated head shots. Results for video (with audio) are more mixed – likely due to differences in the types of videos that have been assessed. While some studies report that performance is only slightly better than chance for videos of people talking, a recent large-scale study investigating how well people could distinguish fabricated political speeches from real ones report a consistent accuracy of 80% and above..”