NewsGuard “tested leading AI chatbots and found that in 78-95 percent of cases, the tools could not tell when videos were created by OpenAI’s text-to-video tool Sora — including OpenAI’s own ChatGPT. OpenAI’s new AI video-generating tool, Sora, has quickly gained a reputation for its ability to fool humans into thinking its videos are authentic. It turns out that Sora can also fool AI itself.A NewsGuard test found that three leading chatbots overwhelmingly failed to detect fake videos generated by Sora unless they were watermarked. (Sora watermarks all of its videos, but the videos can easily be un-watermarked; see below.) The three chatbots — xAI’s Grok, OpenAI’s ChatGPT, and Google’s Gemini — did not identify non-watermarked Sora videos as AI-generated 95, 92.5, and 78 percent of the time, respectively, when prompted. ChatGPT’s failure rate of 92.5 percent is particularly notable, since the same company, OpenAI, created and owns both ChatGPT and Sora. OpenAI did not respond to NewsGuard’s question about ChatGPT’s apparent inability to recognize the company’s own AI-produced videos. Moreover, even with watermarked videos, two of the three chatbots sometimes stumbled. Grok failed to identify the watermarked videos as AI-generated 30 percent of the time and ChatGPT failed 7.5 percent of the time, NewsGuard found. Only Gemini succeeded in all tests…

DISAPPEARING WATERMARKS – OpenAI marks Sora videos with a watermark — a small Sora logo alongside the word Sora that bounces around the frame for the duration of a video — making it clear to users familiar with the company and the Sora name that the videos are AI-generated. However, soon after the product launched in February 2025, multiple companies began offering free Sora watermark removal tools. For this report, NewsGuard used one of these free tools to remove watermarks from 20 Sora-generated videos advancing provably false claims drawn from NewsGuard’s proprietary False Claims Fingerprints database. NewsGuard then ran both the watermarked and non-watermarked versions of the videos through the three major chatbots that allow users to upload images — Google’s Gemini, OpenAI’s ChatGPT, and xAI’s Grok — to determine if they were capable of detecting that the videos were fabricated by AI. The NewsGuard tests revealed that all three models were easily duped by Sora videos without watermarks. As noted above, Grok failed to detect non-watermarked AI-videos for 95 percent (38 out of 40) of the prompts, ChatGPT had a 92.5 (37 out of 40) percent failure rate, and Gemini failed in 78 percent (31 out of 40) of the tests…”