SCWorld: LAS VEGA: “Just as it had at BSides Las Vegas earlier in the week, the risks of artificial intelligence dominated the Black Hat USA 2025 security conference on Aug. 6 and 7. We couldn’t see all the AI-related talks, but we did catch three of the most promising ones, plus an off-site panel discussion about AI presented by 1Password. The upshot: Large language models and AI agents are far too easy to successfully attack, and many of the security lessons of the past 25 years have been forgotten in the current rush to develop, use and profit from AI. We — not just the cybersecurity industry, but any organization bringing AI into its processes — need to understand the risks of AI and develop ways to mitigate them before we fall victim to the same sorts of vulnerabilities we faced when Bill Clinton was president. “AI agents are like a toddler. You have to follow them around and make sure they don’t do dumb things,” said Wendy Nather, senior research initiatives director at 1Password and a well-respected cybersecurity veteran. “We’re also getting a whole new crop of people coming in and making the same dumb mistakes we made years ago.” Her fellow panelist Joseph Carson, chief security evangelist and advisory CISO at Segura, had an appropriately retro analogy for the benefits of using AI. “It’s like getting the mushroom in Super Mario Kart,” he said. “It makes you go faster, but it doesn’t make you a better driver.”

The Black Hat briefings kicked off Aug. 6 with a presentation by Rebecca Lynch and Rich Harang of Nvidia, who detailed how easy it is to feed malicious information to an LLM or an AI agent and what you can do to mitigate (but never quite eliminate) the risk. Lynch, an offensive security researcher, explained that to alter the output, you have to poison the input. Because many, if not all, LLMs have trouble telling the difference between prompts and data, it’s easy to perform the AI equivalent of SQL injection upon them. “The real question is where untrusted data can be introduced,” she said. But fortunately for attackers, she added many AIs can retrieve data from “anywhere on the internet.”…Sharbat showed how he persuaded a customer-service AI agent, built using Microsoft’s Copilot Studio no-code AI developer tool and modeled on a real customer-service bot used by McKinsey and Co., to email him the contents of a customer-relationship management (CRM) database.