AI greatly amplifies threats
getty
Walter White’s fictional meth enterprise in Breaking Bad may have been about nefarious as it gets, but it represented a highly efficient business model – with tightly enforced production, distribution, partnerships, and repeat business. Similarly, today’s digital and AI bad guys run on business models that leverage the latest technologies, particularly AI, and appear to be light years ahead of legitimate businesses.
If anything, businesses aren’t ready to handle the AI-borne tsunami of cybersecurity threats coming their way, warns Patrick Coughlin, author of The Dark Side of the Boom and co-founder and CEO of Savi Security. The security issues with AI come from both the outside and inside organizations themselves.
With the outside AI threat, “while the rest of the world is looking for the productivity gains from AI in the S&P 500, cybercriminal syndicates have already implemented and adopted this and are seeing ROI faster,” Coughlin pointed out in a recent chat.
Ironically, there is more “innovation coming from these cybercriminals syndicates, who are implementing AI faster, and are moving faster,” he said. “The cybercriminal enterprise is actually the earliest and best adopter of AI. The groups that have their on-chain activity linked to integrations, via API to AI tools, are generating 4-1/2 times more revenue, they are generating nine times the amount of velocity in terms of the campaigns that are being sent.”
Criminal organizations are not the best use case for AI by any means, but the implication is that legitimate businesses need to prepare for the onslaught, both from the outside and inside. Employee training and awareness are important strategies, but technology now moves too fast for training to keep up. “In a world of AI, where the threats are moving so quickly, where the evolution is moving so quickly, there’s a real question as to whether we can educate our way out of this problem,” Coughlin says. “Education has to be part of the solution, but it is only a part. And we need technology, innovation, process, and policy change.”
The unchecked proliferation of AI agents now taking place as the “confused deputy problem”: a lot of authority and power, but without enough supervision, Coughlin says. It’s all being rushed without considering the exposures to mistakes or cybersecurity incidents.
For example, “you bring an agent in to help business analysts use actual language queries against the analytics warehouse, so they can write reports faster, or generate metrics faster. Then, a nefarious prompt injection deletes an entire table.”
These days, “a lot of enterprises who are being told by the C-suite and board to get AI into workflows because they’re afraid of getting left behind. So AI is rushed, and moved to production. In many cases, the agents have too much authority, or too much access, for the original job to be done.”
This is where the internal threat rears its head. With the rise of agents and AI systems assuming a range of tasks, “the inside of the enterprise is starting to look like the Star Wars bar,” Coughlin quips. “You have a bunch of non-human identifies running around who don’t know each other. We have to recognize that’s the new normal. And the bouncer at the door doesn’t help you much in that world.”
The key is to treat AI agents with the same levels of caution and restricted access as a new intern, Coughlin advises. “Invest in them, be excited by them, and build amazing programs to attract them. But you don’t give the intern access to the crown jewels. ”Trust them like your newest intern, audit your agents’ surface area, understand where the agents are, implement human in the loop oversight, or visibility for policy writing around these agents, assign responsibilities around agents.”
Existing cybersecurity policies need to be extended to cover non-human identities, he urged. “As agents develop more reasoning capabilities under the hood, they will look and smell more like humans, but they will be a different class of employee,” he added.
Read More
Joe McKendrick