Can AI beat AI? 3 challenges with VulnOps adoption

Emerging AI frontier models could be the impetus that gets security professionals to finally tackle the tough job of fixing broken vulnerability management practices. Security strategists see Mythos and its ilk as engines for dangerously increasing exploitation risks — but  also as the tools that will make it possible to bring to fruition all of those elusive VM improvements that have been out of reach for so long. 

Take the push for continuous threat exposure management (CTEM), which emphasizes doing continuous assessment and basing prioritization on exploit and business context. While the security world has spent buckets of money to uplift VM with CTEM tooling and frameworks, it hasn't managed to operationalize CTEM to effectively automate the remediation of flaws, said Chris Hughes of Resilient Cyber. 

“That’s where most organizations stall. They struggle to execute it at the speed the threat environment demands because their workflows still depend on human analysts to interpret findings, human operators to implement remediations, and human decision-makers to approve changes.”
—Chris Hughes 

But looping agentic swarms into workflows has the potential to drive what security pros are now calling VulnOps. This is the latest shorthand for that mash-up of not only continuous assessment and really meaningful prioritization, but also autonomous triage and remediation. Ideally, it will also help finally bring VM out of its isolation from the rest of SecOps, said Nico Popp, operating partner for Crosspoint Capital Partners.

“I think SecOps needs to swallow VM, at very least for the zero days.”
—Nico Popp 

Popp said he believes that VulnOps will converge VM activities such as threat-driven remediation with higher-tier SOC functions such as threat hunting and control optimization. An effective VulnOps program will revolve around what he calls the seven samurai of VulnOps: shift left, continuous scanning, validation, prioritization, agentic patching, adaptive remediation, and detection and containment. 

“AI beats AI” is a great vision, but a lot of practical AI work has to happen before security teams can reach the autonomous VulnOps nirvana. Here are the biggest challenges that need to be tackled to make VulnOps a reality.  

[ See webinar: How to Build High-Fidelity Threat Intel Feeds for Agentic AI ]

1. Managing AI token costs is essential

IT budgets are always an issue, and AI tokens don’t come cheap. 

“Tokenomics will be a big challenge. If the costs are out of control, it could get to the point where some people will say, ‘If the AI is more expensive than humans, maybe we don’t need the AI.’”
—Nico Popp

In the AI euphoria of just a couple of months ago, companies were encouraging “tokenmaxxing,” or trying to do more with AI by maxing out the number of tokens consumed. But now the bills are coming due, budgets are borked, and the bean counters want to walk back those policies and start real AI cost management.

Tokenomics is a big enough issue that the Linux Foundation has launched the Tokenomics Foundation to define efficient token consumption that doesn’t hold up AI advancement. For CISOs and SOC leaders, the big challenge will be finding a good balance between agentic AI gains and budgetary realities.

2. Choosing the right LLM is key

Another issue tied to cost is deciding on which large language model (LLM) to use and defining the surrounding infrastructure, code, and orchestration logic that will turn that model into a working, autonomous agent — the harness. Models are not one-size-fits-all. Right-sizing the model to whatever problem the AI is supposed to analyze or automate will keep expenses down, Popp said. It’s also crucial for managing the effectiveness of the AI in specific use cases. 

Frontier models are currently getting the most attention, but Stanislav Fort, chief scientist and founder of AISLE, wrote recently that security researchers are showing that “small, cheap models outperform large frontier ones” in a lot of cases. 

Many of the open-source models from China and elsewhere are “good enough at cyber investigations,” said longtime security pro and agentic AI startup founder Jimmy Astle. Kimi K2, from China’s Moonshot AI, for example, costs just one-tenth of more advanced closed-source models but is  effective for many tasks. 

“It’s not as good at the critical thinking stuff, but it’s really good at agentic tool calling and task solving. These open-source models will force the tokenomics down, which will then enable these [autonomous] investigations to proliferate.”
—Jimmy Astle

3. Be the master of your agentic governance

If agentic AI needs free rein to be effective at VulnOps, how do you make overall risk go down and not up? If human oversight of vulnerability remediation is limited, you need to boost threat modeling and controls such as identity and permission structures around the agents. Because they hold write access, they are targets. The architecture has got to be designed deliberately so that the security holds up without slowing down the autonomous action when it needs to be made, wrote AI security consultant Rock Lambros.

“Machine-speed remediation needs pre-approved business-impact authority with bounded autonomy, so the response fires inside agreed limits without a 2 a.m. approval chain.”
—Rock Lambros

Keeping agents hardened from attack is important, but even more crucial is building in the governance and controls that make sure they behave as intended, Popp said.

“You need to control what those swarms of agents are doing. People are going to be even more concerned about the agent going off of the reservation than malicious actors trying to take advantage of vulnerability in the agent.”
—Nico Popp

The biggest concern, Popp said, is action governance rollback. He believes that this is going to be where the human in the loop resides, as security teams phase out of the work of remediating the vulnerabilities and running triage and transition into guiding agents, auditing them, and managing the policies that tell them how to carry out VulnOps and all of the security work around it.

The agentic SOC is the first step

Popp said that for VulnOps to become a reality, getting to to an agentic SOC is going to be the most important piece of the puzzle.

“When you have enemies that can weaponize new vulnerabilities in minutes and it still takes 15 weeks to patch, they have the speed advantage. I tell people, ‘You don’t bring a knife to a gunfight. You have to bring AI to this problem.”
—Nico Popp

Working with agentic AI has its own challenges. Shimon Tolts, co-founder and CEO of Copperhelm, said the core work of SecOps will shift from execution to verification, adding that analysts who continue to spend their days manually triaging alerts will be automated out of relevance because agents are stuff faster and cheaper.

“The durable skill is supervising a fleet of agents and knowing when their conclusions are wrong. That is a judgment skill, not a tooling skill, and most current training still teaches button clicking.”
—Shimon Tolts

Learn more in the recent post, "Working with agentic AI: A SecOps survival guide."