5 lessons from vulnerability management's front lines

What separates organizations that fix their vulnerabilities from those that don’t? That’s what the research team at Semgrep wanted to know when it set out to analyze anonymized remediation patterns across thousands of actively developed repositories and organizations.

In its report, “Remediation at Scale: What High-Performing AppSec Teams Do Differently,” the researchers found that the organizations it classified as AppSec leaders had a fix rate for vulnerabilities that were found using static application security testing (SAST) that was 2.4 times that of other organizations (the bottom 85%, referred to in the report as “the field”) and a fix rate for those found using software composition analysis (SCA) that was 3.3 times higher.

The researchers explained that with vulnerability management (VM), the speed of fixing individual findings is not the differentiator:

“The difference is volume: leaders remediate a far-higher percentage of their total findings. Speed on individual fixes matters less than systematic follow-through on the full backlog.”

Here are five key insights from the research that AppSec teams should know — and why shifting your strategy beyond vulnerabilities is key in the age of AI coding and vulnerability detection.

[ Learn: Why RL Built Spectra Assure Community | Join: RL's free Community ]

1. PR scanning accelerates remediation

The analysis uncovered that AppSec leaders resolve SAST findings that were detected during pull request (PR) reviews in 4.8 days, compared with 43 days for full-scan findings. And of the PR-detected fixes, 63% are completed on the same day — since developers are already looking at the code, the fix can ship in the same PR with no ticket shuffling or context switching.

Ensar Seker, CISO of SOCRadar, explained how PR scanning shortens the feedback loop. 

“Developers see the issue while the code is still fresh in their minds, before it becomes buried under additional commits, sprint changes, or ownership transfers. That immediacy reduces friction between discovery and remediation.”
—Ensar Seker

PR scanning also integrates security into the normal development workflow, thus escaping the normal security queues, he added. “Once vulnerabilities enter backlog systems or ticketing workflows, remediation competes with operational priorities and often loses momentum. Catching issues during code review makes fixing them part of shipping the feature itself,” he said.

But Dimitry Shvarstman, co-founder and chief product officer at Prime Security, cautioned that remediation can slow down when fixes require an understanding of broader business logic and potential downstream impact. 

“Developers don’t want to investigate something they feel the security team should have already validated. And they’re right to feel that way. Speed of remediation is also about confidence in the finding, not just timing.”
—Dimitry Shvarstman

Randolph Barr, CISO of Cequence Security, said that catching an issue while a developer is still in the code, before they’ve moved on to the next thing, makes all the difference between a quick fix and a ticket that ages out. “But it’s one piece, not the whole story,” he said.

“If you look at where remediation actually breaks down, it’s in the things scanners can’t really fix on their own — auth, crypto, authorization. Those don’t move because of a better rule. They move when security, engineering, product, and infrastructure are actually working together instead of throwing tickets over the wall at each other.”
—Randolph Barr

2. High performers set the rules

PR-blocking rules improve AppSec leaders’ fix rate by more than 12-percentage points, while upping the rate for the rest of the field by just five percentage points, the report says. And leaders are more likely to have adopted blocking rules (45% have done so, versus 32% for the field), as well as custom rules (59% versus 50%).

SOCRadar’s Seker said blocking rules turn security measures from recommended actions to release requirements, which keeps security findings from getting deprioritized under delivery pressure. But prioritizing does remain a factor.

“The key is balance. High-performing teams typically do not block everything indiscriminately. They tune policies around exploitability, reachability, and business risk. When blocking rules are well calibrated, they prevent vulnerable code from becoming technical debt in production environments.”
—Ensar Seker

Blocking rules work because they’re built around repeatable mistakes, said Prime Security’s Shvarstman. 

“Repeatable mistakes, like hardcoded secrets, SQL injections, and others, are findable mistakes. Every organization knows to avoid these. They’re low-hanging fruit, and they should be automated away with blocking rules.”
—Dimitry Shvarstman

Complicated attack paths make things more difficult, he said. “The real measure of a mature AppSec program isn’t whether you’re blocking the obvious stuff but whether you can identify complex attack paths built from non-obvious repeatable mistakes,” he said. “That’s where most programs stall. Today, the easy wins are getting easier, but the hard wins are still hard.”

3. Reachability analysis transforms prioritization

Organizations with data showing which vulnerabilities have been confirmed as reachable use that data as a filter to separate the “may be vulnerable” from the “definitely exploitable.” And doing that gives them fix rates 25 percentage points higher than the field.

Validation can be complicated, Shvarstman said. “Validating whether a vulnerability is actually reachable in your environment requires significant resources, and most organizations can’t afford to allocate them at scale,” he said.

“The teams that can are able to dramatically shrink the signal-to-noise problem. Reachability is really just context with a fancy name. It answers the question every developer is already asking: ‘Does this actually matter in our environment?’ When you can answer that, fix rates go up — not because developers got more disciplined, but because the finding became relevant and therefore worth acting on.”
—Dimitry Shvarstman

But Brett Smith, a software developer at SAS, said reachability analysis is getting easier. “With AI tools and clever robots, we can automate reachability analysis,” he said.

“And since robots don’t sleep, we get continuous reachability analysis. This acceleration in reachability analysis allows teams to prioritize vulnerabilities that are exploitable, ultimately leading to higher fix rates.”
—Brett Smith

4. Do-it-once triage is key

Repetitive false positives are a scourge that some organizations are reducing at scale using AI to capture framework protections, customize sanitizers, and provide nonproduction paths as context for their SAST engines.

Eran Kinsbruner, vice president of product marketing at Checkmarx, said AI improves AppSec testing results doing the interpretive work that used to fall on human analysts — and doing it more effectively.

There’s a huge gap between issues that traditional SAST scans flag and those that are actionable, Kinsbruner said, and that gap is what analysts spend most of their time closing manually. But by layering AI over the raw SAST output — validating each finding independently, assessing the attackability, correlating that against the organization’s risk posture, and attacking a confidence rating — “the platform turns a volume problem into a prioritization problem,” which is much easier to solve.

“Being able to reduce a large amount of false positives with AI means that the signal that reaches a developer or analyst has already been stress-tested before it asks for anyone’s time. It’s the compounding outcome of AI that understands context, not just syntax”
—Eran Kinsbruner

5. No one does fixes in the backlog

Findings that have been stuck in the backlog for three or more months are much less likely to be fixed, said Harlan Parrott, vice president of AI innovation at KnowBe4.

“It’s harder to fix a problem when you haven’t looked at the code in a while. Instead of spending just 30 minutes to repair and submit the defect, you must spend a few hours understanding how it’s supposed to work, fixing the defect, and ensuring it didn’t create new bugs.”
—Harlan Parrott

And findings that get backlogged without a fix can be genuinely hard to fix, said Michael King, a senior solutions engineer at Black Duck Software. The Semgrep report notes that auth failures are particularly likely to be deferred. 

“An example of this … is the finding that a stateless auth token can’t be revoked early, such as [when] a user logs out or gets banned. There’s no way to fix that without tracking state on the server, which contradicts design goals, so the finding just sits there until the auth system gets overhauled for unrelated reasons.”
—Michael King

Avoid the fail between detection and remediation

The Semgrep report concludes that the gap between leaders and the rest of the field is not a matter of tools, prioritization frameworks, or severity filters. “Every organization in this data set has access to the same scanning capabilities,” the authors wrote. 

“The gap is systematic execution: whether the findings reach developers with enough context to act, whether there’s a clear owner and a clear timeline, and whether aging findings get an honest decision instead of quiet neglect.”
—Semgrep researchers

The researchers added: “If you take one thing from this report, let it be this: Look at your critical SAST fix rate. If it’s below 50%, the problem is likely not detection quality. It’s that something between detection and remediation is broken. Find where the handoff fails and fix it first. Everything else builds on that foundation.”

Go beyond vulnerabilities with your AppSec

With NIST's move to selective enrichment only for the National Vulnerability Database (NVD) — and the coming "vulnpocalypse" that is expected with frontier AI cyber systems like Anthropic's Mythos, experts say it's time to think well beyond vulnerabilities.

The new selective enrichment policy is expected to create many challenges for security teams, including large backlogs of CVEs with no actionable context, increased manual analysis to determine relevance and risk, greater variability in data quality across public sources, and blind spots in cloud, SaaS, open-source, and third-party ecosystems that fall outside NIST’s prioritization criteria.

Resilient Cyber podcast host Chris Hughes said this presents a serious problem for teams managing AppSec risk.

“If you can’t map vulnerabilities to products in your ecosystem, you’re kind of SOL.”
—Chris Hughes

Nick (Ning) Mo, CEO and co-founder of Ridge Security Technology, said reducing enrichment exemplifies how current VM is failing. 

“Security teams get thousands of CVEs with scores that don’t really tell them what is exploitable in their own environment. A high CVSS number doesn’t [necessarily] mean an attacker can use it in your setup. A medium one can be the one that breaks you. So teams patch by score or theory, and the list keeps growing.”
—Nick (Ning) Mo

Learn about RL's free Spectra Assure Community — and how it can help you focus on malware in your open-source development.