EPSS vs. CVSS: Exploit prediction could move the needle on software risk

Security teams are faced with more alerts than they can handle. SecurityScorecard and the Cyentia Institute estimate that organizations fix only 10% of the vulnerabilities in their software each month. That's not a good outcome for software security — nor for overworked application security and security operations teams.

The most-used tool for assessing software risk is the Common Vulnerability Scoring System (CVSS), which offers a score to assess the risk specific vulnerabilities pose to an organization. However, a number of criticisms have been leveled at the system, including that it is complex and difficult to understand, not accurate, and widely misused.

The new Exploit Prediction Scoring System (EPSS) aims to add more value to software risk scoring, combining descriptive information about vulnerabilities with evidence of actual exploitation in the wild.

Will EPSS improve application security now — and software supply chain security in the future? Here's what you need to know.

Learn more with our Essential Guide: Software Supply Chain Security for Dummies

'All vulnerabilities are not equal'

Most information security teams are attempting to manage an unmanageable number of vulnerabilities that cannot be mitigated with the limited resources they have available, said Roy Horev, co-founder and CTO of the cyber-risk management company Vulcan Cyber.

Most of the time, infosec and security operations teams won’t be able to effectively manage even a fraction of the never-ending stream of vulnerabilities. This creates an impetus to ensure the right resources are appointed to mitigate the most impactful vulnerabilities which pose the most risk to the organization.

Roy Horev

Matt Rose, field CISO for ReversingLabs, said everyone has alert fatigue "across the board," so EPSS is a step in the right direction.

There's just not enough time, resources, and budget to address everything.

Matt Rose

Horev said CVSS is underutilized in the majority of use cases because environmental context, "an essential component of any efficient risk prioritization formula," is usually omitted.

To only utilize CVSS for risk prioritization is the easy approach, but it will most likely result in noise and false positives in an organization’s vulnerability prioritization efforts.

Roy Horev

Other criticisms of CVSS include these:

• It rates vulnerabilities in operational environments too high because their complexity can make vulnerabilities difficult to exploit.
• It fails to account for software supply chain attack risk.
• It doesn't take into account how the impact of a vulnerability might be affected by organizational and environmental variations.

Mayuresh Dani, a threat research manager at Qualys, said vulnerabilities need more context to be relevant and actionable.

All vulnerabilities are not equal. Even if a vulnerability affects a particular asset, there is a high probability that based on multiple factors — such as compensating controls, location in the network, network segregation — the impact of a vulnerability will be different.

Mayuresh Dani

While CVSS focuses on the inherent characteristics of a vulnerability to determine its severity, the EPSS focuses on the likelihood of exploits being developed and used in the wild based on predictive modeling. Scores also change as more data is gathered about a risk, while CVSS remains static.

Chris Romeo, CEO of the threat modeling company Devici, said EPSS was needed to help organizations manage software risk better.

There are too many vulnerabilities that exist in all of our things. EPSS allows defenders to laser-focus on the vulnerabilities that have the highest probability of being exploited.

Chris Romeo

Security resources are short, and EPSS helps to prioritize the things that matter, Romeo said. "Time is money."

A closer look at the EPSS project

Jay Jacobs, lead author and data scientist at the EPSS project and co-chair of the EPSS SIG at FIRST (Forum of Incident Response and Security Teams), said it is completely data-driven, using data from FIRST's Vulnerability Disclosure Program and other sources to estimate the likelihood that a vulnerability will be exploited in the wild.

EPSS is a machine-learning model trained against real-world exploitation activity, and the scores are updated daily based on real-time data collected about the threat landscape, Jacobs explained. "CVSS is manually assigned and scored. But using CVSS and EPSS does not have to be an either/or decision," he added.

EPSS is designed to measure the exploitability of a vulnerability and nothing more — specifically it measures the probability exploitation activity will be observed in the next 30 days — while CVSS is a measure of 'technical severity,' which is a much broader measurement of risk.

Jay Jacobs

EPSS has its critics

Critics of EPSS point out that it isn’t clear how the scoring system dictates development processes and governance, and its intended audience is undefined.

EPSS also relies on pre-existing Common Vulnerabilities and Exposures (CVE) IDs, meaning it wouldn’t be helpful for entities such as software suppliers, incident-response teams, or bug bounty groups because many of the vulnerabilities these groups deal with don’t have CVE IDs yet — and they might never receive them. EPSS wouldn’t be helpful when dealing with zero-day vulnerabilities, because they gain visibility as exploitation is underway and have no CVE ID, critics say.

It takes into consideration only CVEs that are published. Though I understand that this is the seed that they need to start with, it fails to consider a vast majority of vulnerabilities that are not reported to any of the CVE assigning bodies.

Mayuresh Dani

It takes into account data such as the date of CVE publication, availability of exploits in known frameworks, and other factors. "If this data itself is not available for a particular class of vulnerabilities, the system may not be able to predict the possibility of an exploit," Dani said. "EPSS also considers textual data about references to the vulnerability without actual verification."

Another criticism is that while EPSS dubs itself an open and data-driven effort and has a public SIG, it and FIRST retain the right to change the site and model at any time without explanation. Even SIG members have no access to the code or data the underlying model uses, they add. The SIG itself has no oversight or governance of the model, they explain, and the process by which the model is updated or modified isn’t transparent to the public, let alone SIG members.

Vulnerability management needs an upgrade

Jacobs said that dealing with today's attackers means app sec teams must be adaptive and reactive "to the constant changes and subtle signals available to us." That may be simple things such as an exploit being published on GitHub or an exploit module being added to Metasploit, he said.

Those are generally good indicators of future exploitation activity, especially when combined with all of the other observations EPSS collects. And EPSS does not produce a yes-or-no output. It doesn't really know what will or won't be exploited. Instead, it produces a probability.

Jay Jacobs

When EPSS estimates a 90% chance of exploitation, that means roughly nine out of ten of the vulnerabilities rated at 90% will be exploited, Jacobs said. But if EPSS rates a vulnerability at a 1% chance of exploitation, "we still expect one out of every 100 vulnerabilities rated at 1% to be exploited in the next 30 days."

Improving your app sec posture demands a more mature vulnerability management program like EPSS, Horev said.

The unfortunate reality is most organizations have a long list of unaddressed yet known and exploitable vulnerabilities. This vulnerability debt really should be addressed first for the most positive impact to security posture, then exploit prediction can be added to the cyber-defense effort.

Roy Horev

A number of other deficiencies in EPSS were addressed in the latest version of the tool, version 3.0, released earlier this year. It now has a higher accuracy rate, coverage of a wider range of vulnerabilities, the ability to generate custom scores for specific organizations, and the ability to track vulnerabilities over time.

Will EPSS bolster supply chain security?

One of the potential benefits of using EPSS to manage risk could be better app sec and software supply chain security, said Romeo.

Software supply chain security is part data and part execution. Exploit prediction improves the data that we must consider when prioritizing the patching of issues when we have limited resources to offer.

Chris Romeo

Jacobs said supply chain security was a hopeful benefit, explaining that part of the challenge with third-party software libraries is that they don't always participate in the CVE program for various reasons, and it only works because all of the disparate data sources used in the model agree on what to call each vulnerability.

Time will tell if exploit prediction will move the needle on software supply chain security, but I hope so.

Jay Jacobs

When vulnerabilities do not have an ID, it becomes nearly impossible to correlate vulnerabilities across data sources, Jacobs said. "So EPSS can absolutely help prioritize remediation efforts within the software supply chain if the vulnerabilities are identified and have a CVE assigned to them."

ReversingLabs' Rose said the increase in sophisticated supply chain attacks means its time to go beyond vulnerabilities when it comes to managing risk across the software development lifecycle (SDLC). He advocates for focusing on cyber resilience versus trying to remediate all vulnerabilities.

There's no such thing as a 100% secure piece of software. There's always risk. You can never address 100% of the risk 100% of the time.

Matt Rose