The project team for the OWASP Top 10 list of vulnerabilities for large language models (LLMs) has kicked off preparations for releasing version 2.0 of the list. In a newsletter published on LinkedIn, project lead Steve Wilson called version 1.1 a considerable effort with a substantial impact, adding that version 2.0 will represent the first major revision.
Wilson said that a key driver of the next version of the Top 10 for LLMs was a survey that the OWASP project team recently completed. He said in an interview that the team was gathering expert opinion from the front lines of application security (AppSec) about "emerging risks that they're seeing that may not be showing up in our data."
"Since we released versions 1.0 and 1.1, we've been working in the background to do some data gathering about the threat landscape," Wilson said. "Now we've put out a survey asking people for their opinions on the current list and to rank possible areas to explore for the 2.0 list."
For the OWASP Top 10 for LLMs 2.0, Wilson said, "everything is on the table."
"We’ll reevaluate the top vulnerabilities, how we classify them, and decide on relative rankings. Version 2.0 will have a significant new focus on data gathering and using that data to improve our recommendations."
—Steve Wilson
Here are three key focal points for the next version of the OWASP Top 10 for LLMs.
[ See related posts: How software supply chain security tools can protect ML models | Secure AI development guidance: What software teams need to know ]
1. Data privacy
Wilson noted that the team received more than 100 responses to its survey, with about 75% of them from new participants or followers (40%) who were becoming active participants (35%).
"The 1.0 project was very grass-roots. What's interesting with 2.0 is that six or seven months later a lot of people submitted comments saying their companies were interested in the project. A lot of them are new-generation startups around AI security."
—Steve Wilson
Wilson said that some of the themes from the survey will help drive some of the thinking of what to do on the list and provide grist for future projects.
One standout concern for survey respondent was data privacy. "Privacy has always been adjacent to security, but from the folks we interviewed and who responded to the survey, data privacy is at the top of line for AI applications," Wilson said.
2. Safety and bias in LLM systems
Another concern gleaned from the survey is safety and bias in LLM systems. "I was surprised that security people wanted more advice about things like safety and bias," Wilson said.
"They're not things that have fallen into an application security team's purview before, but they're getting asked to help with them. I think we'll be looking to offer some more guidance around those."
—Steve Wilson
3. Mapping out vulnerabilities
Wilson said the project team is dedicated to making the LLM list comparable to other OWASP lists, which are very data-driven and "based on CVEs and sources like that."
"What we've been working on is figuring out: How do we map some of these AI- and LLM-specific risks to some of the data sources out there. It turns out that some of the CVE descriptions and that kind of data are not conducive to recording the risks that we're seeing with large language models."
—Steve Wilson
Wilson said the project team needed to do some bridging of the processes. "We're working with teams at MITRE and other organizations to standardize some of this so we can get a grip on what's going on in the real world," he said.
A SQL injection, for example, is not the same in AI as it is with traditional AppSec risk.
"The security risks with LLM are much harder to pin down and put into the boxes that are in the same shape as some of the more traditional security risks. There's a binary answer to SQL injection: 'Am I vulnerable to a SQL injection attack if I allow untrusted strings into my database?'"
—Steve Wilson
With LLMs, each model is taking in prompts as untrusted data, he said. "Then the questions become: To what degree am I vulnerable, and what kinds of vulnerabilities are there?"
OWASP Top 10 for LLM 2.0: Coming later this year
The project team released a light update in August of its original document. Version 1.1, included a visual illustration of the data flow in a typical LLM application and the potential areas of risk from the vulnerabilities in the Top 10. For example, the data flow between an API and an LLM's production services could be vulnerable to a prompt injection or denial-of-service attack, or an application's plugins might contain insecure design excessive agency vulnerabilities.
Otherwise, the top vulnerabilities list remained the same:
- LLM01: Prompt Injection, which is used to manipulate an LLM through crafty inputs, causing unintended actions
- LLM02: Insecure Output Handling, which occurs when an LLM output is accepted without scrutiny, exposing backend systems
- LLM03: Training Data Poisoning, which occurs when LLM training data is tampered with, introducing vulnerabilities or biases that compromise security, effectiveness, or ethical behavior
- LLM04: Model Denial of Service, which happens when attackers cause resource-heavy operations on LLMs, leading to service degradation or high costs
- LLM05: Supply Chain Vulnerabilities, which take place when an application’s lifecycle is compromised by vulnerable components or services
- LLM06: Sensitive Information Disclosure, which can arise when an LLM inadvertently reveals confidential data in its responses
- LLM07: Insecure Plugin Design, which results in plugins with insecure inputs and insufficient access control, leading to consequences like remote code execution
- LLM08: Excessive Agency, which surfaces when excessive functionality, permissions, or autonomy is granted to the LLM-based systems
- LLM09: Overreliance, which crops up when systems or people become overly dependent on LLMs without oversight
- LLM10: Model Theft, which involves unauthorized access, copying, or exfiltration of proprietary LLM models
The project team is aiming to release the OWASP Top 10 for LLM 2.0 later this year.
"We want to get something out in the next six months. We expect to see drafts in the spring and maybe a new version of the list in the summer."
—Steve Wilson
