The TeamPCP supply chain attack evolves

Updated 3/27 | New day, new compromise. The latest victim in the ongoing TeamPCP supply chain campaign as of Friday includes compromised versions 4.87.1 and 4.87.2 of the telnyx PyPI package, which has been downloaded more than 3.75 million times. The ultimate goal of this latest compromise is exfiltration of cloud secrets like that observed in previous attacks in this campaign. Researcher notes: Malicious code is added to telnyx/_client.py file, and the C2 server is 83[.]142.209.203.

Earlier this week, the TeamPCP attack spread from GitHub Actions and npm to the PyPI ecosystem. The victim: The LiteLLM package, an open-source Python library and proxy server that provides a unified interface to call over 100+ large language model (LLM) APIs — including OpenAI, Anthropic, Bedrock, and VertexAI — using a single, standardized format.

LiteLLM simplifies multi-LLM integration, offering automatic fallbacks, retries, and cost tracking across providers. LiteLLM’s PyPI package has about 480 million downloads, making it a very valuable target. Quick reaction from the PyPI security team moved the affected project to quarantine, reducing the potential impact. 

The compromise affects versions 1.82.7 and 1.82.8. The delivered payload is almost identical to the previous ones described in blogs explaining Trivy and Checkmarx compromise. An infostealer designed to steal a very broad list of secrets from hardcoded file paths and memory based on process names. The C2 domain in this case is models[.]litellm.cloud but in the code responsible for persistence, the checkmarx[.]zone domain can still be found.

The compromise of the PyPI package was likely caused by the initial compromise of a GitHub account belonging to the co-founder & CEO of LiteLLM, Krish Dholakia. The compromise was likely performed on March 23 and on March 24, when the owners of the GitHub repositories were defaced in an automated way at about 14:00 UTC time.

Images 1, 2: LiteLLM co-founder and CEO Krish Dholakia's compromised GitHub profile. And the timestamp showing automated defacing of the LiteLLM repo on GitHub.

Jacob Krell, senior director for secure AI solutions and cybersecurity at Suzu Labs, said the attack was indirect by design. “TeamPCP did not need to attack LiteLLM directly. They compromised Trivy, a vulnerability scanner running inside LiteLLM's CI pipeline without version pinning," he said.

"That single unmanaged dependency handed over the PyPI publishing credentials, and from there the attacker backdoored a library that serves 95 million downloads per month. One dependency. One chain reaction. Five supply chain ecosystems compromised in under a month."
—Jacob Krell

Krell said TeamPCP has exclusively targeted security adjacent tools. A vulnerability scanner, an infrastructure as code analyzer, and now an LLM proxy that handles API keys by design. These tools run with broad access because that is how they function. Compromising one hands the attacker every credential and secret that tool was trusted to touch, he said.

“The pattern is instructive."
—Jacob Krell

Inside the TeamPCP compromise

The initial signs of compromise can be found in commits published on March 23 to both litellm and the litellm-skills repository. On March 23 at 15:30UTC a branch was created and immediately deleted.

Image 3: Malicious activity in litellm-skills repository.

The referenced commit consists of one workflow file containing a malicious payload with a well known RSA public key seen across multiple payloads belonging to this campaign. The code itself looks like a modification to a secrets attack payload from the gato-x tool developed by a well known researcher Adnan Khan specialized in finding vulnerabilities in GitHub workflows. 

Image 4: Malicious GitHub workflow.

Workflow performance metrics provide evidence that the malicious workflow was actually executed in one of the runners, meaning that there is a significant possibility that secrets present in the environment variables have been exfiltrated from the affected runners.

Image 5: Evidence of workflow execution.

How the supply chain attack originally evolved

On Monday, ReversingLabs researchers identified two compromised software packages intended for use by developers that contained malicious code designed to find and steal sensitive developers secrets, tokens and crypto wallet information. 

The Checkmarx software plugins, checkmarx.ast-results version 2.53 and checkmarx.cx-dev-assist version 1.7.0, were both published to the Open VSX Registry on March 23, and are designed for use with VS Code and other VS Code compatible integrated development environments (IDEs) like Cursor, Windsurf and Kiro. 

Checkmarx is a widely used application security testing (AST) platform. The Open VSX Registry is a vendor-neutral, open-source alternative to the official Microsoft Visual Studio Marketplace. Hosted by the Eclipse Foundation, it is a repository for VS Code extensions that is used by platforms like VSCodium, Cursor, and Eclipse Theia, allowing developers to access, share, and manage code extensions without being bound by Microsoft’s license restrictions. Neither of the malicious packages was published to the VS Code Marketplace. 

Inside the VS Code extensions hack

The ast-results package, which has been downloaded about 36,000 times, provides developers with access to the Checkmarx One platform directly from their IDE, running scans from the IDE prior to code commitments, providing recommendations on vulnerability remediation.

The cx-dev-assist package, which has been installed about 500 times, is described as an “advanced security agent” used for real-time context-aware detection, remediation, and guidance to developers from within the IDE. It includes a feature described as an “AI Secure Coding Assistant (ASCA),” described as an AI-powered tool that lets developers identify violations of secure coding best practices in their code. 

On Monday, the RL researcher team detected malicious code in the most recent versions of both packages. Analysis indicated that the code was added to search for developer secrets stored on cloud assets, and to download a malicious payload from an attacker-controlled server. 

Images 6, 7: Malicious code is added to search for cloud secrets and download malicious payload from C2 server checkmarx[.]zone. The downloaded payload is stealing secrets, tokens, and wallet information and exfiltrating it to checkmarx[.]zone/vsx.

The software supply chain is in the hot seat

The Checkmarx packages were compromised as part of the campaign conducted by TeamPCP. The threat actors initially compromised Aqua Security’s Trivy scanner and related GitHub Actions, and has since spread to Checkmarx tools, including KICS GitHub Action and their OpenVSX extensions. 

These incidents follow the compromise of another supply chain security tool, Xygeni GitHub Action earlier this month. Threat researchers have not made any direct connection between the Xygeni compromise and the latest attacks on Trivy and Checkmarx, but the techniques and targets indicate that they could be related. In all three cases, the compromise was made using stolen credentials. How the credentials were stolen remains a mystery.

Once compromised, commits containing malicious code are created and the release tags are pointed to those commits. As of late Monday night, both malicious packages were still available for download on the Open VSX Registry. 

This incident is a textbook example of “blind-trust” engineering, where the convenience of a one-line install command outweighs basic cryptographic integrity, said Noelle Murata, a senior security engineer at Xcape. "The business impact here is a complete loss of environment isolation, as a single unverified pip install can transition an attacker from a developer’s terminal to the core of a Kubernetes cluster in seconds.”

We should care because the industry’s reliance on public repositories without local hash verification or lockfiles has effectively turned the Internet into an unvetted production dependency, Murata said. 

"This breach succeeded because many organizations treat PyPI as a trusted internal mirror rather than a public, high-risk source of untrusted code. If your CI/CD pipelines are pulling directly from the public Internet without validating a requirements.txt against known-good hashes, you have effectively outsourced your root access to anyone who can phish a single package maintainer.”
—Noelle Murata

Installing unvetted packages is essentially the digital equivalent of “eating a sandwich you found on the subway and being surprised when you get food poisoning,”  Murata added.

Why TeamPCP is more serious than earlier attacks

Cory Michal, CISO of AppOmni, said that compared with previous attacks involving AI tools, the TeamPCP attack is more serious because it was not just an abuse of model behavior, prompt injection, or an application bug — it’s a true software supply-chain compromise that led to malicious package publication, credential theft, and persistence on affected hosts. “That puts it in a more consequential category than many earlier AI security attacks, because the risk extends beyond one model or one app into the development and release pipeline itself," Michal said.

“What makes it especially notable is that the LiteLLM compromise appears to have been downstream fallout from the earlier Trivy breach, meaning attackers may have used one trusted CI/CD compromise to poison another widely used AI-layer dependency, which is exactly the kind of cascading, transitive risk security teams worry about most.”
—Cory Michal

Michal added that the big picture issue is that the software supply chain is still built on too much implicit trust and not enough immutability or verification. "Organizations routinely allow third-party actions, packages, and release artifacts into build pipelines because they come from trusted vendors or popular projects, but this incident shows how fragile that model is when an upstream component is compromised," he observed.

How the Trivy attack trickled down to LiteLLM

In the Trivy case, attackers abused trusted CI/CD infrastructure and mutable release references — and the apparent downstream LiteLLM fallout shows how one breach can ripple outward into other projects and other environments, Michal explained.

The broader lesson, Michal said, is that the software supply chain remains insecure by design in too many places, because convenience, automation, and speed still outrank security, leaving many organizations exposed when trust in an upstream source is broken.

“The LiteLLM compromise was not just a CI/CD exposure event. The malicious packages are reported to have installed a credential stealer, Kubernetes lateral-movement tooling, and a persistent system backdoor, so organizations need to treat this as a potential host and cluster compromise, not just a dependency issue."
—Cory Michal

Why this attack is different — and how to manage risk

Michal recommended security teams take immediate steps to identify and remove malicious versions anywhere they were installed, isolate affected hosts, inspect Kubernetes clusters for rogue privileged pods, review network logs for traffic to the reported command-and-control domains, remove persistence mechanisms, and rotate any credentials, tokens, keys, or secrets that were present on those systems.

Teams should also audit whether those environments were exposed because of the earlier Trivy breach, since current reporting indicates the LiteLLM compromise was downstream fallout from that initial CI/CD trust failure, he added.

Rajeev Raghunarayan, head of go-to-market at the security firm Averlon, said that what makes incidents like this dangerous is the scope of access they expose. "Cloud credentials, API tokens, and Kubernetes secrets don’t just represent a single point of compromise. They create pathways into the broader infrastructure those credentials connect to," he said.

“This is the pattern organizations need to plan for. The initial compromise is rarely where the real damage happens. It’s where the attack chain begins.”
—Rajeev Raghunarayan

Incidents like this are a reminder that in the AI era, software supply chain risk is expanding faster than most control models, said Ryan McCurdy, vice president of marketing at the database-change automation company Liquibase. "When widely used developer and AI tooling is compromised, the blast radius can move quickly across credentials, cloud environments, and production systems," he explained.

“The lesson for enterprise teams is bigger than any one package. Governance has to extend across how change is introduced, validated, promoted, and audited. Speed without control is not modernization. It is exposure.”
—Ryan McCurdy

Igor Lasic, SVP of Technology at RL, noted in a post on LinkedIn that the big takeaway for security teams is that the new perimeter isn't your firewall — it's your CI/CD pipeline.

“Last week, we witnessed a masterclass in supply chain weaponization. The threat group TeamPCP didn't just target an application — they targeted the trust we place in security scanners.”
—Igor Lasic

And the shift to LiteLLM was strategic, Lasic said, given the force-multiplier effect of compromising the AI supply chain.

“The attackers chose their target wisely. LiteLLM is the backbone of modern AI infrastructure, acting as a universal proxy for LLM APIs. Its popularity makes it an ideal ‘infection hub’.”
—Igor Lasic

The RL research team did the original analysis. Freelance writer John P. Mello Jr. contributed with additional reporting.

Join RL's free Spectra Assure Community to leverage advanced binary analysis to discover the newest open-source threats and malicious packages like the ones from this campaign.