Compromised npm package threatens developer projects

ReversingLabs’ automated threat detection system discovered a compromise of a popular npm package, eslint-config-prettier, on July 18. The package has more than 3.5 billion downloads and 12,000 dependencies. Several other packages published by the same maintainer were also affected, and malicious versions of eslint-config-prettier were published from the maintainer’s account that was compromised in a well-crafted phishing campaign. The campaign was reported by the Socket research team on the same day as RL’s detection. 

Here’s how that compromise — and the widespread practice of automated dependency updating with tools such as Dependabot — amplified the effects of this serious supply chain incident, spilling the attack over into many GitHub repositories, and almost certainly resulting in the compromise of development build machines. 

How the hack happened

The Socket researchers found the threat actor behind this compromise began their malicious campaign with a phishing email that impersonated npm: spoofing the legitimate support@npmjs.[org] address, and serving up a domain that is a full copy or proxy of npm’s actual website. The phishing domain also used a tokenized URL, indicating that the attackers made an effort to target npm users – potentially those who are active package maintainers with significant reach, Socket researchers noted. 

The maintainer of eslint-config-prettier later confirmed that he fell victim to this exact phishing attack. Other packages belonging to the maintainer, including eslint-plugin-prettier, synckit, @pkgr/core, and napi-postinstall were also targeted. Using the maintainer’s stolen credentials, attackers published multiple unauthorized versions of these packages with malicious code designed to infect Windows machines. 

The first malicious version of the eslint-config-prettier package was published on July 18 at 15:51 GMT. All malicious versions were removed by 18:40 GMT that same day, leaving a time-window of about two hours for the compromise to spread. While that might not sound like a wide exposure window, the prettier packages average more than 36 million weekly downloads — meaning that even a narrow window of compromise can have large repercussions. 

Malicious versions of the compromised packages contained a postinstall script that dropped an PE DLL file containing the Scavenger remote access trojan (RAT) malware. More technical details about the dropped DLL can be found in this research blog post. 

Since this is a configuration for a development tool used for code formatting, it can be expected that it should be declared as a devDependency across packages in which it is used, and, as such, it shouldn’t be automatically installed when the npm install command is executed like with  regular dependencies. 

The RL research team checked the database to see how many packages include the eslint-config-prettier package as a direct dependency and not devDependency. We found that more than 14,000 packages have it declared in that way. That is a pretty large number of packages that could lead to a number of downstream attacks. However, the narrow window of compromise (about two hours) and the various options governing software dependencies (pinned vs. ranged versions) means the number of build and development machines affected by the eslint-config-prettier package breach is likely much smaller. 

Pins vs. carets: Notes on npm code resolution

Version resolution is a complicated mechanism in the npm ecosystem. The semantic versioner (semver) implements a detailed algorithm to enable version resolving and support different types of version declarations. Not just specific version declaration, but also version ranges, minimal and maximal versions, and to support different release types like major, minor and patch releases. 

When you have that three-part version number, the first part represents the major version, the second part the minor version, and the third part represents the patch version. The whole mechanism is a bit complicated, but for this incident, it is sufficient to note that a large portion of version definitions are either pinned to a specific version (for example 10.1.5) or specify a “caret range” like ^10.1.5 for the version.

A brief explanation: The caret before the version number in the example above means that the latest minor or patch version of the dependency will be installed at package installation. So in the previous case, it would include any version greater or equal 10.x.x and smaller than 11.0.0. 

For developers, there are pros and cons for pinning the version number and using a caret range. Pinning to a specific version makes sure you include a version of a dependency your codebase is sure to function with, but it makes dependency updating harder and more resource consuming. It also leaves space for missing some important security updates released for those dependencies.

Specifying a version range with a caret makes your project use the latest version from the defined major version range and by specification that should not include any breaking changes. But that is not guaranteed. An important security consideration to keep in mind is that such an approach leaves your project vulnerable to inclusion of a malicious version of a compromised dependency without your notice. 

Another piece of this complicated mechanism are lock files that make sure the release you published uses a specific version when installed on a machine, to prevent incompatibility that might be introduced with a newer version of some dependency.

The automated update paradox 

In theory, when a new version of a dependency used in your project appears, your team should review it and decide whether to include it in your project. In practice, organizations are typically not willing to spend resources on dependency review, and they try to automate it.

One of the most popular solutions for that task is GitHub’s Dependabot, an automated dependency update management tool. Dependabot is capable of generating automated alerts when a new version of dependency appears, and can also open pull requests (PRs) to update the dependencies used in your project. 

Figure 1: Dependabot opening a version upgrade PR and another bot approving and merging it.

GitHub Actions can also be used to enable automerging of PRs opened by Dependabot. By now, you can probably sense the security concerns related to automerging of dependency updates without reviewing them. During our research, we detected several GitHub repositories that included a dependency on a malicious version of eslint-config-prettier as a result of this type of automated dependency version upgrading.

Pwned and automated: the Dott story

One of the more notable was the GitHub repository of a company named Dott that specializes in public bike fleet management. Dott had Dependabot enabled to make automated version upgrade pull requests (PRs) and another bot to approve and merge them as visible in Figure 1.

Most GitHub workflows contain checks that include building and testing of a pull request (PR) before it can be approved for merge. Running a predefined test usually includes a step that installs the needed dependencies by running package manager commands like npm install. At that step, the malicious dependency gets installed and the malicious code gets executed and (hopefully) detected. 

Figure 2: Installation of dependencies during PR checks by the bike fleet management firm Dott.

If your team is using GitHub-hosted runners, you will get a fresh and clean virtual machine (VM) instance every time you run a test, so malicious code won’t persist between runs and will be contained in a secure environment. However, development organizations that rely on self-hosted runners and follow poor configuration practices may be more vulnerable to a compromise of a build machine, which expands the risks to development organizations. 

The automation of dependency upgrades using tools like Dependabot is the main reason why the compromise of eslint-config-prettier package likely had a significant impact on downstream development organizations, despite the narrow, two hour window of compromise.

That’s because malicious code running on a compromised build machine will likely have access to GitHub tokens used for building and publishing in your GitHub repository. Leaked tokens very often have excessive access rights and get used by attackers to infiltrate build environments and that is why they are becoming a popular target for threat actors.

For Dott, the automated dependency updating mechanism removed the malicious eslint-config-prettier dependency by the time RL detected it and replaced it with a newer, clean version that was published 3 days after the malicious version. 

What was the larger impact of the eslint-config-prettier compromise? The RL team’s GitHub search for the hash value used to verify the integrity of the malicious version of eslint-config-prettier (version 10.1.7) returned 46 package-lock.json files that contained that malicious version. One of those victims was an open-source software project run by Microsoft.

These 46 victims are development projects that had a build triggered during the two-hour window of compromise, and ended up including one of the malicious versions. What we don’t know is the number of users who installed a package during that two hour window and had eslint-config-prettier declared as a dependency and not a devDependency. 

Figure 3: GitHub projects including malicious versions of eslint-config-prettier.

While in most of these cases the malicious version was defined as a devDependency, that doesn’t mean that they wouldn’t get installed on the build machine. Depending on the configuration of the workflow file, devDependencies can also get installed during execution of npm install or npm ci commands. Specifically, the malicious code executes the DLL ‘node-gyp.dll’ – a recognized trojan that as of July 19 had a 19/72 detection score on VirusTotal, highlighting how the malware is still being missed by most antivirus tooling.

Documentation at npm states that the omit option can be used with npm ci command to specify the type of dependencies to omit from the installation on disk, but also expects the NODE_ENV variable to be set to ‘production.’ Based on the complicated nature of GitHub workflow configuration, it is very likely that a good portion of the projects from our list of 46 packages ended up installing these malicious versions to the build machine.           

Among many impacted projects, a Microsoft-owned netperf project also appeared as visible in Figure 4. RL reported the finding to the maintainers and the Microsoft team fixed the issue by removing that dependency in a short time. 

Figure 4: Merged commit in the Microsoft’s netperf GitHub repository.

Issues were also reported on several other projects:

• https://github.com/microsoft/netperf/issues/687
• https://github.com/niaid/monarch-actions-switch-account/issues/971
• https://github.com/w3ux/w3ux-library/issues/321

Lessons learned

Dependency version management is a complicated task that is becoming one of the main security challenges in modern software development. That is because of the heavy reliance of development organizations on open source and public software packages. 

And dependency risks pose a real challenge. On one hand, development teams have a clear need to perform dependency updates to mitigate security issues discovered in software dependencies. However, at the same time, that updating introduces another security risk: integrating a malicious version of a compromised dependency.

Automated version management tools like Dependabot are designed to remove the risk of having dependencies with security issues in your code base, but as this research blog shows, ironically they can end up introducing even bigger security issues like malicious compromise. 

With automated dependency upgrades, you are closing one door for the threat actors, just to open another, even wider one.

With automated dependency upgrades, you are closing one door for the threat actors, just to open another, even wider one.

Compromises of popular packages in open source package repositories like npm are on the rise. In recent weeks, RL has noted the compromise of several very popular npm packages, including the is package with more than 2.5 million weekly downloads that is used by projects from big name firms like Google. At the time of the compromise, the is package hadn’t had a new version released for more than 6 years before the new (compromised) version appeared. (Google quickly addressed the issue and released a new version of their nodejs-bigquery project.)

That version of the is package specified in their package.json file was greater than 3.3.0. One might ask if you should pin such unmaintained projects to a specific version to avoid such risks, or should you even remove such unmaintained projects from your dependency list. 

Another example of a dependency-based compromise happened less than a month ago when a trusted VSCode extension merged a pull request from a freshly created GitHub account. That pull request implemented more than 4,000 lines of useful code, but among them, there were two lines that introduced a malicious dependency into the project. This is another example that rebuilding packages from source code can’t remediate all supply chain security risks — making dependency management an important pillar in supply chain security.

Recommendations

Development organizations concerned about the risks posed by compromised code dependencies should consider the following practices in order to reduce their risk of falling victim to a supply chain attack.

• Do not rush into upgrading a file dependency unless the upgrade is resolving a critical security issue. Most compromises are detected in just a few days, so even a short pause in applying a package update will reduce the chance of a compromise. 
• Separate your team’s dependencies and devDependencies. 
• Properly configure your build workflows to minimize the risk of installing dependencies unnecessary in production. 
• Avoid merging automated version upgrades into your code base without performing security vetting. 

How RL Spectra Assure Community can help

If you lack resources or knowledge for such a task, there are dedicated tools like Spectra Assure Community that can help you spot the threat. 

RL’s Spectra Assure Community provides information about packages published in several public package repositories, including npm, PyPI, RubyGems, NuGet and VS Code marketplace. For each package you can get information about compliance, security and threat related issues. A powerful feature is the list of software quality and threat hunting policies that specific package versions violate. Threat hunting policy violations are a set of rules that can help you detect malicious packages before they can compromise your development environment or code base.

Figure 5: Spectra Assure Community summary for latest version of eslint-config-prettier package.

Figure 5 shows how a package summary looks for the eslint-config-prettier npm package described in this research blog. The summary displays info for the latest version, published after removing the malicious versions from the repository. In “Tampering” related information, you can see the warning that a recent version of this component had a malicious or tampering incident. This warning is triggered by a violation of TH30101 policy and stays visible for the affected packages two years after the incident. 

The biggest benefits of using Spectra Assure Community are that you can recognize potentially malicious packages with the help of RL’s predictive threat hunting ML models for npm and PyPI, and also get information about all the threat incidents RL researchers discovered during their continuous monitoring of public package repositories. Start using it today at secure.software.