One year ago, a vulnerability in Apache’s Log4j turned the security world on its ear. What has changed since then? Here are the key takeaways from Log4Shell's legacy.
On November 25, the U.S. Cybersecurity and Infrastructure Security Agency (CISA) shared a new alert (AA22-320A) detailing an incident response engagement they conducted at a Federal Civilian Executive Branch (FCEB) organization. CISA determined that Iranian government-sponsored APT actors exploited the Log4Shell vulnerability in an unpatched VMware Horizon server.
CISA and the U.S. Federal Bureau of Investigation (FBI) believe that all organizations with affected VMware systems that did not immediately apply available patches or workarounds should assume they have been compromised, and should initiate threat hunting activities immediately.
The alert is just the latest evidence of the fallout from the discovery of Log4Shell, a critical, remote code execution (RCE) flaw in the Log4j Apache library. One year since discovery of the vulnerability, threat actors continue to hit "pay dirt" with attacks that exploit Log4Shell and there is a dawning awareness that Log4Shell will not be going away anytime soon.
Despite that, Log4Shell is leading to some important (and needed) changes in how organizations are managing their cyber risk. Here's a look back at the Log4Shell crisis and some key takeaways from the year that followed it.
A holiday season 'jump-scare'
The flaw that became known as Log4Shell first gained widespread attention following the release of a patch by the video game maker Mojang Studios to its MineCraft game on December 10, 2021. That followed the discovery of the flaw known as "Log4Shell" in late November by researchers at the firm Alibaba. (That company later faced sanctions for not disclosing it to Chinese authorities before reporting it to The Apache Foundation.) The release of an update by the Apache Foundation which patched the flaw followed on December 10, 2021.
Soon after disclosure, the first reports surfaced of active scanning of the Internet for applications that were vulnerable to Log4Shell. That set off a race against the clock to find and patch vulnerable Log4j instances before threat actors could exploit it.
The advantage went to the attackers. According to Check Point, on December 13, 2021, just 72 hours after the release of the patch, there were more than 800,000 attempted attacks seeking to exploit the vulnerability. Emergency patching of the vulnerability became the only option if organizations had any hope in protecting themselves.
By January of 2022, Microsoft warned of continued attempts by nation-state actors and cybercriminals to exploit the vulnerabilities found in Log4j to deploy malware on vulnerable systems. Attacks observed by Microsoft’s research team included a China-based group that exploited the vulnerability in some versions of the VMware Horizon server to gain initial access, allowing them to eventually install the NightSky ransomware.
The flaw was also being used by cybercriminal and other groups engaged in coin mining, establishing remote shells, and red-team activity. Also in January of 2022, researchers at Check Point discovered an Iran-linked threat actor known as APT35 exploiting the Log4j vulnerability in order to deploy modular, PowerShell-based malware. Attempts by APT35 using this same tactic were made on more than 150 organizations.
Clearly, the efforts of security practitioners were high during the period following Log4j’s discovery. But as time went on, it became evident that — despite that — the Log4j threat wasn’t going anywhere.
Log4j is here to stay
As 2022 progressed, evidence mounted that cybercriminals were continuing to find success using the vulnerability. In June of 2022, for example, CISA released an Alert (AA22-174AQ) warning that threat actors, including state-sponsored APT actors, were continuing to exploit Log4Shell in VMware Horizon and Unified Access Gateway servers to obtain initial access to organizations that did not apply patches or workarounds.
Later on, in September, CISA released an advisory warning of an Iranian-backed APT group that was using Log4j to target critical infrastructure organizations across the U.S., U.K., Canada and Australia. Then, in October, a group of hackers with links to the Chinese government used the Log4j vulnerability to attack a variety of targets including the legislature of a U.S. state, the government of a Middle Eastern country and a multinational electronics manufacturer, say researchers at Symantec.
The number and diversity of attacks targeting Log4Shell inspires comparison to other top-tier flaws like “Eternal Blue,” the notorious exploit for a Microsoft SMB flaw (MS-17-10) that powered the NotPetya and WannaCry wiper malware attacks.
Charlie Jones, a Software Assurance Evangelist at ReversingLabs, said he expects the impact of Log4Shell to rival that of Microsoft's MS-17-10.
“Log4Shell presents an even more complex challenge than MS-17-10, as it tends to be deeply nested within application dependencies and therefore more difficult to quickly identify with standard tooling."
Log4Shell puts supply chain risk on the front burner
What can the security community learn from the storm caused by the Log4j vulnerability? We think that Log4Shell has profound implications for both enterprises and software publishers.
As we noted at the time, Log4Shell exposed vulnerabilities inherent to DevOps environments, and among organizations that have embraced agile development methodologies. Namely: Software development teams rely heavily today on both proprietary and open source components to build new applications and services rapidly, but are less attuned to the risks that come with that reliance.
Individual components like Apache’s Log4j library might be used directly by developers (making them fairly easy to identify) or indirectly, via third party platforms and packages. Existing application security testing technology like static code analysis (SAST) and dynamic code analysis (DAST) may not be able to detect this kind of inherited risk.
Downstream consumers of those software and services, conditioned to trust software updates and patches from their suppliers, found themselves in an even more vulnerable position, with no easy way to assess their exposure to threats like Log4Shell.
Log4j spurs SBOM adoption
Those gaps in defenses and risk management give momentum to software supply chain security efforts, including the adoption of Software Bills of Materials (SBOMs).
Acting as a kind of ’“nutrition label” for software packages, SBOMs are a foundational step in securing the software supply chain, giving visibility into software dependencies. In the case of Log4j, for example, an SBOM would document that an organization relies on the vulnerability, and that steps would need to be immediately taken to patch Log4j to prevent attackers from exploiting Log4Shell.
That’s why, as the impact of Log4Shell became clear, interest in SBOMs has grown. Back in February of 2022, just two months after the detection of Log4j, the Linux Foundation found that 78% of organizations expect to produce or consume SBOMs in 2022, which went up by 66% from 2021.
A ReversingLabs study, conducted by Dimensional Research, also confirms widespread SBOM use: 67% of software practitioners surveyed said they generate and review SBOMs to discover if software risk is present within their applications. This demonstrates that there is a clear push among the software industry to utilize SBOMs as a key first step in securing the software supply chain.
Looking to the future, experts believe that visibility into the software supply chain, by using an SBOM, for example, should be mandatory, said Matt Rose, a Field CISO at ReversingLabs.
“To prepare for the next zero-day issue, organizations need to take the approach of when, not if another issue is announced. Automated SBOM creation combined with automated software supply chain security scanning is a huge step to prepare for the unknown."
Regulators raise the bar on software security
Many development organizations may not have an appetite for increased scrutiny of their supply chain. However, changing government rules and regulations mean that the changes wrought by the Log4j vulnerability may be mandatory, rather than optional.
In just the last year, the federal government has taken a number of actions to shore up the security of software used by federal agencies, much of it seemingly in response to the Log4j flaw.
For example, the Biden Administration released several items following the Executive Order on Improving the Nation’s Cybersecurity (14028), released in May of 2021. These include the Federal Government’s Enduring Security Framework Working Panel’s report titled “Securing the Software Supply Chain,” released in September of 2022, which is meant to serve as a practical guide for developers. Also, the Office of Management and Budget issued a memorandum (M-22-18) that same month, mandating all government software vendors to provide attestation of software security to federal agencies.
Learn from the threats, and be prepared
While the threat of Log4j will persist, and other zero-day vulnerabilities could arise in the future, the industry is slowly becoming more suited to handling such software supply chain risks. As with other major security vulnerabilities and attacks, Log4j and Log4Shell will have a mixed legacy, fueling attacks. But the threat raises the profile of threats to the software supply chain in ways that could leave organizations better prepared to identify and fend off such attacks.
- Software Supply Chain Security
- Software Bill of Materials (SBOM)
- Executive Order on Cybersecurity (EO 14028)
- Get up to speed with top software supply chain security posts
- Learn key trends, what's ahead: The State of Supply Chain Security 2022-23
- Get report: Software supply chain and the SOC: Why end-to-end security is key
- NVD Analysis 2022: Why you need to modernize your software security approach
- SBOM: What it is — and why it matters for software supply chain security
- Get a free SBOM and software supply chain risk report