3CX has transformed its software security in the two years since a damaging compromise — and RL was there to help. Here are key takeaways.
Paul Roberts, Director of Content and Editorial at RLPaul Roberts
About two years after 3CX's supply chain compromise, the voice-over-IP vendor has remade its software development process and continuous delivery/continuous integration (CI/CD) pipeline to prioritize the security, integrity, and resilience of its published code. ReversingLabs (RL) has been a part of that transformation. Here's a look at 3CX’s supply chain security transformation — and what your organization can learn from the company’s journey.
Red Flags Fly Over 3CX's Application
In March 2023, following a routine software update by 3CX, endpoint detection tools such as SentinelOne started to identify the 3CXDesktopApp as potentially malicious. Hundreds of thousands of client companies relying on the application for voice and video communication started to block and remove it, disrupting their operations. Initially, 3CX dismissed the alerts as false positives and advised users to whitelist the app.
But SentinelOne and other cybersecurity firms such as CrowdStrike discovered that the warnings were legitimate. The 3CXDesktopApp had been compromised — likely through a breach of 3CX’s development pipeline. On March 29, 3CX acknowledged the breach and urged customers to uninstall the 3CXDesktopApp and switch to its PWA Web Client. 3CX also brought in Mandiant, a Google-owned incident-response firm, to investigate. Mandiant's analysis traced the breach to a compromised employee account, which itself had been targeted through an earlier supply chain compromise involving X_Trader, a financial trading app discontinued in 2018 but breached in 2021.
Researchers at Mandiant, Kaspersky, and other firms eventually linked both incidents to the North Korean state-sponsored Lazarus Group. The attackers appeared to use the 3CX compromise to target a select group of customers in the cryptocurrency sector — revealing a multilayered, targeted campaign that spanned years and industries.
The questions facing 3CX in the immediate aftermath of the damaging 3CXDesktopApp compromise were critical: Should the company simply move on, addressing the immediate problems with the application but otherwise pursuing business as usual? Or should 3CX lean into the breach, using it as the impetus to remake its development and software supply chain security processes? Leaning in could put power in the company’s hands and take it from the attackers. It could actively prevent similar incidents from happening. Tuning out would simply be betting that the company wouldn't wind up in the attackers’ crosshairs again. History has shown us that was unlikely to happen.
3CX chose to lean in, and ReversingLabs was on the short list of security firms that helped it remake its development pipeline and software supply chain security.
Finding the Red Flags
An analysis of the compromised 3CXDesktopApp client published by RL threat researcher Karlo Zanki on March 30, shortly after 3CX acknowledged the compromise, showed that the update for the 3CXDesktopApp — like most other supply chain hacks - set off red flags, if you knew where to look for them.
Among other things, Zanki found that the attackers appended RC4-encrypted shellcode to the d3dcompiler.dll, a standard library used in cross-platform Electron applications such as 3CXDesktopApp. They also modified ffmpeg, a popular open-source module, to execute this code. These alterations were detectable when you compared clean and tampered versions of the software. Furthermore, the malicious modifications used tools such as SigFlip and SigLoader, which are commonly associated with advanced persistent threat groups, Zanki wrote.
How 3CX Tackles Supply Chain Risks with RL
That led to conversations with 3CX in which RL shared its findings and advised 3CX as it looked to secure its CI/CD development pipeline and protect itself from future attacks by addressing software supply chain risks that lurk in both open-source and third-party, commercial software packages.
ReversingLabs is a key component of 3CX’s product security system. Today, every 3CX software binary is scanned using ReversingLabs Spectra technology for hidden malware, evidence of tampering, and other security risks. Working along with vendors such as Coverity and Mandiant, RL helps ensure that 3CX maintains transparency with its customers about the security of its published code and that its applications and supporting services meet the highest standards for software security and integrity.
In the months since the 2023 breach, 3CX has done more than simply button up its breached software. The company has leaned into the challenge of supply chain security. Today, its efforts to maintain the highest standards for the security of its software and its extensive support practices are a selling point rather than an afterthought.
3CX’s Message: Software Transparency is a Requirement
The response of 3CX to its supply chain breach carries a message for other software development organizations: software integrity and transparency are paramount if you want to avoid the kind of devastating attacks that 3CX and its customers endured.
For 3CX, achieving that required wholesale changes to how it developed, built, and released its software. Post-breach, the company invested heavily in application security tools such as static analysis (Coverity) and complex binary analysis (RL) to assess its raw and compiled code for security risks and evidence of tampering or malware. It also engaged with the broader application security community via a bug bounty program managed by HackerOne that incentivizes ethical hackers to identify and fix vulnerabilities in 3CX code before they can be found and exploited by malicious actors.
But are other vendors doing the same? Generally, they are not, say the security experts we asked. While supply chain compromises almost certainly prompt vendors to respond to the immediate risks exposed by the breach, larger changes in their approach to code development, supply chain management, and security tooling are not guaranteed.
“3CX's response was exceptional, but most vendors remain reactive,” said Saša Zdjelar, ReversingLabs' chief trust officer.
After 20-plus years in this industry, I’ve learned ‘trust’ is not a security control; the proper mindset is ‘distrust and scrutinize.’ Technology now lets us independently analyze any software for malware and tampering, not just vulnerabilities. We can stop guessing and start demanding objective proof of security.
Historically, it has been difficult — if not impossible — for organizations to track changes and modifications to the software from their suppliers. But events often tell a story of their own. Starting in 2021, for example, the U.S. Cybersecurity and Information Security Agency (CISA) began issuing notices and alerts about serious, exploitable security flaws in the products of Ivanti, a leading VPN vendor.
In the years that followed, a string of warnings and alerts from CISA and other organizations accompanied sophisticated, nation-state attacks targeting Ivanti’s physical and virtual products as well as the discovery, exploitation, and eventual patching of at least 15 serious software flaws allowing remote code execution, administrative bypass, stack-based buffer overflows, and other attacks.
A steady cadence of attacks and compromises of any software suggests that the vendor in question has adopted a reactive approach to its product security, patching discovered flaws and security holes to address immediate risks without addressing the root causes of those flaws. Without changes to their development, testing, and release processes and to the way they access their software supply chain, vendors cannot cut off the supply of new flaws or short-circuit the malicious campaigns that target them.
The Big Takeaway: Hold Software Producers Accountable
Its track record speaks volumes about a vendor’s commitment to product security, integrity, and resilience. But end-user organizations can’t count on a public record of a given vendor’s misses and misdeeds. That’s why organizations need to be prepared to investigate before signing on to a specific product or renewing an existing license. That means asking tough questions of their suppliers, demanding answers to those questions, and asking for proof that the answers are valid.
Some Questions to Ask Your Software Provider:
Do you publish a software bill of materials (SBOM) — or better yet, a software-as-a-service bill of materials (SaaSBOM) — for your software package?
Can we view the extended bill of materials (xBOM) prior to agreeing to license your software?
Does your organization employ rigorous, secure development practices? If so, please describe them. For example:
Static and dynamic application security testing (SAST, DAST)
Application hardening (ASLR, DEP, etc.)
Does your organization have a software vulnerability disclosure and/or a software bug bounty program? If so, please provide links to any relevant sites related to these programs.
What is your organization’s policy with relation to old and end-of-life software? Could you list any end-of-life or end-of-support software that is used within your organization?
Does your organization conduct complex binary analysis on software builds before they are released to customers? If so, would you be willing to share the findings of your latest scans/reports with us?
Does your organization engage in “red teaming” and penetration testing of its published applications and its larger development environment?
A perfect score on the above questions isn’t needed to win the confidence of potential customers, but a clear pattern of security-focused processes and programs should emerge from the answers to such a questionnaire. A flurry of replies such as “no” and “no comment” should raise red flags.
Follow us
Subscribe
Get the best of RL Blog delivered to your in-box weekly. Stay up to date on key trends, analysis and best practices across threat intelligence and software supply chain security.
The Next MOVEit May Already Be in Your File Transfer Queue
MFT and other trusted systems collect files, access and connections. Patching and inspecting what moves through it are separate defenses.
What RL Found Before Anthropic’s Midnight Blizzard Report
ReversingLabs identified and classified the malware weeks before the report was published — showing why behavioral intelligence and historical telemetry matter in the AI age.
