Suspicious file analysis: Enhance your SOC to fight sophisticated attacks

With threat actors constantly ramping up the sophistication and volume of their campaigns, file triage has become essential for modern security operations (SecOps).

A recent ReversingLabs report revealed that security operations center teams spend an average of two to three hours per day — roughly 45% of their work time — managing suspicious files. Without an efficient triage process for quickly assessing and prioritizing potentially malicious files based on a variety of risk indicators, SOC teams risk being overwhelmed by false positives that cause them to miss real threats or delay their responses, possibly leaving their organizations vulnerable to attacks.

Stuart Philips, digital marketing strategist at ReversingLabs, said the growth in ransomware attacks was one area of particular concern.

As cyber threats become more sophisticated and voluminous, traditional security measures are often overwhelmed, leaving enterprises vulnerable to attacks.

Stuart Phillips

Here's why you need to modernize your SecOps with advanced file analysis.

Get White Paper: Accelerate Your Suspicious File Triage

File threat sophistication and volume are growing

Attackers are using new obfuscation and anti-analysis tactics that, coupled with the sheer diversity and complexity of file types and content, are making file triage more challenging for many organizations, said Ken Dunham, cyberthreat director at Qualys' Threat Research Unit.

Adapting agile SecOps strategies, with multiple layers of automation and strategies, is essential to proactively protect against emergent threats and efficiently triage and combat threats at scale.

Ken Dunham

Modern enterprises currently employ a variety of methods for file analysis and triage. There's static analysis, dynamic analysis, and risk scoring for starters, as well as automated tools such as multi-engine virus scanners, endpoint detection and response platforms, intrusion detection systems, automated sandbox analysis platforms, file integrity monitoring products, and automated unpacking tools.

Static analysis typically involves things such as comparing file hashes against databases of known threats and doing file type verification, code structure analysis, and pattern matching. With dynamic analysis, security teams run files in isolated sandboxes, monitoring for suspicious behaviors and doing API call monitoring, memory analysis during execution, and network traffic analysis.

Risk scoring can include anything from evaluating file properties and behaviors against known threat frameworks to considering potential targets, doing business impact analysis, and looking at file prevalence and reputation before making an assessment.

Protecting against modern threats takes more

While all of these methods and tooling remain essential to file triage, a variety of factors are blunting the effectiveness of some processes. Increasing attack volumes, for example, have overwhelmed manual analysis processes, saddling many organizations with thousands of suspicious files daily across email, web, and endpoint sensors.

Meanwhile, the surge in remote work and cloud adoption has expanded the attack surface, creating more entry points for malicious files. Attackers are using more sophisticated evasion techniques such as designing polymorphic malware that changes its signature to avoid detection, multistage attacks that only reveal malicious behavior after initial execution, and fileless malware that operates solely in memory.

Roger Grimes, data-driven defense evangelist at KnowBe4, said "chained" attacks have started to dominate: One thing leads to another thing, which leads to another thing, and so on — for three to five iterations on average.

So even if the original communication's medium didn't allow a particular dangerous file type, the chained linking will get the user to a platform that does allow it.

Roger Grimes

The problem, Grimes said, is that the defender's tool set has trouble following the entire length of the attack chain. "Attackers know this and are expanding their chains — each additional link drops some percentage of defender tooling. It's just a matter of odds and persistence," Grimes said.

Stephen Kowski, field CTO at SlashNext Email Security+, said the rising use of AI-generated content to create highly convincing but malicious documents and files is another complicating factor. Social engineering tactics have become more sophisticated, making malicious intent harder to detect through traditional means, he said. A 2024 study by VIPRE Group found that a startling 40% of all phishing emails directed at business are generated using artificial intelligence and that 60% of the recipients of these emails fall victim to AI-automated phishing.

Grimes said that between 70% and 90% of successful attacks today use social engineering and phishing across multiple communications mediums, including email, web, SMS, social media, telephone calls, and in-person scams.

Certainly AI-enabled deepfake attacks are making it harder. How can a defending tool accurately tell the difference between a video containing a boss's instructions and a deepfake video that looks like the boss?

Roger Grimes

Qualys' Dunham said that while automated analysis remains crucial to file triage, modern obfuscation methods and encrypted payloads are posing a challenge to its effectiveness. False negatives can result when malicious files evade detection because they are polymorphic, obfuscated, or encrypted. Similarly, fileless attack methods and living-off-the-land techniques often make it harder for security teams to distinguish malicious from benign activity.

Adversaries are constantly evolving their tactics, techniques, and procedures to defeat known blue-team defensive TTPs and configurations.

Ken Dunham

The need to enhance file analysis is clear

Nonetheless, file analysis remains a cornerstone of cybersecurity, said ReversingLabs' Phillips. By implementing advanced file analysis techniques, organizations can more effectively identify and respond to potential threats faster, more accurately, and at a lower cost per mitigation while reducing SOC operator fatigue.

Every SOC operator knows that blocking malware at its origin is the most effective method of preventing it from detonating ransomware in their organization.

Stuart Phillips

Dunham said that a strong cyberthreat intelligence-driven program will consider ongoing changes to TTPs so that it can continually update and counter adversarial tactics for the best outcomes in reducing risk. To counter the growing sophistication and volume of threats, security teams should consider deploying machine-learning (ML) and AI tools to support their broader threat-detection program.

ML and AI tools can be fully automated and combined with specific use cases and playbooks, Dunham said. For example, a ransomware attachment can result in automated notifications to specific teams, along with indicators of compromise (IOCs), information on associated threat-actor tactics, techniques and procedures, and automated remediation and/or flagging. That can all result in improved detection and processing by a security team, Dunham said.

SlashNext Email Security+'s Kowski said AI-driven systems can process files in milliseconds, while maintaining extremely high accuracy rates through continuous learning.

Manual approaches to analyze suspicious file have become unsustainable and will only create bottlenecks that leave organizations more vulnerable to attack. So, said Kowski, the focus should be real-time processing and automated analysis. Teams should focus on security tools that can break down files into components while analyzing intent-based messaging and calls to action, he said.