ReversingLabs Hashing Algorithm

Here's what you need to know about RL's predictive malware detection.

Traditional hash detection algorithms (e.g., MD5, SHA-1) provide an essential tool for security applications. Although commonly used for allowlisting and blocklisting, traditional hashes have significant drawbacks for detecting malware. First, a malicious file must be seen before a hash can be created so polymorphic attacks are not detectable. Second, hashes are fragile, enabling malware authors to make inconsequential changes to files to avoid detection.

ReversingLabs Hashing Algorithm (“RHA”) addresses these issues by intelligently hashing a file’s features rather than its bits. Files have the same RHA hash when they are functionally similar. This makes RHA orders of magnitude better than traditional hashes for malware detection. One RHA hash can potentially identify thousands of functionally similar malware files, even though each has a unique SHA-1 hash. Further, RHA will detect a new and unknown malware variant because it is functionally similar to known malware.

RHA is superior to the traditional similarity algorithms, such as imphash, ssdeep, tlsh, and others, providing superior identification automatically with pre-defined threat matching. With the traditional similarity algorithms, users have to create filesets that they want to match against; with RHA, users can get threat matching directly as that work has been done by ReversingLabs for them. Keep reading to see the results of our proprietary algorithm with a real-world example.

How RHA Works

RHA enables correlation of files based on functional features. These attributes include format-specific header information, file layout, and functional file information (e.g., code and data relationships). RHA calculates functional similarity at four “precision levels,” 25%, 50%, 75%, and 100%, each based on an increasing number of attributes. Precision level represents the degree a file is functionally similar to another file. A higher precision level will match fewer files, but the files will have more functional similarity.

RHA can be applied to any executable file format. First, format-specific features are abstracted into categories such as structure, layout, content, symbols, functionality, and relationships. Then, algorithms are implemented to evaluate the attributes of each category for similarity at each precision level. Algorithms will vary for each format but usually entail data sorting and simplification. The algorithms calculate a hash for each precision level so that functionally related files fall into the same hash group.