Constant Insecurity: Things you didn’t know about (PE) Portable Executable file format

Evolving projects, by their very nature, open up a host of exploit areas and implementation ambiguities that cannot be fixed. As such, complex documentation such as that for PECOFF or PDF are goldmines of possibilities.
In this talk we will disclose our recent findings about never before seen PE or Portable executable format malformations. These findings have serious consequences on security and reverse engineering tools and lead to multiple exploit vectors.
PE is the main executable image file format on Windows operating system since its introduction in Windows NT 18 years ago. PE file format itself can be found on numerous Windows-based devices including PCs, mobile and gaming devices, BIOS environments and others. Its proper understanding is the key for securing these platforms. The talk will focus on all aspects of Portable Executable file format parsing that leads to undesired behavior or prevents security and reverse engineering tools from inspecting malformated files due to incorrect parsing. Special attention will be given to differences between PECOFF documentation and the actual implementation done by the operating system loader. With respect to these differences we will demonstrate existence of files that can't possibly be considered valid from a documentation standpoint but which are still correctly processed and loaded by the operating system. These differences and numerous design logic flaws can lead to PE processing errors that have serious and hardly detectable security implications. Effects of these PE file format malformations will be compared against several reverse engineering tools, security applications and unpacking systems. Special attention will be given to following PE file format aspects and their malformation consequences:

• General PE header layout in respect to data positioning and consequences of different memory model implementations as specified by PECOFF documentation. Use of multiple PE headers in a single file along with self-destructing headers.
• Alignment fields with their impact on disk and memory layout with the section layout issues that can occur due to disk or memory data overlapping or splicing. In addition to this, section table content will be inspected for issues of data hiding and its limits will be tested for upper and lower content boundaries. We will demonstrate how such issues affect existing static and dynamic PE unpacking systems.
• Data tables, including imports and exports, will be discussed in detail to show how their malformated content can break analysis tools but is still considered valid from the operating system loader standpoint. We will demonstrate existence of files that can miss use existing Portable Executable features in order to cloak important file information and omit reverse engineering process. Furthermore based upon these methods a unique undetectable method of API hooking that requires no code for hooks insertion will be presented.
• PE file format will be inspected for integer overflows and we will show how their presence can lead to arbitrary code execution in otherwise safe analysis environments. We will show how Portable Executable fields themselves could be used to deliver code payload resulting in a completely new field of programming; via the file format itself.
• In addition to single field and table malformations more complex ones involving multiple fields and tables will also be discussed. As a demonstration of such use case scenario a unique malformation requiring multiple fields working together to establish custom file encryption will be presented. This simple, yet effective, encryption that is reversed during runtime by the operating system loader itself requires no code in the malformated binary itself to be executed. Its effectiveness is in a unique approach to encryption trough file format features themselves in order to prevent static and dynamic file analysis tools from processing such files.

This talk will be a Black Hat exclusive; Whitepaper accompanying the presentation materials will contain detailed description of all malformations discussed during the talk. This whitepaper aims to be a mandatory reading material for security analysts. It will continue to be maintained as new information on Portable Executable (PE) format malformations are discovered.
More information here.