Unpacking PECompact: Analyzing and Reverse Engineering Packed Binaries
Malware analysts and reverse engineers frequently encounter binaries that resist static analysis. Authors use packers to compress, encrypt, and obfuscate executable files, hiding the true payload from security tools. Among the legacy windows packers, PECompact remains a classic study in structural obfuscation. Understanding how to unpack PECompact binaries manually reveals core concepts of Windows executable structures and dynamic analysis. Understanding the PECompact Architecture
PECompact modifies a standard Portable Executable (PE) file by compressing its original sections into a custom structure. It appends a unique stub entry point that handles decryption and decompression at runtime. Structural Modifications
Section Merging: The original code and data sections (.text, .data, .rdata) are heavily compressed and often consolidated into custom-named sections like .pec1 and .pec2.
Import Table Destruction: The original Import Address Table (IAT) is stripped out. The packer replaces it with a minimal import table containing only a few essential API functions like LoadLibraryA and GetProcAddress.
The Loader Stub: A small, polymorphic piece of code is injected. This stub executes first when the file is launched, acting as the extraction engine. The Runtime Execution Flow
When a PECompact-packed binary executes, it follows a strict sequence to reconstruct itself in memory:
Stub Execution: The Windows OS loader transfers control to the new packed Entry Point (EP).
Environment Allocation: The stub allocates memory and locates the kernel32.dll base address.
Decompression Loop: The stub reads the compressed blocks, decompresses them, and writes the raw data back to their original virtual addresses.
IAT Reconstruction: The stub uses LoadLibraryA and GetProcAddress to dynamically resolve the original external dependencies, filling the original IAT locations.
The OEP Jump: The stub cleans up its temporary structures and executes a tail jump to the Original Entry Point (OEP), handing complete control over to the actual payload. Manual Unpacking Methodology
Manual unpacking relies on identifying the exact moment the loader stub finishes execution and transfers control to the OEP. Once execution reaches the OEP, the process memory can be dumped to a new file. Step 1: Locating the Original Entry Point (OEP)
To find the OEP, load the packed binary into a user-mode debugger like x64dbg. PECompact heavily utilizes structured exception handling (SEH) and push/pop sequences to obscure the transition to the OEP.
A reliable technique for bypassing the decompression loop is the Hardware Breakpoint on Execution method:
Allow the debugger to run past the initial TLS callbacks if present.
Step into the packer stub until you notice a series of decompression loops.
Look for the common tail jump pattern. In PECompact, this is often a PUSH [Address] followed by a RET, or a direct JMP to a register containing a pointer to a completely different memory section.
Place a hardware breakpoint on execution at the suspected OEP address and run the program (F9). Step 2: Dumping the Process Memory
Once the hardware breakpoint hits, the CPU registers and memory space reflect the fully unpacked state of the program. Do not let the application run any further.
Open an unpacking plugin such as Scylla (integrated directly into x64dbg). Select the active process.
Ensure the OEP field matches the current Instruction Pointer (EIP or RIP).
Click Dump to save the raw, expanded memory space into a new PE file on disk. Step 3: Fixing the Import Address Table (IAT)
The dumped file is not yet executable because its PE header pointing to the Import Table is broken. The imports must be reconstructed so the file can resolve Windows APIs when launched independently.
With Scylla still open and attached to the paused process, click IAT Autosearch. Scylla will attempt to locate the start and size of the reconstructed IAT in memory.
Click Get Imports to resolve the API function names from the memory pointers.
Review the resolved tree. If there are invalid pointers, right-click and use Scylla’s built-in automated heuristics to cut or fix the invalid thunks.
Click Fix Dump and select the raw file you dumped in Step 2. Scylla will create a new file, usually appended with _SCY. Static Analysis and Verification
The final phase of reverse engineering a packed binary involves verifying that the obfuscation layer has been successfully stripped away. PE Header Analysis
Load the fixed file into a tool like PEview or Detect It Easy (DIE). The entropy graph should show a significant drop from near-8.0 (indicating high compression/encryption) down to normal levels (typically between 4.0 and 6.5 for code sections). The original section headers and names should now be fully legible and mapped correctly to their virtual addresses. Disassembly and Decompilation
Open the fixed file in a disassembler like IDA Pro or Ghidra. If the unpacking was successful, the tool will auto-analyze the binary starting from the true main function or winmain entry point. Standard library functions, API calls, and clean control-flow graphs should replace the messy, non-linear jump blocks characteristic of the PECompact packer stub. If you are working on a specific sample right now, tell me: The architecture of the binary (32-bit or 64-bit)? The debugger/tools you prefer to use? Any specific errors or anti-debugging tricks you have hit?
I can provide step-by-step debugger commands tailored to your analysis scenario.
Leave a Reply