Thwarting Code-Injection Attacks Using SDT-Based ISR

Defense Advanced Research Projects Agency, Arlington, Virginia

Monday, October 01 2007

This method offers advantages of security and practicality.

An improved method has been devised for defending a server or other computer against a code-injection attack, in which an attacker exploits a hardware or software vulnerability to inject harmful or otherwise unwanted code into an application program that is being executed and then causes the injected code to be executed. The present improved defense method provides for a secure and efficient implementation of instruction-set randomization (ISR), incorporating several advances beyond related prior methods that utilize ISR.

ISR is a theoretically strong approach to defending against a code-injection attack, irrespective of the nature of either the attack or the vulnerability that the attacker exploits. In a computer defended by ISR, an instruction set for the desired application program is created by a randomization algorithm. The encrypted instruction set is sent to an emulator for execution. The emulator is augmented to decrypt the instructions before execution. When an attacker exploits a vulnerability to inject code, the injected code is also subjected to the decryption process. Unless the injected code has been encrypted by means of the same key as that used in the defending computer (in effect, unless the attacker knows the encryption key), the decryption process transforms the injected code into, in essence, a random stream of bytes that give rise to an exception (e.g., invalid operation code or invalid address) when execution is attempted.

The security of ISR depends on the strength of the encryption process, the protection of the encryption key, the security of the underlying execution process, and the probability that an attempt to execute injected code will result in exception. The practicality of ISR is affected by the overheads in execution time and storage space introduced by the encryption and decryption processes. The improvements incorporated into the present method were made with consideration of both security and practicality (of which efficiency is an important component).