Abstract:
A method of dynamically hooking runtime processes without interrupting the flow of execution includes: suspending a thread; hooking a function comprising modifying code of the function; and determining whether the thread was executing the modified code when the thread was suspended. If the thread was not executing the modified code, the thread is resumed. If the thread was executing the modified code, the context of the thread is changed to redirect the thread to a saved copy of the original prologue. In this manner, unpredictable behavior of the thread is avoided.

Description:
BACKGROUND OF THE INVENTION 
   1. Field of the Invention 
   The present invention relates to the protection of computer systems. More particularly, the present invention relates to hooking of runtime processes. 
   2. Description of the Related Art 
   The necessity of hooking runtime processes arises in various scenarios and situations like debugging, troubleshotting, profiling, extending functionality, etc. The challenge is to be able to successfully hook the function without interrupting the flow of execution. 
   When hooking a function via a prologue overwrite, the case that another thread was executing the prologue of the function that was hooked at the time it was hooked should be considered. With modern processors, the cache will become invalidated once the prologue is overwritten so that the CPU will execute the modified instructions. However, if it had executed only part of the prologue, this may create an invalid state that will crash the thread or the process. 
   SUMMARY OF THE INVENTION 
   A method of dynamically hooking runtime processes without interrupting the flow of execution includes: suspending a thread; hooking a function comprising modifying code of the function; and determining whether the thread was executing the modified code when the thread was suspended. If the thread was not executing the modified code, the thread is resumed. If the thread was executing the modified code, the context of the thread is changed to redirect the thread to a saved copy of the original prologue. In this manner, unpredictable behavior of the thread is avoided. 
   Embodiments in accordance with the present invention are best understood by reference to the following detailed description when read in conjunction with the accompanying drawings. 

   
     BRIEF DESCRIPTION OF THE DRAWING 
       FIG. 1  is a flow diagram of a dynamic hooking process in accordance with one embodiment of the present invention. 
   

   Common reference numerals are used throughout the drawings and detailed description to indicate like elements. 
   DETAILED DESCRIPTION 
   In the whole process of suspending the threads, altering the code structure and resuming the threads during hooking of function(s), there lies a possibility that the thread that was suspended was executing one of the functions that was hooked. If part of what the thread was executing was modified while it was suspended, the behavior of the thread after being resumed is unpredictable. 
   Because hooking functions should not disrupt the process, one embodiment of the present invention adds an operation to function hooking through prologue overwrites whereby other threads in the process are checked to ensure they are not affected. A copy of the prologue that is being hooked is saved. If other threads in the process are being affected, the offset of the thread&#39;s instruction pointer into the prologue is calculated and the thread is redirected to the copy of the saved prologue, which then jumps back to the original function. 
   More particularly, a typical target function (to be hooked) looks like the following: [Prologue] [Function body] [Epilogue] 
   Hooking a function typically involves overwriting the usual prologue with a jump or call instruction to the hooking code. When hooing a live process, there will likely be several threads already running. 
   The hooking process typically must suspend any and all threads of the target process. The hooking process further injects the code pages or hooking thread, save copies of the original prologues, overwrite the prologues of the functions to be hooked, and then resume execution of all threads. 
   In the whole process of suspending the threads, altering the code structure and resuming the threads there lies a possibility that the thread that was suspended was executing one of the functions that was hooked. If part of what the thread was executing was modified while it was suspended, the behavior of the thread after being resumed is unpredictable. Most modern processes will invalidate the cache and read in the modified instructions at the thread&#39;s instruction pointer, which may now be garbage. 
   Every thread has a context of execution at any point of time. Whenever the thread is suspended, the thread context needs to be examined. Win32 API GetThreadContext helps to get the context of the running thread. 
   The thread context contains processor relevant information like state of the registers including the location where EIP points to. 
   In accordance with one embodiment, it is verified that the EIP isn&#39;t within a region that was modified during hooking (e.g., check if the EIP lies within the prologue). 
     FIG. 1  is a flow diagram of a dynamic hooking process  100  in accordance with one embodiment of the present invention. 
   From an ENTER OPERATION  102 , flow moves to a SUSPEND THREAD(S) OPERATION  104 . In SUSPEND THREAD(S) OPERATION  104 , any and all threads of the target process to be hooked are suspended. 
   From SUSPEND THREAD(S) OPERATION  104 , flow moves to a GET CONTEXT(S) OF SUSPENDED THREAD(S) OPERATION  106 . In GET CONTEXT(S) OF SUSPENDED THREAD(S) OPERATION  106 , the context of each suspended thread is obtained, e.g., using the Win32 API GetThreadContext. 
   From GET CONTEXT(S) OF SUSPENDED THREAD(S) OPERATION  106 , flow moves to a HOOK FUNCTION(S) OPERATION  207 . In HOOK FUNCTION(S) OPERATION  107 , the desired functions are hooked, for example, by overwriting the usual prologue with a jump or call instruction to the hooking code. 
   From HOOK FUNCTION(S) OPERATION  107 , flow moves to an ARRANGE HOOKED FUNCTION(S) INTO A BINARY SEARCH TREE OPERATION  108 . In ARRANGE HOOKED FUNCTION(S) INTO A BINARY SEARCH TREE OPERATION  108 , each function that is hooked is arranged in a binary search tree with the following information: 
   HOOKED_FUNCTION 
   
       
       
         
           DWORD HookedPrologueStart 
           DWORD HookedPrologueEnd 
           DWORD SavedPrologueStart 
         
       
     
  
   The binary search tree is sorted based on the HookedPrologueStart field. 
   From ARRANGE HOOKED FUNCTION(S) INTO A BINARY SEARCH TREE OPERATION  108 , flow moves to a SELECT FIRST SUSPENDED THREAD OPERATION  110 . In SELECT FIRST SUSPENDED THREAD OPERATION  110 , the first suspended thread is selected. 
   From SELECT FIRST SUSPENDED THREAD OPERATION  110 , flow moves to a GET EIP OF SUSPENDED THREAD OPERATION  112 . In GET EIP OF SUSPENDED THREAD OPERATION  112 , the EIP for the suspended thread, sometimes called the “ThreadEIP”, is obtained, e.g., from the context of the suspended thread obtained in OPERATION  106 . 
   From GET EIP OF SUSPENDED THREAD OPERATION  112 , flow moves to a PrologueStart&lt;=ThreadEIP&lt;PrologueEnd CHECK OPERATION  114 . In PrologueStart&lt;=ThreadEIP&lt;PrologueEnd CHECK OPERATION  114 , a determination is made as to whether PrologueStart&lt;=ThreadEIP&lt;PrologueEnd for any node in the binary search tree, i.e., whether the ThreadEIP is greater than or equal to PrologueStart and less the PrologueEnd. If the ThreadEIP is less than PrologueStart or greater than or equal to PrologueEnd, the suspended thread was not executing the hooked function. Conversely, if the ThreadEIP is greater than or equal to PrologueStart and less than PrologueEnd, the suspended thread was executing the hooked function. 
   Accordingly, if the ThreadEIP is less than PrologueStart or greater than or equal to PrologueEnd, flow moves to RESUME THREAD OPERATION  116  and the thread is resumed. Conversely, if the ThreadEIP is greater than or equal to PrologueStart and less than PrologueEnd, flow moves to CHANGE CONTEXT OF THREAD OPERATION  118 . 
   In CHANGE CONTEXT OF THREAD OPERATION  118 , the context structure of the thread is changed, e.g., using SetThreadContext, as follows:
         OffsetFromPrologueStart=ThreadContext-&gt;EIP−Node-&gt;HookedPrologueStart   ThreadContext-&gt;EIP=Node-&gt;SavedPrologueStart+OffsetFromPrologueStart.       

   From CHANGE CONTEXT OF THREAD OPERATION  118 , flow moves to RESUME THREAD OPERATION  116  and the thread is resumed. 
   From RESUME THREAD OPERATION  116 , flow moves to an ADDITIONAL SUSPENDED THREADS CHECK OPERATION  120 . In ADDITIONAL SUSPENDED THREADS CHECK OPERATION  120 , a determination is made as to other there are any additional suspended threads. 
   If there are no additional suspended threads, flow moves to and exits at an EXIT OPERATION  122 . Conversely, if there are additional suspended threads, flow moves to a SELECT NEXT SUSPENDED THREAD OPERATION  124 . 
   In SELECT NEXT SUSPENDED THREAD OPERATION  124 , the next suspended thread is selected for operation. OPERATIONS  112 ,  114 ,  116 , and sometimes OPERATION  118 , are performed on the thread selected in OPERATION  124 . OPERATIONS  120 ,  124 ,  112 ,  114 ,  116 , and sometimes  118 , are performed until a determination is made that there are no additional suspended threads in ADDITIONAL SUSPENDED THREADS CHECK OPERATION  120 , and flow moves to and exits at EXIT OPERATION  122 . 
   In the above manner, the following operations to the end of the hooking prologue are added:
         1. Enumerate through the list of threads in the process   2. Get the EIP field out of the thread&#39;s context (via GetThreadContext) and search through the binary search tree for a node that has a PrologueStart&lt;=ThreadEIP&gt;PrologueEnd.   3. If no matching nodes are found, resume the thread and repeat operation 1   4. If a matching node was found, it effectively means the code was pulled out from under the thread. We must now change the thread&#39;s context structure (e.g., SetThreadContext) as follows:
           OffsetFromPrologueStart=ThreadContext-&gt;EIP−Node-&gt;HookedPrologueStart   ThreadContext-&gt;EIP=Node-&gt;SavedPrologueStart+OffsetFromPrologueStart   
           5. Resume the thread and repeat at operation 1       

   Operation 4 is calculating the offset of the thread&#39;s instruction pointer into the prologue and then redirecting it to the copy of the saved prologue, which then jumps back to the original function. 
   This disclosure provides exemplary embodiments of the present invention. The scope of the present invention is not limited by these exemplary embodiments. Numerous variations, whether explicitly provided for by the specification or implied by the specification or not, may be implemented by one of skill in the art in view of this disclosure.