Abstract:
A correlation manager correlates network traffic with corresponding file input/output activity. In some embodiments, a correlation manager filters both remote network traffic received by a kernel level fileserver and file input/output operations executed by the kernel level fileserver. The correlation manager correlates a thread requesting performance of a file input/output operation with a worker thread that performs the requested file input/output operation, and is thus able to correlate the remote request to perform the file input/output operation with the resulting performed file input/output operation itself. In some embodiments, the correlation manager correlates a transport driver interface thread requesting a file input/output operation with its corresponding system worker thread that implements the requested file input/output operation.

Description:
TECHNICAL FIELD 
   This invention pertains generally to computer operating system internals, and more specifically to correlating network traffic with it corresponding file input/output activity. 
   RELATED APPLICATION 
   This patent application is related to U.S. patent application Ser. No. 10/902,229, filed on Jul. 28, 2004, titled “Lightweight Hooking Mechanism for Kernel Level Operations,” and having the same assignee (the Lightweight Hooking Mechanism application). The Lightweight Hooking Mechanism application is hereby incorporated by reference in its entirety. 
   BACKGROUND 
   In Microsoft Windows NT®, remote clients can read and write to files across the network. The LanManager redirector (RDR) implements the client side of the protocol for such activity by converting NT file system requests into SMB protocol (recently renamed CIFS by Microsoft) requests. These requests are then sent to the particular LanManager fileserver for processing. Microsoft&#39;s srv.sys is a kernel level operating system component that implements the server side of this interface. As such, srv.sys is not a file system, but rather a fileserver. A system running srv.sys can allow remote users to access any local file system data stored on that particular system. 
   Srv.sys comprises a network based interface and a file system based interface. The network based interface comprises the transport driver interface (TDI) layer and the network protocol drivers (e.g., a TCP/IP driver, a TCP/IPv6 driver or a NetBIOS over TCP/IP driver). Srv.sys communicates with remote computers through the network based interface. 
   The file system based interface is the corresponding interface with the local file system drivers that process the remote file I/O requests. Srv.sys communicates with local file system drivers such as ntfs.sys (the NT file system driver), msfs.sys (the mail slots file system driver) and npfs.sys (the named pipes files system driver) through the file system based interface. 
   Typically, srv.sys receives remote file I/O requests through the TDI interface, and then serves those requests through the file system interface. For example, in response to receiving a remote SMB request to create a file on an NTFS shared folder, srv.sys would issue a new IRP_MJ_CREATE request, and send it to the NT file system driver. 
   Correlating srv.sys network traffic with the corresponding file system traffic would enable many beneficial opportunities. For example, such correlation would allow identification of the remote computer and user issuing a remote file I/O request. This applies to all kinds of remote file I/O requests: write file to the local machine, read file from the local machine, query information of a file on the local machine, set information of a local file, query the security information of a local file, set the security information of a local file, etc. 
   There are three primary technical reasons why such correlation has not been achieved previously. The first reason is that all of the in-kernel interceptors of file I/O operations have been developed as either a file system filter driver or as an API interceptor for the kernel system services dispatch table functions, such as ZwCreateFile, ZwReadFile, ZwWriteFile, etc. Both techniques exist at the destination side of the file I/O request, and due to the in-kernel threading model (which includes native kernel threads, system worker threads and user mode threads), there is no readily discernable way to identify the remote requestor of the file system operation. 
   The second reason concerns the way in which srv.sys itself works. In order to save CPU cycles, srv.sys utilizes a concept of reusable interrupt request packets (IRPs). As such, srv.sys does not call the ZwXXXFile family of functions to request file I/O operations. Rather, srv.sys allocates IRPs by calling IoAllocateIrp and IoInitializelrp, and then reuses those IRPs without using the standard I/O manager services to create separate IRP structures for every file I/O request. Therefore, a kernel dispatch routines interceptor would miss those srv.sys file I/O requests. 
   The third reason is that most of the operations performed by high level device drivers inside the Windows NT kernel are not executed during the time the request is initiated within the original thread context. Instead, they are performed later by system worker threads. Thus, when srv.sys receives an SMB network request, srv.sys will queue a work item. The queued work item&#39;s context field will provide enough information to the callback function to implement the requested file I/O operations. The worker items are de-queued and executed within the context of an arbitrary system worker thread. Therefore, it is not readily feasible to know the network information associated with that specific SMB request since the thread context is already lost. This case is very common, since srv.sys is by definition a stateless network file sharing server, and as such is not required to maintain and manage any kind of state based information. 
   What is needed are methods, systems and computer readable media for correlating srv.sys network traffic with its corresponding file system activity. 
   SUMMARY OF INVENTION 
   The present invention comprises methods, systems and computer readable media for correlating network traffic with corresponding file input/output activity. In some embodiments, a correlation manager filters both remote network traffic received by a kernel level fileserver and file input/output operations executed by the kernel level fileserver. The correlation manager correlates a thread requesting performance of a file input/output operation with a worker thread that performs the requested file input/output operation, and is thus able to correlate the remote request to perform the file input/output operation with the resulting performed file input/output operation itself. In some embodiments, the correlation manager correlates a transport driver interface thread requesting a file input/output operation with its corresponding system worker thread that implements the requested file input/output operation. In some embodiments, the correlation manager executes such a correlation by intercepting system calls made by the kernel level fileserver to place work items in a system queue, each work item comprising a callback function provided by the kernel level fileserver and a context field. The correlation manager replaces queue item callback functions provided by the kernel level fileserver with proxy callback functions, and stores information concerning the transport driver interface thread that issued the request, the callback function provided by the kernel level fileserver, the context field and the proxy callback function. Responsive to the proxy callback function being called by a system worker thread, the proxy callback function locates the stored information concerning the transport driver interface thread that issued the request being addressed by the queue work item, and correlates that transport driver interface thread with the system worker thread that called the proxy callback function. 
   The features and advantages described in this disclosure and in the following detailed description are not all-inclusive, and particularly, many additional features and advantages will be apparent to one of ordinary skill in the relevant art in view of the drawings, specification, and claims hereof. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter, resort to the claims being necessary to determine such inventive subject matter. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
       FIG. 1  is a block diagram illustrating a high level overview of a system for practicing some embodiments of the present invention 
       FIG. 2  is a block diagram illustrating thread correlation, according to some embodiments of the present invention. 
   

   The Figures depict embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein. 
   DETAILED DESCRIPTION 
     FIG. 1  illustrates a high level overview of a system  100  for performing some embodiments of the present invention. A correlation manager  101  correlates network traffic  102  with its corresponding file input/output traffic  107 . It is to be understood that although the correlation manager  101  is illustrated as a single entity, as the term is used herein a correlation manager  101  refers to a collection of functionalities which can be implemented as software, hardware, firmware or any combination of the these. Where a correlation manager  101  is implemented as software, it can be implemented as a standalone program, but can also be implemented in other ways, for example as part of a larger program, as a plurality of separate programs, as one or more device drivers or as one or more statically or dynamically linked libraries. 
   This specification describes performing some embodiments of the present invention within the context of a current Microsoft Windows NT® configuration (including versions of Windows based on NT, such as XP, 2000 and 2003). Thus, the names of system components and system functions used herein are those of the current version of Windows NT®. Of course, the present invention is not tied to these names specifically, and in various embodiments of the present invention the described components can have other names as desired. For example, described system functions might have different names in future or different versions of Windows, all of which would be within the scope of the present invention. 
   In the embodiment illustrated in  FIG. 1 , the correlation manager  101  comprises three different hooking layers: a file I/O interception layer  103 , a TDI traffic filter layer  109  and a work item queue interceptor layer  111 . As illustrated in  FIG. 1 , the hooking layers are installed on top of srv.sys  105 , such that only the traffic related to the operations of srv.sys  105  is affected. 
   The file I/O interception layer  103  filters srv.sys  105  requested file I/O operations  107 . As described in detail below, the correlation manager  101  correlates these operations  107  with their associated remote network requests  102 . The file I/O interception layer  103  is implemented according to the techniques disclosed in the Lightweight Hooking Mechanism application. The correlation manager  101  examines the file I/O services  108  imported by srv.sys  105  to determine which file I/O system calls  108  to hook. 
   The TDI traffic filter layer  109  hooks the TDI callback functions  110  installed by srv.sys  105 , when srv.sys  105  registers those TDI network callback functions  110 . In some embodiments the TDI traffic filter layer  109  also hooks calls  110  to IoCallDriver (TDI Driver, IRP_MJ_INTERNAL_CONTROL) to filter the custom TDI traffic  102 . This allows the TDI traffic filter layer  109  to hook all callbacks  110  and TDI requests  102  coming from srv.sys  105 . Thus, the TDI traffic filter layer  109  can filter remote network traffic  102  received by srv.sys  105 . The implementation mechanics of hooking the TDI callback functions  110  installed by srv.sys  105  and calls  110  to IoCallDriver will be apparent to one of ordinary skill in the relevant art in light of this specification and the Lightweight Hooking Mechanism application. 
     FIG. 2  illustrates the system work item queue interceptor layer  111  according to one embodiment of the present invention. The work item queue interceptor layer  111  correlates the TDI threads  201  with their corresponding system worker threads  203  that implement the file I/O operations  107 . Microsoft Windows NT® has a feature that enables device drivers to allocate a queue item  205  that is essentially a callback function  206  coupled with a context field  208 , and then add this item  205  to a system work queue  207 . The items  205  in the system work queue  207  are processed in first in-first out order, each item  205  being processed by a system worker thread  203  that calls its corresponding callback function  206  with the supplied context field  208 . When srv.sys  105  receives a TDI request  102  to perform file input/output activity  107  (e.g., a request  102  to read a local file), srv.sys  105  uses these system services to place a queue item  205  corresponding to the TDI request  102  in the queue  207 , to be processed by a system worker thread  203 . Srv.sys  105  provides a callback function  206  for the worker thread  203  to call, in order to perform the actual file input/output activity. The Windows I/O manager provides the following functions to enable device drivers to manage the work items: IoAllocateWorkItem, IoQueueWorkItem and IoFreeWorkItem. 
   In some embodiments of the present invention, the work item queue interceptor layer  111  intercepts calls made by srv.sys  105  to these functions in the I/O manager module, which is located in the Windows system kernel. Such interception can be managed according to the techniques disclosed in the Lightweight Hooking Mechanism application. 
   The work queue interceptor layer  111  can create a new proxy callback function  211  for every callback function  206  supplied by srv.sys  105  to queue a work item  205 . By hooking IoQueueWorkItem, the work queue interceptor layer  111  can substitute its own proxy function  211  for the callback function  206  provided by srv.sys  105 , whenever srv.sys  105  adds an item  205  to the queue  207 . The work queue interceptor layer  111  maintains a cache  213  (or other storage mechanism) in which it stores the current TDI thread  201  context (i.e., information concerning the thread making the request  102  resulting in the queuing of the work item  205 ), the corresponding srv.sys  105  callback function  206 , the context field  208 , and the created proxy callback function  211 . When a system worker thread  203  processes the queued item  205 , it calls the proxy function  211 . When the proxy function  211  is called by a system worker thread  203 , the work queue interceptor layer  111  (via its created proxy function  211 ) looks in the cache  213  and locates the original supplied callback function  206 , as well as the associated thread context. With this information, the correlation manager  101  can correlate the requesting TDI thread  201  with the system worker thread  203  servicing the queued item  205 , and hence the local file system operation  107  with the associated network request  102 . The correlation manager  101  can then call the original callback function  206  with its context field  208  to execute the file input/output activity  107 . Typically, the work queue interceptor layer  111  manages the allocation and freeing of the work queue handles to manage the cache  213 . 
   Similarly, the work queue interceptor layer  111  can track the direct creation of system worker threads  203  using the PsCreateSystemThread system function. So, if a worker thread  203  creates a child thread  215 , the work queue interceptor layer  111  can create a new proxy callback function  211  for the original callback function  206  and the context supplied therewith. The work queue interceptor layer  111  can link the entries for both threads  203 ,  215  in the cache  215 , and the correlation logic can subsequently take both threads  203 ,  215  and the relationship between them into account. This will enable the work queue interceptor layer  111  to locate the parent of any currently executing system worker thread  203 , whether the thread was created directly by PsCreateSystemThread or whether it was queued as a work item  205 . 
   As noted above, in some embodiments of the present invention, the work queue interceptor layer  111  replaces callback functions  206  with its own proxy functions  211 . The system functions PsCreateSystemThread and IoQueueWorkItem take a context field  208  as a parameter. In some embodiments, the work queue interceptor layer  111  replaces the context field  208  with one that points directly to the corresponding cache entry  205 , and then uses the supplied context field  208  when calling the original function  206 . This optimization saves the time required to perform the cache lookup operation. 
   Note that the present invention is neutral to the operations of the file system driver and the TDI network protocol driver, as it hooks interfaces provided by the I/O manager and filters TDI traffic. These interfaces are both backward and upward compatible. Therefore, embodiments of the present invention can support any newly developed file system for Windows NT®. Embodiments can also support any newly developed network protocol drivers. 
   Various embodiments of the present invention have numerous benefits. For example, the present invention can act as an intelligent filter that will identify the source computer of every SMB operation. This includes defining the IP address, computer host name, user SID, etc. This kind of functionality can be very beneficial in a variety of contexts, for example for an Anti-virus product to identify computers that are infecting other computers. It would also be desirable for use in a behavior blocking component that catches worms that are quickly propagating through open network shares. Additionally, the present invention can enable development of an intelligent access control system that could control remote access to a local file system, by defining rules based on, e.g., user identity or the source computer issuing the remote file request. This feature would be beneficial during a virus outbreak, when it is essential to isolate infected machines from the network quickly. This feature would also be very important for a behavior blocking system that might decide to block certain user access to the local machine file resources based on any behavior blocking based rules. 
   Some examples of specific usage scenarios of some embodiments of the present invention are detailed in Table 1. 
   
     
       
             
           
         
             
               TABLE 1 
             
             
                 
             
           
           
             
               srv.sys establishes TDI connections to serve a SMB request: 
             
             
               Through the TDI filter layer, get all the remote computer network 
             
             
               information. Store the network information in internal cache, and 
             
             
               associate that information with that particular thread. 
             
             
               srv.sys queues a work item to serve the new SMB request: 
             
             
               Through the work queue interceptor layer, generate a new proxy 
             
             
               callback function. Queue the proxy callback function, the original 
             
             
               callback function and the context field 208 to the right record in 
             
             
               the internal cache. Call the IoQueueWorkItem function with the proxy 
             
             
               function instead of the original function. 
             
             
               A system worker thread proxy function is called: Do a cache lookup in 
             
             
               the internal cache using the proxy callback as a hash index, and locate the 
             
             
               original cache entry which contains the original callback function, the 
             
             
               original thread-id and the network connection information. Store the 
             
             
               system worker queue thread-id in the located cache entry. Call the original 
             
             
               callback function. 
             
             
               The original callback function serves file I/O operations: 
             
             
               Through the file I/O filter layer, capture the file I/O traffic and get the 
             
             
               worker thread-id. By doing a cache lookup into the internal cache using 
             
             
               the worker thread-id as a cache lookup key, locate the corresponding cache 
             
             
               entry. Correlate the served file I/O operations with the original network 
             
             
               connection information stored in the cache entry. 
             
             
               srv.sys calls PsCreateSystemThread to serve a file 110 request: 
             
             
               Create a new proxy function to replace the original specified thread 
             
             
               callback function and its associated context. Create/modify the 
             
             
               corresponding cache entry. 
             
             
               A file I/O request is issued from a 
             
             
               thread created by PsCreateSystemThread: 
             
             
               Use the proxy function and the context information to locate the 
             
             
               previous thread cache entry. Repeat until the original main thread is found. 
             
             
               Associate the network connection information associated with the ancestor 
             
             
               threads with the currently performed file I/O operation. Call the original 
             
             
               thread function. 
             
             
               A file I/O request is issued and a behavior blocking system is installed: 
             
             
               Identify the network identity of the requestor of this file I/O operation, 
             
             
               through the different sequence of operations mentioned previously. Apply 
             
             
               rules controlling remote access rights. 
             
             
                 
             
           
        
       
     
   
   As will be understood by those familiar with the art, the invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Likewise, the particular naming and division of the modules, managers, functions, layers, features, attributes, methodologies and other aspects are not mandatory or significant, and the mechanisms that implement the invention or its features may have different names, divisions and/or formats. Furthermore, as will be apparent to one of ordinary skill in the relevant art, the modules, managers, functions, layers, features, attributes, methodologies and other aspects of the invention can be implemented as software, hardware, firmware or any combination of the three. Of course, wherever a component of the present invention is implemented as software, the component can be implemented as a script, as a standalone program, as part of a larger program, as a plurality of separate scripts and/or programs, as a statically or dynamically linked library, as a kernel loadable module, as a device driver, and/or in every and any other way known now or in the future to those of skill in the art of computer programming. Additionally, the present invention is in no way limited to implementation in any specific programming language, or for any specific operating system or environment. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims.