Abstract:
An application independent, in-kernel cache is implemented. In-kernel caches provide applications with commonly used data quickly and efficiently. Methods and instruments for storing content, cache objects, are implemented leveraging operating system supplied mechanisms. Every operating system has one or more specific means of in-kernel storage usable by kernel extensions. A system that acquires storage for the cache objects is implemented. Techniques for allowing multiple varying sources of data are defined. Multiple data sources may provide application specific and/or application independent data using various protocols. The aforementioned cache object storage mechanisms are independent of the source of the data and vise versa. Techniques for fast lookup and management of cache objects are defined. Mechanisms for moving the data from the data source into the cache object storage mechanism are implemented.

Description:
FIELD OF THE INVENTION  
         [0001]    The present invention relates generally to operating systems and, more specifically, to the caching of application data inside an operating system kernel environment.  
         BACKGROUND OF THE INVENTION  
         [0002]    Caching is found in many areas of computer science, wholly consistent with the dictionary definition of “a safe place of hiding or storing things” (see, e.g., in  Webster&#39;s New World Dictionary of the American,  2nd college ed. 1976). Processors can have many levels of caches (see Hennessy, John and David A Patterson,  Computer Architecture A Quantitative Approach  2nd ed. San Francisco: Morgan Kaufmann, 1996, p. 375), while some file systems can also have caches (see Solomon, David and Mark Russinovich,  Inside Microsoft Windows  2000 3rd ed. Redmond: Microsoft Press, 2000, p. 646). Applications, such as HTTP servers (Fielding, R., et al.,  RFC  2068, 1997, p. 70, http://www.ietf.org/rfc/rfc2068.txt), may have them as well. Further examples abound. Some applications even have application specific caches in the operating system kernel (Hu, Elbert, et al., “Adaptive Fast Path Architecture”, IBM Journal of Research and Development, March 2001, p. 191-206). The term “kernel” is commonly understood to refer to that portion of the operating system that operates in privileged mode; it is the essential part of an operating system that performs basic functions and services.  
           [0003]    However, to date there has not been known to exist any universal operating system kernel environment cache that is independent of the source of the data. A need has thus been recognized in connection with overcoming such a shortcoming.  
         SUMMARY OF THE INVENTION  
         [0004]    There is broadly contemplated, in accordance with at least one presently preferred embodiment of the present invention, a universal operating system kernel cache capable of caching both application specific and application independent data and delivering such data via essentially any application protocol. Data may come from a variety of sources and may be stored in the kernel cache using numerous kernel specific mechanisms. Systems for managing the cache are also broadly contemplated herein.  
           [0005]    In summary, the present invention provides, in one aspect, a system for caching application data in the kernel environment of an operating system, the system comprising: an arrangement for creating at least one cache object for storing cached content in the kernel environment; at least one arrangement for looking up at least one cache object; and an arrangement for collecting data from plurality of data sources and populating at least one cache object with the collected data.  
           [0006]    In another aspect, the present invention provides a method of caching application data in the kernel environment of an operating system, the method comprising the steps of: creating at least one cache object for storing cached content in the kernel environment; looking up at least one cache object; and collecting data from plurality of data sources and populating at least one cache object with the collected data.  
           [0007]    Furthermore, in an additional aspect, the present invention provides a program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for caching application data in the kernel environment of an operating system, the method comprising the steps of: creating at least one cache object for storing cached content in the kernel environment; looking up at least one cache object; and collecting data from plurality of data sources and populating at least one cache object with the collected data.  
           [0008]    For a better understanding of the present invention, together with other and further features and advantages thereof, reference is made to the following description, taken in conjunction with the accompanying drawings, and the scope of the invention will be pointed out in the appended claims. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0009]    [0009]FIG. 1 is a block diagram illustrating a runtime environment.  
         [0010]    [0010]FIG. 2 is an instance diagram showing different instantiated objects and their references.  
         [0011]    [0011]FIG. 3 is a block diagram which illustrates interaction between various modules and objects.  
         [0012]    [0012]FIG. 4 is a graphical diagram illustrating memory layout and references for cache objects.  
         [0013]    [0013]FIG. 5 is a graphical diagram illustrating memory layout and references for data sources.  
         [0014]    [0014]FIG. 6 is a block diagram which illustrates communication between objects and modules relating to a request for data.  
         [0015]    [0015]FIG. 7 is a block diagram illustrating various kernel components.  
         [0016]    [0016]FIG. 8 is a block diagram which illustrates communication between a file data source and its associated kernel component.  
         [0017]    [0017]FIG. 9 is a block diagram which illustrates communication between an HTTP data source and its associated kernel component.  
         [0018]    [0018]FIG. 10 is a block diagram which illustrates communication between user space and kernel space. 
     
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0019]    Generally, at least one presently preferred embodiment of the present invention relates to a kernel run-time environment for authoring high performance network servers. One aspect of the present invention relates to an N-source In-kernel Cache (NICache). NICache makes it possible for one or more applications to extend the operating system for high performance. The application discussed here which is enhanced with NICache is Adaptive Fast Path Architecture (AFPA). AFPA is an in-kernel network server accelerator that serves data from many disparate sources. It should be understood that NICache is not a stand-alone application; it must be employed by one or more applications.  
         [0020]    The following are detailed definitions of some useful terms utilized herein.  
         [0021]    A “callback function” may be understood as a reference to discrete code, that is, functions that are specified and called at a later time when desirable. This technique is often employed by programmers who wish for a function to be called as a result of an event or wish not for functions to be called synchronously. For example, a signal handler executes a function written by a programmer when a signal occurs.  
         [0022]    A “hash table” may be understood as a data structure used by programmers to store data. Hash tables provide fast direct addressing of elements in a table, without having to use a storage mechanism large enough to store K elements of a set. Hash tables apply a function, typically called a hash function, to each data element to determine the index into a table. Hash tables are usually fixed in length.  
         [0023]    A “linked list” may be understood as a data structure used by programmers to store data. Linked lists store data linearly, typically with access to the front or rear of the list. Linked lists are not fixed in length and are easily grown. The amount of time required to search a linked list generally grows linearly with the number of items in the list.  
         [0024]    Turning now to a description of preferred embodiments of the present invention, a native kernel environment  110  as illustrated in FIG. 1 may include address space, execution context, security attributes, etc. of an operating system which are typically hidden from user level applications. The native kernel environment will implement services exposed to applications in user space and may provide services for writing kernel extensions.  
         [0025]    The term “native”, used as a modifier to “kernel environment,” refers to a particular kernel environment. For instance the AIX, Linux, and Windows operating systems all have distinct native kernel environments. They are distinct because they each have a specific set of application program interfaces (APIs) for writing subsystems (such as network adapter drivers, video drivers, or kernel extensions). Windows provides hundreds of APIs within its native kernel environment. AIX and Linux each have different APIs. Furthermore, the environments among these operating systems differ in how the file system and TCP/IP subsystems interact.  
         [0026]    N-source In-kernel Cache (NICache)  130  makes use of these APIs as a kernel extension. Typically, NICache is used with or within one or more applications  120 . An application, whatever its purpose, also runs in the native kernel environment  110 . In a preferred embodiment of the present invention, an HTTP server accelerator is the application, with NICache linked with the application. That is, NICache cannot be executed outside of the application as a standalone executable unit. The application itself may have more then one execution unit, including an extension running in user space. The application extension  140  preferably does not run in the native kernel environment but in user space.  
         [0027]    The application extension  140  could be part of an HTTP server that generates dynamic content based on the HTTP request received. It is useful for applications to separate servers into kernel and user components. This approach allows the server writer to pick and choose what pieces of the server need to be in user space and what pieces need to be in the kernel. For example, HTTP servers invariably want to run client Common Gateway Interface (CGI) requests at user space and not in the kernel, because CGIs require user space protection.  
         [0028]    In a preferred embodiment of the present invention, the application handles HTTP requests including GET, HEAD, and POST requests. If a response resides in the cache, then the request is serviced from the kernel. However, if the HTTP request is a request for dynamic content (e.g., a CGI request), then it is deferred to the application extension  140  for processing. The application extension executes any part of the request requiring a user space execution context.  
         [0029]    NICache  130  provides an API to its accompanying application  120 . All calls into this API are funneled through the cache object manager  210  (see FIG. 2). Continuing with FIG. 2, the cache object manager  210  proxies preferably every call into NICache and manages all of the internal components and logic. To perform its duties optimally, the cache object manager may query  260  the state of the hardware of the machine such as physical memory utilization or CPU utilization. It may also inquire  260  as to the state of the native kernel environment, i.e., non-paged pool usage or number of free page descriptors.  
         [0030]    Cached content is the data that NICache is caching on behalf of applications. More specifically, it is the data required by the application to be stored in a system in which the data is easy to retrieve. This data can be in any format, such as ASCII or binary, generated from any source. The limitations on cached content are that of the native kernel environment. Each piece of data is determined by its source. The source could include the output from a user-level program, a response to an HTTP GET request, or the contents of a local or remote file. Cache content is stored in cache objects  230 ,  231 ,  232 . Cache objects are the structures and procedures for storing or caching the required data using specific kernel mechanisms. These mechanisms may include the file system cache, pinned memory, or pageable memory and represent various cache object types or classes. Each cache object may be associated with a descriptor or search key possibly provided by the application.  
         [0031]    To maintain and locate the cache content, the cache object manager  210  utilizes any number of container data structures  220 . These containers preferably hold references to the cache objects  230  in a manner that is specific to each container. For example, containers could include a linked list  221 , a hash table  220 , a binary tree, or an array. The linked list container  221  preferably has a reference  250  to the head cache object of a doubly linked list. The hash table container  220  holds references  270  to all cache objects stored in the hash table. A reference to each cache object must be stored in at least one container. The cache object manager  210  preferably has sole discretion as to which containers hold which cache objects. Other containers may be used for sorting cache objects based on the source of the data, the amount of certain system resources used, or other criteria.  
         [0032]    Each cache object is preferably associated with a single data source. The data source, which will be described later in more detail, describes where the cache object&#39;s data originated from. The data sources communicate  215 ,  225 ,  235  with the native kernel environment  110  on behalf of the cache object. To locate its data source  240 ,  241 ,  242 , each cache object  230 ,  231 ,  232  preferably maintains a reference  280  to the data source. Likewise, the data source preferably possesses a reference  290  back to the cache object.  
         [0033]    Preferably, NICache  130  supports any number of cache objects of any type, simultaneously. Furthermore, NICache  130  may support data from many sources simultaneously. Any type of cache object can be populated from any source. For example, a cache object storing its content in pinned memory could receive the content from a local file or perhaps from an HTTP response. To clarify, the cache object stores cached content. The data&#39;s source identifies where that content came from and is configured for retrieving the content. Each cache object is associated with a single data source and vice versa. There are preferably no restrictions on which types of cache objects can be paired with which type of data sources and vice versa.  
         [0034]    Turning to FIG. 3, NICache  130  may typically react to requests from one or more applications  120 . Being a cache, NICache  130  responds to Lookup/Find  310 , Store/Create  350 , and Remove/Delete  380  requests. All requests for cache object operations, such as the aforementioned requests, are made preferably into the cache object manager  210 . The cache object manager  210  preferably manages all cache objects and thus handles all external requests. When an application  120  has data to be cached, it preferably issues a create command to NICache  130 , passing it a reference to a data source (though not the data). Based on fields within the data source, information gathered  260  about the current state of the system and the current state of NICache, the cache object manager  210  preferably decides which storage mechanism to use to store the data (i.e. the type of cache object to use). After the cache object type is determined, a cache object  330  of that type is preferably created  360 , and a reference to that cache object is stored  361 ,  362  in one or more appropriate containers  220 ,  221 . A reference to the newly created cache object is returned  370  to the application  120 .  
         [0035]    Conversely, when the application  120  aims to remove content from the cache, it preferably makes a call to NICache  130  to delete  380  a cache object passing in a reference to the cache object to be removed. The cache object manager  210  handles the request  380  from the application  120  by invoking the remove procedure  390 ,  391  on any containers  220 ,  221  that contain a reference to the cache object and then by issuing a delete command to the cache object  330 . The cache object will preferably clean up any volatile and possibly nonvolatile storage it was using to store the cached content. Upon completion of these tasks, the cache object manager  210  preferably deletes all volatile storage used by the cache object data structure itself.  
         [0036]    When data stored in the cache NICache  130  is needed by the application, a lookup request  310  is preferably made for the cache object based on an identifier. In an exemplary embodiment of the present invention, this identifier includes an HTTP Universal Resource Identifier (URI) (Berners-Lee, T., et al.,  RFC  1945, 1996, http://www.ietf.org/rfc/rfc1945.txt). If the cache object is in at least one of NICache&#39;s containers, then it is returned  312  to the application  120 . Also included in this embodiment is the notion of a “fast search container”—that is a container in which lookups on the identifier are done in constant time and the cache object manager knows how to locate this container. A preferred implementation of NICache uses a hash table  220  for this purpose. Of course, the cache object manager issues a lookup call  320  to this fast container to find the data stored under the identifier. Once located, the container returns  340  the cache object to the cache object manger  210 , which in turn returns  330  the cache object (or not found) to the application. These actions could take place synchronously or asynchronously.  
         [0037]    A preferred implementation of NICache  130  includes a linked list container  221  as one of two containers  221 ,  220 . This linked list is preferably sorted using a weighted Least Recently Used (LRU) algorithm. Each time a cache object is returned as a result of a lookup request  310 , the cache object is preferably moved to the head of the linked list. This insures that the least recently accessed cache objects are towards the tail of the list. This information becomes useful when cache objects must be removed from the cache, such as in a resource constrained environment.  
         [0038]    Many different classes of cache objects lead to many disparate implementations of cache objects; however, all cache objects are generally required to have certain pieces. Turning now to FIGS. 4 and 5, a reference  450  to a cache object manager  210  is preferably maintained in case the cache object needs to inform the cache object manager that the source of the data has been removed thus the cache object itself must be removed. A meta data structure  410  will preferably possess information that describes this cache object. Information required to generate network protocol headers would be found here. Every cache object preferably contains a reference  460  to a single meta data. Likewise, each cache object preferably maintains a reference  470  to a utilization  420 , which describes the system resource utilization of its corresponding cache object  230 . Cache objects preferably contain a reference  440  to a list of included cache objects  430 . When the data for a cache object is requested of by the application, the data from the included cache objects is returned in addition to the original cache object&#39;s data. The included cache objects structure  430  may contain rules, such as ordering, for sending included cache object&#39;s data. The type-specific portion  430  of the cache object will preferably contain any and all data and references used by a cache object to maintain cached content in a type-defined way. Data sources  240  may maintain references  290  to their individual cache objects  230  and vice versa  280 . FIGS. 4 and 5 show this association.  
         [0039]    In addition to the reference  290  to the cache object, every data source preferably holds the size of the data (if any), and the time the data expires (if any). Information specific to the source of the data and how to retrieve content from the source is preferably contained in the type-specific  510  portion of the data source  240 .  
         [0040]    Turning to FIG. 6, when an application  120  intends to extract information from a cache object  230 , it will preferably issue a request  610 , optionally including an offset and length, for the data. In a presently preferred implementation of NICache, the application will only issue such a request  610  when the content is needed in order to send to a client. In addition, the cache object will not pre fill the contents. Therefore, when a request  610  for data is made into the cache object, the data may not be present (i.e., stored within the cache object). If the data is present, then the cache object returns  620  the content immediately. Based on the rules in the cache object&#39;s included cache object&#39;s structure  430 , the cache object may then forward the request  620  to included cache objects. If the content is not present in the cache object, then the call  610  returns with a pending status code, and the cache object makes a call  630 , optionally including an offset and length, to its data source requesting the data. When the data source already has the requested information from the source, then the request  630  is returned immediately, which in turn passes  620  the data back to the application  120 . There are instances, however, where the data source must contact  615  the source of the data via kernel mechanisms to get a recent copy of the data. In this case, the cache object&#39;s call  630  is returned with a pending status code, while the data source waits for the source to return the data. Once the data source receives some or all the content required by the cache object, the cache object is called back  640  with the data, which causes the cache object to return  620  the data to the application  120 . The cache object callback  640  may be called more than once for a single request  630  into the data source. This double layered asynchronous interface prevents the application and cache object from waiting. This permits the application  120  to submit a request for the data  610  at interrupt level or elevated interrupt request levels (IRQLs).  
         [0041]    As mentioned previously, the data source communicates with the native kernel environment  110  to obtain a recent copy of the data as depicted by item  615 . FIG. 7 focuses in on different sub-components that may be found in a native kernel environment  110 . Though the names and potential groupings vary between operating systems, these components all characteristically exist in commercial operating systems such as AIX, Linux, and Windows. Many of these components may potentially be employed by NICache.  
         [0042]    The file system  710  is the part of the operating system that deals with the transfer of data to and from nonvolatile storage. This storage could be physical or logical disk drives located in the same physical machine or located across the network. The data from these files can be used to populate cache objects. NICache data sources can use the file system API for communication  715  to create files, open files, retrieve file handles, read data, and write data. FIG. 8 shows how a file data source  800  in an exemplary embodiment issues an file open request  810  to the file system  710 , and a file handle  820  is returned.  
         [0043]    The memory manager  720  is a set of system services responsible for allocating, deallocating, and managing virtual memory, resolving hardware memory exceptions, and governing the page table (see Solomon and Russinovich, supra, p. 380). A data source may need to allocate  725  memory (pinned memory, non-paged pool, etc.) or access shared memory in order to satisfy a cache object&#39;s Get Data request  630 .  
         [0044]    A kernel&#39;s network stack  730  typically sends and receives data over a network, as explained in detail by W. Richard Stevens, TCP/IP Illustrated, Volume 1 The Protocols Addison-Wesley, 1994. A data source may obtain its data from interaction  735  with the network stack  730 . FIG. 9 shows an exemplary embodiment of NICache in which an HTTP data source  900  utilizes the kernel&#39;s network stack  730  to send HTTP requests  910  (see Berners-Lee, T., et al.,  RFC  1945, 1996, http://www.ietf.org/rfc/rfc1945.txt) to an HTTP server. The response to these HTTP requests, in the form of a data stream  920 , is routed back to the HTTP data source  900 . A variety of data sources could be constructed based on other network protocols requiring the use of the network stack  730 .  
         [0045]    Processes and threads  740  refer to any program running on the same machine as NICache but in a different context. A data source could communicate  745  with another process or thread via inter-process communication mechanism provided by the native kernel environment  110 .  
         [0046]    [0046]FIG. 10 shows a class of data source, a User Level Data Source  1000  in an exemplary embodiment, interacting with another program or thread of execution  1030 . The request for the data  1010  and the response containing the data  1020  are communicated via kernel mechanisms for inter process communication, including sockets, streams, shared memory, signals, or pipes.  
         [0047]    It is to be understood that the present invention, in accordance with at least one presently preferred embodiment, includes an arrangement for creating at least one cache object for storing cached content in a kernel environment, at least one arrangement for looking up at least one cache object, and an arrangement for collecting data from a plurality of data sources and populating at least one cache object with the collected data. Together, these elements may be implemented on at least one general-purpose computer running suitable software programs. These may also be implemented on at least one Integrated Circuit or part of at least one Integrated Circuit. Thus, it is to be understood that the invention may be implemented in hardware, software, or a combination of both.  
         [0048]    If not otherwise stated herein, it is to be assumed that all patents, patent applications, patent publications and other publications (including web-based publications) mentioned and cited herein are hereby fully incorporated by reference herein as if set forth in their entirety herein.  
         [0049]    Although illustrative embodiments of the present invention have been described herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various other changes and modifications may be affected therein by one skilled in the art without departing from the scope or spirit of the invention.