Multi-level cache system in a software application

A system, method, and computer-readable medium are disclosed for performing a multi-level application cache operation, comprising: defining a first application level cache; defining an intermediate second application level cache; communicating with a last memory level, the last memory level including a source for a plurality of data objects; and, accessing a data object via the first application level cache when the data object is present and valid within the first application level cache.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to information handling systems. More specifically, embodiments of the invention relate to executing a software application which includes a multi-level cache system.

Description of the Related Art

It is known to user information handling systems to execute applications which perform external service calls to gather information for the application may require from that service. An issue can arise when many service calls are required, the external service is lacking in performance, or when the external service is not responding. If any of these cases are met, then the end user may run into hindrances in the experience.

SUMMARY OF THE INVENTION

A system, method, and computer-readable medium are disclosed for performing a multi-level application cache operation comprising: defining a first application level cache; defining an intermediate application level cache; communicating with a last memory level, the last memory level including a source for a plurality of data objects; and, accessing a data object via the first application level cache when the data object is present and valid within the first application level cache.

In various embodiments, the multi-level application cache operation provides a way to define multiple providers that work fluidly to enhance reliability and scalability of information. In certain embodiments, the multi-level application cache operation preemptively updates a cache object in the background based upon a cache update value. In certain embodiments, the cache update value includes an update service level agreement (SLA) value. Such a multi-level application cache operation enhances a customer experience by reducing the need for run-time calls to a source before a user is presented with information.

DETAILED DESCRIPTION

Various aspects of the present disclosure include a recognition that challenge facing many development teams in the information technology (IT) industry is the large number of service calls that sometimes perform poorly, or in some cases can cripple the application when the service load is high. Issues relating to the large number of service calls can cause a cascading effect where a client application is negatively affected by the degraded performance of the service or API that is being called.

Various aspects of the disclosure include an appreciation that a system that uses a memory cache will generally perform better than a system that does not assuming that cache objects are in memory. However, with limited storage available to a memory cache, a downside of using a memory cache can potentially mean that more calls to source systems may need to occur.

Various aspects of the disclosure include an appreciation that a system that takes advantage of a persistent cache repository will generally also perform better than a system which needs to always access a source system. Although using a persistent cache repository may be slightly slower than using memory cache, the persistent cache repository provides a large (relative to a memory cache) amount of storage thus reducing the overall load on source systems since the persistent cache repository can hold more information.

FIG. 1is a generalized illustration of an information handling system100that can be used to implement the system and method of the present invention. The information handling system100includes a processor (e.g., central processor unit or “CPU”)102, input/output (I/O) devices104, such as a display, a keyboard, a mouse, and associated controllers, a hard drive or disk storage106, and various other subsystems108. In various embodiments, the information handling system100also includes network port110operable to connect to a network140, which is likewise accessible by a service provider server142. The information handling system100likewise includes system memory112, which is interconnected to the foregoing via one or more buses114. System memory112further comprises operating system (OS)116and in various embodiments may also comprise multi-tier application cache system118.

The multi-tier application cache system118performs a multi-tier application cache operation. The multi-tier application cache operation improves processor efficiency (and thus the efficiency of the information handling system100) by improving the efficiency of accessing information from within an application.

As will be appreciated, once the information handling system100is configured to perform the multi-tier application cache operation, the information handling system100becomes a specialized computing device specifically configured to perform the multi-tier application cache operation and is not a general purpose computing device. Moreover, the implementation of the multi-tier application cache operation on the information handling system100improves the functionality of the information handling system and provides a useful and concrete result of reducing the time for obtaining information from a remote memory location.

In various embodiments, the multi-level software cache operation provides an extensible solution that integrates multiple cache providers with different capabilities to gain the advantages of the capabilities of each type of cache. Providing a multi-level cache system that integrates these or other types of cache providers, enables applications to take advantage of the best of each type of cache provider.

An additional advantage of such a multi-level cache system is present when updated information is necessary after the information has been semi-permanently stored in a database system or source of information. In these cases, the multi-level cache system allows development teams to make less frequent calls to external services to retrieve refreshed information while update existing cache objects with the new data retrieved from the service response object. For the purposes of this disclosure, a service may be defined as software functionality (or a set of software functionalities) that can be reused by different clients for different purposes, together with the policies that control usage of the functionality. Examples of software functionality include retrieval of specified information and execution of a set of operations.

FIG. 2shows a simplified block diagram of multi-level software cache environment200. In various embodiments, the multi-level software cache environment200includes a host system210and at least one client system212. The host system210executes the multi-level software cache system118on a hardware processor. In certain embodiments, the multi-level software cache system118includes a multi-level software cache. The host system210may also execute an application222on a hardware processor. It will be appreciated that the host system210and each client system212may be respective information handling systems. Each client system212includes an application240executing on a hardware processor. In various embodiments, the multi-level cache system118may be coupled to a plurality of cache providers, where each of the plurality of cache providers provides one or more levels of a multi-level cache. In various embodiments, the cache providers can be in memory local to the host system210as well as another shared memory, file system, etc.

In certain embodiments, the network140may be a public network, such as the Internet, a physical private network, a wireless network, a virtual private network (VPN), or any combination thereof. Skilled practitioners of the art will recognize that many such embodiments are possible and the foregoing is not intended to limit the spirit, scope or intent of the invention.

In various embodiments, the multi-level application cache system118includes capabilities to retrieve, store, update, and remove cache objects from any number of levels in the multi-level cache system. In various embodiments, the multi-level application cache system118combines or integrates caches from any number of cache providers. For example, by combining providers a memory cache and a persistent cache repository cache, the application gains performance (when retrieving items from the memory provider), puts low load on a source system (by rarely requiring access to it) and has virtually unlimited extendable storage as an option. For the purposes of this disclosure, virtually unlimited indicates that the storage space for each cache level can be extended by increasing an amount of memory in the cache provider itself or by adding new nodes to elastic search. The method by which each cache level's storage can be extended can vary based upon the technology used within the cache provider.

Referring toFIG. 3, a block diagram of a multi-level application cache architecture300is shown. In various embodiments, multi-level application cache architecture300is integrated with existing applications to provide a multi-level application cache system that can store, retrieve, update, and remove cache objects from any level within the multi-level cache system. As requests from client systems212arrive to the multi-level cache system118, the multi-level cache system118performs a hierarchical search for the item in all the cache levels. If the item is located in a level and it's found to be valid, then that item is returned to the client. If the item is not located, then a call is made to the source external service. The results of the call to the source external services is used to populate the cache levels and to send a response to the client system212.

In various embodiments, the multi-level application cache architecture300includes a plurality of storage levels. In various embodiments, at least some of the plurality of storage levels comprise application cache levels provided via a cache provider. For the purposes of this disclosure, an application cache level may be defined as a level of storage that includes an assigned provider, an update value and a valid trigger value. For the purposes of this disclosure, a cache provider may be defined as a system which provides cache functionality. In certain embodiments, the cache functionality includes a GetValue function, a GetValues function, a SetValue function and a RemoveValue function. For the purposes of this disclosure, a cache may be defined as a temporary storage repository, whether local or remote, which is designed to enhance the speed of retrieving information stored within the repository.

In various embodiments, the plurality of storage levels include first cache level310(Level 0), intermediate cache level312(Level N) and a last memory level314(Level Z). In certain embodiments, the last level314is a wrapper around a source system to make it appear like intermediate cache level. In certain embodiments, the first cache level310and the intermediate cache level312reside locally with respect to a system executing an application240. In certain embodiments, the first cache level310and the intermediate cache level312are integrated with the application240. In certain embodiments, the first cache level310is accessed directly by the application240. In certain embodiments, the first cache level310corresponds to application memory storage. In certain embodiments, the application240accesses the intermediate cache level312via an enterprise search engine such as the Elasticsearch search engine. In certain embodiments, the last memory level314is semi-permanent and resides remotely with respect to the system executing the application240. In certain embodiments, the last memory level314is included within the file system214associated with the host system210. In certain embodiments the application240accesses the last memory level314via a service call. In various embodiments, the last memory level314is the ultimate source for each data object used by the application240. In various embodiments, the plurality of storage levels are in addition to and function independently of any cache levels directly controlled by the hardware processor.

In various embodiments, the multi-level application cache system118includes a software process that provides the multi-level cache architecture with ‘n’ levels where a cache object320is stored in object storage within a particular level. Each cache object320includes a reference key (key), update information (LastUpdated) and a data value (Value). A cache object210is referenced via a key. When the key is present, the multi-level cache architecture returns the cache object In certain embodiments, a cache object may be valid, as indicated by a checkmark, or invalid, as indicated by an “x.” Upon finding an invalid item in a level. The system calls the next level to obtain a “fresh” value to update the current level. If the item in the next level is invalid, the process repeats until a valid item is found (which in the worst case scenario will involve calling the external service). When making a call to a source system to retrieve cacheable information, upon retrieval of a null, empty or invalid response, the multi-level cache system removes the cache object from the current cache level.

Use of caching response objects from external services enables application development teams to focus on application performance, reliability and improving the customer experience. Such a multi-level application cache system118provides a user of the application240with a cost saving due to the reduction in time spent making service calls to external services.

The multi-level application cache system118tracks a plurality of first cache values having associated first cache level cache values and a plurality of intermediate cache level cache having associated intermediate cache level cache values. More specifically, the plurality of first cache level cache values includes a first valid value and a first update value. A valid value represents an amount of time that an item in an associated level is considered valid or fresh. After that time, the item is considered invalid or stale. An update value is always less than the valid value and represents the amount of time an item must be in cache before a preemptive update is triggered. A cache item has an updated timestamp. The timestamp is used to determine whether an item is still valid (if the duration value has not already elapsed) or whether the item needs to be preemptively updated (if the update value has already elapsed). For example, if a valid value for the memory cache is one hour, each object that was cached more than one hour ago is considered invalid since it has now expired. The update value defines when the multi-level cache system can use the value from the retrieved cache, but will update the cached information in the background by calling the next cache level. In certain embodiments, the first valid value includes a first valid service level agreement value and the first update value includes a first update service level agreement value. More specifically, the plurality of intermediate cache level values includes an intermediate valid value and an intermediate update value. In certain embodiments, the intermediate valid value includes an intermediate valid service level agreement value and the intermediate update value includes an intermediate update service level agreement value. For the purposes of this disclosure, a valid service level agreement value is a service level agreement value that determines the amount of time that a cache item is considered valid in a particular cache level. For the purposes of this disclosure, an update service level agreement value is a service level agreement value that determines the amount of time that an item must be in a cache before a background update is triggered.

In one example, with information served up quicker through a multi-level cache system integrated with an application such as a unified search application, the application experiences on average over a 91% cache usage rate. Accordingly, with the example, calls to external services were only necessary 8-9% of the time a user interacts with the application. Thus, the multi-level application cache operation serves the data to the application quicker and reduces the load on the external services.

The optimal case where multi-level cache performs best is when cached data is retrieved from the best performing cache level. With the multi-level application cache architecture300, the best performing cache level is the first cache level310. For the purposes of this disclosure, cache freshness is the length of time a cache object has been stored in the cache level relative to the expiry time. The higher the cache level, the slower the cache level performs but the fresher the cached data is. In one example, the application240obtains information from the first level cache310in 0-20 milliseconds, from the intermediate level cache in 10-100 milliseconds and from the last memory level in 300 or more milliseconds.

A block diagram of a multi-level application cache operation when a valid cache object is stored within the first level cache310is shown inFIG. 4. For the purposes of this disclosure, a valid cache object is a cache item whose expiration time has not passed and an invalid cache object is a cache item whose expiration time has already passed. With this example, the object associated with key K1is present and valid at each memory level. When the application requests information (such as Service Level Agreement (SLA) information) by generating the key410associated with the information. For the purposes of this disclosure a service level agreement may be defined as an amount of time that a cache object is valid for use, i.e., SLA information is a date time value that defines the expiration of a cache object within a given cache level. If a cache item is outside of a valid SLA, the multi-tier application cache operation takes action to ensure that the cache is updated or the item is removed to reflect the latest information. This key410is provided to the first level cache310. Because the object associated with the key is present in the first level cache310and valid, the object is provided to the application240from the first level cache, significantly improving the response time of the application with respect to the information contained in the object. Because the object associated with the key is present in the first level cache310and valid, the proceeding levels are then ignored and the data is returned from the first level.

A block diagram of a multi-level application cache operation when an invalid cache object is stored within the first level cache310is shown inFIG. 5. With this example, the object associated with key K1is present but the version stored within the first level cache is invalid, but the version stored within the intermediate level cache is valid. For example, the first level cache object is invalid because the LastUpdated property value is “00:00.” The valid SLA for this cache level is 60 minutes. If the current time is 1:00 or after, then the cache object is considered invalid since it is now expired. For example, the valid time for the intermediate cache level is 240 minutes. If the current time is 01:00, then the object in this cache level will remain valid until 04:00. After that, updated cache information will be retrieved from the source and then recursively updated in the other cache levels. The version of the object that is stored in the last memory level314is always valid as this is the source of the object. When the application requests information by generating the key410associated with the information. This key410bypasses the first level cache310and is provided to the intermediate level cache312. Because the object associated with the key is present in the intermediate level cache312and valid, the object is provided to the application240from the intermediate level cache312, significantly improving the response time of the application with respect to the information contained in the object. Because the object associated with the key is present in the intermediate level cache310and valid, the proceeding levels are then ignored and the data is returned from the intermediate level.

A block diagram of a multi-level application cache operation when an invalid or missing cache object is stored within the first level cache310and the intermediate level cache312is shown inFIG. 6. With this example, the object associated with key K1is not present within the first level cache310. Additionally, the object associated with key K1is not present in the intermediate level cache312. The version of the object that is stored in the last memory level314is valid as this is the source of the object. When the application requests information by generating the key410associated with the information, the multi-level application cache system118determines that the object is not present in the first level cache310and is not present in the intermediate level cache312. Thus the key410bypasses the first level cache310and the intermediate level cache312and is provided to the last memory level314(i.e., the source of the object). Once the value is retrieved from the source, the source value is returned to the application240. The source value is also stored and/or updated in all lower cache levels.

A block diagram of a multi-level application cache operation when an invalid object is stored within the first level cache310is shown inFIG. 7. With this example, the object associated with key K1is present within the first level cache310, but there is a more recent version of this object stored in the remote level314. More specifically, the object associated with key K1is within the first update value, but not the valid update time value. Thus, the value associated with key K1stored within the first level cache310is returned from the cache item, but an update request to the source is generated. This update request causes the source to preemptively update the first level cache310in the background so that the next cache request is ready to pull an updated cache item from the first level cache310. When a cache object is retrieved and deemed to be within the update value, the system triggers an event to retrieve updated and fresh cache information from the next level and update the current level. The current cached values are returned to the user thus preventing the user from waiting for updated cache information while the new values are set in the background. Thus, the background does not block the user of the system for obtaining the object. Additionally, a valid object associated with key K1is present within the level cache312; this valid object is used to update the first cache level. When the application requests information by generating the key410associated with the information, the multi-level application cache system determines that the object is out of date (i.e., there is a more recent version of the object at the source) in the first level cache310. Because the object is out of date, there is no need for the application to attempt to retrieve the information from the second level cache312. The source value is used to update the object stored in the first level cache310and the object stored in the second level cache312.

Referring again toFIG. 3, the multi-level application cache operation implement caching for a service or API, by providing a request class that has all parameters needed to build the service call and a response class. Both classes implement a public interface (IHasKey). The public interface IHasKey provides a logical point at which the application interacts. The multi-level application cache operation also creates data provider(s) that accept request(s) and return a response to be stored, retrieved, updated, or removed in cache. The cache values are referenced with the ‘Key’ string field which is equivalent in both the request and response item objects thus both classes implement IHasKey to inherit the Key property.

More specifically, the detailed implementation and usages are shown as: public interface IHasKey {string Key {get;}}}

The request object is set forth as:

The response object is set forth as:

After the request and response objects have been created, the multi-level application cache operation creates one or more providers to retrieve information from the cache. These providers implement the IProvider interface. In this interface, a provider is where the multi-level application cache operation adds code to retrieve a response object from a service or API call.

This interface contains two methods “Get” which will either take a single cache request and “List” which will take a collection of cache requests.

The multi-level application cache operation also declares cache levels. Each cache level must be initialized with a provider, an update SLA value, and a valid SLA value. Cache levels are declared statically, but the multi-level application cache operation traverses through the levels recursively at run-time to retrieve valid cache objects. var level1=new CacheLevel<SnpPriceRequest, SnpPriceItem>{Provider=provider1, UpdateSla=59.Minutes( ), ValidSla=1.Hours( )}; var level2=new CacheLevel<SnpPriceRequest, SnpPriceItem>{Provider=provider2, UpdateSla=240.Minutes( ), ValidSla=120.Minutes};

The multi-level application cache operation also create a new instance of multilevel cache by passing level and level2 as cache levels. In this case, the multi-level application cache operation sets OverrideCache to false. This allows cache validation checks to be overwritten given a certain condition.

The multi-level application cache operation also implements a cache request. In the class where the cache request is executed, a provider field should be declared. private readonly IProvider<SnpPiceRequest, SnpPriceItem>_provider;

The cache request is initialized to pass into the Get( ) method to retrieve cache information from any of the levels included in the provider. The cache key is built using required fields from this request. The key is used to reference the cache item in any cache level. If the cache key is found in a cache level and is valid, then the cache item is returned from that level. The following is an example of a cache request.

FIG. 8shows a chart800of a multi-level application cache operation having three memory levels. The chart also shows the cache retrieval rate for each level along with the cache type retrieval rate within each level. As an example, the first cache level310(identified in the chart as Memory) is used ˜58% of the time. Within that level, two large cache type usages become apparent, a “SnpPriceItem” and “SystemPriceItem.” The intermediate cache level312(identified in the chart as Elasticsearch) is used ˜32% of the time. Within that level, two large cache type usages become apparent, a “SnpPriceItem” and “SystemPriceItem.” The intermediate cache level312(identified in the chart as Service Call) is used ˜32% of the time. Within that level, two large cache type usages become apparent, a “SnpPriceItem” and “SystemPriceItem.” From this chart it may be determined that the cache usages of the intermediate cache level and last memory level are progressively less than those of the first cache level. Such a chart enables a user to can gauge how often a cache is being used for each item which allows a user to determine how much of an impact the usage has on the overall system.