LATENCY VIRTUALIZATION

A technique involves placing a data acceleration engine between an end user device and a host device. The host device provides data associated with a client application to the data acceleration engine, which provides the data to the end user device. If the data acceleration engine is on the host device, content from a datastore is served to the data acceleration engine as if the data acceleration engine were a client running the client application locally; therefore, latency normally associated with a network between the content datastore and the client device is eliminated. If the data acceleration engine is on the end user device and has received at least some data in advance of a relevant query, responses to the query also do not have latency associated with a network. The data acceleration engine can be implemented as a series of data acceleration engines between end user and host devices.

DETAILED DESCRIPTION

FIG. 1depicts a diagram100of an example of a system for latency virtualization. In the example ofFIG. 1, the diagram100includes a computer-readable medium102, a data server104, a datastore106, data consumers108-1to108-N (collectively, the data consumers108), a data accelerator110, and an application intelligence table (AIT)112.

In the example ofFIG. 1, the computer-readable medium102can include a networked system that includes several computer systems coupled together, such as the Internet, or a device for coupling components of a single computer, such as a bus. The term “Internet” as used herein refers to a network of networks that uses certain protocols, such as the TCP/IP protocol, and possibly other protocols such as the hypertext transfer protocol (HTTP) for hypertext markup language (HTML) documents that make up the World Wide Web (the web). Content is often provided by content servers, which are referred to as being “on” the Internet. A web server, which is one type of content server, is typically at least one computer system which operates as a server computer system and is configured to operate with the protocols of the web and is coupled to the Internet. The physical connections of the Internet and the protocols and communication procedures of the Internet and the web are well known to those of skill in the relevant art. For illustrative purposes, it is assumed the computer-readable medium102broadly includes, as understood from relevant context, anything from a minimalist coupling of the components illustrated in the example ofFIG. 1, to every component of the Internet and networks coupled to the Internet.

A computer system, as used in this paper, is intended to be construed broadly. In general, a computer system will include a processor, memory, non-volatile storage, and an interface. A typical computer system will usually include at least a processor, memory, and a device (e.g., a bus) coupling the memory to the processor.

The processor can be, for example, a general-purpose central processing unit (CPU), such as a microprocessor, or a special-purpose processor, such as a microcontroller.

The memory can include, by way of example but not limitation, random access memory (RAM), such as dynamic RAM (DRAM) and static RAM (SRAM). The memory can be local, remote, or distributed. The term “computer-readable storage medium” is intended to include physical media, such as memory.

The bus can also couple the processor to the non-volatile storage. The non-volatile storage is often a magnetic floppy or hard disk, a magnetic-optical disk, an optical disk, a read-only memory (ROM), such as a CD-ROM, EPROM, or EEPROM, a magnetic or optical card, or another form of storage for large amounts of data. Some of this data is often written, by a direct memory access process, into memory during execution of software on the computer system. The non-volatile storage can be local, remote, or distributed. The non-volatile storage is optional because systems can be created with all applicable data available in memory.

The bus can also couple the processor to the interface. The interface can include one or more of a modem or network interface. It will be appreciated that a modem or network interface can be considered to be part of the computer system. The interface can include an analog modem, isdn modem, cable modem, token ring interface, satellite transmission interface (e.g. “direct PC”), or other interfaces for coupling a computer system to other computer systems. The interface can include one or more input and/or output (I/O) devices. The I/O devices can include, by way of example but not limitation, a keyboard, a mouse or other pointing device, disk drives, printers, a scanner, and other I/O devices, including a display device. The display device can include, by way of example but not limitation, a cathode ray tube (CRT), liquid crystal display (LCD), or some other applicable known or convenient display device.

In one example of operation, the computer system can be controlled by operating system software that includes a file management system, such as a disk operating system. File management systems are typically stored in non-volatile storage and cause the processor to execute the various acts required by the operating system to input and output data and to store data in the memory, including storing files on the non-volatile storage. One example of operating system software with associated file management system software is the family of operating systems known as Windows® from Microsoft Corporation of Redmond, Wash., and their associated file management systems. Another example of operating system software with its associated file management system software is the Linux operating system and its associated file management system. Another example of operating system software with associated file management system software is VM (or VM/CMS), which refers to a family of IBM virtual machine operating systems used on IBM mainframes System/370, System/390, zSeries, System z, and compatible systems, including the Hercules emulator for personal computers.

The algorithms and displays presented herein are not necessarily inherently related to any particular computer or other apparatus. Various general purpose systems may be used with programs to configure the general purpose systems in a specific manner in accordance with the teachings herein, or it may prove convenient to construct specialized apparatus to perform the methods of some embodiments. The required structure for a variety of these systems will appear from the description below. In addition, the techniques are not described with reference to any particular programming language, and various embodiments may thus be implemented using a variety of programming languages.

Referring once again to the example ofFIG. 1, the data server104is coupled to the computer-readable medium102. The data server104can be implemented on a known or convenient computer system. Only one data server104is illustrated inFIG. 1, but it should be understood that specific implementations could have multiple servers. Moreover, partial functionality might be provided by a first device and partial functionality might be provided by a second device, where together the first and second devices provide the full functionality attributed to the data server104.

The datastore106and other datastores described in this paper, can be implemented, for example, as software embodied in a physical computer-readable medium on a general- or specific-purpose machine, in firmware, in hardware, in a combination thereof, or in an applicable known or convenient device or system. Datastores described in this paper are intended, if applicable, to include any organization of data, including tables, comma-separated values (CSV) files, traditional databases (e.g., SQL), or other known or convenient organizational formats.

In an example of a system where the datastore106is implemented as a database, a database management system (DBMS) can be used to manage the datastore106. In such a case, the DBMS may be thought of as part of the datastore106or as part of the data server104, or as a separate functional unit (not shown). A DBMS is typically implemented as an engine that controls organization, storage, management, and retrieval of data in a database. DBMSs frequently provide the ability to query, backup and replicate, enforce rules, provide security, do computation, perform change and access logging, and automate optimization. Examples of DBMSs include Alpha Five, DataEase, Oracle database, IBM DB2, Adaptive Server Enterprise, FileMaker, Firebird, Ingres, Informix, Mark Logic, Microsoft Access, InterSystems Cache, Microsoft SQL Server, Microsoft Visual FoxPro, MonetDB, MySQL, PostgreSQL, Progress, SQLite, Teradata, CSQL, OpenLink Virtuoso, Daffodil DB, and OpenOffice.org Base, to name several.

Database servers can store databases, as well as the DBMS and related engines. Any of the datastores described in this paper could presumably be implemented as database servers. It should be noted that there are two logical views of data in a database, the logical (external) view and the physical (internal) view. In this paper, the logical view is generally assumed to be data found in a report, while the physical view is the data stored in a physical storage medium and available to a specifically programmed processor. With most DBMS implementations, there is one physical view and an almost unlimited number of logical views for the same data.

A DBMS typically includes a modeling language, data structure, database query language, and transaction mechanism. The modeling language is used to define the schema of each database in the DBMS, according to the database model, which may include a hierarchical model, network model, relational model, object model, or some other applicable known or convenient organization. An optimal structure may vary depending upon application requirements (e.g., speed, reliability, maintainability, scalability, and cost). One of the more common models in use today is the ad hoc model embedded in SQL. Data structures can include fields, records, files, objects, and any other applicable known or convenient structures for storing data. A database query language can enable users to query databases, and can include report writers and security mechanisms to prevent unauthorized access. A database transaction mechanism ideally ensures data integrity, even during concurrent user accesses, with fault tolerance. DBMSs can also include a metadata repository; metadata is data that describes other data.

In the example ofFIG. 1, the data consumers108are coupled to the computer-readable medium102. The data consumers108can be implemented as clients of the data server104. Regardless of how the relationship with the data server104is characterized, the data consumers108receive data from the datastore106, which can include executable software, served by the data server104.

Multiple data consumers108can introduce issues when they are capable of multi-user access to datastores because latency virtualization can result in serving improper data to a second data consumer108-2after a first data consumer108-1has modified the data. Advantageously, the data accelerator110knows what queries have been made and by whom. So if the first data consumer108-1modifies a first portion of the datastore106, the data accelerator110can send a downstream notification to the second data consumer108-2if the second data consumer108-2is known (to the data accelerator110) to have data associated with the first portion of the datastore106in cache. In a specific implementation, a sequence number of notification is maintained in the AIT112and the data accelerator110does not serve data if the sequence number is not proper. Optionally, if not notification is received for a period of time, a request for the notification can be sent. In a specific implementation, high priority notifications can be sent with a response to prevent the notification from being lost (e.g., put something in a pending state).

In the example ofFIG. 1, the data accelerator110is coupled to the computer-readable medium102. The data accelerator110is, at least logically, located between the datastore106and the data consumers108. In a specific implementation, the data accelerator110has a client component and a server component. The client component can be located at one or more of the data consumers108, which includes a client application that treats the data accelerator110as if it were the data server104. By “located at one or more of the data consumers108,” what is meant is the data accelerator110can be on a same relatively local network as one or more of the data consumers108, or on same devices as the one or more data consumers108. A network is “relatively local” if the network is smaller than a network coupling the data server104to a relevant one of the data consumers108. For example, if the data server104is coupled to a relevant one of the data consumers108through a wide area network (WAN), then a relatively local network could include a personal area network (PAN), local area network (LAN), a campus area network (CAN), a municipal area network (MAN), or some other network smaller than a WAN. The server component can be located at the data server104, which treats the data accelerator110as if it were the client application. By “located at the data server104,” what is meant is the data accelerator110can be on a same relatively local network as the data server104, or on same device as the data server104.

In a specific implementation, the data accelerator110is located at a subset of the data consumers108. In such an implementation, a client application at data consumers of the subset treats the data accelerator110as if it were the datastore from which data is being consumed. Thus, the data accelerator110can act as a proxy for the datastore106(though not necessarily in the technical sense). Generally, the data accelerator110will try to serve from cache if possible, but can wait for one of the data consumers108to, for example, request a first time, prefetch initially, prefetch based on a recent query, or recache expired data if commonly known.

In another specific implementation, the data accelerator110is located at the data server104. In such an implementation, the data server104treats the data accelerator110as if it were the client application receiving data from the datastore106. Thus, the data accelerator110can act as a proxy for a data consumer (though not necessarily in the technical sense).

The data accelerator110and various devices described in this paper can be implemented with engines. Engines, as described below and in this paper generally, refer to computer-readable media coupled to a processor. The computer-readable media have data, including executable files, that the processor can use to transform the data and create new data. An engine can include a dedicated or shared processor and, typically, firmware or software modules that are executed by the processor. Depending upon implementation-specific or other considerations, an engine can be centralized or its functionality distributed. An engine can include special purpose hardware, firmware, or software embodied in a computer-readable medium for execution by the processor. As used in this paper, a computer-readable medium is intended to include all mediums that are statutory (e.g., in the United States, under 35 U.S.C. 101), and to specifically exclude all mediums that are non-statutory in nature to the extent that the exclusion is necessary for a claim that includes the computer-readable medium to be valid. Known statutory computer-readable mediums include hardware (e.g., registers, random access memory (RAM), non-volatile (NV) storage, to name a few), but may or may not be limited to hardware.

In the example ofFIG. 1, the AIT112is coupled to the data accelerator110. The AIT112is a datastore that stores effects of queries that have been worked out by the data accelerator110. For example, if a query will change data in the datastore106, the AIT112can include an indication that some cached data needs to be expired. The AIT112can facilitate latency virtualization when, for example, executing applications that assume no latency. The AIT112can also include operational parameters, such as instructions to never compress/use certain compression, never cache, recache if expired, force prefetch, or cache based on last cache (e.g., prior state on shutdown), to name several.

FIG. 2depicts a diagram200of an example of a system with a client-side data accelerator. In the example ofFIG. 2, the diagram200includes a network202, an end user (EU) device204, and a client-server application host device206. The network202can be implemented as described with reference toFIG. 1.

The EU device204includes a client application engine208, an EU-side data acceleration engine interface210, an EU-side data acceleration engine212, a data acceleration datastore214, and a network interface216. The client application engine208, in operation, executes at least a portion of an application (“the client application”) on the EU device204.

For illustrative purposes, the client application needs data during its execution, which the client application engine208is configured to request from the client-server application host device206. Rather than sending the request immediately over the network interface216, the data request is first sent via the EU-side data acceleration engine interface210to the EU-side data acceleration engine212. Conceptually, the client application engine208acts as if the EU-side data acceleration engine interface210is the network interface216. Thus, the client application engine208can run a client application without a traditional client-server relationship with an application server.

The data acceleration datastore214may or may not include the requested data. The EU-side data acceleration engine212can respond to the data request using data in the data acceleration datastore214if the data acceleration datastore214includes the requested data. The EU-side data acceleration engine212can forward the request over the network interface216to the client-server application host device206, or to an intermediary device (not shown) capable of responding to such requests, if the data acceleration datastore214does not include the requested data.

The client-server application host device206includes a network interface218, a master datastore interface engine220, and a master datastore222. The client-server application host device206receives data requests from the EU device204over the network interface218. In a specific implementation, receipt of a data request is an indication that the data acceleration datastore214did not include the requested data. For illustrative purposes, the master datastore222is expected to include any data the client application engine208requests, though it should be understood that the master datastore222could be distributed across multiple machines (e.g., on an intermediary device that includes non-redundant data or as part of a distributed system, to name two examples) or the client application engine208might be capable of requesting data that is not available. In response to a data request, the master datastore interface engine220provides the requested data from the master datastore222over the network interface218.

The EU device204receives the requested data on the network interface216, which can be implemented as an applicable convenient device for coupling the EU device204to the network202. Depending upon configuration- and implementation-specific factors, the EU-side data acceleration engine212may or may not store the requested data in the data acceleration datastore214to satisfy future requests for the same data locally, though it is believed to be advantageous to store at least some such data locally to satisfy future requests. In any case, the EU-side data acceleration engine212provides the requested data to the client application engine208to satisfy the client application engine208's request.

Data from the master datastore222can be downloaded to the data acceleration datastore214prior to a request for the data being generated. If lucky or if the download is suitably predictive of future data requests, requested data can be in the data acceleration datastore214when the data is requested. This can eliminate the latency associated with requesting data over the network202. Depending upon the resources of the EU device204, the data acceleration datastore214may be incapable of storing all data of the master datastore222, and an applicable convenient caching algorithm can be used to free up storage. To the extent future requests can be predicted, it would be most desirable to leave data that will be the subject of future requests in the data acceleration datastore214when freeing up resources, though predictive caching is not required.

In a specific implementation, the client application engine208is coupled to the EU-side data acceleration engine212via a TCP connection. Because the client application engine208can act as if the data acceleration datastore214is actually a server-side datastore (with latency associated with the network202eliminated when the data is locally available), it can be useful to employ a known client-to-server connection technique for connecting the client application engine208to the EU-side data acceleration engine212. In such an implementation, the EU-side data acceleration interface210can be referred to as a TCP-compatible interface. As used in this paper, “compatible interface” is intended to mean an interface capable of operating within at least one parameter defined by a relevant protocol. More generally, the EU-side data acceleration interface210can be referred to as a client-to-server protocol-compatible interface.

Advantageously, despite the apparent client-to-server relationship between the client application engine208and the EU-side data acceleration engine212, the system illustrated by way of example inFIG. 2enables client-server applications to be hosted (e.g., in the cloud) without server-based computing. It should be understood that the term “server” can generally be applied to any device (even a cloud-based device) that serves content to another device. In this broadest sense, the client-server application host device206can be referred to as a server. However, it is theoretically possible in some implementations for the client-server application host device206to receive no requests from the EU device204where data is provided to the EU-side data acceleration engine212prior to any such request, thereby eliminating at least an aspect of server-based computing.

FIG. 3depicts a diagram300of an example of a system with a server-side data accelerator. In the example ofFIG. 3, the diagram300includes a network302, a client device304, and a server device306. The network302can include any applicable devices capable of coupling the client device304to the server device306.

In the example ofFIG. 3, the client device304is depicted as the client-side of a client-server relationship between the client device304and the server device306. The client device304can include an applicable device capable of running an application served at least in part by the server device306. The client device304may or may not include a data acceleration engine.

The server device306includes a network interface308, a server-side data acceleration engine310, a server-side data acceleration engine interface312, an application server engine314, a content datastore interface316, a content datastore318, and an acceleration datastore320. The network interface308can be implemented as an applicable convenient device for coupling the server device306to the network302.

In a specific implementation, the application server engine310is coupled to the server-side data acceleration engine310through the server-side data acceleration engine interface312. In a specific implementation, the server-side data acceleration engine interface312includes a TCP connection. Because the application server engine314can act as if the server-side data acceleration engine310is actually a client application (with the latency associated with an intervening network eliminated), it can be useful to employ a known server-to-client connection technique for connecting the application server engine314to the server-side data acceleration engine310. In such an implementation, the server-side data acceleration engine interface312can be referred to as a TCP-compatible interface. More generally, the server-side data acceleration interface312can be referred to as a server-to-client protocol-compatible interface.

In the example ofFIG. 3, the application server engine314is capable of serving application data from the content datastore318to the server-side data acceleration engine310of the same type that an application server would normally serve to a client. Depending upon implementation-specific factors, the content datastore318can be accessed through the content datastore interface316, though the content datastore interface316can be thought of as a logical interface that may or may not have actual database interface features, drivers, or the like.

In the example ofFIG. 3, the server-side data acceleration engine310stores data from the content datastore318, and/or a derivative thereof, in the data acceleration datastore320. The server-side data acceleration engine310can also analyze queries before passing a query onto the application server engine314to work out the target and action of a query. Data associated with the analysis of queries can also be stored in the data acceleration datastore320. Depending upon the query, it may be necessary to resolve the query by sending the query to a master datastore that responds appropriately to the query. This may be required in the case where the query has a locally unidentifiable target or action, for example.

Queries generally include a first portion that identifies a target of the query and a second portion that identifies an action associated with the query. The second portion can include a target category and specific (variable) identifier. An example of a query might include an action, select, a target, table, and an identifier, table_id (which could be an array of tables). After working out a query, it becomes possible to generate a unique key or query template for a set of queries that affects multiple targets. A query template generally strips out variables, such as specific table_id's. A query can be its own key. When a query has been worked out, it becomes possible to determine what affect a query will have on other queries. This knowledge can be used to determine what locally stored data (e.g., cache data) might be out of date.

An out of date (or expired) designation is one possible cache state for data (“expired state”). Expired state can be explicitly set using knowledge to determine whether a query has expired data that is locally stored. Expired state can also be set based upon a time-out. Another possible state is servable state, which is state of locally stored data that is presumed valid. Another possible state is pending state; when the relevant data is determined to be servable, serve and set state to servable and otherwise set state to expired. State can be on a table or page basis, file or block basis, or some other applicable basis.

Working out a query enables one to determine what, e.g., table will be impacted. With such knowledge, all queries for the table could be expired to avoid serving stale local data. With such knowledge, it is possible to determine whether a page is up-to-date, which can make it desirable to set to pending state until what, e.g., table is effected is known. In a specific implementation slow, secure queries, tables, etc. have no chance of serving stale local data, but in an alternative implementation some of the reliability is traded off in favor of speed. In a specific implementation, a user is notified when data might be stale.

FIG. 4depicts a diagram400of an example of a system with a series of data accelerators. In the example ofFIG. 4, the diagram400includes an application engine402, an application data acceleration engine interface404, a series of data acceleration engines406, a host data acceleration engine interface408, and an application datastore410.

In the example ofFIG. 4, the application engine402runs a hosted client application. The hosted client application may or may not be hosted by a server in a client-server relationship with a device on which the application engine402is associated.

In the example ofFIG. 4, the application data acceleration engine interface404can couple the application engine402to a local data acceleration engine in the manner illustrated by way of example inFIG. 2(with or without a data acceleration engine that is local relative to the application datastore410). Alternatively, the application data acceleration engine interface404can couple the application engine402to a data accelerator that is some degree of remote from the application engine402(e.g., on a same LAN, on a same CAN, on a same WAN, or the like), in which case the application data acceleration engine interface404can be implemented substantially as a network or other applicable convenient interface.

In the example ofFIG. 4, the series of data acceleration engines406includes a first data acceleration engine406-1that is coupled to the application engine402. As was previously mentioned, the first data acceleration engine406-1can be local relative to the application engine402or some degree of remote from the application engine402. The first data acceleration engine406-1can also be local or some degree of remote from other ones of the data acceleration engines406.

In a specific implementation, the data acceleration engines406communicate with one another using UDP. This works because the data acceleration engines406know order and can control retransmits with low overhead. The cloud cannot “drop” UDP. Also, additional reliability checks can be added. In a specific implementation, a unique UDP packet with a sequence number (of packets) data field and TCP connection ID field is used to facilitate simplified (relative to TCP) requests/responses. The unique UDP packet can be referred to as a “sequenced TCP connection identifying UDP packet.” In a specific implementation, the application data acceleration interface404flips a TCP message to a UDP message that includes the payload of the TCP message. The data acceleration engines406send the sequenced TCP connection identifying UDP packet from one to the next, and eventually to the host data acceleration interface408where the sequenced TCP connection identifying UDP message is flipped back to TCP. The TCP/UDP/TCP conversion can also take place from the host data acceleration interface408to the application data acceleration interface404. Generalizing, the data acceleration engines406use an asynchronous protocol (e.g., UDP) over a network for an order-sensitive activity.

In the example ofFIG. 4, the host data acceleration engine interface408is coupled to a last data acceleration engine406-N. The host data acceleration engine interface408can be local or some degree of remote from the last data acceleration engine406-N. The last data acceleration engine406-N can also be local or some degree of remote from other ones of the data acceleration engines406.

In the example ofFIG. 4, the application datastore410can be coupled to a local data acceleration engine by the host data acceleration engine interface408in the manner illustrated by way of example inFIG. 3(with or without data acceleration engine that is local relative to the application engine402). The application datastore410includes data appropriate for provisioning to a client running a served application. In the example ofFIG. 4, the host data acceleration engine interface408can include a datastore interface410, an interface may not be needed, or a distinct datastore interface can be provided (not shown).

FIGS. 5A and 5B(collectively,FIG. 5), depict a flowchart500of an example of a method for latency virtualization. The example ofFIG. 5includes serially-arranged modules, but it should be understood that the modules of the flowchart500and other flowcharts described in this paper could be reordered and/or arranged for parallel execution, if applicable.

In the example ofFIG. 5, the flowchart500starts at module502with installing a data acceleration interface on an end-user device. The installation can be part of a manufacturing process, included as part of another software installation, accomplished by downloading and installing an “app” on a device, or in some other applicable manner.

In the example ofFIG. 5, the flowchart500continues to module504with installing an instance of a latency virtualization data accelerator on a computer. As used in this paper, latency virtualization refers to modifying execution parameters of client applications such that any implied reliance upon a minimum latency threshold is addressed in the context of data acceleration. For example, if a client application is configured to send a request to a server and await the response from the server, the client application may assume that a response will not be received before a next instruction is executed. This is because sending and receiving a request over a network normally requires substantially more time than moving to an immediate next instruction of a program.

The instance of the latency virtualization data accelerator installed on the computer can be referred to as a data acceleration engine because the software instance is implemented in hardware (of the computer) and will result in accelerated data provisioning in at least some circumstances. A data accelerator can include a “client-side” latency virtualization instance that is on an EU device with which a relevant application is associated; a “server-side” latency virtualization instance that is on a host device that includes a datastore with data for provisioning to application clients; or a series of latency virtualization instances implemented on devices extending across a link between the EU device and host device (including a series of two, where one is on the end user device and one is on the host device).

In the example ofFIG. 5, the flowchart500continues to module506with configuring the data acceleration interface to connect to the instance of the latency virtualization data accelerator. Depending upon implementation- and/or configuration-specific factors, the data acceleration interface may or may not be configured to connect to a predetermined instance. If so, the module504may be reordered to occur before the module502. (As was previously mentioned, if applicable, a module can be reordered and/or arranged for parallel execution.)

In the example ofFIG. 5, the flowchart500continues to decision point508with determining whether a host endpoint is a next connection for the instance of the latency virtualization data accelerator. If it is determined that a host endpoint is not a next connection (508-N), then the flowchart500continues to module510with configuring the instance of the latency virtualization data accelerator to connect to another instance of the latency virtualization data accelerator and returns to decision point508. In this way, a series of data accelerator instances of the data accelerator can be chained together. Each instance of the chain will be installed at some point (see, e.g., module504), and can accordingly be preinstalled relative to reaching decision point508, or installed on the fly for a subset of the iterations of the module510.

If, on the other hand, it is determined that the host endpoint is a next connection (508-Y), then the flowchart500continues to module512with configuring the data accelerator to connect to the host endpoint. If there were no iterations of module510, then module512entails configuring the instance of the latency virtualization data accelerator to the host endpoint. If there were any iterations of module510, then module512entails configuring a last instance of the data accelerator chain to connect to the host endpoint. It may be noted that there is no particular reason that an instance closer to the data server endpoint be configured after an instance that is farther away; so the last configuration in time may be different from the configuration of the last instance of the data accelerator chain.

In the example ofFIG. 5, the flowchart500continues to module514with receiving a request on behalf of an application associated with the end-user device. Depending upon implementation- and/or configuration-specific factors, the application on the end-user device can have been initiated before or after the start of the flowchart500. In a specific implementation, the data accelerator can “take over” while the application is running by interjecting the instance of the latency virtualization data accelerator between the application and a datastore of a data server.

In the example ofFIG. 5, the flowchart500continues to module516with identifying at the data accelerator a target and action associated with the request. By working out the target and action, the data accelerator can in some instances respond to the request using a relatively local datastore. In some cases, it may be necessary for the data accelerator to forward the request to a data server and waiting for the response before details of the request can be worked out. In a specific implementation, the data accelerator can save a query or information associated with the request for future reference. In a specific implementation, the data accelerator can share a query or information associated with the request with other data accelerators. In a specific implementation, the data accelerator can consolidate multiple queries into a single query.

In the example ofFIG. 5, the flowchart500continues to decision point518where it is determined whether the data accelerator can independently respond to the request. If it is determined that the data accelerator cannot independently respond to the request (518-N), then the flowchart500continues to module520with sending the request to a host server and to module522with receiving a response to the request from the host server. If the request includes a write instruction, the host server may or may not send a response to the request. For some known databases, write confirmations are used, but that is not a requirement. Thus, the module522is optional in at least some cases. If, on the other hand, it is determined that the data accelerator can independently respond to the request (518-Y), then the flowchart500skips the modules520and522.

In the example ofFIG. 5, the flowchart500continues to module524with responding to the request from a relatively nearby location. As used in this paper, a relatively nearby location in this context is a location that is closer than the host server. In a specific implementation, if latency is lower from a first location than a second location, then the first location is relatively nearer than the second location. It may be desirable to send information to the host server even if the data accelerator can independently respond to the request. Advantageously, such reporting can be handled over a channel that is different from the one used by the application on the end-user device such that latency associated with requests from the application is not increased thereby. In an implementation in which a data accelerator is located entirely at the host server, the text of module524can be replaced with “responding to the request by the data accelerator.”

FIG. 6depicts a state diagram600of an example of states of an application with virtualized latency. In the example ofFIG. 6, a learning state602is a starting state of the state diagram600. In learning mode, a latency virtualizing data accelerator analyzes queries from an application to work out a target and action associated with the queries. Data received from a master server in response to the queries can be cached locally. After a learning mode threshold is passed, state transitions to a hybrid state604. The learning mode threshold can be based upon time, number of queries, size of cache, cache utilization, or some other applicable metric. In hybrid mode, if an unknown query is received, state transitions to the learning state602(and state transitions back to the hybrid state604after the query is addressed); if a known query is received, state transitions to an accelerated state606. Addressing a query entails responding to the query (from a local datastore if the query is known) and, if the query is unknown, working out a target and action of the query. In accelerated mode, a latency virtualizing data accelerator can satisfy a known query from a local cache that includes data obtained during the learning state602. After satisfying a known query, state transitions from the accelerated state606back to the hybrid state604. In an alternative to the example ofFIG. 6, the starting state could be the hybrid state604. In this alternative, there would be no initial learning mode threshold.

The detailed description discloses examples and techniques, but it will be appreciated by those skilled in the relevant art that modifications, permutations, and equivalents thereof are within the scope of the teachings. It is therefore intended that the following appended claims include all such modifications, permutations, and equivalents. While certain aspects of the invention are presented below in certain claim forms, the applicant contemplates the various aspects of the invention in any number of claim forms. For example, while only one aspect of the invention is recited as a means-plus-function claim under 35 U.S.C sec. 112, sixth paragraph, other aspects may likewise be embodied as a means-plus-function claim, or in other forms, such as being embodied in a computer-readable medium. (Any claims intended to be treated under 35 U.S.C. §112, ¶6 will begin with the words “means for”, but use of the term “for” in any other context is not intended to invoke treatment under 35 U.S.C. §112, ¶6.) Accordingly, the applicant reserves the right to add additional claims after filing the application to pursue such additional claim forms for other aspects of the invention.