Patent Description:
Many client applications access network-based services using version <NUM> or <NUM> ("<NUM>. x") of the Hypertext Transfer Protocol ("HTTP") over Transmission Control Protocol ("TCP"). In so doing, a client application interacts with a network-based service via HTTP transactions that are transmitted over TCP connection(s) between the client application and the network-based service. A typical client application initiates a relatively small number of interactive HTTP transactions and a relatively large number of informational HTTP transactions. In an interactive HTTP transaction, the client application downloads data, such as display information for a home page, that needs to be provided to a user who is interacting with the client application. In an informational HTTP transaction, a client application uploads data to one or more server machines that the network-based service uses for informational purposes. For example, the client application could upload event logs, metric logs that indicate the performance of the client application, and/or "heartbeats" that indicate that the client application is properly executing.

One drawback to using HTTP/<NUM>. x over TCP is that the volume of informational HTTP transactions can significantly delay the transmission and processing of the interactive HTTP transactions associated with a given client application. In HTTP/<NUM>. x over TCP, each HTTP transaction requires a dedicated TCP connection. Further, the client platforms on which client applications execute oftentimes limit the number of concurrent TCP connections to between two and six connections. And, as is well-understood, upload speeds usually are slower than download speeds. Consequently, at various points in time, all of the available TCP connections can be allocated to only informational HTTP transactions. Any pending interactive HTTP transactions must wait until one of the informational HTTP transactions, including the relative slow upload portion of the HTTP transaction, completes. The resulting delay in downloading data associated with the pending interactive HTTP transaction can cause a degradation in the overall user experience.

In an effort to address the above problems, some client applications allocate at least one of the available TCP connections to informational HTTP transactions, while allocating the other TCP connections to interactive HTTP transactions. However, with this approach, the number of TCP connections available to interactive HTTP transactions is automatically reduced irrespective of the actual volume of informational HTTP transactions. In cases where the volume of interactive HTTP transactions is relatively low, such reductions can be undesirable. Further, if the capacity of the TCP connection(s) allocated to informational HTTP transactions is exceeded, then the client application may end up dropping the information being transmitted to the network-based service via the informational HTTP transactions.

In another approach to addressing the above problems, some client applications implement HTTP/<NUM>, where multiple HTTP transactions can be multiplexed over each TCP connection. Because a client application can initiate any number of interactive HTTP transactions concurrently with any number of informational HTTP transactions over each TCP connection when implementing HTTP/<NUM>, delays in downloading data that needs to be provided to the user via one or more interactive HTTP transactions can be reduced. However, some client platforms do not support HTTP/<NUM>, which reduces the efficacy of this avenue of solution. <CIT> discloses systems and methods to allow a series of HTTP request/response transactions to be used to emulate a more fully bi-directional communications channel between a client and a server in a communications network. <CIT> discloses acceleration systems that operate as intermediaries between the application programming interface (API) processing system and the clients to reduce API call roundtrip latencies.

As the foregoing illustrates, what is needed in the art are more effective techniques for processing requests associated with network-based services.

<CIT> describes a method for fetching a content from a web server to a client device is disclosed, using tunnel devices serving as intermediate devices. The client device access an acceleration server to receive a list of available tunnel devices. The requested content is partitioned into slices, and the client device sends a request for the slices to the available tunnel devices. The tunnel devices in turn fetch the slices from the data server, and send the slices to the client device, where the content is reconstructed from the received slices. A client device may also serve as a tunnel device, serving as an intermediate device to other client devices. Similarly, a tunnel device may also serve as a client device for fetching content from a data server. The selection of tunnel devices to be used by a client device may be in the acceleration server, in the client device, or in both. The partition into slices may be overlapping or non-overlapping, and the same slice (or the whole content) may be fetched via multiple tunnel devices.

<CIT> describes systems and methods of increasing the performance of computer networks, especially networks connecting users to the Web. Performance is increased by reducing the latency the client experiences between sending a request to the server and receiving a response. A connection cache may be maintained by an agent on the network access equipment to more quickly respond to request for network connections to the server. Additionally, the agent may maintain a cache of information to more quickly respond to requests to get an object if it has been modified. These enhancements and other described herein may be implemented singly or in conjunction to reduce the latency involved in sending the requests to the server by saving round-trip times between computer network components.

In an aspect there is provided a computer implemented method as defined in claim <NUM>. In a second aspect there is provided a computer implemented method as defined in claim <NUM>. In a third aspect there is provided a computer implemented method as defined in claim <NUM>. In a fourth aspect there is provided a computer implemented method as defined in claim <NUM>.

At least one technical advantage of the disclosed techniques relative to the prior art is that, with the disclosed techniques, interactive HTTP transactions are less likely to be delayed by informational HTTP transactions for a client application that implements HTTP/<NUM>. x over TCP. In particular, as soon as the proxy server responds to an informational HTTP transaction transmitted from the client application, the client application can close or reuse the associated TCP connection without having to wait for a response from the back-end server. Accordingly, the client application is less likely to use all available TCP connections for informational HTTP transactions and delay the transmission and processing of interactive HTTP transactions. These technical advantages represent one or more technological advancements over prior art approaches.

So that the manner in which the above recited features of the various embodiments can be understood in detail, a more particular description of the inventive concepts, briefly summarized above, may be had by reference to various embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of the inventive concepts and are therefore not to be considered limiting of scope in any way, and that there are other equally effective embodiments.

In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one of skilled in the art that the inventive concepts may be practiced without one or more of these specific details.

Users typically interact with a media streaming service via client applications executing on Internet-connected client devices. For example, a client application executing on a smart television could allow users to browse, search, select, download, and stream media content provided by the media streaming service. Many client applications access the media streaming service using HTTP/<NUM>. x over TCP. In so doing, a client application interacts with the media streaming service via HTTP transactions. A typical client application initiates a relatively small number of interactive HTTP transactions and a relatively large number of informational HTTP transactions. In an interactive HTTP transaction, the client application downloads data, such as display information for a home page, that needs to be provided to a user who is interacting with the client application. In an informational HTTP transaction, a client application uploads data to one or more server machines that the media streaming service uses for informational purposes. For example, the client application could upload event logs, metric logs that indicate the performance of the client application, and/or "heartbeats" that indicate that a streaming session is active.

One drawback to using HTTP/<NUM>. x over TCP is that the volume of informational HTTP transactions can significantly delay the transmission and processing of the interactive HTTP transactions associated with a given client application. In HTTP/<NUM>. x over TCP, each HTTP transaction requires a dedicated TCP connection. Further, the client platforms on which client applications execute oftentimes limit the number of concurrent TCP connections to between two and six connections. And, as is well-understood, upload speeds usually are slower than download speeds. For example, an Internet Service Provider ("ISP") could provide a tier of service with upload speeds up to <NUM> Megabits per second ("Mbps") vs download speeds up to <NUM> Mbps. Consequently, at various points in time, all of the available TCP connections can be allocated to only informational HTTP transactions. Any pending interactive HTTP transactions must wait until one of the informational HTTP transactions, including the relative slow upload portion of the HTTP transaction, completes. The resulting delay in downloading data associated with the pending interactive HTTP transaction can cause a degradation in the overall user experience. For example, instead of waiting <NUM> milliseconds to view information associated with a selected video, a user could wait <NUM> second.

With the disclosed techniques, however, interactive HTTP transactions are less likely to be delayed by informational HTTP transactions for client applications that implement HTTP/<NUM>. x over TCP. In one embodiment, a proxy application executes on a proxy server that acts as an intermediary between one or more client applications and a back-end sever that provides the media streaming service. The proxy server and the back-end server communicate via HTTP/<NUM> over TCP, where multiple HTTP transactions can be multiplexed over each TCP connection.

In some embodiments, the client application adds a "fire-and-forget" header to each request that initiates an informational HTTP transaction. Upon receiving a request, the proxy application determines whether to activate offloading for the request based on whether the request includes a fire-and-forget header. If the proxy application activates offloading for the request, then the proxy application transmits a successful generic response to the client application prior to transmitting the request to the back-end server. The successful generic response intentionally and erroneously indicates that the back-end server has successfully processed the request. Irrespective of whether offloading is active for the request, the proxy application transmits the request to the back-end server. If offloading is active, then the proxy application discards the response received from the back-end server. Otherwise, the proxy application transmits the response received from the back-end server to the client application.

At least one technical advantage of the disclosed techniques relative to the prior art is that, with the disclosed techniques, interactive HTTP transactions are less likely to be delayed by informational HTTP transactions for a client application that implements HTTP/<NUM>. x over TCP. In particular, as soon as the proxy server responds to an informational HTTP transaction transmitted from the client application, the client application can close or reuse the associated TCP connection without having to wait for a response from the back-end server. Accordingly, the client application is less likely to use all available TCP connections for informational HTTP transactions and delay the transmission and processing of interactive HTTP transactions. As a result, a typical user experience provided by the media streaming service via the client application is improved. These technical advantages represent one or more technological advancements over prior art approaches.

<FIG> is a conceptual illustration of a system <NUM> configured to implement one or more aspects of the present invention. As shown, the system <NUM> includes, without limitation, a network-based service system <NUM>, any number of client devices <NUM>, and any number of acceleration systems <NUM>. For explanatory purposes, multiple instances of like objects are denoted with reference numbers identifying the object and parenthetical numbers identifying the instance where needed.

The network-based service system <NUM>, the client devices <NUM>, and the acceleration systems <NUM> communicate over a communications network (not shown). The communications network includes a plurality of network communications systems, such as routers and switches, configured to facilitate data communication. Persons skilled in the art will recognize that many technically feasible techniques exist for building the communications network, including technologies practiced in deploying the well-known Internet communications network.

The network-based service system <NUM> includes, without limitation, a network of interconnected nodes that are distributed across the globe and receive, transmit, process, and/or store data associated with a network-based service (e.g., a streaming media service). The interconnected nodes may include any suitable combination of software, firmware, and hardware to perform these desired functions. In particular, the network-based service system <NUM> includes multiple computing devices that may be co-located or physically distributed from one another. For example, these computing devices could include one or more general-purpose PCs, Macintoshes, workstations, Linux-based computers, server computers, one or more server pools, or any other suitable devices. The computing devices store and execute one or more programs that are remotely accessible in any technically feasible fashion, such as via corresponding application programming interfaces ("APIs"). In various embodiments, any number of the computing devices may be implemented in one or more cloud computing environments (i.e., encapsulated shared resources, software, data, etc.).

Each client device <NUM> may be any type of device that is capable of executing software applications and communicating with other devices via the communication network. For example, the client device <NUM>(<NUM>) could be a tablet, a set-top box, a smart television, a game console, a streaming media player, a mobile device such as a smart phone, etc The client devices <NUM> may be distributed across any number of physical locations. Each of the client devices <NUM> may include any appropriate input devices (such as a keypad, touch screen, mouse, or other device that can accept information), output devices, mass storage media, or other suitable components for receiving, processing, storing, and communicating data. Both the input device and output device may include fixed or removable storage media such as a magnetic computer disk, CD-ROM. Each client device <NUM> may include, without limitation, any number of processors and any number of memories in any combination. Any number of the client devices <NUM> may provide a multiprocessing environment in any technically feasible fashion.

Each client device <NUM> includes computer hardware and/or computer software that relies on the network-based service system <NUM> for certain operations. In particular, each client device <NUM> may include any number of client platforms that each execute any number of software applications. Examples of client platforms include, without limitation, web browsers, smart television operating systems (OSs), mobile phone OSs, video game console OSs, etc. Software applications that communicate with the network-based service system <NUM> over the communications network to perform various operations are referred to herein as "client applications.

In some embodiments, a client application operates by issuing requests to the network-based service system <NUM>. The client device <NUM> establishes a network connection with the network-based service system <NUM> and then transmits the request to the network-based service system <NUM> via the network connection. In response to receiving the request, the network-based service system <NUM> processes the request and generates a response that is transmitted back to the client application via the network connection. The process of issuing a request and receiving a corresponding response is referred to herein as a "transaction. " The round trip between the client application executing on the client <NUM>, and the portion of the network-based service system <NUM> that processes the request, is referred to herein as the transaction round trip. In general, the farther the client <NUM> is from the portion of the network-based service system <NUM> that processes the request, the higher the latency of the transaction round trip. Further, the higher the congestion of the network connection, the higher the latency of the transaction round trip.

The acceleration systems <NUM> operate as an intermediary between the network-based service system <NUM> and the client devices <NUM> to reduce the transaction round trip latencies. The acceleration systems <NUM> include a network of interconnected systems that are distributed across the globe and that each operates as an intermediary between the client devices <NUM> and the network-based service system <NUM>. A given acceleration system <NUM> establishes a network connection with a given client device <NUM> and receives a request over the connection. The acceleration system <NUM> facilitates the processing of the request over a network connection with the network-based service system <NUM>.

In various embodiments, any number of the acceleration systems <NUM> may be embedded within a network associated with an Internet Service Provider ("ISP"). In some such embodiments, if the acceleration system <NUM>(x) is embedded within a network associated with an ISP, then the acceleration system <NUM>(x) is accessible only by the client devices <NUM> that are associated with and/or subscribe to the ISP. In the same or other embodiments, any number of the acceleration systems <NUM> may operate within or in association with an Internet exchange point and independent of an ISP. An Internet exchange point is a physical infrastructure through which ISPs and content delivery networks ("CDNs") exchange Internet traffic.

When an acceleration system <NUM> operates as an intermediary between the network-based service system <NUM> and the client device <NUM>, the time required to execute transactions is reduced for at least two reasons. First, in some embodiments, the acceleration system <NUM> is generally physically closer to the client device <NUM> relative to the network-based service system <NUM>. Thus, any round trip times needed to establish the network connection between the client device <NUM> and the acceleration system <NUM> are shorter relative to if the network connection needed to be established between the client device <NUM> and the network-based service system <NUM>. Second, in some embodiments, due to the acceleration system <NUM> having a large volume of requests originating from multiple client devices <NUM>, the acceleration system <NUM> has a consistent, pre-established, and pre-authenticated network connection with the network-based service system <NUM>. Thus, a network connection with the network-based service system <NUM> need not be established and authenticated for each request.

It will be appreciated that the system <NUM> shown herein is illustrative and that variations and modifications are possible. For example, the connection topology between the various components of the system <NUM> may be modified as desired. Note that the techniques described herein are illustrative rather than restrictive, and may be altered without departing from the scope of the invention. In various embodiments, any number of the techniques disclosed herein may be implemented while other techniques may be omitted in any technically feasible fashion.

For explanatory purposes only, <FIG> describe the functionality of the system <NUM> in the context of TCP connections and specific versions of the HTTP protocol. However, the techniques described herein are illustrative rather than restrictive, and may be altered without departing from the scope of the invention. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments and techniques.

<FIG> is a more detailed illustration of one of the acceleration systems <NUM> of <FIG> during operation, according to various embodiments of the present invention. More precisely, <FIG> illustrates interactions between any number of client applications <NUM>(<NUM>)-<NUM>(M) executing on any number of client devices <NUM>(<NUM>)-<NUM>(N), a proxy server <NUM> included in the acceleration system <NUM>(<NUM>), and a back-end server <NUM> included in the network-based service system <NUM>.

The back-end server <NUM> includes, without limitation, any number of the computing devices included in the network-based service system <NUM>. Each of the computing devices may execute any number and type of software applications that process and respond to HTTP requests for the network-based service system <NUM>.

The proxy server <NUM> is a computing device included in the acceleration system <NUM>(<NUM>). In alternate embodiments, the proxy server <NUM> may include any number of the computing devices included in the acceleration system <NUM>(<NUM>). As shown, the client devices <NUM> and the proxy server <NUM> communicate using HTTP/<NUM> over client/proxy TCP connections <NUM>. The proxy server <NUM> and the back-end server <NUM> communicate using HTTP/<NUM> over proxy/back-end TCP connections <NUM>. In operation, the proxy server <NUM> acts an intermediary between the client devices <NUM>(<NUM>)-<NUM>(N) and the back-end server <NUM> to reduce the time required to execute HTTP transactions. The proxy server <NUM> is sometimes referred to as a "reverse proxy server.

The proxy server <NUM> reduces the time required to execute HTTP transactions for at least three reasons. First, the proxy server <NUM> is generally physically closer to the client devices <NUM>(<NUM>)-<NUM>(N) relative to the back-end server <NUM>. In particular, in some embodiments, the proxy server <NUM> is embedded within a network associated with an ISP that is also associated with the client devices <NUM>(<NUM>)-<NUM>(N). Thus, any round trip times needed to establish the network connection between the client device <NUM> and the proxy server <NUM> are shorter relative to if the network connection needed to be established between the client device <NUM> and the back-end server <NUM>. Second, in some embodiments, due to the proxy server <NUM> having a large volume of requests originating from multiple client devices <NUM>, the proxy server <NUM> has any number of consistent, pre-established, and pre-authenticated proxy/back-end TCP connections <NUM> with the back-end server <NUM>. Thus, a TCP connection with the back-end server <NUM> need not be established and authenticated for each HTTP request.

Third, the proxy server <NUM> and the back-end server <NUM> communicate using HTTP/<NUM>. As is well known, HTTP/<NUM> implements a wide variety of performance improvements relative to HTTP/<NUM>. x In particular, in HTTP/<NUM> multiple HTTP transactions are multiplexed over each connection (e.g., the proxy/back-end TCP connection <NUM>(<NUM>)). Accordingly, the number of concurrent HTTP interactions is not necessarily limited. In some embodiments, the proxy server <NUM> establishes a concurrency limit that is not likely to be reached during normal operation. For instances, in some embodiments, the proxy server <NUM> connects to the back-end server <NUM> via two proxy/back-end TCP connections <NUM>(<NUM>) and <NUM>(<NUM>) and establishes a concurrency limit of fifty concurrent HTTP transactions per TCP connection. Accordingly, up to <NUM> HTTP transactions between the proxy server <NUM> and the back-end server <NUM> may execute concurrently.

As shown, the proxy server <NUM> includes, without limitation, a processor <NUM> and a memory <NUM>. The processor <NUM> may be any instruction execution system, apparatus, or device capable of executing instructions. For example, the processor <NUM> could comprise a central processing unit (CPU), a graphics processing unit (GPU), a controller, a microcontroller, a state machine, or any combination thereof. The memory <NUM> stores content, such as software applications and data, for use by the processor <NUM> of the compute instance <NUM>.

The memory <NUM> may be one or more of a readily available memory, such as random access memory (RAM), read only memory (ROM), floppy disk, hard disk, or any other form of digital storage, local or remote. In some embodiments, a storage (not shown) may supplement or replace the memory <NUM>. The storage may include any number and type of external memories that are accessible to the processor <NUM>. For example, and without limitation, the storage may include a Secure Digital Card, an external Flash memory, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In alternate embodiments, the proxy server <NUM> may include, without limitation, any number of processors <NUM> and any number of memories <NUM> in any combination. In the same or other embodiments, any number of computing devices included in the proxy server <NUM> may provide a multiprocessing environment in any technically feasible fashion.

The proxy server <NUM> is configured to implement one or more applications that enable the client applications <NUM> to access the network-based service via the back-end server <NUM>. Each application is described as residing in a single memory <NUM> of the proxy server <NUM> and executing on a single processor <NUM> of the proxy server <NUM>. However, as persons skilled in the art will recognize, the functionality of each application may be distributed across any number of other applications that reside in the memories <NUM> of any number of compute devices and execute on the processors <NUM> of any number of compute devices. Further, the functionality of any number of applications may be consolidated into a single application or subsystem.

Each of the client applications <NUM> accesses the network-based service system <NUM> using HTTP <NUM>. x over TCP. In so doing, each of the client applications <NUM> interacts with the proxy server <NUM> over one or more of the client/proxy TCP connections <NUM>. Each client/proxy TCP connection <NUM> connects one of the client applications <NUM> executing on one of the client devices <NUM> to the proxy server <NUM>. As shown for the client application <NUM>(<NUM>), each of the client applications <NUM> is associated with a maximum TCP connections <NUM>. The maximum TCP connections <NUM>(<NUM>) specifies the maximum number of TCP connections that the client application <NUM>(<NUM>) can have to each network based service system <NUM>. Typically, the maximum TCP connections <NUM> is set and enforced by the client platform (e.g., web browser, smart television operation system, etc.) that executes the client application <NUM>. For explanatory purposes only, the maximum TCP connections <NUM> for the client application <NUM>(<NUM>) is three. Accordingly, the client application <NUM>(<NUM>) may establish at most three client/proxy TCP connections <NUM>.

In HTTP/<NUM>. x over TCP, each HTTP transaction requires a dedicated TCP connection. A typical client application <NUM> initiates a relatively small number of interactive HTTP transactions and a relatively large number of informational HTTP transactions. In an interactive HTTP transaction, the client application <NUM> downloads data, such as display information for a home page, that needs to be provided to a user who is interacting with the client application <NUM>. In an informational HTTP transaction, the client application <NUM> uploads data to the back-end server <NUM> that the network-based service system <NUM> uses for informational purposes. For example, the client application <NUM> could upload event logs, metric logs that indicate the performance of the client application <NUM>, and/or "heartbeats" that indicate that the client application <NUM> is properly executing.

One drawback to using HTTP/<NUM>. x over TCP is that the volume of informational HTTP transactions can significantly delay the transmission and processing of the interactive HTTP transactions associated with a given conventional client application. Not only is the conventional client application limited to a maximum number of TCP connections, but also upload speeds usually are slower than download speeds. Consequently, at various points in time, all of the available TCP connections can be allocated to only informational HTTP transactions. Any pending interactive HTTP transactions must wait until one of the informational HTTP transactions, including the relative slow upload portion of the HTTP transaction, completes. The resulting delay in downloading data associated with the pending interactive HTTP transaction can cause a degradation in the overall user experience.

As described previously herein, efforts to address the above problems automatically reduce the number of TCP connections available to interactive HTTP transactions, increase the risk of dropping information being transmitted to the network-based service via the informational HTTP transactions, and/or have limited applicability.

To more effectively address the above problems, the proxy server <NUM>, includes, without limitation, a proxy application <NUM>. As shown, the proxy application <NUM> resides in the memory <NUM> and executes on the processor <NUM> of the proxy server <NUM>. While acting as an intermediary between the client applications <NUM> and the back-end server <NUM>, the proxy application <NUM> selectively offloads, from the back-end server <NUM>, responding to any number of HTTP requests for informational uploads. An HTTP request for an informational upload and the corresponding HTTP response are referred to herein as an informational HTTP transaction.

When generating an HTTP/<NUM>. x request <NUM>(x), if a corresponding HTTP/<NUM>. x response <NUM>(x) is unimportant to the client application <NUM>, then the client application <NUM> can add a "fire-and-forget" header to the HTTP/<NUM>. x request <NUM>(x). As persons skilled in the art will recognize, the HTTP/<NUM>. x request <NUM>(x) for which the corresponding HTTP/<NUM>. x response <NUM>(x) is unimportant to the client application <NUM> typically specifies an HTTP method used to upload data, such as "POST" or "PUT.

The fire-and-forget header indicates to the proxy application <NUM> that responding to the HTTP/<NUM>. x request <NUM>(x) can be offloaded from the back-end server <NUM> to the proxy application <NUM>. Note that the only impact that an informational HTTP interaction typically has on the associated client application <NUM> is potentially delaying other HTTP interactions, thereby reducing the performance of the client application <NUM>. For this reason, in some embodiments, the client applications <NUM> add a fire-and-forget header to each HTTP/<NUM>. x request <NUM> that initiates an informational HTTP interaction.

The fire-and-forget header includes, without limitation, a name "Fire-and-Forget" and a success code list (one or more values for a success code parameter). The name "Fire-and-Forget" indicates that the corresponding HTTP/<NUM>. x response <NUM>(x) is unimportant. The success code list specifies one or more status codes that each indicate that the HTTP/<NUM>. x response <NUM>(x) is successful. For example, the success code list for a fire-and-forget header in the HTTP/<NUM>. x request <NUM>(x) could specify "<NUM>, <NUM>," and the success code list of a fire-and-forget header in the HTTP/<NUM>. x request <NUM>(y) could specify "<NUM>".

If the HTTP/<NUM>. x request <NUM>(x) includes a fire-and-forget header, then the HTTP/<NUM>. x request <NUM>(x) is associated with a persistence level that correlates to a level of importance associated with the HTTP/<NUM>. x request <NUM>(x). The persistence level for the HTTP1. x request <NUM>(x) can be specified via a "persistence" header that is included in the HTTP1. x request <NUM>(x). If the HTTP1. x request <NUM>(x) does not include a persistence header, then the HTTP1. x request <NUM>(x) is associated with a default persistence level.

The persistence header includes, without limitation, a name "Persistence" and a persistence level (a value for a persistence parameter). Each allowed persistence level is associated with a different error-handling process that the proxy application <NUM> is to perform if the back-end server <NUM> does not successfully process a version of the associated HTTP request <NUM>. More precisely, the persistence level for the HTTP request <NUM>(x) specifies the error handling process that the proxy application <NUM> is to perform if the back-end server is unable to successfully process an HTTP. <NUM> request <NUM>(x). The HTTP/<NUM> request <NUM>(x) is an HTTP/<NUM> version of the HTTP/<NUM>. x request <NUM>(x).

Upon receiving the HTTP/<NUM>. x request <NUM>(x) from one of the client applications <NUM>, the proxy application <NUM> determines whether to activate offloading for the HTTP/<NUM>. x request <NUM>(x). In some embodiments, the proxy application <NUM> activates offloading for the HTTP/<NUM>. x request <NUM>(x) if the HTTP/<NUM>. x request <NUM>(x) includes a fire-and-forget header. In other embodiments, the proxy application <NUM> may determine whether to activate offloading for the HTTP/<NUM>. x request <NUM>(x) based on whether the HTTP/<NUM>. x request <NUM>(x) includes a fire-and-forget header and any number of additional criteria.

For instance, in various embodiments, the proxy application <NUM> determines whether to activate offloading for the HTTP/<NUM>. x request <NUM>(x) based on whether the HTTP/<NUM>. x request <NUM>(x) includes a fire-and-forget header and a maximum concurrent offloads (not shown in <FIG>). The maximum concurrent offloads specifies a maximum number of HTTP/<NUM>. x requests <NUM> for which offloading may be active at any given time. If the HTTP/<NUM>. x request <NUM>(x) includes the fire-and-forget header and the total number of HTTP1. x requests <NUM> for which offloading is active is less than the maximum concurrent offloads, then the proxy application <NUM> activates offloading for HTTP/<NUM>. x request <NUM>(x). Otherwise, the proxy application <NUM> does not activate offloading for the HTTP/<NUM>. x request <NUM>(x). The proxy application <NUM> may track and activate offloading in any technically feasible fashion.

For instance, in some embodiments and as described in greater detail in conjunction with <FIG>, the proxy application <NUM> is written in the Go language and each HTTP/<NUM>. x request <NUM> is processed via a different execution thread. The offloading functionality is included in an HTTP. handler that limits the number of threads executing the offloading functionality to the maximum concurrent offloads.

If offloading is active for the HTTP/<NUM>. x request <NUM>(x), then the proxy application <NUM> transmits a "successful" generic HTTP/<NUM>. x response <NUM>(x) to the client application <NUM> using the client/proxy TCP connection <NUM> over which the HTTP/request <NUM>(x) was received. The client application <NUM> may then reuse or close the client/proxy TCP connection <NUM>. The successful generic HTTP/<NUM>. x response <NUM>(x) purposely and erroneously indicates that the back-end server <NUM> has successfully processed the HTTP/<NUM>. x request <NUM>(x). The proxy application <NUM> may generate the successful generic HTTP/<NUM>. x response <NUM>(x) in any technically feasible fashion.

For instance, in some embodiments, the proxy application <NUM> generates a successful generic HTTP/<NUM>. x response <NUM>(x) that includes one of the status codes specified in the success code list of the fire-and-forget header of the HTTP/<NUM>. x request <NUM>(x). For example, if the success code list includes the status code of <NUM>, then the proxy application <NUM> could generate the successful generic HTTP/<NUM>. x response <NUM>(x) having a "<NUM> OK" HTTP status line and an empty body. As persons skilled in the art will recognize, an "HTTP status line" is a status code (e.g., <NUM>) accompanied by an associated reason phrase (e.g., "OK"). The status code of <NUM> indicates that the back-end server <NUM> has successfully processed the HTTP/<NUM>. x request <NUM>(x). In another example, if the success code list includes the status code of <NUM>, then the proxy application <NUM> could generate the successful generic HTTP/<NUM>. x response <NUM>(x) having a "<NUM> No Content" HTTP status line. The status code of <NUM> indicates that the back-end server <NUM> has successfully processed the HTTP/<NUM>. x request <NUM>(x) and is not returning any content.

Subsequently, irrespective of whether offloading is active for the HTTP/<NUM>. x request <NUM>(x), the proxy application <NUM> converts the HTTP/<NUM>. x request <NUM>(x) to the HTTP/<NUM> request <NUM>(x). The HTTP/<NUM> request <NUM>(x) is an HTTP/<NUM> version of the HTTP/<NUM>. x request <NUM>(x). The proxy application <NUM> may convert the HTTP/<NUM>. x request <NUM>(x) to the HTTP/<NUM> request <NUM>(x) in any technically feasible fashion. The proxy application <NUM> then attempts to transmit the HTTP/<NUM> request <NUM>(x) to the back-end server <NUM> via one of the proxy/back-end TCP connections <NUM>. In contrast to the HTTP/<NUM>. x transactions, because any number of the HTTP/<NUM> requests <NUM> and any number of HTTP/<NUM> responses <NUM> can share each of the proxy/back-end TCP connections <NUM>, any number of HTTP/<NUM> transactions may execute concurrently.

If offloading is not active for the HTTP/<NUM> request <NUM>(x) and the proxy application <NUM> successfully receives an HTTP/<NUM> response <NUM>(x) from the back-end server <NUM>, then the proxy application <NUM> converts the HTTP/<NUM> response <NUM>(x) to the HTTP1. x response <NUM>(x). The HTTP/<NUM>. x response <NUM>(x) is an HTTP/<NUM>. x version of the HTTP/<NUM> response <NUM>(x). The proxy application <NUM> may convert the HTTP/<NUM> response <NUM>(x) to the HTTP1. x response <NUM>(x) in any technically feasible fashion. The proxy application <NUM> then transmits the HTTP/<NUM>. x response <NUM>(x) to the proxy application <NUM> using the client/proxy TCP connection <NUM> over which the HTTP/request <NUM>(x) was received. The client application <NUM> may then reuse or close the client/proxy TCP connection <NUM>.

If offloading is not active for the HTTP/<NUM> request <NUM>(x) and the proxy application <NUM> does not receive the HTTP/<NUM> response <NUM>(x) from the back-end server <NUM>, then the proxy application <NUM> generates a "server error" HTTP/<NUM> response <NUM>(x). The proxy application <NUM> may not receive the HTTP/<NUM> response <NUM>(x) from the back-end server <NUM> for a variety of reasons. For example, the back-end server <NUM> could be overloaded. The server error HTTP response <NUM>(x) indicates that the HTTP/<NUM>. x request <NUM>(x) has not succeeded because of a server error. For instance, in some embodiments, the server error HTTP response <NUM>(x) has a "<NUM> Bad Gateway" HTTP status line. The status code of <NUM> indicates that the proxy server <NUM> received an invalid response from the back-end server <NUM>.

If offloading is active for the HTTP/<NUM> request <NUM>(x) and the proxy application <NUM> receives the HTTP/<NUM> response <NUM>(x) indicating success from the back-end server <NUM>, then the proxy application <NUM> discards the HTTP/<NUM> response <NUM>(x). The proxy application <NUM> may determine whether the HTTP/<NUM> response <NUM>(x) indicates success in any technically feasible fashion. For instance, in some embodiments, the proxy application <NUM> determines whether the HTTP/<NUM> response <NUM>(x) indicates success based on the success code list associated with the HTTP/<NUM> request <NUM>(x). If the status code included in the HTTP/<NUM> response <NUM>(x) matches one of the status codes included in the success code list, then the proxy application <NUM> determines that the HTTP/<NUM> response <NUM>(x) indicates success. Otherwise, the proxy application <NUM> determines that the HTTP/<NUM> response <NUM>(x) does not indicate success.

If offloading is active for the HTTP/<NUM> request <NUM>(x) and the proxy application <NUM> does not receive an HTTP/<NUM> response from the back-end server <NUM>, then the proxy application <NUM> executes an error-handling process as per the persistence level. Similarly, if offloading is active for the HTTP/<NUM> request <NUM>(x) and the proxy application <NUM> receives the HTTP/<NUM> response <NUM>(x) that does not indicate success from the back-end server <NUM>, then the proxy application <NUM> executes an error-handling process as per the persistence level.

The proxy application <NUM> may implement any number and type of error-handling processes based on any number and type of acceptable persistence levels in any technically feasible fashion. For instance, in some embodiments, the acceptable persistence levels are "low," "medium," "high" (the default persistence level), and "durable. " If the persistence level for the HTTP/<NUM>. x request <NUM>(x) is low, then the proxy application <NUM> performs no further operations with respect to the HTTP/<NUM>. x request <NUM>(x). If the persistence level is medium, then the proxy application <NUM> performs at most three re-transmission attempts at relatively short intervals. A re-transmission attempt is an attempt to re-transmit the HTTP/<NUM> request <NUM>(x) to the back-end server <NUM>.

If the persistence level is high, then the proxy application <NUM> performs re-transmission attempts until the proxy application <NUM> receives an HTTP/<NUM> response <NUM>(x) indicating success from the back-end server <NUM>. Note that if the maximum concurrent offloads is reached, then the proxy application <NUM> no longer activates offloading and instead performs synchronous proxy operations. Consequently, a persistence level of high does not cause a backlog in the proxy application <NUM>.

If the persistence level is durable, then the proxy application <NUM> writes the HTTP/<NUM> request <NUM>(x) to persistent storage (e.g., disk) to survive across restarts and performs re-transmission attempts until the proxy application <NUM> receives an HTTP/<NUM> response <NUM>(x) indicating success from the back-end server <NUM>. If, at any point during the error handling process, the proxy application <NUM> receives an HTTP/<NUM> response <NUM>(x) indicating success from the back-end server <NUM>, then the proxy application <NUM> discards the HTTP/<NUM> response <NUM>(x) and terminates the error-handling process.

Each client application <NUM> may determine the persistence level for each HTTP/<NUM>. x request <NUM> in any technically feasible fashion. For instance, in some embodiments, since heartbeats occur relatively frequently, the client applications <NUM> include a persistence header specifying a persistence level of low in each HTTP/<NUM>. x request <NUM> that involves uploading heartbeats. Because event logs and metric logs are typically generated less frequently, the client applications <NUM> do not include a persistence header in HTTP/<NUM>. x requests <NUM> that involve uploading event logs or metric logs, thereby defaulting the associated persistence levels to high.

For explanatory purposes only, <FIG> depicts a series of interactions between the client application <NUM>(<NUM>), the proxy application <NUM>, and the back-end server <NUM> as a series of numbered bubbles. The client application <NUM>(<NUM>) executes on the client device <NUM>(<NUM>) and has the max connections <NUM>(<NUM>) of three.

To interact with the network-based service system <NUM>, the client application <NUM>(<NUM>) generates four HTTP/<NUM>. x requests <NUM>(<NUM>)-<NUM>(<NUM>). Each of the HTTP/<NUM>. x requests <NUM>(<NUM>)-<NUM>(<NUM>) specifies the network-based service system <NUM> as the host. Each of the HTTP/<NUM>. x requests <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>) is a request for a download and does not include a fire-and-forget header. By contrast, the HTTP/<NUM>. x request <NUM>(<NUM>) is a request for an upload of an event log and includes a fire-and-forget header. The client application <NUM>(<NUM>) then attempts to transmit the HTTP/<NUM>. x requests <NUM>(<NUM>)-<NUM>(<NUM>) to the proxy application <NUM>.

Because the client application <NUM>(<NUM>) is limited to at most three TCP connections, the client device <NUM> generates three client/proxy TCP connections <NUM>(<NUM>)-(<NUM>) for, respectively, the HTTP/<NUM>. x requests <NUM>(<NUM>)-<NUM>(<NUM>). Because each of the HTTP/<NUM>. x requests <NUM> requires a dedicated client/proxy TCP connection <NUM> and all of the available client/proxy TCP connections <NUM> are in use, the transmission of the HTTP/<NUM>. x request <NUM>(<NUM>) is delayed.

To generate the client/proxy TCP connection <NUM>(x) for the HTTP/<NUM>. X request <NUM>(x), the client device <NUM>(<NUM>) and the proxy server <NUM> perform a TCP handshake followed by a transport layer security ("TLS") handshake. The TCP handshake is the mechanism by which the client device <NUM> and the proxy server <NUM> negotiate and start a TCP communication session for communicating with one another. The TLS handshake is the mechanism by which the client device <NUM>(<NUM>) and the proxy server <NUM> exchange the security keys needed to establish a secure communication session.

As depicted with the bubbles numbered <NUM>-<NUM>, the proxy application <NUM> receives the HTTP/<NUM>. x requests <NUM>(<NUM>)-<NUM>(<NUM>) over, respectively, the client/proxy TCP connections <NUM>(<NUM>)-<NUM>(<NUM>). The proxy application <NUM> determines that the HTTP/<NUM>. x request <NUM>(<NUM>) includes a fire-and-forget header and that the total number of current concurrent offloads is less than the maximum concurrent offloads. Accordingly, as depicted with the bubble numbered <NUM>, the proxy application <NUM> activates offloading for the HTTP/<NUM>. x request <NUM>(<NUM>) and transmits the HTTP/<NUM>. x response <NUM>(<NUM>) having the HTTP status line "<NUM> OK" to the client application <NUM>(<NUM>) over the client/proxy TCP connection <NUM>(<NUM>).

The proxy application <NUM> converts the HTTP/<NUM>. x requests <NUM>(<NUM>)-<NUM>(<NUM>) to, respectively, the HTTP/<NUM> requests <NUM>(<NUM>)-<NUM>(<NUM>), and attempts to transmit the HTTP/<NUM> requests <NUM>(<NUM>)-<NUM>(<NUM>) to the back-end server <NUM>. Advantageously, the proxy server <NUM> and the back-end server <NUM> have previously performed the TCP handshake and the TLS handshake to generate any number of pre-established and pre-authenticated proxy/back-end TCP connections <NUM>. Further, due to the relatively high volume of traffic between the proxy server <NUM> and the back-end server <NUM>, the proxy/back-end TCP connections <NUM> are persistent. Consequently, the proxy server <NUM> begins to transmit the HTTP/<NUM> requests <NUM>(<NUM>)-<NUM>(<NUM>) over the client/proxy TCP connection <NUM>(<NUM>) to the back-end server <NUM> without performing any additional handshakes. More precisely, the proxy server <NUM> multiplexes the HTTP/<NUM> requests <NUM>(<NUM>)-<NUM>(<NUM>) along with any number of other HTTP/<NUM> requests <NUM> and/or HTTP/<NUM> responses <NUM> over the proxy/back-end TCP connection <NUM>(<NUM>).

When the client application <NUM>(<NUM>) receives the HTTP/<NUM>. x response <NUM>(<NUM>), the client device <NUM>(<NUM>) re-generates (e.g., closes, re-establishes, and reauthenticates) the client/proxy TCP connection <NUM>(<NUM>) for the delayed HTTP/<NUM>. x request <NUM>(<NUM>). In alternate embodiment's, the client device <NUM>(<NUM>) may re-use the client/proxy TCP connection <NUM>(<NUM>) for the delayed HTTP/<NUM>. x request <NUM>(<NUM>) without closing, re-establishing, and re-authenticating the client/proxy TCP connection <NUM>(<NUM>). The client device <NUM>(<NUM>) then begins to transmit the HTTP/<NUM>. x request <NUM>(<NUM>) to the proxy server <NUM> via the client/proxy TCP connection <NUM>(<NUM>).

Advantageously, the proxy server <NUM> is physically closer to the client device <NUM>(<NUM>) relative to the back-end server <NUM>. As a result, and as depicted with the bubble numbered <NUM>, the proxy application <NUM> receives the HTTP/<NUM>. x request <NUM>(<NUM>) before the HTTP/<NUM> requests <NUM>(<NUM>)-<NUM>(<NUM>) reach the back-end server <NUM>. The proxy application <NUM> converts the HTTP/<NUM>. x request <NUM>(<NUM>) to the HTTP/<NUM> request <NUM>(<NUM>) and attempts to transmit the HTTP/<NUM> request <NUM>(<NUM>) to the back-end server <NUM>. The proxy server <NUM> multiplexes the HTTP/<NUM> request <NUM>(<NUM>) with the HTTP/<NUM> requests <NUM>(<NUM>)-<NUM>(<NUM>) and any number of other HTTP/<NUM> requests <NUM> and/or HTTP/<NUM> responses <NUM> over the proxy/back-end TCP connection <NUM>(<NUM>).

Subsequently, and as depicted with the bubbles numbered <NUM>-<NUM>, the back-end server <NUM> receives the HTTP/<NUM> requests <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>) via the proxy/back-end TCP connection <NUM>(<NUM>). Because the HTTP/<NUM> request <NUM>(<NUM>) involves an upload, the HTTP/<NUM> request <NUM>(<NUM>) requires a longer period of time to reach the back-end server <NUM>. The back-end server <NUM> processes the HTTP/<NUM> requests <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>) and generates, respectively, the HTTP/<NUM> responses <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>). The back-end server <NUM> initiates the transmission of the HTTP/<NUM> responses <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>) to the proxy application <NUM> over the proxy/back-end TCP connection <NUM>(<NUM>).

As depicted with the bubbles number <NUM>-<NUM>, the proxy application <NUM> successfully receives the HTTP/<NUM> responses <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>) over the proxy/back-end TCP connection <NUM>(<NUM>). Because offloading is not active for the HTTP/<NUM>. x requests <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>), the proxy application <NUM> converts the HTTP/<NUM> responses <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>) to, respectively, the HTTP/<NUM>. x responses <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>). The proxy application <NUM> then initiates the transmission of the HTTP1. x response <NUM>(<NUM>) over the client/proxy TCP connection <NUM>(<NUM>), the transmission of the HTTP1. x response <NUM>(<NUM>) over the client/proxy TCP connection <NUM>(<NUM>), and the transmission of the HTTP1. x response <NUM>(<NUM>) over the client/proxy TCP connection <NUM>(<NUM>).

While the HTTP/<NUM>. x responses <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>) are traveling to the client application <NUM>(<NUM>), the back-end server <NUM> receives the HTTP/<NUM> request <NUM>(<NUM>) (depicted with the bubble numbered <NUM>). The back-end server <NUM> processes the HTTP/<NUM> request <NUM>(<NUM>), generates the HTTP/<NUM> response <NUM>(<NUM>), and initiates the transmission of the HTTP/<NUM> response <NUM>(<NUM>) to the proxy application <NUM> over the proxy/back-end TCP connection <NUM>(<NUM>).

As depicted with the bubbles numbered <NUM>-<NUM> and while the HTTP/<NUM> response <NUM>(<NUM>) is traveling to the proxy application <NUM>, the client application <NUM>(<NUM>) receives the HTTP/<NUM>. x responses <NUM>(<NUM>), <NUM>(<NUM>), and <NUM>(<NUM>). Subsequently, as depicted with the bubble numbered <NUM>, the proxy application <NUM> receives the HTTP/<NUM> response <NUM>(<NUM>). Because offloading is active for the HTTP/<NUM>. x request <NUM>(<NUM>) and the HTTP/<NUM> response <NUM>(<NUM>) indicates that the back-end server <NUM> successfully processed the HTTP/<NUM>. x request <NUM>(<NUM>), the proxy application <NUM> discards the HTTP/<NUM> response <NUM>(<NUM>).

For explanatory purposes only, <FIG> describe the functionality of the system <NUM> in the context of TCP connections and specific versions of the HTTP protocol. However, the techniques described herein are illustrative rather than restrictive, and may be altered without departing from the scope of the invention. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments and techniques.

As a general matter, the techniques outlined herein are applicable to returning successful generic responses in response to selected requests for network-based services before transmitting the requests to the network-based services. In alternate embodiments, the requests and the responses may comply with any number and type of transmission protocols and be transmitted over any number and type of connections. In general, when responding to an HTTP request received from the client application <NUM>, the proxy application <NUM> generates or relays the HTTP response to the client application <NUM> in the HTTP format of the HTTP request. When relaying an HTTP request to the back-end server <NUM>, the proxy application may perform any number of conversion operations on the HTTP request to ensure the most efficient transmission of a version of the HTTP request to the back-end server <NUM>.

For instance, in some alternate embodiments, any number of the client applications <NUM> implement HTTP/<NUM>. When generating the HTTP/<NUM> request <NUM>(x), the client application can add a fire-and-forget header to the HTTP/<NUM> request <NUM>(x) to indicate that the corresponding HTTP/<NUM> response <NUM>(x) is unimportant. The proxy application <NUM> determines whether to activate offloading for the HTTP/<NUM> request <NUM>(x) based on whether the HTTP/<NUM> request <NUM>(x) has the fire-and-forget header and the maximum concurrent offloads. If offloading is active for the HTTP/<NUM> request <NUM>(x), then the proxy application <NUM> transmits a successful generic HTTP/<NUM> response <NUM> to the client application <NUM>. Irrespective of whether offloading is active for the HTTP/<NUM> request <NUM>(x), the proxy application <NUM> attempts to transmit the HTTP/<NUM> request <NUM>(x) to the back-end server <NUM>. If offloading is active for the HTTP/<NUM> request <NUM>(x), then the proxy application <NUM> discards any HTTP/<NUM> response <NUM>(x) received from the back-end server <NUM>. If offloading is not active for the HTTP/<NUM> request <NUM>(x) and the proxy application <NUM> is unable to transmit the HTTP/<NUM> request <NUM>(x) to the back-end server <NUM>, then the proxy application <NUM> transmits a server error HTTP/<NUM> response to the client application <NUM>. Otherwise, if offloading is not active for the HTTP/<NUM> request <NUM>(x), then the proxy application <NUM> transmits the HTTP/<NUM> response <NUM>(x) received from the back-end server <NUM> to the client application <NUM>.

In alternate embodiments, the fire-and-forget header and the persistence header may be specified in any technically feasible fashion and include any amount and type of relevant information. For instance, in some embodiments, the fire-and-forget header does not specify a success code list. In such embodiments, the proxy application <NUM> may determine whether each HTTP/<NUM> response <NUM> indicates success in any technically feasible fashion. For example, the proxy application <NUM> could implement a predetermined success code list. In the same or other embodiments, persistence headers are not implemented and the fire-and-forget header optionally specifies a persistence level.

<FIG> is a more detailed illustration of the proxy application <NUM> of <FIG>, according to various embodiments of the present invention. For explanatory purposes only, <FIG> describes an example of the proxy application <NUM> written in the Go programming language. However, the techniques described herein are illustrative rather than restrictive, and may be altered without departing from the scope of the invention. In particular, the proxy application <NUM> may execute any number and type of algorithms and may be written in any number (including one) of technically feasible programming languages and/or scripting languages.

As shown, the proxy application <NUM> includes, without limitation, an HTTP library layer <NUM>, a fire-and-forget proxy handler <NUM>, any number of handlers <NUM>, and a proxy handler <NUM>. The HTTP library layer <NUM> is configured to receive each HTTP/<NUM>. x request <NUM> to the proxy server <NUM> as a different execution thread. Upon receiving the HTTP/<NUM>. x request <NUM>(x), the HTTP library layer <NUM> transfers control of the associated execution thread to a bidirectional pipeline of nested HTTP handlers.

The bidirectional pipeline includes, sequentially in a forward direction, the fire-and-forget proxy handler <NUM>, any number of the handlers <NUM>, and the proxy handler <NUM>. In alternate embodiment's, any number of the handlers <NUM> may precede the fire-and-forget proxy handler <NUM> in the pipeline and any number of the handlers <NUM> may follow the proxy handler <NUM> in the pipeline.

Each of the HTTP handlers is of a type http. HandlerFunc that has two arguments, an http. ResponseWriter and a pointer to an http. Request is a data structure that represents the HTTP/<NUM> request <NUM>(x). responseWriter generates and transmits the HTTP/<NUM>. x response <NUM>(x) to the HTTP/<NUM> request <NUM>(x). If an HTTP handler writes data to the http. responseWriter, then the http. responseWriter transmits a corresponding HTTP/<NUM>. x response <NUM>(x) via the client/proxy TCP connection <NUM> over which the HTTP/<NUM> request <NUM>(x) was received. In addition, various HTTP handlers in the pipeline access an http. Response data structure that represents the HTTP response to the HTTP/<NUM> request <NUM>(x).

Each of the HTTP handlers may perform any number and type of operations associated with the HTTP/<NUM> request <NUM>(x) and/or the HTTP/<NUM> request <NUM>(x) via the http. Similarly, each of the HTTP handlers may perform any number and type of operations associated with the HTTP/. <NUM> response <NUM>(x) and/or the HTTP/<NUM> response <NUM>(x) via the http. Each of the HTTP handlers may also write to the http. responseWriter, transfer control to the next HTTP handler in the pipeline in either direction, stop control propagating through the pipeline, etc..

When the execution thread associated with the HTTP/<NUM> request <NUM>(x) enters the HTTP library layer <NUM>, the HTTP library layer <NUM> generates the http. responseWriter and the http. Request representing the HTTP/<NUM> request <NUM>(x). The HTTP library layer <NUM> then transfers control to the first HTTP handler in the pipeline to initiate a propagation in the forward direction In the embodiment depicted in <FIG>, the first HTTP handler in the pipeline is the fire-and-forget handler <NUM>.

As shown, the fire-and-forget handler <NUM> includes, without limitation, an HTTP header trigger <NUM>, a maximum concurrent offloads <NUM>, and offloading operations <NUM>. The HTTP header trigger <NUM> specifies the name "Fire-and-Forget" that distinguishes a fire-and-forget header. The maximum concurrent offloads <NUM> specifies the maximum number of HTTP/<NUM>. x requests <NUM> (i.e., threads) for which offloading can be active at any given time. The offloading operations <NUM> are the operations in the fire-and-forget handler <NUM> that execute when offloading is active for the execution thread.

When the fire-and-forget handler <NUM> receives control of the execution thread, the fire-and-forget handler <NUM> determines whether the HTTP/<NUM> request <NUM>(x) includes a fire-and-forget header based on the HTTP header trigger <NUM>. If the fire-and-forget handler <NUM> determines that the HTTP/<NUM> request <NUM>(x) does not include a fire-and-forget header, then the fire-and-forget handler <NUM> acts as a pass-through in the forward direction. More precisely, the fire-and-forget handler <NUM> transfers control to the handler <NUM>(<NUM>) that is next in the pipeline in the forward direction without executing the offloading operations <NUM>.

If, however, the fire-and-forget handler <NUM> determines that the HTTP/<NUM> request <NUM>(x) includes a fire-and-forget header, then the fire-and-forget handler <NUM> determines whether to activate offloading for the HTTP/<NUM> request <NUM>(x) based on the maximum concurrent offloads <NUM>. In general, the fire-and-forget handler <NUM> limits the total number of HTTP/<NUM> requests <NUM> for which offloading is active to the maximum concurrent offloads <NUM>.

The fire-and-forget handler <NUM> may track and limit the total number of HTTP/<NUM>. x requests <NUM> for which offloading is active in any technically feasible fashion. For instance, in some embodiments, fire-and-forget handler <NUM> may use concurrency features of the GO language (e.g., channels) to limit the total number of threads for which offloading is active at any given time. In other embodiments, the fire-and-forget handler <NUM> may maintain a count of the execution threads for which offloading is active and compare the count to the maximum concurrent offloads <NUM>. If the count is equal to the maximum concurrent offloads <NUM>, then the fire-and-forget handler <NUM> does not enable offloading for the HTTP/<NUM>,<NUM> request <NUM>(x) and acts as a pass-through. in the forward direction. More precisely, the fire-and-forget handler <NUM> transfers control to the handler <NUM>(<NUM>) that is next in the pipeline in the forward direction without executing the offloading operations <NUM>.

In alternate embodiments, the fire-and-forget handler <NUM> may activate offloading for the HTTP/request <NUM>(x) in any technically feasible fashion. For instance, the fire-and-forget handler <NUM> may initialize an offload flag associated with the HTTP/request <NUM>(x) to false. If the HTTP/request <NUM>(x) includes a fire-and-forget header and the total number of HTTP requests for which offloading is active is less than the maximum concurrent offloads <NUM>, then the fire-and-forget handler <NUM> sets the offload flag to true. If the offload flag is false, then the fire-and-forget handler <NUM> does not execute the offloading operations <NUM>. Otherwise, the fire-and-forget handler <NUM> executes the offloading operations <NUM>.

If the fire-and-forget handler <NUM> retains control of the execution thread, then offloading is active for the HTTP/<NUM>,<NUM> request <NUM>(x) and the fire-and-forget handler <NUM> begins to execute the offloading operations <NUM>. The fire-and-forget handler <NUM> generates the successful generic HTTP/<NUM>. x response <NUM>(x). The fire-and-forget handler <NUM> then writes the successful generic HTTP/<NUM>. x response <NUM>(x) to the http. responseWriter. In response, the http. responseWriter transmits the successful generic HTTP/<NUM>. x response <NUM>(x) via the client/proxy TCP connection <NUM> over which the HTTP/<NUM> request <NUM>(x) was received. Note that writing to the http. responseWriter is a non-blocking operation with respect to the execution thread. After writing to the http. responseWriter, the fire-and-forget handler <NUM> transfers control of the execution thread to the handler <NUM>(<NUM>) that is next in the pipeline in the forward direction.

Each of the handlers <NUM> may perform any number and type of operations before transferring control of the execution thread to the next HTTP handler in the pipeline. When the control of the execution thread is transferred to the proxy handler <NUM>, the proxy handler <NUM> converts the HTTP/<NUM>. x request <NUM>(x) to the HTTP/<NUM> request <NUM>(x). The proxy handler <NUM> then attempts to transmit the HTTP/<NUM> request <NUM>(x) to the back-end server <NUM>. Transmitting the HTTP/<NUM> request <NUM>(x) to the back-end server <NUM> blocks the execution thread until the proxy handler <NUM> receives the HTTP/<NUM> response <NUM>(x) or the proxy handler <NUM> determines that the back-end server <NUM> is unable to transmit a corresponding HTTP/<NUM> response. If the proxy handler <NUM> determines that the back-end server <NUM> is unable to transmit a corresponding HTTP/<NUM> response, then the proxy handler <NUM> generates the server error HTTP/<NUM> response <NUM>(x) having a status code that is not included in the success code list associated with the HTTP/<NUM>. x request <NUM>(x). Subsequently, the proxy handler <NUM> returns control of the execution thread to the HTTP handler from which the proxy handler <NUM> received control, thereby initiating a propagation of the pipeline in the reverse direction.

When control of the execution thread is returned to the fire-and-forget handler <NUM>, the fire-and-forget handler <NUM> resumes executing either outside or within the offloading operations <NUM>. If offloading is not active for the HTTP/request <NUM>(x), then the fire-and-forget handler <NUM> resumes executing outside the offloading operations <NUM>. The fire-and-forget handler <NUM> converts the HTTP/<NUM> response <NUM>(x) to the HTTP/<NUM>. x response <NUM>(x). The fire-and-forget handler <NUM> then writes the HTTP/<NUM>. x response <NUM>(x) to the http. responseWriter. In response, the http. responseWriter transmits the HTTP/<NUM>. x response <NUM>(x) via the client/proxy TCP connection <NUM> over which the HTTP/<NUM>. x request <NUM>(x) was received.

If, however, offloading is active for the HTTP/<NUM>. x request <NUM>(x), then the fire-and-forget handler <NUM> resumes executing within the offloading operations <NUM>. If the status code of the HTTP/<NUM> response <NUM>(x) indicates success, then the fire-and-forget handler <NUM> terminates the execution thread. Otherwise, the fire-and-forget handler <NUM> executes the error handling procedure corresponding to the persistence level associated with the HTTP/<NUM>. x request <NUM>(x).

As described previously herein, if the status code of the HTTP/<NUM> response <NUM>(x) matches one of the status codes included in the success code list associated with the HTTP/<NUM>. x request <NUM>(x), then the fire-and-forget handler <NUM> determines that the HTTP/<NUM> response <NUM>(x) indicates success. Otherwise, the fire-and-forget handler <NUM> determines that the HTTP/<NUM> response <NUM>(x) does not indicate success. In general, the back-end server <NUM> may include any status code in the HTTP/<NUM> response <NUM>(x). Consequently, if the HTTP/<NUM> response <NUM>(x) is generated by the back-end server <NUM>, then the HTTP/<NUM> response <NUM>(x) may or may not indicate success as per the success code list. By contrast, if the HTTP/<NUM> response <NUM>(x) is generated by the proxy handler <NUM> to indicate that the back-end server <NUM> was unable to respond to the HTTP/<NUM> request <NUM>(x), then the HTTP/<NUM> response <NUM>(x) does not indicate success as per the success code list.

As part of executing the error handling procedure, the fire-and-forget handler <NUM> may cause the execution thread to repeatedly propagate back and forth between the fire-and-forget handler <NUM> and the proxy handler <NUM>. The error handling procedure eventfully terminates the execution thread.

It will be appreciated that the proxy application <NUM> described herein is illustrative and that variations and modifications are possible. For instance, in alternate embodiments, the fire-and-forget handler <NUM> may indicate to the proxy handler <NUM> whether offloading is active for the HTTP/request <NUM>(x) in any technically feasible fashion. If offloading is not active for the HTTP/request <NUM>(x), then the proxy handler <NUM> generates and writes the HTTP/<NUM>. x response <NUM>(x) to the http. responseWriter. The proxy handler <NUM> then terminates the execution thread.

For explanatory purposes only, <FIG> depicts the control of the execution thread associated with the HTTP/<NUM>. x request <NUM>(<NUM>) of <FIG> that is received from the client application <NUM>(<NUM>) as a series of numbered bubbles. The HTTP/<NUM>. x request <NUM>(<NUM>) includes, without limitation, a request line, a HOST header, a fire-and-forget header, and a body. The request line specifies the POST method, the request target "/upload," and the protocol version HTTP/<NUM>. The POST method sends the body of the HTTP/<NUM>. x request <NUM>(<NUM>) (not shown) to the back-end server <NUM> to create or modify the request target "/upload. " The HOST header specifies the domain name "www. com" of the back-end server <NUM>. The fire-and-forget header specifies the name "Fire-and-Forget" and the success code list that includes the status code of <NUM>. Because the HTTP/<NUM>. x request <NUM>(<NUM>) does not include a persistence header, the HTTP/<NUM>. x request <NUM>(<NUM>) is associated with the default persistence level of high.

As depicted with the bubble numbered <NUM>, the execution thread associated with the HTTP/<NUM>. x request <NUM>(<NUM>) enters the HTTP library layer <NUM>. The HTTP library layer <NUM> generates the http. responseWriter and the http. Request that represents the HTTP/<NUM>. x request <NUM>(<NUM>) and then transfers control to the fire-and-forget handler <NUM>. Because the HTTP/<NUM>. x request <NUM>(<NUM>) includes a fire-and-forget header, the fire-and-forget handler <NUM> determines whether to activate offloading for the HTTP/<NUM>. x request <NUM>(<NUM>) based the maximum concurrent offloads <NUM>. For explanatory purposes only, the total number of HTTP/<NUM>. x requests <NUM> for which offloading is active is less than the maximum concurrent offloads <NUM>. Therefore, the fire-and-forget handler <NUM> activates offloading for the HTTP/<NUM>. x request <NUM>(<NUM>) and begins to execute the offloading operations <NUM>.

As depicted with the bubble numbered <NUM>, the fire-and-forget handler <NUM> configures the http. responseWriter to transmit the successful generic HTTP response <NUM>(x) to the client application <NUM>(<NUM>) in a non-blocking manner. The fire-and-forget handler <NUM> then causes the execution thread to propagate through the pipeline in a forward direction until the proxy handler <NUM> receives control of the execution thread. The proxy handler <NUM> converts the HTTP/<NUM>. x request <NUM>(<NUM>) to the HTTP/<NUM> request <NUM>(<NUM>).

As depicted with the bubble numbered <NUM>, the proxy handler <NUM> transmits the HTTP/<NUM> request <NUM>(<NUM>) to the back-end server <NUM>. The execution thread then receives the HTTP/<NUM> response <NUM>(<NUM>) specifying a "<NUM> Gateway Timeout" HTTP status line depicted with the bubble numbered <NUM>). Subsequently, the proxy handler <NUM> causes the execution thread to propagate through the pipeline in a reverse direction until reaching the fire-and-forget handler <NUM>.

The fire-and-forget handler <NUM> resumes execution within the offloading operations <NUM>. Because the HTTP/<NUM> response <NUM>(<NUM>) does not specify a successful status code, the fire-and-forget handler <NUM> executes the error handler process corresponding to the persistence level of high. More specifically and as depicted with the bubble numbered <NUM>, the fire-and-forget handler <NUM> performs a re-transmission attempt <NUM>. To execute the re-transmission attempt <NUM>, the fire-and-forget handler <NUM> transfers control of the execution thread to the proxy handler <NUM>(<NUM>), and the execution thread re-propagates through the pipeline in a forward direction until reaching the proxy handler <NUM>.

As depicted with the bubble numbered <NUM>, the proxy handler <NUM> attempts to re-transmit the HTTP/<NUM> request <NUM>(<NUM>) to the back-end server <NUM>. Subsequently and as depicted with the bubble numbered <NUM>, the proxy handler <NUM> receives the HTTP/<NUM> response <NUM>(<NUM>') specifying the HTTP status line of "<NUM> OK. " The proxy handler <NUM> causes the execution thread to re-propagate through the pipeline in a reverse direction until reaching the fire-and-forget handler <NUM>. The fire-and-forget handler <NUM> resumes executing within the error handling process. Because the status code of the HTTP/<NUM> response <NUM>(<NUM>') indicates success, the fire-and-forget handler <NUM> terminates the execution thread.

Note that the techniques described herein are illustrative rather than restrictive, and may be altered without departing from the scope of the invention. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments and techniques. In particular, the functionality of the proxy application <NUM> may be implemented in any technically feasible fashion. For instance, in alternate embodiments, the proxy application <NUM> may include any number (including one) and type of functions instead of the pipeline of HTTP handler functions.

<FIG> is a flow diagram of method steps for processing a request associated with a network-based service, according to various embodiments of the present invention. Although the method steps are described with reference to the systems of <FIG>, persons skilled in the art will understand that any system configured to implement the method steps falls within the scope of the present invention.

As shown, a method <NUM> begins at step <NUM>, where the proxy application <NUM> receives an HTTP request from one of the client applications <NUM>. The HTTP request may be in an HTTP/<NUM>. x format (i.e., the HTTP/<NUM>. x request <NUM>(x)) or in the HTTP/<NUM> format (i.e., the HTTP/<NUM> request. At step <NUM>, the fire-and-forget handler <NUM> determines whether to activate offloading for the HTTP request based on whether the HTTP request includes a fire-and-forget header and, optionally, the maximum concurrent offloads <NUM> At step <NUM>, the fire-and-forget handler <NUM> determines whether offloading is active for the HTTP request. If, at step <NUM>, the fire-and-forget handler <NUM> determines that offloading is active for the HTTP request, then the method <NUM> proceeds to step <NUM>.

At step <NUM>, the fire-and-forget handler <NUM> transmits a successful generic HTTP response in the version of HTTP associated with the HTTP request to the client application <NUM>. At step <NUM>, the proxy handler <NUM> attempts to transmit the HTTP/<NUM> request <NUM> corresponding to the HTTP request to the back-end server <NUM>. The HTTP/<NUM> request <NUM> corresponding to the HTTP request may be the HTTP request. At step <NUM>, the fire-and-forget handler <NUM> determines whether the proxy handler <NUM> received the HTTP/<NUM> response <NUM> indicating success. If, at step <NUM>, the fire-and-forget handler <NUM> determines that the proxy handler <NUM> received the HTTP/<NUM> response <NUM> indicating success, then the method <NUM> terminates.

If, however, at step <NUM>, the fire-and-forget handler <NUM> determines that the proxy handler <NUM> did not receive the HTTP/<NUM> response <NUM> indicating success, then the method <NUM> proceeds to step <NUM>. At step <NUM>, the fire-and-forget handler <NUM> executes an error handling process based on the persistence level. The persistence level is either specified in a persistence header of the HTTP request or is equal to a default persistence level. The error handling process may involve any number (including zero) of re-transmission attempts <NUM>. The method <NUM> then terminates.

Returning now to step <NUM>, if the fire-and-forget handler <NUM> determines that offloading is not active for the HTTP request, then the method <NUM> proceeds directly to step <NUM>. At step <NUM>, the proxy handler <NUM> attempts to transmit the HTTP/<NUM> request <NUM> corresponding to the HTTP request to the back-end server <NUM>. The HTTP/<NUM> request <NUM> corresponding to the HTTP request may be the HTTP request.

At step <NUM>, the proxy handler <NUM> determines whether the corresponding HTTP/<NUM> response <NUM> was received from the back-end server <NUM>. If, at step <NUM>, the proxy handler <NUM> determines that the corresponding HTTP/<NUM> response <NUM> was received from the back-end server <NUM>, then the method <NUM> proceeds directly to step <NUM> If, at step <NUM>, the proxy handler <NUM> determines that the corresponding HTTP/<NUM> response <NUM> was not received from the back-end server <NUM>, then the method <NUM> proceeds to step <NUM>. At step <NUM>, the proxy handler <NUM> generates the server error HTTP/<NUM> response <NUM> and the method <NUM> proceeds to step <NUM>.

At step <NUM>, the fire-and-forget handler <NUM> sends a version of the HTTP/<NUM> response <NUM> to the client application <NUM>. If the HTTP request received from the client application <NUM> is in the HTTP/<NUM> format, then the fire-and-forget handler <NUM> transmits the HTTP/<NUM> response <NUM> to the client application <NUM>. Otherwise, the fire-and-forget handler <NUM> converts the HTTP/<NUM> response <NUM> to the HTTP/<NUM>. x response <NUM> and transmits the HTTP/<NUM>. x response <NUM> to the client application <NUM>. The method <NUM> then terminates.

In sum, using the disclosed techniques, a proxy application may enable a client application that implements HTTP/<NUM>. x over TCP to efficiently interact with a back-end server to access a network-based service. The client application can add a "fire-and-forget" header to an HTTP request for an informational upload to indicate that the corresponding HTTP response is unimportant. The fire-and-forget header specifies a success code list of status codes that are to be considered successful. The fire-and-forget header is associated with a default persistence level, and the client application can also add a "persistence" header to the HTTP request that explicitly specifies a persistence level. The persistence level correlates to the level of importance for the back-end server receiving the HTTP request.

The proxy application executes on a proxy server that acts as an intermediary between the client application and the back-end server. Upon receiving an HTTP request from the client application, the proxy application determines whether to activate offloading for the HTTP request. If the HTTP request has a fire-and-forget header and a maximum number of concurrent offloads has not been reached, then the proxy application activates offloading for the HTTP request. Otherwise, the proxy application does not activate fire-and-forget for the HTTP request.

If offloading is active for the HTTP request, then the proxy application transmits a successful generic HTTP response to the client. The successful generic HTTP response specifies one of the status codes included in the success code list, thereby intentionally and erroneously indicating that the back-end server has successfully received and processed the HTTP request. The proxy application then attempts to transmit an HTTP/<NUM> version of the HTTP request to a back-end server. If the proxy application receives an HTTP/<NUM> response indicating success as per the success code list, then the proxy application discards the HTTP/<NUM> response. Otherwise, the proxy application executes an error handling process based on the persistence level. The error handling process can involve any number of re-transmission attempts.

If offloading is not active for the HTTP request, then the proxy application attempts to transmit an HTTP/<NUM> version of the HTTP/<NUM>. x request to the back-end server. If the proxy application does not receive an HTTP/<NUM> response from the back-end server, then the proxy application transmits a server error HTTP/<NUM>. x response to the client application. Otherwise, the proxy application transmits an HTTP/<NUM>. x version of the HTTP/<NUM> response received from the back-end server to the client application.

At least one technical advantage of the disclosed techniques relative to the prior art is that, with the disclosed techniques, interactive HTTP transactions are less likely to be delayed by informational HTTP transactions for a client application that implements HTTP/<NUM>. x over TCP. In particular, as soon as the proxy server responds to an informational HTTP transaction transmitted from the client application, the client application can close or reuse the associated TCP connection without having to wait for a response from the back-end server. Accordingly, the client application is less likely to use all available TCP connections for informational HTTP transactions and delay the transmission and processing of interactive HTTP transactions. Furthermore, because the proxy server subsequently attempts to upload the information associated with an informational HTTP transaction to the back-end server, the likelihood that the information ends up being dropped is not increased. And since the proxy server communicates with the back-end server via HTTP/<NUM> using pre-established and pre-authenticated TCP connections, communication times between the client application and the back-end server are shortened,. These technical advantages represent one or more technological advancements over prior art approaches.

Aspects of the present embodiments may be embodied as a system, method or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "module," a "system," or a "computer. " In addition, any hardware and/or software technique, process, function, component, engine, module, or system described in the present disclosure may be implemented as a circuit or set of circuits. Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

Claim 1:
A computer-implemented method, the method comprising:
determining (<NUM>) that a first request (<NUM>) received from a client application (<NUM>) indicates a response to the first request can be offloaded from a server machine (<NUM>); and
prior to transmitting (<NUM>) the first request to the server machine, transmitting (<NUM>) a first response (<NUM>) to the first request to the client application indicating that the server machine has successfully processed the first request,
wherein, upon receiving the first response, the client application is able to initiate a second request; the method characterised in that the first request includes a status code that indicates that the server machine has successfully processed the first request.