Patent Publication Number: US-11044335-B2

Title: Method and apparatus for reducing network resource transmission size using delta compression

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a continuation of U.S. application Ser. No. 15/233,157, filed Aug. 10, 2016, which is hereby incorporated by reference. 
    
    
     TECHNICAL FIELD 
     Embodiments of the invention relate to the field of networking; and more specifically, to reducing network resource transmission size using delta compression. 
     BACKGROUND ART 
     Network resources are commonly requested and retrieved using the Hypertext Transport Protocol (HTTP). A common approach to improving performance is caching network resources closer to the clients (often in several different data centers are differently geographically located). For example, content delivery networks (CDNs) are commonly used to cache resources such as static resources (e.g., resources that change infrequently such as images, videos, etc.) closer to clients. 
     Resources that frequently change (e.g., web sites for news, sports, etc.) may not be cached since the cached version may quickly become stale. Also, other types of resources may not be cached. As an example, HTTP includes a Cache-Control header field that is used to control the caching mechanisms along the request/response chain. The Cache-Control header field includes a “no-cache” directive that can be set by an origin server to specify that a cached version of the resource is not to be used to respond to a request without successful revalidation with the origin server. 
     To decrease the size of transmissions of resources, whether from a cache server or from an origin server, compression is typically used. For example, gzip is a common compression algorithm used to send compressed HTTP responses to clients. 
     Delta encoding in HTTP has been proposed in RFC 3229, “Delta encoding in HTTP”, January 2002, as a way to reduce the size of resource transmission between an origin server and the client. The delta encoding proposal described in RFC 3229 recognizes that if a web resource changes, the new version is often similar to the older version. RFC 3229 proposes that the difference between the versions be sent to the client (e.g., the browser on a client computing device such as a desktop computer, notebook, tablet, smartphone, etc.), and the client applies the delta to construct the new version. The approach described in RFC 3229 has not been widely adopted because in order to be effective for a resource that changes often, the origin server would have to store many versions of the website (e.g., an extreme would be a single version for each single client). 
     Another proposal for compression is the use of a shared dictionary compression over HTTP. In the shared dictionary compression, the client and server agree on a set of predefined elements that will be the same across the resources (referred to as cross-payload redundancy). For example, the header, footer, and/or other elements of the resource may be the same. With this proposal, the shared dictionary is downloaded to the client that contains strings that are likely to appear in subsequent HTTP responses. The server can then substitute those elements with a reference to the dictionary when sending the HTTP response and the client can use its dictionary to reconstruct the resource, thereby reducing the payload of the transmission. This approach has not been widely adopted in part because it is administratively difficult to maintain the dictionaries. For example, if a website is changed (e.g., it is redesigned), the references in the dictionary will likely need to be changed and disseminated to the clients. In addition, this approach requires support on the client devices. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The invention may best be understood by referring to the following description and accompanying drawings that are used to illustrate embodiments of the invention. In the drawings: 
         FIG. 1  illustrates an exemplary system for reducing network resource transmission size using delta compression according to one embodiment; 
         FIG. 2  illustrates the exemplary system of  FIG. 1  in more detail according to one embodiment; 
         FIG. 3  illustrates a sequence of exemplary operations performed when a network resource is not in the dynamic dictionary of the near end network optimizer according to one embodiment; 
         FIG. 4  illustrates a sequence of exemplary operations performed when a network resource is included in the dynamic dictionary of the near end network optimizer and the same version is not included in the dynamic dictionary of the far end network optimizer according to one embodiment; 
         FIG. 5  illustrates a sequence of exemplary operations performed when a network resource is included in the dynamic dictionary of the near end network optimizer and the same version is included in the dynamic dictionary of the far end network optimizer according to one embodiment; 
         FIG. 6  is a flow diagram illustrating exemplary operations performed for a delta compression technique for compressing the payload size of network resources according to one embodiment; 
         FIG. 7  is a flow diagram illustrating exemplary operations performed when the requested resource is not in the dynamic dictionary of the near end network optimizer and/or the far end network optimizer according to one embodiment; 
         FIG. 8  illustrates an embodiment where a client network application of the client device includes a near end network optimizer according to one embodiment; 
         FIG. 9  illustrates an alternative embodiment where a client network application of the client device includes a near end network optimizer according to one embodiment; 
         FIG. 10  illustrates an embodiment where a first point of presence of a cloud proxy service includes a near end network optimizer according to one embodiment; 
         FIG. 11  illustrates a sequence of exemplary operations for reducing network resource transmission size using delta compression between points of presence of a cloud proxy service, according to one embodiment. 
         FIG. 12  illustrates a sequence of exemplary operations performed when a network resource is not in the dynamic dictionary of the near end network optimizer of a first point of presence of a cloud proxy service, according to one embodiment; 
         FIG. 13  illustrates a sequence of exemplary operations performed when a network resource is included in the dynamic dictionary of the near end network optimizer of a first point of presence of a cloud proxy service and the same version is not included in the dynamic dictionary of the far end network optimizer of a second point of presence of the cloud proxy service, according to one embodiment; 
         FIG. 14  illustrates a sequence of exemplary operations performed when a network resource is included in the dynamic dictionary of the near end network optimizer of a first point of presence of a cloud proxy service and the same version is included in the dynamic dictionary of the far end network optimizer of the second point of presence of the cloud proxy service, according to one embodiment; 
         FIG. 15A  is a flow diagram illustrating exemplary operations performed for reducing network resource transmission using a delta compression technique between points of presence of a cloud proxy service, according to one embodiment; 
         FIG. 15B  is a flow diagram illustrating exemplary operations performed for reducing network resource transmission using a delta compression technique between points of presence of a cloud proxy service, according to one embodiment; and 
         FIG. 16  illustrates an exemplary computer system which may be used in embodiments. 
     
    
    
     DETAILED DESCRIPTION 
     In the following description, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail in order not to obscure the understanding of this description. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation. 
     References in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described. 
     In the following description and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. “Coupled” is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other. “Connected” is used to indicate the establishment of communication between two or more elements that are coupled with each other. 
     A method and apparatus for reducing network resource transmission size using delta compression is described. The techniques described herein reduce the network resource transmission size between at least two endpoints between client devices and origin servers. A first endpoint, which is referred herein as a near end network optimizer, is coupled with client devices and may be incorporated into a proxy server that receives requests (e.g., HTTP requests) from client devices on behalf of the origin servers. The first endpoint is also coupled with a second endpoint, which is referred herein as a far end network optimizer. The far end network optimizer may incorporated into a server of a hosting company, which itself is coupled to an origin server. The far end network optimizer may alternatively be incorporated into the origin server. The far end network optimizer may be incorporated into a point of presence of a cloud proxy service that is closer to an origin server hosting a network resource. In some embodiments, the near end network optimizer is coupled with the far end network optimizer through a wide area network (WAN) such as the Internet. The near end network optimizer is referred to as a near end since it is closer to the client devices relative to the far end network optimizer, which is closer to the origin server relative to the client devices. 
     In one embodiment, when a near end network optimizer transmits a request for a network resource to a far end network optimizer, it also sends a version identifier of the latest version of that network resource that it has stored (this version is sometimes referred herein as the initial version). The far end network optimizer determines whether it has access to that version and if it has, the far end network optimizer retrieves the most recent version of the network resource (this version is sometimes referred herein as the current version and is typically different than the initial version but it can also be the same as the initial version), determines the differences between the versions (sometimes referred herein as the delta), and replies to the near end network optimizer with the differences between the versions (and not the entire network resource). The near end network optimizer applies the differences to its version of the network resource to generate the latest version of the resource, and transmits the latest version of the resource to the client device. 
     Since the difference between versions is transmitted to the near end network optimizer instead of the entire resource, the size of the response can be reduced (often substantially). Thus, bandwidth usage is reduced between the near end network optimizer and the far end network optimizer. In addition, performance is increased (in particular the amount of time that the resource is delivered to the client) as the near end network optimizer does not have to wait until the entire resource is returned. 
       FIG. 1  illustrates an exemplary system for reducing network resource transmission size using delta compression according to one embodiment.  FIG. 1  includes the client device  110  that is coupled with the near end network optimizer  120  which is itself coupled with the far end network optimizer  140 . The client device  110  is a computing device that is capable of accessing network resources (e.g., laptops, workstations, smartphones, palm tops, mobile phones, tablets, gaming systems, set-top boxes, etc.). The client device  110  includes a client network application (e.g., web browser, FTP client, SSH client, Telnet client, etc.) that is capable of establishing connections for the purpose of sending requests and receiving network resources. For example, a user of the client device  110  requests network resources (e.g., HTML pages, images, word processing documents, PDF files, movie files, music files, or other computer files) through the client network application. Although  FIG. 1  illustrates a single client device, it should be understood that typically there are many more client devices that are coupled with the near end network optimizer  120 . 
     In some embodiments, the near end network optimizer  120  is coupled with the far end network optimizer  140  through a wide area network (WAN) such as the Internet. In a specific example, the near end network optimizer  120  may be part of a proxy server that receives requests (e.g., HTTP requests) from client devices (including the client device  110 ) on behalf of one or more origin servers. By way of a specific example, the proxy server may receive network traffic destined for origin servers (e.g., HTTP requests) as a result of a Domain Name System (DNS) request for the domains of the origin servers resolving to the proxy server. The proxy server may include a cache for returning network resources, as well as providing other services (e.g., protecting against Internet-based threats (e.g., proactively stopping botnets, cleaning viruses, trojans, and worms, etc.), performance services (e.g., acting as a node in a CDN and dynamically caching customer&#39;s files closer to clients, page acceleration, etc.), and/or other services). 
     The far end network optimizer  140  may incorporated into a server of a hosting company, which itself is coupled to an origin server (not illustrated for purposes of convenience). The far end network optimizer may alternatively be incorporated into the origin server. 
     The near end network optimizer  120  includes the dynamic dictionary  124  and the far end network optimizer  140  includes the dynamic dictionary  144 . The dynamic dictionaries are used by the network optimizers to store version(s) of resources and optionally version identifier(s). For example, the dynamic dictionary  124  stores version(s) of resources  126  and optionally the version identifiers  128 , and the dynamic dictionary  144  stores version(s) of resources  146  and optionally the version identifiers  148 . In one embodiment the dynamic dictionaries  124  and/or  144  store only a single version of a resource; in other embodiments the dynamic dictionaries  124  and/or  144  are configured to store multiple versions of a resource.  FIG. 1  illustrates the logical structure of the dynamic dictionaries  124  and  144  and the structure may be different in different embodiments. In a specific example, the resources fields  126  and  146  include a URL portion and a resource portion. When determining whether a dynamic dictionary includes a requested resource, the URL of the requested resource may be used when accessing the dynamic dictionary. 
     In another embodiment, the dynamic dictionary  144  may store the differences between versions of a resource, in lieu of or in addition to the actual resource version. The stored differences may be linked or associated in order to be used to reconstruct the resource versions (e.g., the initial versions), which will be described in greater detail later herein. For example, the stored differences may be in a chain from oldest to newest or may be in another format (e.g., a tree structure). Storage space is reduced by storing the differences between versions of a resource instead of copies of the resource, with a trade-off for the time necessary for reconstruction. 
     In some embodiments the dynamic dictionaries are built dynamically as requests for resources are received and retrieved. By way of example, the client device  110  transmits a request  160  for a resource (e.g., an HTTP request), which is received by the request module  122  of the near end network optimizer  120 . The request may be received at the request module  122  as a result of a cache mechanism of the proxy server determining that the requested resource cannot be returned to the requesting client device (e.g., the resource has been set as “no-cache”, the cache has expired, etc.). 
     The request module  122  accesses the dynamic dictionary  124  to determine whether it includes a copy of the requested resource. If it does not, then the request module  122  causes a request to be transmitted to the far end network optimizer  140  for the requested resource. The far end network optimizer  140  retrieves the requested resource (e.g., the far end network optimizer  140  makes a request to the origin server for the requested resource, or if locally available, the far end network optimizer  140  accesses the local version), updates its dynamic dictionary  144  to include the latest version of the resource, and replies to the near end network optimizer  120  with the requested resource. Updating its dynamic dictionary  144  may include storing a version identifier, which is a value that identifies a version of a network resource, and associating the version identifier with the resource version. By way of a specific example, the version identifier generator module  155  may generate the version identifier by hashing the contents of the resource (e.g., using a suitable hashing algorithm such as Secure Hash Algorithm (SHA)), with the results being the version identifier. As another example, the last-modified header may be used as the version identifier. As yet another example, a value may be chosen and included in the ETag header field. The response module  150  of the far end network optimizer  140  causes a response (e.g., an HTTP response) that includes the requested resource to be transmitted to the near end network optimizer  120 , which is received by the response module  130 . The response may, in some embodiments, include the version identifier. The response may include a no-cache directive. 
     The response module  130  causes the resource to be stored in the dynamic dictionary  128 , along with the version identifier if one was provided in the response from the far end network optimizer  140 . In some embodiments, the response module  130  causes the resource version to be stored in the dynamic dictionary  124  even if it has a no-cache directive set. Thus, by way of example, a version of a news website, which changes frequently, is stored in the dynamic dictionary  124  even if the no-cache directive is set. 
     In some embodiments, if the version identifier was not included in the response from the far end network optimizer  140 , the version identifier generator module  135  generates a version identifier to store in the dynamic dictionary  124  (e.g., it hashes the resource and stores the resulting value as the version identifier). 
     After receiving a request for a resource that is the dynamic dictionary  124 , the near end network optimizer  120  transmits a request  162  for that resource that includes the version identifier of the last known version to the far end network optimizer  140 . The version identifier may be previously stored and associated with the resource, or in some embodiments, the version identifier is generated by the version identifier generator module  135  (e.g., the resource is hashed). The request may also indicate (e.g., in a header) that it is requesting the delta between the version identified in the version identifier and the latest version of the resource. 
     The request module  142 , upon receiving the request  162 , accesses its dynamic dictionary  144  to determine whether it has access to the version of the resource identified by the version identifier included in the request  162 . In one embodiment, the request module  142  determines has access to the version of the resource if a same copy of that resource is stored in the dynamic dictionary  144 . In another embodiment, the request module  142  has access to the identified resource version if that version of the resource can be reconstructed using the stored differences between versions of a resource. If the request module  142  does not have access to that version, then the far end network optimizer  140  retrieves the requested resource (e.g., the far end network optimizer  140  makes a request to the origin server for the requested resource, or if locally available, the far end network optimizer  140  accesses the local version), updates its dynamic dictionary  144  to include the latest version of the resource, and replies to the near end network optimizer  120  with the requested resource, which may be compressed using a suitable non-delta compression algorithm such as gzip. 
     If the request module  142  has access to the version (e.g., the dynamic dictionary  144  of the far end network optimizer  140  includes the same version of the resource identified by the version identifier included in the request  162  or that version can be reconstructed using the stored differences between versions of that resource), the far end network optimizer  140  retrieves the requested resource (e.g., the far end network optimizer  140  makes a request to the origin server for the requested resource, or if locally available, the far end network optimizer  140  accesses the local version), updates its dynamic dictionary  144  to include the latest version of the resource, and determines the differences between the versions. 
     For example, the difference determination module  152  determines the differences between the versions. A number of algorithms may be used to determine the differences between the versions, including Xdelta3, VCDIFF (described in RFC 3284, “The VCDIFF Generic Differencing and Compression Data Format”, June 2002), Bentley/McIlroy compression, bsdiff, etc. 
     In embodiments where the differences between versions of a resource are stored, the difference determination module  152  may traverse the differences to reconstruct the initial version (the version identified in the request  162 ) in order to determine the differences between the initial version and the current version. By doing so, storage space may be reduced as the versions of the resource may not need to be stored, with a trade off for the time necessary for reconstructing. 
     The response module  150  causes a response  164  (e.g., an HTTP response) to be transmitted to the near end network optimizer  120 , the response  164  including the differences between the versions (between the initial version and the current version). The response  164  does not contain the entire resource. In one embodiment, a header in the response  164  indicates that it contains the differences between versions (not the entire resource). The header may also indicate the algorithm used to determine the differences. In one embodiment, the far end network optimizer  140  also transmits the version identifier to the near end network optimizer  120 . 
     In one embodiment, the far end network optimizer  140  also compresses the response using a suitable compression algorithm. For example, the response may be compressed using gzip. As another example, the response may be compressed using zlib compression that is pre-warmed using the version of the resource identified in the request  162 . Pre-warming allows the compression algorithm to understand more about the data that is being compressed and therefore typically provides a more accurate and faster compression. The far end network optimizer  140  may also perform pre-warmed compression (e.g., zlib compression) of the HTTP headers of the response  164  based on the version of the resource in the dynamic dictionary  144  and identified in the request  162 . 
     The response module  130  of the near end network optimizer  120  receives the response  164 . The difference application module  132  of the response module  130  applies the differences indicated in the response  164  to the resource version included in the dynamic dictionary  124  to generate the latest version of the resource. The response module  130  causes a response  166  (e.g., an HTTP response) with the requested resource to be transmitted to the client device  110 . The response module  130  also causes the updated version to be stored in the dynamic dictionary  124 , and optionally a version identifier associated with the updated version. 
       FIG. 2  illustrates the exemplary system of  FIG. 1  in more detail according to one embodiment. The near end network optimizer  120  receives a request  210  for a resource A. The resource A may be a HyperText Markup Language (HTML) file, a text document, or any other type of resource. The near end network optimizer  120  determines that is has a version of the resource X in its dynamic dictionary  124 . As illustrated in  FIG. 2 , the dynamic dictionary  124  includes the resource X, which is associated with the version identifier  7 . The near end network optimizer  120  causes a request  220  for resource X to be transmitted to the far end network optimizer  140 , the request including the version identifier  7 . 
     The far end network optimizer  140  determines that it has access to the same version (version 7) identified by the version identifier included in the request  220  for the resource X in its dynamic dictionary  144 . As illustrated in  FIG. 2 , version 7 of the resource is stored in the dynamic dictionary  144 . The far end network optimizer  140  retrieves the latest version of the resource (referred to in  FIG. 2  as version 8) (e.g., the far end network optimizer  140  makes a request to the origin server for the requested resource, or if locally available, the far end network optimizer  140  accesses the local version). The difference determination module  152  determines the differences between version 7 and version 8 of the resource X. The far end network optimizer  140  causes a response  225  to be transmitted to the near end network optimizer  120  that identifies the differences between version 7 and version 8. The differences file that is transmitted to the near end network optimizer  120  is such that the near end network optimizer  120  can apply those differences to its latest version of the resource (version 7) to generate the current version (version 8). The far end network optimizer  140  may also include a version identifier. The far end network optimizer  140  updates its dynamic dictionary  144  with the latest version of resource X (version 8). 
     Responsive to receiving the differences, the difference application module  132  applies the differences to the version 7 of the resource X, to generate the version 8 of the resource X. The differences may indicate to delete content, add content, move content, etc. After generating the latest version of the resource X, the near end network optimizer  120  causes a response  230  to be transmitted to the requesting client with the version 8 of the resource X. The near end network optimizer  120  also updates its dynamic dictionary  124  with the updated version of the resource X (version 8). 
       FIG. 3  illustrates a sequence of exemplary operations performed when a network resource is not in the dynamic dictionary of the near end network optimizer according to one embodiment. At operation  310 , the near end network optimizer  120  receives a request for a resource. Next, at operation  315 , the near end network optimizer  120  determines that the requested resource is not in its dynamic dictionary  124 . As a result, at operation  320 , the near end network optimizer  120  transmits a request for the resource  320  to the far end network optimizer  140 . 
     The far end network optimizer  140  receives the request, and retrieves the most current version of the resource at operation  325 . For example, the far end network optimizer  140  transmits a request for the resource to the origin server, or if locally available, the far end network optimizer  140  accesses the local version of the resource. 
     Next, the far end network optimizer  140  may generate a version identifier for the retrieved resource at operation  330 . For example, the far end network optimizer  140  may hash the resource to generate a value that is used for the version identifier. In other embodiments, the far end network optimizer  140  does not generate a version identifier. For example, the value of the version identifier may be set by the origin server (e.g., the value of the last-modified header, the value of the ETag header field). The far end network optimizer  140  stores the version of the resource in its dynamic dictionary  144  at operation  335 , and may associate it with the version identifier. In one embodiment, the far end network optimizer  140  may determine the differences between the current version (retrieved at operation  325 ) and the most immediately previous version (if one exists), store those differences in the dynamic dictionary  144 , and may remove older versions of the resource from the dynamic dictionary  144 . 
     The far end network optimizer  140 , at operation  340 , transmits a response to the near end network optimizer  120  that includes the requested resource and optionally includes a version identifier for that resource. While  FIG. 3  illustrates the operation  340  being performed after the operations  330  and  335 , the operations  330  and/or  335  may be performed after operation  340  in some embodiments. 
     The near end network optimizer  120  receives the response that includes the requested resource and transmits a response to the client device  110  with the requested resource at operation  345 . The near end network optimizer  120  stores the requested resource in its dynamic dictionary  124  at operation  350 . The near end network optimizer  120  may also generate a resource version identifier for the resource at operation  355 , and may store the resource version identifier at operation  360 . 
       FIG. 4  illustrates a sequence of exemplary operations performed when a network resource is included in the dynamic dictionary of the near end network optimizer and the same version is not included in the dynamic dictionary of the far end network optimizer according to one embodiment. 
     At operation  410 , the near end network optimizer  120  receives a request for a resource. Next, at operation  415 , the near end network optimizer  120  determines that the requested resource is in its dynamic dictionary  124 . As a result, at operation  420 , the near end network optimizer  120  transmits a request for the resource to the far end network optimizer  140 , where the request identifies the version identifier of the most recent version of the resource that is stored in the dynamic dictionary  124  of the near end network optimizer  120 . 
     The far end network optimizer  140  receives the request and determines, in operation  425 , that it does not have access to the version of the resource identified in the received request in. For example, its dynamic dictionary  144  does not include the same version of the requested resource and that version cannot be reconstructed. Since the far end network optimizer  140  does not have access to the version of the resource identified in the received request, it will not be able to determine the differences between the version in the dynamic dictionary  124  of the near end network optimizer  120  and the latest version of the resource. At operation  430 , the far end network optimizer retrieves the most current version of the resource. For example, the far end network optimizer  140  transmits a request for the resource to the origin server, or if locally available, the far end network optimizer  140  accesses the local version of the resource. 
     Next, the far end network optimizer  140  may generate a version identifier for the retrieved resource at operation  435 . For example, the far end network optimizer  140  may hash the resource to generate a value that is used for the version identifier. In other embodiments, the far end network optimizer  140  does not generate a version identifier. For example, the value of the version identifier may be set by the origin server (e.g., the value of the last-modified header, the value of the ETag header field). The far end network optimizer  140  stores the version of the resource in its dynamic dictionary  144  at operation  340 , and may associate it with the version identifier. In one embodiment, the far end network optimizer  140  may determine the differences between the retrieved version (retrieved at operation  430 ) and the most immediately previous version (if one exists), store those differences in the dynamic dictionary  144 , and may remove older versions of the resource from the dynamic dictionary  144 . 
     The far end network optimizer  140 , at operation  445 , transmits a response to the near end network optimizer  120  that includes the requested resource and optionally includes a version identifier for that resource. While  FIG. 4  illustrates the operation  445  being performed after the operations  435  and  440 , the operations  435  and/or  440  may be performed after operation  445  in some embodiments. 
     The near end network optimizer  120  receives the response that includes the requested resource and transmits a response to the client device  110  with the requested resource at operation  450 . The near end network optimizer  120  stores the requested resource in its dynamic dictionary  124  at operation  455 . The near end network optimizer  120  may also generate a resource version identifier for the resource at operation  460  and may store the resource version identifier at operation  465 . 
       FIG. 5  illustrates a sequence of exemplary operations performed when a network resource is included in the dynamic dictionary of the near end network optimizer and the same version is included in the dynamic dictionary of the far end network optimizer according to one embodiment. 
     At operation  510 , the near end network optimizer  120  receives a request for a resource. Next, at operation  515 , the near end network optimizer  120  determines that the requested resource is in its dynamic dictionary  124 . As a result, at operation  520 , the near end network optimizer  120  transmits a request for the resource to the far end network optimizer  140 , where the request identifies the version identifier of the most recent version of the resource that is stored in the dynamic dictionary  124  of the near end network optimizer  120 . 
     The far end network optimizer  140  receives the request and determines, in operation  525 , that it has access to the version of the resource identified in the request of operation  520 . For example, the dynamic dictionary  144  includes the same version or that version can be reconstructed using stored version differences. This version of the resource may or may not be the most current version of the resource. Therefore, at operation  530  the far end network optimizer retrieves the most current version of the resource. For example, the far end network optimizer  140  transmits a request for the resource to the origin server, or if locally available, the far end network optimizer  140  accesses the local version of the resource. 
     The far end network optimizer  140  determines the differences between the versions of the resource at operation  535 . For example, the Xdelta3, VDDIFF, Bentley/McIlroy compression, bsdiff, or other algorithm that can be used to determine the differences between the versions of the resource, is used to determine the differences. In a situation where the version identified in the request of operation  520  (the initial version) is not stored but a chain or tree of differences is, the far end network optimizer  140  may traverse the chain or tree of differences to reconstruct the initial version and use that reconstructed version when determining the differences between the initial version and the current version. 
     Next, the far end network optimizer  140  may generate a version identifier for the retrieved resource at operation  540 . For example, the far end network optimizer  140  may hash the resource to generate a value that is used for the version identifier. In other embodiments, the far end network optimizer  140  does not generate a version identifier. For example, the value of the version identifier may be set by the origin server (e.g., the value of the last-modified header, the value of the ETag header field). The far end network optimizer  140  stores the version of the resource in its dynamic dictionary  144  at operation  545 , and may associate it with the version identifier. In one embodiment, the far end network optimizer  140  may determine the differences between the current version (retrieved at operation  530 ) and the most immediately previous version, store those differences in the dynamic dictionary  144 , and may remove the older versions of the resource from the dynamic dictionary  144 . 
     The far end network optimizer  140 , at operation  550 , transmits a response to the near end network optimizer  120  that includes the differences between the versions and does not include the complete network resource. The response may also include a version identifier associated with the network resource (that is to identify the version of the resource once updated by the near end network optimizer  120 ). The response may also indicate the algorithm used to generate the differences. In one embodiment, if there is not a difference between versions of the resource, the far end network optimizer  140  transmits a NULL difference or other message to the near end network optimizer  120  that indicates that there are no differences. While  FIG. 5  illustrates the operation  550  being performed after the operations  540  and  545 , the operations  540  and/or  545  may be performed after operation  550  in some embodiments. 
     The near end network optimizer  120  receives the response, and applies the differences to its last known version to generate a current version of the network resource at operation  555 . At operation  560 , the near end network optimizer  120  transmits a response to the client device  110  with the requested resource (the updated resource). The near end network optimizer  120  stores the updated resource in its dynamic dictionary  124  at operation  565 . The near end network optimizer  120  may also generate a resource version identifier for the resource at operation  560  (e.g., if one was not included in the response  550 ) and may store the resource version identifier at operation  575 . 
       FIG. 6  is a flow diagram illustrating exemplary operations performed for a delta compression technique for compressing the payload size of network resources according to one embodiment. The operations of this and other flow diagrams will be described with reference to the exemplary embodiments of  FIG. 1 . However, it should be understood that the operations of the flow diagrams can be performed by embodiments of the invention other than those discussed with reference to  FIG. 1 , and the embodiments of the invention discussed with reference to  FIG. 1  can perform operations different than those discussed with reference to the flow diagrams. 
     At operation  610 , the near end network optimizer  120  receives a request for a network resource. By way of example, the request is an HTTP request sent from the client device  110  and the resource is identified at a Uniform Resource Locator (URL) included in the request. Flow then moves to operation  615  where the near end network optimizer  120  determines whether the resource is included in the dynamic dictionary  124 . For example, the request module  122  accesses the dynamic dictionary  124  to determine whether the URL for the requested resource is included and whether there is a corresponding version of the resource. If there is not, then flow moves to operation  710 , which will be described with reference to  FIG. 7 . If there is a version of the resource in the dynamic dictionary of the near end network optimizer  124 , then flow moves to operation  635 . 
     At operation  635 , the near end network optimizer  120  transmits a request for the resource to the far end network optimizer  124 . The request includes a version identifier for the requested resource. In one embodiment, the version identifier that is included in the request is the version identifier that is associated with the latest version of the resource that is included in the dynamic dictionary  124 . By way of a specific example, the request may be an HTTP request, and the request may also indicate (e.g., in a header) that it is for the delta between the version identified through the included version identifier and the latest version of the resource. Flow then moves to operation  640 . 
     At operation  640 , the far end network optimizer  140  receives the request and determines whether it has access to the identified version of the requested resource. In one embodiment, the far end network optimizer  140  has access to the version of the resource if a same copy of that resource is stored in the dynamic dictionary  144 . In another embodiment, the request module  142  has access to the identified resource version if that version of the resource can be reconstructed using the stored differences between versions of a resource. For example, the request module  142  searches for the URL of the requested resource in the dynamic dictionary  144 . If found, the far end network optimizer  140  compares the version identifier included in the request with the version identifier(s) stored for the resource (if version identifier(s) are stored locally) or stored for the version differences. In embodiments where the version identifier(s) are not stored, a version identifier may be generated by the version identifier generation module  155  and compared with the version identifier included in the request. If the version is not accessible, then flow moves to operation  715 , which will be described with reference to  FIG. 7 . However, if the identified version of the resource is accessible, then flow moves to operation  645 . 
     At operation  645 , the far end network optimizer  140  retrieves the most current version of the requested resource. In some implementations the far end network optimizer  140  is located in a different geographical location than the origin server, where the resource originally resides or is created (the far end network optimizer  140  may be part of the same Local Area Network (LAN) as the origin server, or may be located across a WAN). In these implementations, the far end network optimizer  140  transmits a request towards the origin server (e.g., an HTTP request) requesting the resource (with an expectation that the origin server will reply with the most current version of the resource). In other implementations, the far end network optimizer  140  is part of a system that has local access to the network resources (e.g., it is implemented in conjunction with the origin server, it is being provided with resources as they are being updated/created by the origin server, etc.). In these implementations, the far end network optimizer  140  accesses the latest version through local methods. Flow moves from operation  645  to operation  650 . 
     At operation  650 , the far end network optimizer  140  determines the difference between the versions. For example, the difference determination module  152  applies an algorithm such as Xdelta3, VCDIFF, Bentley/McIlroy compression, bsdiff, or other algorithm suitable for determining the differences between versions of files, to determine the differences (if any) between the versions. In a situation where the version identified in the request of operation  635  (the initial version) is not stored but a chain or tree of differences is, the far end network optimizer  140  may traverse the chain or tree of differences to reconstruct the initial version and use that reconstructed version when determining the differences between the initial version and the current version. 
     Flow then moves to operation  655  where the far end network optimizer  140  transmits the difference to the near end network optimizer  120  (and not the complete resource). For example, the response module  150  formats a response (e.g., an HTTP response) that includes a file containing the differences between the versions. If there is not any difference, then the response includes a NULL difference or otherwise indicates that the version stored in the dynamic dictionary  124  of the near end network optimizer is the most current version. The response may also include a header that indicates that it contains the differences between versions, and not the entire resource. Also, the response may indicate the algorithm used to determine the differences. In one embodiment, the response is also compressed using gzip or other suitable compression techniques. For example, the in some embodiments the far end network optimizer  140  performs pre-warmed compression (e.g., zlib compression) of the response and/or the HTTP headers of the response (based on the version of the resource identified in the request from the near end network optimizer). 
     The response may also include a version identifier associated with the network resource that is to identify the version of the resource once updated by the near end network optimizer  140 . The version identifier may be generated by the far end network optimizer  140  (e.g., a hash of the resource version). As another example, the version identifier may be set by the origin server (e.g., the value of the last-modified header, the value in an ETag header field, a hash of the resource, etc.). Flow then moves to operation  660 , where the far end network optimizer  140  stores the resource in its dynamic dictionary  144  and optionally associates it with the version identifier. In one embodiment, the far end network optimizer  140  may determine the differences between the current version (retrieved at operation  645 ) and the most immediately previous version, store those differences in the dynamic dictionary  144 , and may remove the older versions of the resource from the dynamic dictionary  144 . Flow moves from operation  660  to operation  665 . 
     At operation  665 , the near end network optimizer  120  applies the differences specified in the differences file received from the far end network optimizer  140  to the version of the network resource in its dynamic dictionary  124  to generate an updated version of the network resource. For example, the difference file may specify to add content, delete content, move content, etc. 
     Flow then moves to operation  670  where the near end network optimizer  120  transmits the latest version (the updated version of the requested resource) to the requesting device (e.g., the client device  110 ). Flow then moves to operation  675  where the near end network optimizer stores the latest version in its dynamic dictionary  124  and optionally stores a version identifier for that version in its dynamic dictionary  124  (which it may generate, depending on whether the response from the far end network optimizer included the version identifier). 
       FIG. 7  is a flow diagram illustrating exemplary operations performed when the requested resource is not in the dynamic dictionary of the near end network optimizer and/or the far end network optimizer according to one embodiment. When the requested resource is not in the dynamic dictionary  124  of the near end network optimizer  120 , the near end network optimizer  120  transmits a request (e.g., an HTTP request) for the requested resource to the far end network optimizer  140  at operation  710 . Flow then moves to operation  715  and the far end network optimizer  140  retrieves the latest version of the resource as previously described. Next, at operation  720 , the far end network optimizer  140  transmits the requested resource to the near end network optimizer  120 , the response optionally including a version identifier of the requested resource. Flow then moves to operation  725  where the far end network optimizer  140  stores the resource in its dynamic dictionary  144  (or updates its dynamic dictionary) and optionally stores a version identifier for that version of the resource. In one embodiment, the far end network optimizer  140  may determine the differences between the current version and the most immediately previous version (if one exists), store those differences in the dynamic dictionary  144 , and may remove older versions of the resource from the dynamic dictionary  144 . 
     Flow then moves to operation  730 , where the near end network optimizer  120  transmits the resource to the requesting device. Next, flow moves to operation  735  where the near end network optimizer  120  stores the resource in its dynamic dictionary  124  and optionally stores a version identifier for that version of the resource. 
     Since the difference between versions is transmitted to the near end network optimizer instead of the entire resource, the size of the response can be reduced (often substantially). Thus, bandwidth usage is reduced between the near end network optimizer and the far end network optimizer. In addition, performance is increased (in particular the amount of time that the resource is delivered to the client) as the near end network optimizer does not have to wait until the entire resource is returned. 
     In addition, the technique described herein can be used for resources that have been designated as being un-cacheable. As an example, origin servers may include a “no-cache” directive in the Cache-Control header field of an HTTP response that indicates that a cached version of the resource is not to be used to respond to a request without successful revalidation with the origin server. Previously when the no-cache directive was used, proxy servers checked with the origin server to determine whether the resource has been updated and if it has, the proxy server returned with the entire resource, even if there was a minimal change. However, with the techniques described herein, the near end network optimizer makes a request for the differences (if any) between its version and the most recent version and if there is a difference, the far end network optimizer returns the differences and not the entire resource. 
     Also, at least some of the techniques described herein do not have to be supported by the clients. Thus, in some embodiments, client software (e.g., client network applications) does not have to be updated to support the techniques described herein. In addition, unlike the delta encoding proposal in RFC 3229 which requires coordination between the client devices and the origin servers and would require the origin servers to store many versions of the resources (e.g., potentially up to one version for each different client), the techniques described in some embodiments herein allow the far end network optimizer to store only a single version of each resource. In other embodiments described herein, the differences between versions of a resource are stored in lieu of the actual versions, which reduces the amount of storage space required. 
     While embodiments have been described with respect to the near end network optimizer being incorporated in a device that is coupled to a client computing device (e.g., a proxy server), in other embodiments, similar functionality described herein with reference to the near end network optimizer may be incorporated into the client computing device 
     For example,  FIG. 8  illustrates an embodiment where the client network application  815  of the client device  810  includes the near end network optimizer  820  that performs similar operations as the near end network optimizers previously described herein. For example, the near end network optimizer  820  includes the dynamic dictionary  822 , which is similar to the dynamic dictionary  124  and stores version(s) of resources and optionally version identifier(s). The near end network optimizer  820  also includes the difference application module  824 , which operates in a similar way as the difference application module  132 . 
     The proxy server  830  includes the far end network optimizer  840 , which operates in a similar way as the far end network optimizer  140 . For example, the far end network optimizer  840  includes the dynamic dictionary  842 , which is similar to the dynamic dictionary  144  and stores version(s) of resources and optionally version identifier(s). The far end network optimizer  840  also includes the difference determination module  844 , which operates in a similar way as the difference determination module  152 . 
     The client device  810  transmits requests for network resources of the origin server  850  to the proxy server  830  and receives responses from the proxy server  830 . In one embodiment, requests (e.g., HTTP requests) are received at the proxy server  830  as a result of a DNS request for the domain of the origin server  850  resolving to the proxy server  830 . In some embodiments, the authoritative name server of the domain of the origin server is changed and/or individual DNS records are changed to point to the proxy server (or point to other domain(s) that point to a proxy server of the service). For example, owners and/or operators of the domain of the origin server  850  may change their DNS using a CNAME record that points to the proxy server  830 . While  FIG. 8  illustrates a single proxy server  830 , in some embodiments the service has multiple proxy servers that are geographically distributed. For example, in some embodiments, there are multiple point of presences (PoPs). A PoP is a collection of networking equipment (e.g., authoritative name servers and proxy servers) that are geographically distributed to decrease the distance between requesting client devices and content. The authoritative name servers have the same anycast IP address and the proxy servers have the same anycast IP address. As a result, when a DNS request is made, the network transmits the DNS request to the closest authoritative name server. That authoritative name server then responds with a proxy server within that PoP. Accordingly, a visitor will be bound to that proxy server until the next DNS resolution for the requested domain (according to the TTL (time to live) value as provided by the authoritative name server). In some embodiments, instead of using an anycast mechanism, embodiments use a geographical load balancer to route traffic to the nearest PoP. 
     The near end network optimizer  820  dynamically builds the dynamic dictionary  822  in a similar way as previously described herein, according to one embodiment, and is capable of receiving the differences of network resources and applying those differences to previously stored files. 
     For example, upon determining to transmit a request for a network resource that is stored in the dynamic dictionary  822  (e.g., as a result of the network resource being set as “no-cache”, the cached version of the network resource in the cache of the client network application  815  being expired, etc.), the client network application  815  transmits a request  860  for the resource, along with the version identifier of the last known version of that resource to the proxy server  830 . 
     The far end network optimizer  840  receives the request and determines whether it has access to the identified version of the network resource. Assuming that it does, the far end network optimizer  840  retrieves the most current version of the resource. As illustrated in  FIG. 8 , the proxy server  830  transmits the request  865  to the origin server  850  for the requested resource in order to receive the most current version of the resource. The proxy server  830  receives the response  870 , which includes the requested resource. 
     The difference determination module  844  receives the requested resource and determines the differences between the versions of the resource as previously described. The far end network optimizer  840  causes the response  875  to be transmitted to the client network application  815  that identifies the differences between the versions. The response  875  does not include the entire network resource. The response  875  may also include a version identifier in some embodiments. The far end network optimizer  840  updates its dynamic dictionary  842  with the most current version of the resource and may store the differences in the dynamic dictionary  842  and remove old versions of the resource. 
     Responsive to receiving the differences, the difference application module  824  applies the differences to its previous version to generate the most current version of the resource. The most current version may then be displayed by the client network application  815 . The near end network optimizer  820  also updates its dynamic dictionary  822  with the updated version of the resource. 
       FIG. 9  illustrates another embodiment where the client network application  915  of the client device  910  includes the near end network optimizer  920  that performs similar operations as the near end network optimizers previously described herein. For example, the near end network optimizer  920  includes the dynamic dictionary  922 , which is similar to the dynamic dictionary  124  and stores version(s) of resources and optionally version identifier(s). The near end network optimizer  920  also includes the difference application module  924 , which operates in a similar way as the difference application module  132 . 
     Unlike  FIG. 8 ,  FIG. 9  illustrates a proxy server  930  that includes functionality of a far end network optimizer (with respect to the client device  910 ) and of a near end network optimizer (with respect to the hosting provider server  935 ). The proxy server  930  includes the dynamic dictionary  942 , which is similar to the dynamic dictionary  144  and stores version(s) of resources and optionally version identifier(s). 
     The client device  910  transmits requests for network resources of the origin server  950  to the proxy server  930  and receives responses from the proxy server  930 . In one embodiment, requests (e.g., HTTP requests) are received at the proxy server  930  as a result of a DNS request for the domain of the origin server  950  resolving to the proxy server  930  in a similar was as described with reference to  FIG. 8 . For example, upon determining to transmit a request for a network resource that is stored in the dynamic dictionary  922  (e.g., as a result of the network resource being set as “no-cache”, the cached version of the network resource in the cache of the client network application  915  being expired, etc.), the client network application  915  transmits a request  960  for the resource, along with the version identifier of the last known version of that resource to the proxy server  930 . 
     The network optimizer  940  receives the request and determines whether it has access to the identified version of the network resource. Assuming that it does, the network optimizer  940  retrieves the most current version of the resource. As illustrated in  FIG. 9 , the proxy server  930  transmits the request  965  to the hosting provider server  935 , along with a version identifier of the resource. In one embodiment, the version identifier included in the request  965  is the same version identifier as included in the request  960 . In other embodiments, the version identifier included in the request  965  is the version identifier of the most recent network resource stored in the dynamic dictionary  942 , which may or may not correspond with the version identifier included in the request  960 . 
     The far end network optimizer  952  receives the request and determines whether it has access to the identified version of the network resource (as identified by the version identifier in the request  965 ). Assuming that it does, the far end network optimizer  952  retrieves the most current version of the resource. As illustrated in  FIG. 9 , the hosting provider server  935  transmits the request  970  to the origin server  950  for the requested resource in order to retrieve the most current version of the resource. The hosting provider server  935  receives the response  975 , which includes the requested resource. 
     The difference determination module  956  of the far end network optimizer  952  receives the requested resource and determines the differences between the versions of the resource as previously described. The far end network optimizer  952  causes the response  980  to be transmitted to the proxy server  930  that identifies the differences between the versions. The response  980  does not include the entire network resource. The response  980  may also include a version identifier in some embodiments. The far end network optimizer  952  updates its dynamic dictionary  956  with the most current version of the resource and may store the differences in the dynamic dictionary  956  and remove old versions of the resource. 
     Responsive to receiving the differences, the difference application module  946  applies the difference to its previous version (identified in the request  965 ) to generate the most current version of the resource. The difference determination module  944  may also determine the difference between the most current version of the resource, and the version identified in the request  960  if the version identified in the request  960  is different than that of the version identified in the request  965 . The network optimizer  940  causes the response  985  to be transmitted to the client network application  915  that identifies the differences between the latest version of the resource and the version identified in the request  960 . The response  985  may also include a version identifier in some embodiments. The network optimizer  940  updates its dynamic dictionary  942  with the most current version of the resource and may store the differences in the dynamic dictionary  942  and remove old versions of the resource. 
     Responsive to receiving the differences, the difference application module  924  applies the differences to its previous version to generate the most current version of the resource. The most current version may then be displayed by the client network application  915 . The near end network optimizer  920  also updates its dynamic dictionary  922  with the updated version of the resource. 
     Reducing Network Resource Transmission Size Using Delta Compression Between Points of Presence (PoPs) of a Cloud Proxy Service: 
     A method and apparatus for reducing network resource transmission size using delta compression between points of presence of a cloud proxy service is described. The techniques described herein reduce the network resource transmission size between at least two endpoints between client devices and origin servers. A first endpoint, which is referred herein as a near end point of presence (PoP) of a cloud proxy service is coupled with client devices. The near end PoP receives requests (e.g., HTTP requests) from client devices on behalf of the origin servers and is coupled with a second endpoint, which is referred herein as a far end PoP. The far end PoP is coupled to an origin server hosting network resources. In some embodiments, the two end points (i.e., the near end PoP and the far end PoP) are coupled through a wide area network (WAN) such as the Internet. The near end PoP is referred to as a near end since it is closer to the client devices relative to the far end PoP that is closer to the origin server relative to the client devices. 
     In one embodiment, upon receipt of a request for a resource from a client device, the near end PoP identifies a far end PoP from a plurality of PoPs of the cloud proxy service, and determines whether to perform a delta compression technique for reducing network resource transmission size between the near end PoP and the far end PoP. Upon determining that the delta compression technique is to be performed, the near end PoP transmits a request for the network resource to a far end PoP. In one embodiments, when the near end PoP transmits the requests, it also sends a version identifier of the latest version of that network resource that it has stored (this version is sometimes referred herein below as the initial version). The far end PoP determines whether it has access to that version and if it has, the far end PoP retrieves the most recent version of the network resource (this version is sometimes referred herein as the current version and is typically different than the initial version but it can also be the same as the initial version), determines the differences between the versions (sometimes referred herein as the delta), and replies to the near end PoP with the differences between the versions (and not the entire network resource). The near end PoP applies the differences to its version of the network resource to generate the latest version of the resource, and transmits the latest version of the resource to the client device. 
     Since the difference between versions is transmitted to the near end PoP instead of the entire resource, the size of the response can be reduced (often substantially). Thus, bandwidth usage is reduced between the near end PoP and the far end PoP. In addition, performance is increased (in particular the amount of time that the resource is delivered to the client) as the near end PoP does not have to wait until the entire resource is returned. Further the identification of the far end PoP may be performed based on respective response latency measures between each one of a plurality of far end PoPs and an origin server hosting the network resource, consequently increasing the performance of the system by reducing the response time for a request. Thus, the embodiments described herein provides reduction of bandwidth usage between a client device and an origin server by enabling delta compression techniques between points of presence of a cloud proxy service. In some embodiments, this is performed without any burden or additional computation resources at the client device and/or origin server, while providing reduced bandwidth usage and improved performance in response times between the two devices. 
       FIG. 10  illustrates an embodiment where a first point of presence of a cloud proxy service includes a near end network optimizer and a second point of presence of the cloud proxy server includes a far end network optimizer for enabling reduction of network resource transmission size using delta compression according to one embodiment.  FIG. 10  includes the client device  1010  that is coupled with a near end PoP  1030 . The near end PoP is coupled with a far end PoP  1035 . The client device  1010  is a computing device that is capable of accessing network resources (e.g., laptops, workstations, smartphones, palm tops, mobile phones, tablets, gaming systems, set-top boxes, etc.). The client device  1010  includes a client network application (e.g., web browser, FTP client, SSH client, Telnet client, etc.) that is capable of establishing connections for the purpose of sending requests and receiving network resources. For example, a user of the client device  1010  requests network resources (e.g., HTML pages, images, word processing documents, PDF files, movie files, music files, or other computer files) through the client network application. Although  FIG. 10  illustrates a single client device, it should be understood that typically there are many more client devices that are coupled with the near end PoP  1030 . 
     The near end PoP and the far end PoP are part of a cloud proxy service which provides a set of one or more services for the benefit of customers upon their registration for the service. The near end PoP  1030  receives requests (e.g., HTTP requests) from client devices (including the client device  1010 ) on behalf of one or more origin servers. By way of a specific example, the near end PoP  1030  may receive network traffic destined for origin servers (e.g., HTTP requests) as a result of a Domain Name System (DNS) request for the domains of the origin servers resolving to a server of the PoP. The near end PoP may include a cache for returning network resources, as well as providing other services (e.g., protecting against Internet-based threats (e.g., proactively stopping botnets, cleaning viruses, trojans, and worms, etc.), performance services (e.g., acting as a node in a CDN and dynamically caching customer&#39;s files closer to clients, page acceleration, etc.), and/or other services). A PoP (e.g., near end PoP or far end PoP) is a collection of networking equipment (e.g., authoritative name servers and proxy servers) that are geographically distributed to decrease the distance between requesting client devices and content. In some embodiments, the authoritative name servers have the same anycast IP address and the servers have the same anycast IP address. As a result, when a DNS request is made, the network transmits the DNS request to the closest authoritative name server. That authoritative name server then responds with a server within that PoP. Accordingly, a visitor will be bound to that server (e.g., client device  1010  may be bound to a server of the near end PoP  1030 ) until the next DNS resolution for the requested domain (according to the TTL (time to live) value as provided by the authoritative name server). In some embodiments, instead of using an anycast mechanism, embodiments use a geographical load balancer to route traffic to the nearest PoP. In some embodiments, a hybrid anycast/unicast mechanism may be used, such that the DNS server would have a list of unicast networks, with their destination PoP. If the network is not on the unicast list, the DNS responds to the DNS request with standard anycast addresses. In other embodiments, a region is first selected then a PoP within that region is identified using an anycast mechanism. 
     The near end PoP  1030  is coupled with the far end PoP  1035  through a wide area network (WAN) such as the Internet. The far end PoP  1035  is coupled to an origin server  1050  that hosts network resources. Although  FIG. 10  illustrates a single origin server, it should be understood that typically there are many more origin servers that are coupled with the far end PoP  1035 . The far end PoP  1035  may alternatively be coupled to a hosting provider server that is communicatively coupled with the origin server  1050 . The near end PoP is referred to as a near end since it is closer to the client devices relative to the far end PoP, which is closer to the origin server relative to the client devices. 
     The near end PoP  1030  includes the near end network optimizer  1020  and the far end PoP  1035  includes the far end network optimizer  1040 . The near end network optimizer  1020  includes the dynamic dictionary  1024  and the far end network optimizer  1040  includes the dynamic dictionary  1044 . The dynamic dictionaries are used by the network optimizers to store version(s) of resources and optionally version identifier(s). For example, the dynamic dictionary  1024  stores version(s) of resources  1026  and optionally the version identifiers  1028 , and the dynamic dictionary  1044  stores version(s) of resources  1046  and optionally the version identifiers  1048 . In one embodiment the dynamic dictionaries  1024  and/or  1044  store only a single version of a resource; in other embodiments the dynamic dictionaries  1024  and/or  1044  are configured to store multiple versions of a resource.  FIG. 10  illustrates the logical structure of the dynamic dictionaries  1024  and  1044  and the structure may be different in different embodiments. In a specific example, the resources fields  1026  and  1046  include a URL portion and a resource portion. When determining whether a dynamic dictionary includes a requested resource, the URL of the requested resource may be used when accessing the dynamic dictionary. 
     In another embodiment, the dynamic dictionary  1044  may store the differences between versions of a resource, in lieu of or in addition to the actual resource version. The stored differences may be linked or associated in order to be used to reconstruct the resource versions (e.g., the initial versions), which will be described in greater detail later herein. For example, the stored differences may be in a chain from oldest to newest or may be in another format (e.g., a tree structure). Storage space is reduced by storing the differences between versions of a resource instead of copies of the resource, with a trade-off for the time necessary for reconstruction. 
     In some embodiments the dynamic dictionaries are built dynamically as requests for resources are received and retrieved. By way of example, the client device  1010  transmits a request  1060  for a resource (e.g., an HTTP request), which is received by the request module  1021  of the near end PoP  1030 . The request may be received at the request module  1021  as a result of a cache mechanism of the server determining that the requested resource cannot be returned to the requesting client device (e.g., the resource has been set as “no-cache”, the cache has expired, etc.). In some embodiments, a PoP includes a set of one or more proxy servers that are operative to receive requests for network resources from multiple client devices. In one embodiment, the dynamic dictionary may be shared between the various proxy servers of the PoP such that versions of a resource can be shared between the multiple proxy servers. In other embodiments, each proxy server of a PoP may have a respective dynamic dictionary such that each server maintains versions of a resource independently of other servers within the same PoP. 
     The request module  1021  includes a PoP identifier  1022 , which is operative to identify/select a point of interest of the cloud proxy service to which a request for the resource can be transmitted. The selected PoP is a far end PoP that is relatively closer to the origin server  1050  hosting the requested resource, when compared with the near end PoP  1030  relative to the origin server. In one embodiment, the identification of the far end PoP may be performed through a variety of techniques, without departing from the spirit and scope of the present invention. In one embodiment, the far end PoP can be selected by a user of the cloud based service that is the owner or administrator of the domain of the requested resource through a user interface of the cloud proxy service. In another embodiments, the far end PoP may be automatically selected based on its geographic location (e.g., closer to the origin server hosting the resource, etc.). In another embodiment, the selection of the far end PoP may be based on measures of response latency of various PoPs. For example, the response latency from each one of multiple PoPs to an origin server can be measured and based on these measures, the PoP providing the best response times is selected. 
     The request module  1021  further includes a performance analyzer  1023 , which is operative to determine whether a delta compression mechanism is to be used or not. The performance analyzer  1023  tracks performance and bandwidth usage measures for network resources and multiple points of presence to determine for each PoP and requested resource whether it is more advantageous to perform the delta compression mechanism or receive the resources without any compression. For example, in one embodiment, the performance analyzer  1023  can track a measure of the average delta compression achieved for any specific geographical zone (or for a given resource (e.g., as identified by a Uniform Resource Identifier (URI)). Further the performance analyzer  1023  tracks the bandwidth between the two PoPs (near end PoP  1030  and far end PoP  1035 ). The performance analyzer  1023  is further to track the average time needed to perform the compression for a given far end PoP (or for a given resource). The performance analyzer  1023  may determine if the time taken to compress is greater than the time saved by not transmitting the entire resource. Therefore according to these measures, the performance analyzer  1023  determines whether it is more advantageous to perform the compression technique or not and whether the technique should be performed for a request. In some embodiments, this decision is made for each URI, in alternative embodiments the decision is performed for each far end PoP to which the request is to be sent. 
     Once the request module  1021  determines that the delta compression technique is to be used, it accesses the dynamic dictionary  1024  to determine whether it includes a copy of the requested resource. If it does not, then the request module  1022  causes a request to be transmitted to the far end PoP  1035  for the requested resource. The far end PoP  1035  retrieves the requested resource (e.g., the far end PoP  1035  makes a request to the origin server for the requested resource, or if locally available, the far end  1035  accesses the local version), updates its dynamic dictionary  1044  to include the latest version of the resource, and replies to the near end PoP  1030  with the requested resource. Updating its dynamic dictionary  1044  may include storing a version identifier, which is a value that identifies a version of a network resource, and associating the version identifier with the resource version. By way of a specific example, the version identifier generator module  1049  may generate the version identifier by hashing the contents of the resource (e.g., using a suitable hashing algorithm such as Secure Hash Algorithm (SHA)), with the result being the version identifier. As another example, the last-modified header may be used as the version identifier. As yet another example, a value may be chosen and included in the ETag header field. The response module  1047  of the far end PoP  1035  causes a response (e.g., an HTTP response) that includes the requested resource to be transmitted to the near end PoP  1030 , which is received by the response module  1027 . The response may, in some embodiments, include the version identifier. The response may include a no-cache directive. For example, the response may include an HTTP header field which is used to specify instructions to be obeyed by caching mechanisms along the request/response chain of network resources. In general, the instructions specify a behavior that is intended to prevent caches (e.g., the near end PoP) from adversely interfering with the request or response. In some embodiments, these directives/instructions override the default caching algorithms traversed by the request/response. For example, as defined in Request for Comment (RFC) 7234, in HTTP/1.1, a cache-control header field may include a “no-cache” response directive which indicates that the response must not be used to satisfy a subsequent request without successful validation on the origin server (e.g., origin server  1050 ). This allows the origin server to prevent a cache from using the response to satisfy a request without contacting it, even by caches that have been configured to send stale responses (e.g., here the near end PoP). 
     The response module  1027  causes the resource to be stored in the dynamic dictionary  1024 , along with the version identifier if one was provided in the response from the far end PoP  1035 . In some embodiments, the response module  1027  causes the resource version to be stored in the dynamic dictionary  1024  even if it has a no-cache directive set. Thus, by way of example, a version of a website, which changes frequently, is stored in the dynamic dictionary  1024  even if the no-cache directive is set. 
     In some embodiments, if the version identifier was not included in the response from the far end PoP  1035 , the version identifier generator  1029  generates a version identifier to store in the dynamic dictionary  1024  (e.g., it hashes the resource and stores the resulting value as the version identifier). 
     After receiving a request for a resource that is the dynamic dictionary  1024 , the near end network optimizer  120  transmits a request  162  for that resource that includes the version identifier of the last known version to the far end PoP  1035 . The version identifier may be previously stored and associated with the resource, or in some embodiments, the version identifier is generated by the version identifier generator  1029  (e.g., the resource is hashed). The request may also indicate (e.g., in a header) that it is requesting the delta between the version identified in the version identifier and the latest version of the resource. 
     The request module  1041 , upon receiving the request  1065 , accesses its dynamic dictionary  1044  to determine whether it has access to the version of the resource identified by the version identifier included in the request  1065 . In one embodiment, the request module  1041  determines that it has access to the version of the resource if a same copy of that resource is stored in the dynamic dictionary  1044 . In another embodiment, the request module  10141  has access to the identified resource version if that version of the resource can be reconstructed using the stored differences between versions of a resource. If the request module  1041  does not have access to that version, then the far end PoP  1035  retrieves the requested resource (e.g., the far end PoP  1035  makes a request to the origin server  1050  for the requested resource, or if locally available, the far end PoP  1035  accesses the local version), updates its dynamic dictionary  1044  to include the latest version of the resource, and replies to the near end PoP  1030  with the requested resource, which may be compressed using a suitable non-delta compression algorithm such as gzip. 
     If the request module  1041  has access to the version (e.g., the dynamic dictionary  1044  of the far end PoP  1035  includes the same version of the resource identified by the version identifier included in the request  1065  or that version can be reconstructed using the stored differences between versions of that resource), the far end PoP  1035  retrieves the requested resource (e.g., the far end PoP  1035  makes a request to the origin server for the requested resource, or if locally available, the far end PoP  1035  accesses the local version), updates its dynamic dictionary  1044  to include the latest version of the resource, and determines the differences between the versions. 
     For example, the difference determination module  1045  determines the differences between the versions. A number of algorithms may be used to determine the differences between the versions, including Xdelta3, VCDIFF (described in RFC 3284, “The VCDIFF Generic Differencing and Compression Data Format”, June 2002), Bentley/McIlroy compression, bsdiff, etc. 
     In embodiments where the differences between versions of a resource are stored, the difference determination module  1045  may traverse the differences to reconstruct the initial version (the version identified in the request  1065 ) in order to determine the differences between the initial version and the current version. By doing so, storage space may be reduced as the versions of the resource may not need to be stored, with a trade-off for the time necessary for reconstructing. 
     The response module  1047  causes a response  1080  (e.g., an HTTP response) to be transmitted to the near end PoP  1030 , the response  1080  including the differences between the versions (between the initial version and the current version). The response  1080  does not contain the entire resource. In one embodiment, a header in the response  1080  indicates that it contains the differences between versions (not the entire resource). The header may also indicate the algorithm used to determine the differences. In one embodiment, the far end PoP  1035  also transmits the version identifier to the near end PoP  1030 . 
     In one embodiment, the far end PoP  1035  also compresses the response using a suitable compression algorithm. For example, the response may be compressed using gzip. As another example, the response may be compressed using zlib compression that is pre-warmed using the version of the resource identified in the request  1065 . Pre-warming allows the compression algorithm to understand more about the data that is being compressed and therefore typically provides a more accurate and faster compression. The far end PoP  1035  may also perform pre-warmed compression (e.g., zlib compression) of the HTTP headers of the response  1080  based on the version of the resource in the dynamic dictionary  1044  and identified in the request  1065 . 
     The response module  1027  of the near end PoP  1030  receives the response  1080 . The difference application module  1025  of the response module  1027  applies the differences indicated in the response  1080  to the resource version included in the dynamic dictionary  1024  to generate the latest version of the resource. The response module  1027  causes a response  1085  (e.g., an HTTP response) with the requested resource to be transmitted to the client device  1010 . The response module  1027  also causes the updated version to be stored in the dynamic dictionary  1024 , and optionally a version identifier associated with the updated version. 
     In some embodiments, the near end network optimizer  1020  and the far end network optimizer  1040  are operative to perform operations as performed by the near end network optimizer  120  and far network optimizer  140  of  FIG. 2 . Thus, according to some embodiments, when the far end PoP is identified and it is determined that the delta compression technique is to be used, the operations described with reference to  FIG. 2  are performed between the near end PoP  1030  and the far end PoP  1035  respectively including the near end network optimizer  1020  and the far end network optimizer  1040 . 
       FIG. 11  illustrates a sequence of exemplary operations for reducing network resource transmission size using delta compression between PoPs of a cloud proxy service, according to one embodiment. At operation  1110 , the near end PoP  1030  receives a request for a network resource from client device  1010 . Next, at operation  1115 , the near end PoP  1030  identifies a far end PoP from a plurality of PoPs of the cloud proxy service to which the request is to be sent. In some embodiments, when the second PoP includes multiple servers, the identification of the second PoP includes identifying a server from the set of servers forming the far end PoP. The identification of the far end PoP may be performed via various mechanisms without departing from the scope of the present invention. For example, the far end PoP can be selected by a user of the cloud based service that is the owner or administrator of the domain of the requested resource through a user interface of the cloud proxy service. In another embodiments, the far end PoP may be automatically selected based on its geographic location (e.g., closer to the origin server hosting the resource, etc.). In another embodiment, the selection of the far end PoP may be based on measures of response latency of various PoPs. For example, the response latency from each one of multiple PoPs to an origin server can be measured and based on these measures, the PoP providing the best response times is selected. 
     At operation  1120 , the near end PoP  1030  determines whether to perform a delta compression technique for reducing network resource transmission size for the received request. The performance analyzer  1023  tracks performance and bandwidth usage measures for network resources and multiple points of presence to determine for each PoP and requested resource whether it is more advantageous to perform the delta compression mechanism or receive the resources without any compression. 
     While  FIG. 11 , is described with respect to operations performed by a near end PoP and a far end PoP, in some embodiments, each operation at a PoP is performed by one of multiple proxy servers that is part of the PoP. For example, the request  1110  is received at a server from the near end PoP  1030  as a result of a DNS request. The request  1130  is transmitted to a server from the far end PoP, following the identification of a server from the far end PoP that may receive the request. Thus the operation performed at the PoP are performed within a server part of that PoP. In some embodiments, the dynamic dictionary of each PoP may be shared between the various servers of a single PoP such that versions of a network resource may be shared between these servers to respond to requests for that resource. Alternatively, each server of a PoP may include a dynamic dictionary independent of other dynamic dictionaries of other servers within the same PoP. 
     When it is determined that the delta compression technique is to be performed, the near end PoP  1030  and the far end PoP  1035  perform operations of the delta compression technique  1130  in response to the receipt of the request for the network resource at operation  1110 .  FIGS. 12-14  illustrate sequences of exemplary operations performed when a delta compression technique is performed between two points of presence of a cloud proxy service.  FIG. 12  illustrates a sequence of exemplary operations performed when a network resource is not in the dynamic dictionary of the near end point of presence of the cloud proxy service, according to one embodiment. Upon receipt of a request for a network resource (operation  1210 ), and following the determination that the delta compression technique is to be performed (e.g., as described with reference to  FIG. 11 ), the near end PoP  1030  determines at operation  1235  that a version of the network resource is not stored in the near end PoP (i.e., it is not in the dynamic dictionary  1024  of the near end PoP). As a result, at operation  1030 , the near end PoP  1030  transmits a request  1040  for the network resource to the far end PoP  1035 . 
     The far end server  1030  receives the request, and retrieves the most current version of the resource at operation  1245 . For example, the far end PoP  1035  transmits a request for the resource to the origin server, or if locally available, the far end PoP  1035  accesses the local version of the resource. 
     Next, the far end PoP  1035  may generate a version identifier for the retrieved resource at operation  1250 . For example, the far end PoP  1035  may hash the resource to generate a value that is used for the version identifier. In other embodiments, the far end PoP  1035  does not generate a version identifier. For example, the value of the version identifier may be set by the origin server (e.g., the value of the last-modified header, the value of the ETag header field). The far end PoP  1035  stores the version of the resource in its dynamic dictionary  1044  at operation  1255 , and may associate it with the version identifier. In one embodiment, the far end PoP  1035  may determine the differences between the current version (retrieved at operation  1245 ) and the most immediately previous version (if one exists), store those differences in the dynamic dictionary  1044 , and may remove older versions of the resource from the dynamic dictionary  1044 . 
     The far end PoP  1035 , at operation  1260 , transmits a response to the near end PoP  1030  of the near end PoP that includes the requested resource and optionally includes a version identifier for that resource. While  FIG. 11  illustrates the operation  1260  being performed after the operations  1250  and  1255 , the operations  1250  and/or  1255  may be performed after operation  1260  in some embodiments. 
     The near end PoP  1030  receives the response that includes the requested resource and transmits a response to the client device  1010  with the requested resource at operation  1265 . The near end PoP  1030  stores the requested resource in its dynamic dictionary  1024  at operation  1270 . The near end PoP  1030  may also generate a resource version identifier for the resource at operation  1275 , and may store the resource version identifier at operation  1280 . 
       FIG. 13  illustrates a sequence of exemplary operations performed when a network resource is included in the dynamic dictionary of the near end point of presence and the same version is not included in the dynamic dictionary of the far end point of presence, according to one embodiment. At operation  1310 , the near end PoP  1030  receives a request for a resource. Next, at operation  1335 , the near end PoP  1035  determines that the requested resource is in its dynamic dictionary  1024 . As a result, at operation  1340 , the near end PoP  1030  transmits a request for the resource to the far end PoP  1035 , where the request identifies the version identifier of the most recent version of the resource that is stored in the dynamic dictionary  1024  of the near end PoP  1030 . 
     The far end PoP  1035  receives the request and determines, in operation  1345 , that it does not have access to the version of the resource identified in the received request. For example, its dynamic dictionary  1044  does not include the same version of the requested resource and that version cannot be reconstructed. Since the far end PoP  1035  does not have access to the version of the resource identified in the received request, it will not be able to determine the differences between the version in the dynamic dictionary  1024  of the near end PoP  1030  and the latest version of the resource. At operation  1350 , the far end network optimizer retrieves the most current version of the resource. For example, the far end PoP  1035  transmits a request for the resource to the origin server, or if locally available, the far end PoP  1035  accesses the local version of the resource. 
     Next, the far end PoP  1035  may generate a version identifier for the retrieved resource at operation  1355 . For example, the far end PoP  1035  may hash the resource to generate a value that is used for the version identifier. In other embodiments, the far end PoP  1035  does not generate a version identifier. For example, the value of the version identifier may be set by the origin server (e.g., the value of the last-modified header, the value of the ETag header field). The far end PoP  1035  stores the version of the resource in its dynamic dictionary  1044  at operation  1360 , and may associate it with the version identifier. In one embodiment, the far end PoP  1035  may determine the differences between the retrieved version (retrieved at operation  1350 ) and the most immediately previous version (if one exists), store those differences in the dynamic dictionary  1044 , and may remove older versions of the resource from the dynamic dictionary  1044 . 
     The far end PoP  1035 , at operation  1365 , transmits a response to the near end PoP  1030  that includes the requested resource and optionally includes a version identifier for that resource. While  FIG. 13  illustrates the operation  1365  being performed after the operations  1355  and  1360 , the operations  1355  and/or  1360  may be performed after operation  1365  in some embodiments. 
     The near end PoP  1030  receives the response that includes the requested resource and transmits a response to the client device  1010  with the requested resource at operation  1370 . The near end PoP  1030  stores the requested resource in its dynamic dictionary  1024  at operation  1375 . The near end PoP  1030  may also generate a resource version identifier for the resource at operation  1380  and may store the resource version identifier at operation  1385 . 
       FIG. 14  illustrates a sequence of exemplary operations performed when a network resource is included in the dynamic dictionary of the near end point of presence and the same version is included in the dynamic dictionary of the far end point of presence of the cloud proxy service, according to one embodiment. At operation  1410 , the near end PoP  1030  receives a request for a resource. Next, at operation  1435 , the near end PoP  1030  determines that the requested resource is in its dynamic dictionary  1024 . As a result, at operation  1440 , the near end PoP  1030  transmits a request for the resource to the far end PoP  1035 , where the request identifies the version identifier of the most recent version of the resource that is stored in the dynamic dictionary  1024  of the near end PoP  1030 . 
     The far end PoP  1035  receives the request and determines, in operation  1445 , that it has access to the version of the resource identified in the request of operation  1440 . For example, the dynamic dictionary  1044  includes the same version or that version can be reconstructed using stored version differences. This version of the resource may or may not be the most current version of the resource. Therefore, at operation  1450  the far end network optimizer retrieves the most current version of the resource. For example, the far end PoP  1035  transmits a request for the resource to the origin server, or if locally available, the far end PoP  1035  accesses the local version of the resource. 
     The far end PoP  1035  determines the differences between the versions of the resource at operation  1455 . For example, the Xdelta3, VDDIFF, Bentley/McIlroy compression, bsdiff, or other algorithm that can be used to determine the differences between the versions of the resource, is used to determine the differences. In a situation where the version identified in the request of operation  1440  (the initial version) is not stored but a chain or tree of differences is, the far end PoP  1035  may traverse the chain or tree of differences to reconstruct the initial version and use that reconstructed version when determining the differences between the initial version and the current version. 
     Next, the far end PoP  1035  may generate a version identifier for the retrieved resource at operation  1460 . For example, the far end PoP  1035  may hash the resource to generate a value that is used for the version identifier. In other embodiments, the far end PoP  1035  does not generate a version identifier. For example, the value of the version identifier may be set by the origin server (e.g., the value of the last-modified header, the value of the ETag header field). The far end PoP  1035  stores the version of the resource in its dynamic dictionary  1044  at operation  1465 , and may associate it with the version identifier. In one embodiment, the far end PoP  1035  may determine the differences between the current version (retrieved at operation  1450 ) and the most immediately previous version, store those differences in the dynamic dictionary  1044 , and may remove the older versions of the resource from the dynamic dictionary  1044 . 
     The far end PoP  1035 , at operation  1470 , transmits a response to the near end PoP  1030  that includes the differences between the versions and does not include the complete network resource. The response may also include a version identifier associated with the network resource (that is to identify the version of the resource once updated by the near end PoP  1030 ). The response may also indicate the algorithm used to generate the differences. In one embodiment, if there is not a difference between versions of the resource, the far end PoP  1035  transmits a NULL difference or other message to the near end PoP  1030  that indicates that there are no differences. While  FIG. 14  illustrates the operation  1470  being performed after the operations  1460  and  1465 , the operations  1460  and/or  1465  may be performed after operation  1365  in some embodiments. 
     The near end PoP  1030  receives the response, and applies the differences to its last known version to generate a current version of the network resource at operation  1475 . At operation  1480 , the near end PoP  1030  transmits a response to the client device  1010  with the requested resource (the updated resource). The near end PoP  1030  stores the updated resource in its dynamic dictionary  1024  at operation  1485 . The near end PoP  1030  may also generate a resource version identifier for the resource at operation  1490  (e.g., if one was not included in the response  1470 ) and may store the resource version identifier at operation  1495 . 
       FIG. 15A-15B  illustrate a flow diagram of exemplary operations performed in a near end point of presence of a cloud proxy service for reducing network resource transmission size, according to one embodiment. At operation  1510 , a near end PoP  1030  of a cloud proxy service receives a first request for a network resource from a first one of a plurality of client devices  1010 . The first client device is physically separate from the near end PoP. At operation  1015 , the near end PoP identifies a far end PoP from a plurality of PoPs of the cloud proxy service. The far end PoP may be selected through various techniques. For example, the far end PoP can be selected by a user of the cloud based service that is the owner or administrator of the domain of the requested resource through a user interface of the cloud proxy service. In another embodiments, the far end PoP may be automatically selected based on its geographic location (e.g., closer to the origin server hosting the resource, etc.). In another embodiment, the selection of the far end PoP may be based on measures of response latency of various PoPs. For example, the response latency from each one of multiple PoPs to an origin server can be measured and based on these measures, the PoP providing the best response times is selected. 
     Flow then moves to operation  1020 , at which the near end PoP determines whether to perform a delta compression technique for reducing network resource transmission size for the first request. Responsive to determining that the delta compression technique is to be performed, the near end PoP performs the operations described with reference to  FIG. 15B . Next the near end PoP determines whether the resource is stored at the near end PoP (e.g., at operation  1535 , the near end PoP determines if the resource is stored in the dynamic dictionary). Responsive to determining that a version of the network resource is not stored in the near end PoP, the near end PoP transmit, at operation  1540 , a second request for the network resource to the far end PoP. The near end PoP then receives at operation  1545 , from the far end PoP, a first response that includes the requested network resource, and transmits, to the first client device, a second response that includes the requested network resource (operation  1550 ). At operation  1555 , the near end PoP stores the requested network resource as a first version of the network resource. 
     Alternatively, responsive to determining that the first version of the network resource is stored in the near end PoP, the near end PoP transmits, at operation  1560 , a fourth request for the network resource to the far end PoP, the fourth request including a first version identifier that identifies the first version of the network resource stored in the near end PoP. The near end PoP receives, at operation  1565 , from the far end PoP, a third response that includes a differences file that specifies a set of one or more differences between the first version of the network resource stored in the near end PoP with a most current version of the network resource, where the third response does not include the entire network resource. At operation  1570 , the near end PoP applies the set of differences specified in the differences file to the first version of the network resource stored in the near end PoP to generate an updated version of the network resource, and transmits, to the second client device, a fourth response that includes the updated version of the network resource (operation  1570 ). 
     As illustrated in  FIG. 16 , the computer system  1600 , which is a form of a data processing system, includes the bus(es)  1650  which is coupled with the processing system  1620 , power supply  1625 , memory  1630 , and the nonvolatile memory  1640  (e.g., a hard drive, flash memory, Phase-Change Memory (PCM), etc.). The bus(es)  1650  may be connected to each other through various bridges, controllers, and/or adapters as is well known in the art. The processing system  1620  may retrieve instruction(s) from the memory  1630  and/or the nonvolatile memory  1640 , and execute the instructions to perform operations described herein. The bus  1650  interconnects the above components together and also interconnects those components to the display controller &amp; display device  1670 , Input/Output devices  1680  (e.g., NIC (Network Interface Card), a cursor control (e.g., mouse, touchscreen, touchpad, etc.), a keyboard, etc.), and the optional wireless transceiver(s)  1690  (e.g., Bluetooth, WiFi, Infrared, etc.). In one embodiment, the client devices  110 , the server including the near end network optimizer  120 , the server including the far end network optimizer  140 , and/or origin servers, can take the form of the computer system  1600 . 
     The techniques shown in the figures can be implemented using code and data stored and executed on one or more computing devices (e.g., client devices, servers, etc.). Such computing devices store and communicate (internally and/or with other computing devices over a network) code and data using machine-readable media, such as machine-readable storage media (e.g., magnetic disks; optical disks; random access memory; read only memory; flash memory devices; phase-change memory) and machine-readable communication media (e.g., electrical, optical, acoustical or other form of propagated signals—such as carrier waves, infrared signals, digital signals, etc.). In addition, such computing devices typically include a set of one or more processors coupled to one or more other components, such as one or more storage devices, user input/output devices (e.g., a keyboard, a touchscreen, and/or a display), and network connections. The coupling of the set of processors and other components is typically through one or more busses and bridges (also termed as bus controllers). The storage device and signals carrying the network traffic respectively represent one or more machine-readable storage media and machine-readable communication media. Thus, the storage device of a given computing device typically stores code and/or data for execution on the set of one or more processors of that computing device. Of course, one or more parts of an embodiment of the invention may be implemented using different combinations of software, firmware, and/or hardware. 
     While embodiments described herein refer to a single near end network optimizer coupled with a single far end network optimizer, embodiments are not so limited. For example, in some embodiments, multiple near end network optimizers are coupled with a single far end network optimizer. In such embodiments, the far end network optimizer may use the same dynamic dictionary for all of the near end network optimizers in some implementations, and in other implementations may use a separate dynamic dictionary for each separate near end network optimizer. 
     In some embodiments there may be multiple pairs of near end network optimizers and far end network optimizers between a client computing device and the origin server. For example, in such embodiments, a device implementing a far end network optimizer may also be acting as a near end network optimizer for another far end network optimizer. 
     While the flow diagrams in the figures show a particular order of operations performed by certain embodiments of the invention, it should be understood that such order is exemplary (e.g., alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, etc.). 
     While the invention has been described in terms of several embodiments, those skilled in the art will recognize that the invention is not limited to the embodiments described, can be practiced with modification and alteration within the spirit and scope of the appended claims. The description is thus to be regarded as illustrative instead of limiting.