Abstract:
A method, system and computer program product for transmitting content across a link in a communications network. In an embodiment, the method comprises examining traffic in the network at a sender end of the link to determine if the traffic is carrying compressed content; when the traffic is carrying compressed content, decompressing at least one portion of the compressed content; and checking a cache to determine if the cache has content matching the at least one decompressed portion of the content. In an embodiment, the method further comprises when the cache has content matching the at least one decompressed portion of the content, determining an identifier associated with the at least one decompressed portion of the content; and using the identifier to reconstruct, at a receiving end of the link, the at least one decompressed portion of the content. In one embodiment, the at least one decompressed portion of the content from the traffic is replaced with an index, the index is compressed to form the associated identifier, and the associated identifier is sent to the receiving end of the link.

Description:
STATEMENT OF GOVERNMENT INTEREST 
     This invention was made with Government support under Contract No.: W911NF-06-3-0001 awarded by U.S. Army. The Government has certain rights in this invention. 
    
    
     BACKGROUND 
     This invention generally relates to communication networks, and more specifically, to saving bandwidth on links in communication networks. 
     In the operation of a communications network, the network, or a part of the network, may become congested with data. This may happen for any one or more of a number of reasons. For example, congestion may occur when there is an increase in the use of the network of in the data traffic in the network. Congestion may also result from changes in the topology of the network or from changes in the equipment or devices within the network. Congestion may cause lost or dropped data packets, delays in the data traffic moving through the network, or otherwise result in a significant Quality of Service (QoS) degradation. 
     A number of procedures and mechanisms may be used to prevent or to eliminate network congestion. For instance, object caching is a technique to save bandwidth on communication links in a network when similar content is transmitted multiple times on the link. In object catching, an intermediary network node caches content when it is first served; and for subsequent accesses to the same content, the content is served from the intermediary network node instead of the original content provider. Object caching reduces overall network load. 
     However, in many situations, a congested bottleneck link is somewhere in the middle of the network. While an object cache reduces overall network traffic, a congested link cannot solely rely on the presence of object caches as a solution to the congestion on the link since: (a) object caches may not be deployed or they may be out of service—an object cache out of service will impact a limited set of applications but a congested link will impact all applications; (b) there may be no object cache in the path for many users due to network technology; (c) object cache in general does not provide bandwidth savings when mirrors are used or when similar or same files are downloaded via different protocols. 
     In such a case, a set of synchronized caches at both ends of a congested link provides a better solution that is targeted specifically at the bottleneck link. Before content enters one end of the congested link, that content is passed through the cache at the end of the link to determine if the content (or part thereof) already exists in the cache. If so, then a short index is sent to the other end of the link instead of the matched content; and at the other end of the link, matched content is recovered from the synchronized cache at that end of the link. The synchronized caching is completely complementary and transparent to object caching. 
     The content is often transmitted in a compressed format. For example, a server may send compressed content to a browser that is capable of decompressing content (html:Content_Encoding: gzip is a standard and all standard browsers such as firefox, chrome, IE, support compressed content). Large software are often packaged and transmitted over computer networks in a compressed format (.cab, .zip). Many document formats such as PDF are compressed. After compression, even small difference in two content C1 and C2 will result in vastly different compressed content C1.zip and C2.zip as compressed output is largely random (if it was not, it could be further compressed). For example, for 2 Microsoft Word documents, while the difference between C1.doc and C2.doc is only 1 character (i.e., ˜0.6 KB bytes in binary form), the difference between the zipped version of the documents, C1.zip and C2.zip, is around 170 KB. 
     However, as a result of compressed content, caching techniques may be ineffective for about 15%-25% of the total network traffic. This fraction may increase in the future as storage moves to remote sites in a cloud computing environment and more compressed formats are used to save storage and bandwidth. 
     BRIEF SUMMARY 
     Embodiments of the invention provide a method, system and computer program product for transmitting content across a link in a communications network. In an embodiment, the method comprises examining specified traffic in the network at a sender end of the link to determine if the traffic is carrying any compressed content; when the specified traffic is carrying compressed content, decompressing at least one portion of the compressed content; and checking a sender cache to determine if the sender cache has content matching the at least one decompressed portion of the content from the specified traffic In an embodiment, the method further comprises when the sender cache has content matching the at least one decompressed portion of the content from the specified traffic, determining an identifier associated with said at least one decompressed portion of the content; and using the associated identifier to reconstruct, at a receiving end of the link, the at least one decompressed portion of the content from the specified traffic. 
     In one embodiment, the at least one decompressed portion of the content from the specified traffic is replaced with the associated identifier. 
     In an embodiment, an index is determined for identifying the at least one decompressed portion of the content from the specified traffic, that index is compressed to form the associated identifier, and the associated identifier is sent over the link to the receiving end of the link. 
     In one embodiment, the associated identifier is decompressed at the receiving end of the link to reconstruct the index for identifying the at least one decompressed portion of the content from the specified traffic. 
     In an embodiment, the reconstructed index is used to obtain, from a receiver cache, a copy of the at least one decompressed portion of the content from the specified traffic. 
     In one embodiment, the associated identifier is only used to reconstruct the at least one decompressed portion of the content from the specified traffic when one or more given conditions are satisfied. 
     In an embodiment, when the one or more conditions are not satisfied, the compressed content in the specified traffic is sent to the receiving end of the link. 
     In one embodiment, the specified traffic in the network is examined at the sender end of the link before this traffic enters the link. 
     In one embodiment, the method further comprises determining if any content of the specified traffic is cached at the receiver end of the link. 
     Embodiments of the invention provide a network decompression/compression capability that works in coordination with synchronized caching appliances on either side of a congested link. In embodiments of the invention, before the traffic enters a congested network link, the traffic is examined to determine if the traffic is carrying compressed content—the determination of which can be done either by software or hardware or a combination thereof. If the content is compressed, the content is decompressed and passed through a cache. The cache examines whether it has matching files (or portions thereof); and if the cache does, those matching files (or portions thereof) are replaced by a short index that uniquely identifies a particular content. The output of the cache is then compressed. If this process does not reduce content size, then the original compressed content is transmitted. If the process does reduce content size, the output of the cache is transmitted to the other end of the link. 
     At the other end of the congested link, if the received content is the compressed output of the cache at the transmitting end of the link, then the content is recovered by inversing the above-described steps. The output is decompressed and the decompressed content is parsed to identify cache indices. These cache indices are replaced by appropriate content from the cache at the receiving end of the link, reconstructing the original uncompressed byte stream. The uncompressed byte stream is then compressed and sent out in its original form. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  depicts a network in which embodiments of the invention may be implemented. 
         FIG. 2  shows a link of the network of  FIG. 1 , with a synchronized caching sender and caching receiver in accordance with an embodiment of the invention. 
         FIG. 3  illustrates a process employing the synchronized caching sender and caching receiver of  FIG. 2  in accordance with an embodiment of the invention. 
     
    
    
     DETAILED DESCRIPTION 
     As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, method or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, embodiments of the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium. 
     Any combination of one or more computer usable or computer readable medium(s) may be utilized. The computer-usable or computer-readable medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CDROM), an optical storage device, a transmission media such as those supporting the Internet or an intranet, or a magnetic storage device. Note that the computer-usable or computer-readable medium could even be paper or another suitable medium, upon which the program is printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory. In the context of this document, a computer-usable or computer-readable medium may be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-usable medium may include a propagated data signal with the computer-usable program code embodied therewith, either in baseband or as part of a carrier wave. The computer usable program code may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc. 
     Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user&#39;s computer, partly on the user&#39;s computer, as a stand-alone software package, partly on the user&#39;s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user&#39;s computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). 
     The present invention is described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks. 
     The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. 
     The present invention, generally, relates to saving bandwidth in the transmission of compressed data in a communications network. As mentioned above, a network, or a part of a network, may become congested with data, and this may adversely affect network performance. A number of techniques may be used to prevent or to eliminate network congestion. For instance, it is common practice to use data compression techniques on network data bytes in order to conserve bandwidth. 
     An example of potential savings for software downloads is illustrated by Tables I and II below (with numbers from July 2011) for the then latest version of Eclipse IDE). 
     
       
         
               
               
               
               
             
               
               
               
               
             
           
               
                 TABLE I 
               
               
                   
               
               
                   
                 (1) eclipse- 
                 (2) eclipse- 
                 (3) eclipse- 
               
               
                   
                 cpp-indigo- 
                 jee-indigo- 
                 SDK-3.7- 
               
               
                   
                 incubation-win32 
                 win32 
                 win32 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                 Zipped (MB) 
                 107 
                 240 
                 190 
               
               
                 Unzipped (MB) 
                 120 
                 212 
                 174 
               
               
                 # of files 
                 1290 
                 2850 
                 1219 
               
               
                   
               
             
          
         
       
     
     
       
         
               
               
               
               
             
               
               
               
               
             
           
               
                 TABLE II 
               
               
                   
               
               
                   
                 Among  
                 Among  
                 Among  
               
               
                   
                 (1) and (2) 
                 (2) and (3) 
                 (1) and (3) 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                 # of same files 
                 1121 
                 950 
                 868 
               
               
                 Size of the same  
                 82 
                 104 
                 63 
               
               
                 files (MB) 
                   
                   
                   
               
               
                   
               
             
          
         
       
     
     Table II shows that even if three different downloads of Eclipse IDE zip archives have completely different byte patterns (as tested by diff utility), once uncompressed, one-third to two-thirds of the bytes are exactly the same. These similarities are hidden due to compression which ironically is applied to save network bandwidth (among other reasons). 
     A set of synchronized caches at both ends of a congested link may also be used to prevent or to alleviate link congestion. It is difficult, however, to use compressed data with synchronized caches. Embodiments of the invention add decompression and compression capability in order to facility caching on either side of the congested network link. 
       FIG. 1  depicts a communications network  100  in which embodiments of the invention may be implemented. Network  100  includes a multitude of nodes  102  and a multitude of network devices  104  connected together by a multitude of links  106  to form the network. As an example nodes  102  may include personal computers, servers, workstations, laptop computers, personal digital assistants (PDAs), mobile phones, or any other entities that are operable to transmit data to or in and to receive data from or in network  100 . 
     Network devices  104  are used to route data packets from and throughout the network  100 . These devices may be computer network equipment such as switches, network bridges, or routers. The network devices can be connected together in any configuration to form network  100 , as will be appreciated by those skilled in the art. 
     Network links  106  may be established over wired networks such as, for example, local area networks (LAN) and wide area networks (WAN), or over wireless networks such as, for example, IEEE 802.11b (WiFi), General Packet Radio Service (GPRS), Universal Mobile Telecommunications Service (UMTS), Personal Communications Service (PCS), Global System for Mobile communications (GSM), Digital-Advanced Mobile Phone Service (DAMPS), Interim Standard (IS)-136, IS-95, Code Division Multiple Access (CDMA)  2000 , Wide-band CDMA (W-CDMA), or Universal Mobile Telecommunications Service (UMTS) standards, or any other Personal Communications Services (PCS), Time Division Multiple Access (TDMA) or CDMA wireless network. Network links, also, may be or include suitable optical data transmission devices. 
     With reference to  FIGS. 1 and 2 , in the operation of network  100 , a sender, represented at  202 , and a receiver, represented at  204 , exchange data packets over one of the links, represented at  206 . This exchange may be, for example, between one of the nodes  102  and one of the network devices  104 , or between a pair of the network devices  104 . In a conventional packet exchange, sender  202  transmits data packets  210  to receiver  204 , and the receive sends acknowledgment packets  212  back to the sender  202 . 
     In network  100 , due to changes in traffic, topology changes, equipment changes, or for other reasons, a link may become congested. This can affect the data passing through the network and cause a decrease in the Quality of Service provided by the network. 
     It is common practice to use compression techniques on network data bytes in order to conserve bandwidth. However, even with the use of compression, links in the network may still become congested. 
     Embodiments of the invention provide a network decompression/compression capability that works in coordination with synchronized caching appliances on either side of a congested link. Before the traffic enters a congested network link  206 , the traffic is examined to determine if the traffic is carrying compressed content—the determination of which can be done either by software or hardware or a combination thereof. If the content is compressed, the content is decompressed and passed through a cache  214 . The cache examines whether it has matching files (or portions thereof); and if the cache does, those matching files (or portions thereof) are replaced by a short index that uniquely identifies a particular content. The output of the cache  214  is then compressed. If this process does not reduce content size, then the original compressed content is transmitted. If the process does reduce content size, the output of the cache is transmitted to the other end of the link. 
     At the other end of the congested link, if the received content is the compressed output of the cache at the transmitting end of the link, then the content is recovered by inversing the steps. The output is decompressed at  216  and the decompressed content is parsed to identify cache indices. These cache indices are replaced by appropriate content from the cache  220  at the receiving end of the link, reconstructing the original uncompressed byte stream. The uncompressed byte stream is then compressed and sent out in its original form. 
     In an embodiment of the invention, both ends of the link, the sender and the receiver, use the uncompressed data in order to update their cache content, i.e., all the uncompressed data (as well as other data that were not originally compressed by the end-points of the link) are stored in the sender and receiver caches and get replaced using known replacement policies. 
       FIG. 3  shows the main steps involved in embodiments of the invention. These embodiments use the synchronized caching sender  202  and receiver  204  on either side of the congested network link  206 . Normal (i.e. uncompressed) data  302  is examined by the caching sender in order to determine if the data is new, and hence needs to be transmitted or if the data is cached data that only requires an identifier to be transmitted. The synchronized caching receiver resides on the other end of the network link and communicates to the sender if the data (or portions of the data) is cached. 
     Embodiments of the invention enable compressed data to be cached by introducing a decompression module  216  before the caching sender  202 . This module, at  304 , decompresses compressed data  306 , and passes the decompressed data on, at  310 , to the caching sender, which then, at  312 , performs the same operations as it did on the normal, uncompressed data. 
     The caching sender  202 , in embodiments of the invention, sends uncompressed or decompressed data to the caching receiver  204 . Compressed data is decompressed at  314 , the caching receiver, represented at  316 , distinguishes, at  320 , between data that was and data that was not originally compressed, and the caching receivers is also equipped with the logic  322  to compress originally compressed data, reconstructing at  324  the originally compressed data while serving from its cache  220 . Normal (originally uncompressed) data is served, at  326  without any modification. 
     If the data examined by the caching sender was uncompressed, the caching sender sends this data, or an identifier, to the caching receiver. If the original, uncompressed data is sent to the receiver, the receiver can output this data, and also cache a copy of the data for possible future use. If the identifier was sent to the receiver, the receiver uses this identifier to retrieve the original, uncompressed data from the receiver cache. If the receiver receives data that had been decompressed by the sender, the receiver can recompress this data and output the data in that recompressed form. The receiver may also store a copy of this data—in a compressed form or a decompressed form—for possible future use. 
     The description of the present invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or to limit the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the invention. The embodiments were chosen and described in order to explain the principles and application of the invention, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.