Abstract:
A client device transmits requests via a gateway to a server in a network environment. The requests indicate specific portions of a file on a server to be transmitted as part of the download process. The gateway receives into its memory the requested portions of the file and assembles the received portions into an assembly file. The gateway continuously scans the largest contiguous sequence of the portions in the assembly file for viruses while the requested portions of the file are being received and become available before feeding the received portions to the client computer. By scanning the largest consecutive sequence while new portions become available, the time to complete the scan is reduced thereby increasing the throughput of the gateway.

Description:
BACKGROUND 
     There are two generally accepted methods for a computer application to download a file from a remote location. In the first method, a client computer connects to a server either directly or via a network gateway. The client computer transmits a request for the server to transmit a whole file at once. The server responds to the request by sending the whole file, typically in packet form, to the client computer. In the second method, the client computer transmits a sequence of requests to the server, or a peer computer (such as in a pear to pear network), where each request asks for specific portions of the file. 
     Requesting specific portions of the file is preferable to requesting the entire file at once because it allows for resuming of the download if the connection is broken. Requesting specific portions of the file also allows for more efficient utilization of available bandwidth because the client computer can request more portions when more bandwidth is available and fewer portions when less bandwidth is available. 
     The downloaded portions of the same file can be of different sizes, can be received out of order or can overlap previously downloaded portions. This brings a challenge to an anti-virus (AV) application disposed in the network gateway (or host computer) when the application performs anti-virus scanning and inspection of routed traffic. The scanning process involves an AV application that contains a virus reference signature or heuristical pattern that is compared against the content being downloaded from the remote location. In many cases, anti-virus application must scan an entire file to ensure that no viruses are embedded in the file. 
     Usually, it is not possible to download the whole file for anti-virus scanning after a portion of the file is requested. The whole file can be very large, requiring significant network bandwidth and time to complete the download. Existing AV solutions either attempt to download the entire file before scanning or limit the scanning to the content of the downloaded portions. Inspecting only a portion of the content downloaded is not sufficient to detect a virus. The virus signature may be spread over two or more portions of the file and may not be identifiable when each file portion is scanned separately. 
     SUMMARY 
     This Summary is provided to introduce concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. 
     Described herein are, among other things, embodiments of various technologies for use in anti-virus scanning of content. In accordance with one embodiment, client devices transmit requests for a download of content via a network gateway to a server. The requests indicate specific portions of the content to be transmitted and the order that the portions should be transmitted. The gateway receives in its memory the requested portions of the content. The portions are assembled into blocks and are arranged in the same sequence as when they were stored on the server. This arrangement may be different than the sequence that the portions were received via the network. The gateway scans the block with the largest contiguous number of portions for a virus as the requested portions of the file are received. 
     By scanning the block with the largest contiguous number of portions, a virus can quickly be identified across portion boundaries so that if a virus is detected the remainder of the download can be aborted. This can be done while avoiding the traditional less effective solution of either completely blocking files that are downloaded in portions, or not scanning them for viruses, thus increasing the security provided by the gateway while maintaining the normal end user experience and allowing such downloads to occur. 
     Because the gateway inspects small combinations of file portions before the entire file is available at the gateway. Once the final portion is received, a final scan is performed on the entire file. The final scan faster would then be faster than if the gateway had waited for the entire file to be available because only a subset of virus signatures needs to be compared. Further the gateway able to process more requests because it can abort downloads of infected files. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference number in different figures indicates similar or identical items. 
         FIG. 1  illustrates an exemplary architecture in which anti-virus scanning of partially available content may be implemented. 
         FIG. 2   a - c  is a diagram illustrating the content being received, assembled and scanned by the gateway in  FIG. 1 . 
         FIG. 3  is a block diagram depicting selected modules in a network gateway in the anti-virus scanning system. 
         FIG. 4  is a flow diagram of an exemplary process used to scan partially available content while the content is being routed in a gateway. 
     
    
    
     DETAILED DESCRIPTION 
     Overview 
     Described herein are, among other things, embodiments of various technologies for use in anti-virus scanning of content. In accordance with one embodiment described herein, an anti-virus scanning system transfers portions of content via a network gateway (or any host computer) from a server (or any pear computer) to a client computing device in response to requests from the computing device. The portions are received by the gateway and are assembled into an assembly file. The gateway scans a block of the largest contiguous number of the portions in the assembly file for viruses while the requested portions of the file are being received. 
     Example System Architecture 
     Illustrated in  FIG. 1  is a virus detection system  100  including client computing devices  102   a - 102   n  coupled via a network gateway  104  through a network  106  to servers  108   a - 108   n . Although gateway  104  is shown, any type of network processing device that can scan for viruses may be substituted for gateway  104 . Examples of such a processing device include a proxy server and a general purpose computer. 
     Stored in server  108   a  is a content file  109 . The content file has portions  1 - 10  arranged in an original sequential order. In one embodiment, client computer device  102   a  transmits a request indicating which portions  110  of content file  109  stored on server  108   a  are to be downloaded. Specifically device  102   a  requests that portions  110  be transferred in requested order 1, 2, 3, 6, 7, 9, 10, 4, 8 and 5. Gateway  104  then feeds portion request  112  via network  106  to server  108   a . Server  108   a  responds by transmitting portions  109  in the requested order as portions  114  to gateway  104 . 
     Gateway  104  includes one or more processors  122  and memory  124 . Stored in memory  124  are assembly file  126  and datastore  128 . Portions  114  are received by gateway  104  and stored using processor(s)  122  in memory  124 . In an exemplary embodiment, as portions  114  are received, processor  122  arranges the received portions into blocks within an assembly file  126  in their original sequential order. Also as portions are received, the largest block of contiguous portions in assembly file  126  is scanned to determine if a virus is present. Examples of the scanning process are described in  FIGS. 2   a - 2   c.    
     Virus signatures are stored in a datastore  128 . The datastore  128  is periodically updated with new virus signatures. In one embodiment, scanning is performed by comparing portions of the assembly file  126  against the virus signature. In another embodiment, antivirus scanning is not limited to signature comparison. Scanning may be performed by first determining the type of the content, then performing both regular expression matching (looking for a signature) and behavior analysis, executing portions of the file in an isolated environment to observe what the executed portions tries to do. 
     In one embodiment, certain portions of the assembly file  126  known not to contain viruses and know to contain viruses may be identified. The portions known not to contain virus may be skipped when scanning. In one embodiment, the AV engine will determine that it needs to scan the whole file to inspect the content (e.g. when archives that cannot be unpacked unless the whole file is present). In these cases, the AV engine will scan the files only once, e.g. when all the portions are available. 
     If a virus is not detected when scanning, gateway  104  would continue to arrange the received portions in sequential order within blocks for the entire requested (or partially requested) content file. The size of the virus signature may be larger than a combination of many received portions. Thus the gateway scans all the requested portions before they are disassembled from the assembly file  126  and fed as disassembled portions  116  to device  102   a . Specifically once the all the portions in the request (or partial request) are received, gateway  104  feeds the dissembled portions  116  to the device  102   a  in the order requested by device  102   a.    
     After each portion is received, assembled, and the largest available portions of the assembly file  126  is scanned and if no viruses are detected, the last received portion is fed to client computing device  102   a . This ensures clients don&#39;t have to wait while gateway  104  is assembling or fetching additional portions. 
     In one embodiment, if a virus is detected in assembly file  126 , the infected portion of the assembly file is purged and is not fed to client device  102   a . Also upon virus detection, an indication may be provided to client  102   a  of the portion of the file  126  that is infected. Further, in another embodiment, an indication of an infected file may be fed to client device  102   a  by embedding a virus indication with the portions  116  sent to the client device  102   a , or such indication may be sent to a system administrator (not shown). 
       FIGS. 2   a - 2   c  illustrate memory  104  comprising assembly file  126  in memory  104  of gateway  104  at sequential points in time after receiving content in response to a single request from device  102   a . In one embodiment, gateway  104  may detect the requests from device  102   a , and assemble the response from multiple requests into blocks within a single assemble file  126  for scanning. In another embodiment the gateway  104  may detect the response from a single request before forwarding scanned content to device  102   a . Assembly file  126  is depicted in  FIGS. 2   a  and  2   b  as containing partially available content. Assembly file  126  is depicted in  FIG. 2   c  to include the entire content file  109 . 
     Referring to  FIG. 2   a , portions  114  ( FIG. 1 ) of content file  109  are received and stored in assembly file  126 . As the portions  114  are received, they are arranged in their original sequential order (See also  FIG. 1 ). Also as more portions  114  are received, the block with the largest number of portions that are in a contiguous order is scanned to determine if a virus is present. In one embodiment, either a minimum threshold number of portions or a number of portions in aggregate having a minimum size must be received before the portions are scanned. Assembly file  126  may be scanned after reception of each new portion is received. Further when the entire content file is stored in the assembly file  126 , the entire file may be scanned to determine if a virus is present. 
     For example, in  FIG. 2   a  received portions  1 - 3 ,  6 ,  7 ,  9  and  10  are assembled in memory  124  to create assembly file  126 . In this example the minimum threshold number of portions may be set to four. Thus once portion  4  is received, portion  4  is assembled with portions  1 - 3 ,  6 ,  7 ,  9  and  10  in assembly file  126  to create a block having portions  1 - 4 , a block having portions  6 - 7  and a block having portions  9 - 10 . The largest block of contiguous portions, e.g. the block containing portions  1 - 4 , is scanned to determine if a virus is present. If a virus is not present, then additional portions would be received. 
     Referring to  FIG. 2   b , portion  8  is received. Portion  8  is then assembled with portions  1 - 4 ,  6 - 7 , and  9 - 10  in assembly file  126  to create a block having portions  1 - 4  and a block having portions  6 - 10 . The largest block of contiguous portions, e.g. the block containing portions  6 - 10 , is scanned to determine if a virus is present. If a virus is not present, then additional portions would again be received. 
     Referring to  FIG. 2   c , portion  5  is received and is assembled with portions  1 - 4 , and  6 - 10  in assembly file  126  to create a block having portions  1 - 10 . The largest block of contiguous portions, e.g. the block containing portions  1 - 10 , is scanned. Although an exemplary content file  109  having ten portions is shown, larger files having additional portions may be assembled and scanned in a similar manner, e.g. by scanning the largest block of contiguous portions in the assembly file  126 . 
     In  FIG. 3  illustrates selected modules in gateway  104  of the virus detection system  100  shown in  FIG. 1 . Gateway  104  has process capabilities and memory suitable to store and execute computer-executable instructions. In one example, gateway  104  includes one or more processors  122  and memory  124 . 
     The memory  124  may include volatile and nonvolatile memory, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules or other data. Such memory includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, RAID storage systems, or any other medium which can be used to store the desired information and which can be accessed by a computer system. 
     Stored in memory  124  of the gateway  104  are a transceiver component  306 , an assemble module  308 , a scan module  310 , a disassemble module  312 , and a datastore  314 . These modules and components  306 - 314  may be implemented as software or computer-executable instructions that are executed by the one or more processors  122 . 
     Transceiver component  306  receives information and requests from client computer devices  102 ( a - n ) and feeds those requests to servers  108 ( a - n ) via network  106 . In one embodiment, such requests may conform to a hyper-text transfer protocol (HTTP) and a Transmission Control Protocol/Internet Protocol (TCP/IP). Transceiver component  306  transfers content received by gateway  104  from servers  108 ( a - n ) to client computer devices  102 ( a - n ) and transfers content from client devices  102 ( a - n ) to servers  108 ( a - n ). In one embodiment such content is transferred directly from memory  124 . 
     Assemble module  308  accumulates and stores in memory  124  portions received from servers  108 ( a - n ). The portions are stored sequentially in data blocks within an assembly file. The largest contiguous portions may be scanned for viruses. Scan module  310  scans the block with the largest number of contiguous portions in the assembly file as the portions are being received by gateway  104 . The block is scanned to detect viruses including viruses having a signature that extends across portion boundaries. By scanning the largest contiguous portions, the downloading process may be quickly terminated if a virus is detected and the likelihood of detecting the virus in partial content increases. 
     Disassemble module  312  disassembles the assembly file into disassembled portions for transmission to client computer devices  102 ( a - n ). Upon disassembly of the assembly file, the disassembled portions  116  are arranged to be transmitted to client computer devices  102 ( a - n ) in the order indicated by request  110 . In a simpler embodiment, every portion is sent to the client  102  when it&#39;s received, immediately after the largest contiguous portion is scanned. 
     Contained in datastore  314  are the signatures of the viruses that may be updated from time to time. In another embodiment, inspection logic is included in the AV engine and the AV engine is updated frequently to detect new malware. Also stored in datastore  314  may be names or network addresses (such as a uniform resource locator URLs) for content files from server  108 ( a - n ) in which viruses were previously detected. In one embodiment, the assembly file may be stored in datastore  314 . 
     Exemplary Process 
     The exemplary process in  FIG. 4  is illustrated as a collection of blocks in a logical flow diagram, which represents a sequence of operations that can be implemented in hardware, software, and a combination thereof. In the context of software, the blocks represent computer-executable instructions that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order and/or in parallel to implement the process. For discussion purposes, the processes are described with reference to system  100  of  FIG. 1 , although it may be implemented in other system architectures. 
       FIG. 4  illustrates a flow diagram of an exemplary process  400  used by gateway  104  (see  FIG. 1 ) of the virus detection system  100 , to scan partially available content for viruses. Although the flow diagram is depicted in the order of blocks shown, blocks  402 - 424  do not have to be implemented in any particular order. 
     At block  402 , gateway  104  connects with one of the client devices  102 ( a - n ) and one of the servers  108 ( a - n ). Gateway  104  also receives a request from one or more of the client computer devices  102 ( a - n ) for portions  110  of content file  109 . In one embodiment the client computer devices  102 ( a - n ) actually specify the order in which the portions are to be transmitted from server  108  ( a - n ). In another embodiment, the client computer devices  102 ( a - n ) may transmit a file name or file address as the request. For the purpose of example, in process  400  gateway  104  receives requests from client  102   a  for transmission to server  108   a.    
     In block  404 , the gateway  104  determines, by comparing information in the datastore  214  against the client computer device&#39;s  102   a  request, whether the received request for the portion of the content file has previously been made. Such a determination may be made by examining the request to identify the address or the name of the requested file. If such identification has been made previously (“yes” to block  404 ), then in block  420  a determination is made as to whether the previously requested file contained a virus. If the file has not previously been requested (“no” to block  404 ), then the client device&#39;s request  110  for the portion of the content file  109  is transmitted to server  108   a  in block  406 . 
     Next in block  408 , the gateway  104  receives a next portion of the content file  109  from server  108   a . In block  410 , the gateway  104  stores the received portion in memory  124  and assembles the portion in sequential order into a block within an assembly file. Then in block  412 , the gateway  104  scans the block with the largest number of contiguous portions in the assembly file by comparing one or more portions against the virus signatures retrieved from the datastore  214 . Such scanning may be done continuously across portion boundaries while other portions of the content file are being received by gateway  104 . If the virus signature matches the assembly file, a virus would be detected. In one embodiment, a minimal size of the block to be scanned and a maximum number of times an assembly file will be scanned may be configured in gateway  104 . If the minimum block size is not present, then the assembly file may not be scanned. 
     In another embodiment, the file format of the assembly file is identified by the gateway  104  (by combining information about the object name or type conveyed by the delivery protocol, or by looking at contiguous portions that have been assembled and determining the file format based on actual content). Certain portions of the assembly file known to not contain a virus may also be identified and thus would not be scanned. For example, if the assembly file in question was identified as a JPEG file, the gateway  104  will assume that the malicious code might be found in an EXIF portions and pass the other file portions (Non-EXIF portions) without scanning them. 
     In block  414 , the gateway  104  determines if a virus was detected. If it was (“yes” to block  414 ), gateway  104  provides a virus indication to an administrator device (not shown) or to the client computer in block  422 . If a virus was not detected (“no” to block  414 ), the portions are disassembled from the assembly file in block  415  and fed as disassembled portions  116  ( FIG. 1 ) to the client computer device  102   a  in the order that the portions were requested in request  110 . The current portion (or portions) is sent to the client computer device  102   a  before all the portions are received by gateway  104 . Then a determination is made whether all the portions of the requested file have been received by gateway  104  (e.g. whether a gateway  104  has received a complete assembly file) in block  416 . If all the portions have not been received (“no” to block  416 ), upon a request from the client computer the next portions of the content file is received in block  408 . If all the portions of the content file have been received (“yes” to block  414 ), then the last received portion from server  108   a  is sent to client computer device  102   a  in block  418 . 
     After a virus is indicated in block  422 , in one embodiment, the viral portions of the assembly file are purged and transmission of the portion of the file containing the virus to client computer device  102   a  is terminated in block  424 . In block  424  in another embodiment, the portion of the file containing the virus may be flagged before that portion is fed to the client computer device  102   a.    
     If the previously requested file was already determined to contain a virus (“yes” to block  420 ), then a virus indication is provided in block  422 . If the file was not determined to contain a virus (“no” to block  420 ), then the portion request  112  is fed to server  108   a  in block  406 . 
     Conclusion 
     In closing, although the invention has been described in language specific to structural features and/or methodological acts, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed invention.