Backup procedure with transparent load balancing

In an embodiment of the invention, an apparatus and method provides a backup procedure with transparent load balancing. The apparatus and method perform acts including: performing a preamble phase in order to determine if a file will be backed up from an agent to a portal; and applying a chunking policy on the file, wherein the chunking policy comprises performing chunking of the file on an agent, performing chunking of the file on the portal, or transmitting the file to the portal without chunking.

TECHNICAL FIELD

Embodiments of the invention relate generally to a backup procedure with transparent load balancing.

BACKGROUND

Data in a node that is connected to a communication network (e.g., WAN or LAN) can be backed up (i.e., mirrored) in another node that is also connected to the communication network. Incremental backup is often used in order to optimize the data backup procedure. Incremental backup involves only backing up files (or directories) that have been modified or added since the last backup, or backing up the modified chunks of a file if the chunk-based compare-by-hash technique is used. The chunks that are part of a file being backed up in a source node will be transmitted to the destination node, depending on the hash comparison results. In the chunking procedure, the file would be divided into chunks and a hash value is calculated for each chunk. For every file that is backed up the destination node maintains a list of hash chunk pairs that compose the file being backed up in a hash chunk database. During a subsequent incremental backup if the compared hash values in the hash chunk database for a file that was previously backed up on to the destination node differs, only the chunks that differ are transmitted to the destination node and the deltas are applied to the existing version of the file on the destination node and a new version of the file is created. For the cases where a file is being backed up to the destination node for the first time a heuristic resemblance detection method is used, where only the first few hashes of chunks in a file are compared with the hashes of chunks of other files that have already been stored on the destination node. If there is a match then the chunks that are being shared by the two files need not be transmitted from the source node to the destination node. Instead only the chunks that differ in the file that is being backed up needs to be transmitted. This procedure is called chunk level single instancing where chunks can be shared between unrelated files.

However, the chunking of files consumes significant resources (e.g., CPU cycles, memory spaces, I/O resources, and network bandwidth) in the source node, particularly if the files are large in sizes (e.g., large megabyte or gigabyte sizes) and/or are numerous in number. For example, the calculation of hash values will consume CPU cycles and require an amount of time to perform. Therefore, the current technology is limited in its capabilities and suffers from at least the above constraints and deficiencies.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1is a block diagram of an apparatus (system)100in accordance with an embodiment of the invention. One or more agents105are connected to a communication network110. In the example ofFIG. 1, agents105(1) to105(x) are shown, although the number of agents can vary from one to a plurality of agents. An agent105can run on a client machine (generally referred to as client machine106) such as, for example, a client computer or other suitable types of computing device. In the example ofFIG. 1, the agent105(1) is running on the client machine106(1). The number of clients in the system100can vary. A portal machine115is also connected to the network110. The portal machine115can be, for example, a server. A storage device121is also coupled to the portal machine115. The network110can be, for example, a wide area network, local area network, or other types of communication networks. The operation of the agent105on a client machine and that of the portal machine115are discussed in detail below. As discussed further below, an embodiment of the invention provides various advantages such as, for example, the ability to perform a chunking operation for a file (or for an entire backup session) at an agent or at the portal for purposes of achieving load balancing in the system100during the backup of files from an agent to a storage device connected to a portal, so that use of resources in the agents are optimized. Other features or advantages in an embodiment of the invention are also discussed below.

FIG. 2is a block diagram that shows additional details of the system100ofFIG. 1, in accordance with an embodiment of the invention. A file change monitor201in a client machine106can determine which files in a file system202have been modified (i.e., changed, added, or deleted). The file change monitor201can be (or perform the function of) a file tree walker (which compares the last modification time of every file in the file system202to determine which files have been modified), a file change detector (e.g., a WINDOWS NT® journal) which records the type of operation that is performed on a file, and/or a filter driver (which detects reads and writes on a file). The file change monitor201can use other known methods for detecting changes in a file system201.

To initiate the process of backing up the data from the client106to a storage device121(FIG. 1) via the portal115, a command and control architecture would launch a backup task204on the client106and a backup task205on the portal115. This architecture is implemented by various elements in the client and portal as discussed further below. Typically, the backup task204is launched in accordance with a data backup policy that determines when backup operations are to be performed. The data backup policy can be implemented in, for example, a software program. The backup task205on the portal115is launched by the command and control architecture based on a schedule in the backup policy and there is handshaking involved between the client machine106and portal machine115. The client106will initiate all management communication to the portal via a suitable protocol such as, for example SOAP (Simple Object Access Protocol) and the data processing operation is performed via XDP (transmission data protocol) with the features that will be discussed below. The backup tasks204and205are launched with the same session identifier.

In an embodiment of the invention, the backup task204of agent105typically launches the following software elements. A file change producer206receives the metadata of files that have been modified in the file system202since the last backup occurrence. Alternatively, the file change producer206will contain or perform the functions of the file change monitor201. The file change producer206places the file name and the associated metadata of a modified file as an entry in a change log207. The change log207contains a list of entries219that have metadata of files that have been modified since the last backup. An agent send file thread208in a change log consumer209reads the metadata and filename for an entry219in the change log207, where the metadata and filename is for a file that has been modified in the file system202. Typically, multiple send file threads208can be implemented in the change log consumer209so that the change log consumer209can read entries in parallel in the change log207as an optimization.

The send file208, hash manager210, chunk manager211, and chunk control212are typically software threads in the change log consumer209. Assume that file217is a file that has been modified in the file system202since the last backup. The metadata218of modified file217is placed by the file change producer206in an entry219in the change log207. The send file thread208reads the metadata218in entry219of change log207. The send file208can also flag each entry219that has been read as being successful or failed to indicate the status of the file backup. The producers remove the entry from the change log207as soon as they receive a notification from the change log.

The file217also has a client identifier221which identifies the file217and is typically the identifier (ID) of the row219(in change log219) that contains the metadata218of that file217. The transport220of XDP passes the session ID216and file ID221as part of every call that the transport220makes to the portal110. The transport220is responsible for delivering hashes and chunks from the agent105in the client106to the portal115, batches the hashes (into hash batches) and chunks (into chunk batches) from various files being computed on the agent, and transports the batches across the network110(FIG. 1) to the portal115. The transport220also transmits the communications from the portal115to the agent105. The batching of hashes and chunks produces a pipelining effect that ensures effective utilization of bandwidth between the agent105and portal115. As will be discussed further below, the chunk level and object level single instancing and the load balancing feature (i.e., ability to chunk a file at the agent or at the portal) provides considerable performance improvement over traditional data archiving methods.

The agent backup task204and portal backup task205are launched with the same session ID216. Each file217that is being backed up is also identified by a specific client ID221which is the ID of the row219in the changelog database207for that file217on the client106. The transport220serves as a channel of communication between the client106and portal110and typically defines commands, messages, and listeners (receiving elements) that are used in a data backup or data restoration procedure. In addition to a backup task205that is launched in a portal115, there is another corresponding portal task created internally that allows the portal stack to be set up for portal chunking as will be discussed below. The stacks for portal chunking and agent chunking are running in parallel.

In an embodiment of the invention, the computation of hashes on a client is optimized by having a preamble phase before performing the computation and transmission of hashes. In the preamble phase, the metadata218pertaining to a file217to be backed up is sent by the send file component208to the portal115via preamble222. If the file was previously backed up, the upper shim layer251(that is launched by the portal backup task205) compares the incoming metadata with the metadata of the previous version of the file existing on the portal to determine if the metadata has changed since the last backup. This is a proof safe way of ensuring that we start the chunking operation only if the file being backed up has really changed and is more recent than the one that has previously been backed up. This is very important in the case of continuous data protection where the file change monitor is continuously capturing changes being done to a file and it is possible that it could report those operations in an incorrect order resulting in lost versions or incorrect latest versions. The preamble response226indicates if the metadata218has changed since the last backup. The preamble response226is sent from the upper shim layer251via transport220to the send file208in the agent105. If the metadata218did not change, then the backup agent105running on the client106will not start to chunk the file217as the contents of a file cannot change without its metadata getting affected. The preamble session in the agent chunking scenario, portal chunking scenario, and no-chunking scenario (whole file transfer from agent to portal) will be discussed below in additional details. By not performing the chunking of file unless the file contents have changed, tremendous cost savings is achieved in the case of an incremental backup. Incremental backup involves transmitting only the deltas (changed data in a file) since the previous backup, instead of transmitting the entire file contents.

Three different cases will now be discussed. These cases involve chunking is not performed for a file, chunking of a file is performed at the client106, and chunking of a file is performed at the portal115. By providing the capability to chunk a file either on the agent105or on portal115(i.e., ability to perform load balancing), an XDP in an embodiment of the invention advantageously prevents the client106from getting overwhelmed with the computationally intensive chunking operations that are required for some files. Therefore, the load balancing feature permits improved optimization of the use of resources290(FIG. 2) in the client106because chunking is not necessarily performed at the client106at all times, or if the agent105is running on a slower client machine that is not suitable to support the computationally intensive chunking operation. The resources290include, for example, processor291cycles, disk I/O bandwidth292, memory resources293, and OS resources and other software resources203.

FIG. 3is a block diagram that illustrates the components for determining a chunking policy to be applied to a file, as used in an embodiment of the invention. The user can set the chunking policy301that is used by the chunk control212. For example, the user can set a file size threshold305on whether a file will be chunked at an agent or portal, or will not be chunked. For example, the policy301will indicate that any file310that is below a file size threshold305will not be chunked and will be transmitted as a whole file from the agent105to the portal115. Assume that this threshold is 100 kilobytes as a specific example. If a file310that is less than 100 kilobytes, then the file310will not be chunked by the chunk manager211, and will instead be directly transmitted as a whole file by the agent send file component208to the portal115using transport. The backup task205in the portal115will then store the file310into the storage device121directly by, for example, streaming the file contents. This transmission of the file310from the client106to the portal115is performed after the preamble phase that has been described above, and assumes that the metadata of the file310has changed since the last backup occurrence.

The user can set other criteria in the chunking policy301such as, for example, MIME type (Multi-Purpose Internet Mail Extensions)307, or the metadata308of the file. For example, the policy301will indicate that any file310that is a pdf-type file will not be chunked. This restriction on pdf-type files can be set in the chunk control212. Therefore, if the file310is a pdf-type file, then the file310will not be chunked by the chunk manager211, and will instead be directly streamed as a whole file by the backup task204to the portal task205.

As another specific example, larger sized files (e.g., greater than 20 megabyte or other example sizes) are chunked at the agent105in client106, as opposed to performing the chunking operation on the portal. The transmission of these larger sized files across the network will consume more time. By chunking the file in agent, and transmitting the chunks in batches to the portal without waiting for the entire file to be chunked, a pipelining effect is achieved that improves the performance of the backup. In contrast, if a file is less than, e.g., 20 megabyte or other example sizes, the whole file can be transmitted from the agent to the portal and then chunked at the portal115as will be discussed below. While this file is being chunked at the portal, other files can be concurrently chunked at the agent105, and in this case, both backup tasks204and205are performing chunking of files. Additionally, chunking of smaller sized files does not provide additional performance benefits, and therefore, these files are directly streamed to the portal.

Additionally, chunking is not performed for certain MIME types (e.g., gif images or zip files) because chunking of these compressed files do not provide additional performance benefits, and are instead transmitted as a whole file.

As known to those skilled in the art, chunking algorithms are used for dividing a file into chunks which are portions of the file. A background on chunking algorithms is discussed in, for example, commonly-assigned U.S. Pat. No. 7,269,689. As also known to those skilled in the art, chunking algorithms are also used in Low Bandwidth File Systems (LBFS) in order to divide a file into chunks.

As shown inFIG. 4, an embodiment of the invention provides a chunk manager211that would chunk a file by use of the fixed chunking algorithm. The chunk manager211will chunk the file405into the chunks410(1) to410(Z) where the size of the file will determine the value of Z. In an embodiment of the invention, a minimal set of chunks in a file are first obtained and a hash manager212will calculate corresponding hash values for this minimal set of chunks in the file405. This minimal set can be set by a programmable threshold value412. As an example, assume that file405(FIG. 4) will be backed up. Initially, the metadata of the file405is sent to the portal during the preamble phase and it is determined if the metadata has changed since the last backup. If the metadata has changed as indicated by the preamble response, then one of the three scenarios can occur: agent chunking, portal chunking, or no chunking, as will be discussed in additional details below. In an agent chunking scenario, the first minimal set of chunks of the file is first obtained (e.g., one-hundred chunks per set). The hashes for these one-hundred chunks in this minimal set is sent to the portal without waiting for the chunking to finish for the next one-hundred chunks of the file. As a result, the first one-hundred hash values of the first one-hundred chunks are being compared in the portal side. In the case of files being backed up for the first time and which do not have any previous versions on the portal resemblance detection can be performed rather quickly. In the transmitted first-one hundred hash values, a resemblance detection heuristic method is performed where a subset of the hashes (e.g., first 10 hashes) in the first one-hundred hash values is compared with a subset of hash values (e.g., first 10 hashes) of first 10 chunks of stored files on the portal side. The content repository1001(FIG. 10) can perform this comparison of hash values in the resemblance detection heuristic method. Assume that a stored file (candidate file) on a portal side was found to match the file405in the first 10 chunks. Hash comparison between the rest of the chunks in the stored file and file405are made, and only the chunks of non-matching hashes of the file405are sent to the portal. As an achieved benefit, the chunks of the matching hash values will not be transmitted from the agent to the portal. By performing chunk level single instancing on the portal side, the backup time is advantageously reduced. Additionally, in the case of whole file backup, where chunking the file does not provide any added benefit, a hash value is computed for the contents of the entire file. This hash value is sent as part of the metadata during the preamble phase. The whole file is not transmitted from the agent to the portal if a hash match is determined on the portal side. This is called Whole File Single Instancing.

By first chunking and hashing a minimal set of chunks, the XDP protocol in an embodiment of the invention allows the hash comparison to be performed, typically at the storage device121, without the need to wait for the entire file405to be chunked. Compare-by-hash algorithms are known to those skilled in the art. When a minimal set of chunks have been produced by the chunking process (as set in the configuration parameter412), a hash value can be generated for each of the minimal set of chunks and the portal can then request the agent105for missing chunks while the chunking of the remaining portion of file405is still in progress. The hashes and requested chunks are transmitted to the portal in one or more batches until the modified chunks of the file405is backed up and versioned via portal115at the storage device121.

An embodiment of the invention also permits multiple files to be chunked by the chunk manager211in parallel on the agent105, while the send file208(FIG. 2) sends hashes to the portal115so that the portal115is able to request for and receive the missing chunks based on the hash comparison. This parallel operation of chunking and hash comparison and missing chunks transmission to the portal provides a pipelining effect that improves data transfer by an embodiment of the XDP protocol.

In another embodiment of the invention, the transmission of the requested chunks are interleaved between the previous batch of hashes that were submitted by the agent105to portal115and the next batch of hashes to be submitted by the agent105to portal115. This interleaving between the requested chunks and batches of hashes takes advantages of available idle time when transmission across the network110would otherwise not be occurring. The transport220can accumulate the hashes in batch and the missing chunks in batch and also interleave the transmission of missing chunks between transmissions of hash batches.

As a simplified example, assume that the hash manager210calculates the hash values414(1) to414(3) for chunks410(1) to410(3), respectively. Assume also that the preamble phase has indicated that the file405has been modified since the last backup. Hash values can be calculated by a hash algorithm such as, for example, MD5 or a Secure Hash Algorithm (SHA) such as, for example, SHA1 or SHA256. The send file component208in backup task204will then send the hashes414(1) to413(3) to the portal115. The agent105can be optionally optimized so that the transport220accumulates a given number of hashes so that the send file208can send the hashes in batches. A lower shim layer in the portal115can compare the received hashes414(1)-414(3) with stored hashes which are mapped to stored chunks in the storage device121, for the specific file405in question. The stored hashes (and/or mapping of the hashes to chunks) can be, for example, in a hash database in the storage device. As discussed below, different files can optionally share the same chunks in the storage device121.

FIG. 5is a block diagram of resource sharing that can be used by versions of a file, in an embodiment of the invention. This resource sharing method is used in the storage device121(FIG. 1). A file501contains a node505that points to the metadata507of the file501and has a resource508that points to the chunks509that contain the content of the file501. The file502has a node506that points to the metadata510of the file502. If the file502has the same contents as file501, then the resource511of file502will point to the same chunks509as shown inFIG. 5.

Chunks can be shared across different versions of a file and shared across different files. If two files are mostly identical in chunks and if one of the files has already been backed up on the storage server121(FIG. 1), then the agent105need to only send to the portal115the differing chunks for the second file.

FIG. 6is a block diagram that illustrates the functionality of the agent send file component208, transport220and upper shim251when a whole file is transferred from the agent105to the portal115, in an embodiment of the invention. In an embodiment of the invention, the XDP protocol from the agent's105perspective is the same for both the no-chunking case and the portal chunking case, except for the time line (time value) when the file stored completion message reaches the agent from the portal. The agent send file208will send, via transport220portion on the agent105, the sendPreamble call601which contains the metadata pertaining to a file to be backed up, as discussed above. The transport220portion on the portal115will then send the receivepreamble call602to the upper shim layer251which checks if the metadata in the sendPreamble call601has changed since the last backup. The sendPreamble response603indicates if the metadata has changed since the last backup. The transport220forwards the call603as a preambleResponse604to the agent send file208. If the metadata has changed, then the agent send file208sends the whole file via storefile call605which is forwarded by the transport220as receiveFile call605to the upper shim251. The whole file is sent by the upper shim251to lower shim1002and eventually streamed as a set of bytes to a content repository1001, and the file contents are stored into storage device121.

FIG. 7is a block diagram that illustrates the functionality of the agent send file component208, transport220and upper shim251when a chunking of a file is performed on the agent105, in an embodiment of the invention. The various calls in the preamble phase (sendpreamble, receivepreamble, sendPreambleResponse, and preambleResponse) have been discussed above. If the metadata of a file has changed, as will be indicated by the preamble response701to the agent send file component208, then the chunk manager211will perform chunking of the file and hash manager210will calculate hash values of the chunks, as has been discussed above. In an embodiment of the invention, the features of the chunk manager and hash manager can be integrated with or be separate from the agent send file component208. The sendHash calls702from the send file208to the transport220will send the hash values to the transport. The transport then batches the hashes and sends the hash batch via receiveHashes call703to the portal, and the receivehashes call703from the transport220to upper shim251results in the hash comparisons as discussed above. The send file208sends the sendLastSequenceNumber call704to the transport220when the last hash for the entire file has been sent by the send file208to the transport220. The transport220sends the receiveLastSequenceNumber call705to the upper shim251to indicate that the last hash for the file has been sent from the agent205to portal115.

Based on the hash comparisons, the upper shim251sends a sendChunkRequest call706listing the hashes, for each batch of hashes that were submitted to the portal from the agent105, for chunks whose hash values do not match any stored hashes in the Hash chunk database on the portal115side for the file in question (e.g., stored hashes in a database in the storage device121or in the portal115itself). The transport220forwards the call706as a chunkRequest707to the send file208. In response to each chunkRequest707, the send file208will send all chunks708as requested in the chunkRequest707. The transport220will send the chunks708via the receive chunks call709to the upper shim251.

The chunks708are then eventually forwarded by the upper shim251to lower shim1002and then streamed to the content repository1001and then stored in the storage device121. An example of the content repository1001is in the RISS products from HEWLETT-PACKARD COMPANY. For each batch of chunks, submitted by send file to the portal, that is stored in the storage device121, the upper shim251sends a sendChunkStoreResponse710to the transport220which then forwards this call710as a chunkStoreResponse711to the send file208, in order to confirm that the chunks has been successfully stored on the portal115side (e.g., in storage device121). The calls710and711can be transmitted in a batch712. When all of the chunks sent from the agent105to portal115have been stored on the portal side, then the upper shim251will send a sendstoreopertioncomplete call714which is forwarded by the transport220as a storeOperationComplete call715to the send file208. The call715is a confirmation message to the send file208in order to confirm that the backing of the file on the portal side is complete

As similarly discussed above, the sending of hash batches via sendHash calls702and sending of chunks batches via sendChunk calls708can be performed asynchronously and transmission of a chunk batch can also be interleaved between the transmission of hash batches.

FIG. 8is a block diagram that illustrates the functionality of the agent send file component208, transport220and upper shim251when chunking of a file is performed on the portal116. As mentioned above, in an embodiment of the invention, the XDP protocol from the agent's105perspective is the same for both the no-chunking case and the portal chunking case, except for the time line (time value) when the file stored completion message reaches the agent from the portal. Therefore, the calls in the preamble phase801in the portal chunking case is similar to the calls in the preamble phase610(FIG. 6) in the whole file transfer case. However, the time occurrence between the receivefile call802and sendStoreOperationComplete803(which confirms successful storage on the portal115side of all transmitted chunks) in the portal chunking case is longer than the time occurrence between the receivefile call605and sendStoreOperationComplete call612in the whole file transfer case because the process of portal chunking will occur in the portal115between call802and call803.

FIG. 9is a block diagram that illustrates additional details of a portal chunking scenario in accordance with an embodiment of the invention. For convenience, the previously discussed details of the agent chunking scenario ofFIG. 7is also shown on the left side ofFIG. 9to assist in describing the portal chunking scenario. In the portal chunking scenario, an entire file will be sent from the agent105to the portal115as discussed above. A preamble phase is performed where the agent send file208sends the sendPreamble (1,2) call905to the transport220which forwards the call905as receivePreamble (1,2) call906. The notation (1,2) for calls905and906means (backup session1, client (file) ID2). Hence, notation (1,2) means that the call is using a backup session ID1and the file2is involved in the call. The sendPreambleResponse (1,2) call907and preambleResponse (1,2) call908indicates if there is a metadata mismatch on the portal115side for the metadata of the file2. If the file ID2metadata does not match with metadata on the portal side, then the file2needs to be backed up on the portal side. The agent send file208makes the storeFile (1,2) call910to the transport220to transmit the file2, and the transport220forwards the call910as receiveFile (1,2) call911to the upper shim251. The upper shim251sends a ChangeLogEntry (P1,2) call912to the change log902, and this call912will insert the metadata of file2into an entry in change log database902. Therefore, the upper shim251is acting in similar to the file change producer206(FIG. 2) which is inserting a metadata of a file into the change log207. The parameter P1means that the call912is using a second backup session (with ID P1) which is a different session from the backup session1. The backup session1launches the upper shim251and front end transport220, while the portal backup session P1launches the changelog207, changelog consumer209and proxy transport904on the portal.

The portal send file component903makes the getNextEntry (P1,2) call914to the change log database902in order to process the entry with the metadata of file2. The portal send file component903then begins a portal preamble phase which involves the following calls. The portal send file component903sends a sendPreamble (P1,2) call915(which uses the portal session P1and which has the metadata for file2). A proxy transport904will pass through (e.g., perform a “U-turn”) on the call915, so that the call915is passed by the proxy transport904as receivePreamble (P1,2) to the upper shim251. The upper shim251then sends a sendPreambleResponse (P1,2) call918In a similar manner as discussed above, in the agent chunking scenario, the upper shim251would receive a receivepreamble call720(FIG. 7) and send a sendPReambleResposnse call721during the preamble phase.

The proxy transport904would pass the call918as a preambleResponse (P1,2) call920to the portal send file903. In response to call920, the portal send file903would call (or perform) a chunk manager211function to chunk the file2, as similarly discussed inFIG. 4and would call (or perform) a hash manager function210to calculate hash values for chunks in the file2. A threshold value412(FIG. 4) can also be set so that only the first few chunks of a file are chunked and hashed at a time.

The portal send file903sends the hashes via sendHash (P1,2) calls922which are passed923by proxy transport904as a call924the upper shim251. The call925is passed926by the transport904as call927to upper shim251to indicate that all hashes in the entire file have been transmitted by the portal send file903.

Based on the hash comparisons, the upper shim251sends a sendChunkRequest (P1,2) call928which is passed929to the portal sendfile903. The sendChunkRequest (P1,2) call928is for chunks of hashes from the submitted batch where the hashes are not stored on the portal side. The transport904passes929the call928as a chunkRequest (P1,2) call930to send file903. In response to each call930, the send file903will send all chunks931as requested in call930. The chucks are forwarded by the portal send file903to the lower shim1002and then to the content repository1001and then stored in the storage device121.

The upper shim passes the chunks to the lower shim1002and the chunks are then forwarded to the content repository1001and then stored in the storage device121. The transport904will pass932the chunks as receiveChunks (P1,2) call933to the upper shim251, and this call933delivers the chunks.

For a batch of chunks that is stored in the storage device121, the upper shim251sends a sendChunkStoreResponse (P1,2) call934to the transport904which passes935the call934as call936to the portal send file903. The calls934and936can be transmitted in a batch937. The upper shim251sends a sendstoreopertioncomplete (P1,2) call938which is passed by the transport904as call940to the send file904to confirm that the file has been stored on the portal side. The send file903also updates941the change log database902to remove the entry for portal session P1and file2, i.e., the parameter (P1,2). The calls942-944are performed so that the agent send file component208receives confirmation of the successful backup of the file2.

Note inFIG. 9for the agent chunking scenario on the left side of the drawing, the parameter (1,1) is added to each call such as, for example, sendPreamble (1,1) call945. All calls in the agent chunking scenario were previously discussed above with reference toFIG. 7. The parameter (1,1) means that the call is using backup session1for a different file1. Since sessions1and P1can be occurring concurrently, an embodiment of the invention advantageously permits the agent chunking of file1(via backup session1) and portal chunking of file2(via backup session P1) to occur concurrently or asynchronously.

It is also within the scope of the present invention to implement a program or code that can be stored in a machine-readable or computer-readable medium to permit a computer to perform any of the inventive techniques described above, or a program or code that can be stored in an article of manufacture that includes a computer readable medium on which computer-readable instructions for carrying out embodiments of the inventive techniques are stored. Other variations and modifications of the above-described embodiments and methods are possible in light of the teaching discussed herein.