Abstract:
An apparatus and method by which repeaters are able to temporarily or permanently store data in a local repository called a depot. Two uses for the depots are checkpoint restart functionality and the ability to store commonly installed software distributions on nodes closer to their destinations. Large software programs such as Office 95™ can be stored on local repeaters for fast and efficient distribution. A particular distribution can be stored in more than one depot. Depots will provide an interface to allow administrators to list the contents of a depot, store new distributions and delete old distributions. Data may be added to a depot by either an explicit administrator command or by retaining data sent as part of a distribution. Applications can decide what data is appropriate for depoting, and mark those distribution segments as “storable”.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     Related subject matter may be found in the following commonly assigned, U.S. Patent Applications which are hereby incorporated by reference herein: 
     Ser. No. 09/460,855, entitled “APPARATUS FOR DATA DEPOTING AND METHOD THEREFOR” 
     Ser. No. 09/460,853, entitled “APPARATUS FOR RELIABLY RESTARTING INTERRUPTED DATA TRANSFER AT LAST SUCCESSFUL TRANSFER POINT AND METHOD THEREFOR” 
     Ser. No. 09/438,437, entitled “AN APPARATUS AND METHOD FOR DISTRIBUTING AND COLLECTING BULK DATA BETWEEN A LARGE NUMBER OF MACHINES” and filed concurrently herewith; 
     Ser. No. 09/458,268, entitled “COMPUTER NETWORK CONTROL SYSTEMS AND METHODS” and filed concurrently herewith; 
     Ser. No. 09/460,852; entitled “METHODS OF DISTRIBUTING DATA IN A COMPUTER NETWORK AND SYSTEMS USING THE SAME”; 
     Ser. No. 09/458,269, entitled “SYSTEMS AND METHODS FOR REAL TIME PROGRESS MONITORING IN A COMPUTER NETWORK”; 
     Ser. No. 09/460,851, entitled “APPARATUS FOR AUTOMATICALLY GENERATING RESTORE PROCESS DURING SOFTWARE DEPLOYMENT AND METHOD THEREFOR”; 
     Ser. No. 09/460,854, entitled “AN APPARATUS FOR JOURNALING DURING SOFTWARE DEPLOYMENT AND METHOD THEREFOR”; and 
     Ser. No. 09/438,436, entitled “AN APPARATUS FOR CONNECTION MANAGEMENT AND METHOD THEREFOR”. 
    
    
     TECHNICAL FIELD 
     The present invention relates generally to data processing systems, and in particular, to temporary or permanent storage of bulk data distributions within networked data processing systems. 
     BACKGROUND INFORMATION 
     Present day data processing systems are often configured in large multi-user networks. Management of such networks may typically include the need to transfer bulk data to an endpoint system from a source system (or, simply, “a source”) and the collection of information, for example, error reports from a multiplicity of endpoints systems (or, simply, “endpoints”). 
     Such large data transfers may occur within a network, for example, to distribute software updates. The system administrator may need to allocate a specific period of time for the data transfer to more efficiently utilize network resources. This may typically occur when the communication load on the system is lowest, usually at night when most endpoint users are not working at their stations. The system administrator may load the bulk data and the corresponding transfer instructions onto the network system&#39;s source, or server, in preparation for the transfer. At the predetermined time set by the administrator, the server will push the data while ensuring that the bulk data is successfully transferred to each of the desired endpoint locations. However, during the transfer a portion of the system server is dedicated to the data transfer and thus unavailable for other networking tasks. Moreover, as the number of endpoints which must be simultaneously serviced by the bulk data distribution increases, network bandwidth demands are concomitantly increased. This complicates scalability of the bulk-distribution systems. 
     Therefore, a need exists in the art for a bulk distribution mechanism that can transfer large amounts of data between network connected subsystems (or nodes) while maintaining scalability. Additionally, there is a need in such distribution mechanisms for methods and apparatus to distribute bulk data to a multiplicity of endpoints and to collect bulk data, including large log files, from the endpoints. These methods and apparatus may facilitate the bulk distribution of data by storing the data at intermediate nodes in the network, thereby reducing network bandwidth demand. 
     SUMMARY OF THE INVENTION 
     As described in the co-pending, commonly owned U.S. Patent Application entitled “A Method and Apparatus for Distributing and Collecting Bulk Data Between Data Processing Systems,” Ser. No. 438,437, incorporated herein by reference bulk data transfers are mediated by one or more repeaters between the server and endpoint. Each repeater includes a local repository which may be referred to as a “depot” for the storage of data being transferred between servers and endpoints. Two uses of the depots are checkpoint restart functionality which allow an interrupted delivery to be automatically restarted from a predetermined point, and the ability to store commonly installed software distributions on nodes closer to their destinations. (Checkpoint restart is discussed in detail in the commonly-owned, co-pending U.S. Patent Application entitled “Apparatus for Restarting Interrupted Data Transfer and Method Therefor,” Ser. No. 09/460,853, incorporated herein by reference.) For example, large software packages like Office 95™ can be stored on local repeaters or gateway repeaters for fast and efficient distribution. A particular distribution can be stored in more than one repeater. For purposes of this application, the term depot is used interchangeably with repeater; however, it would be understood by an artisan of ordinary skill that, in general, repeaters incorporate functionality for depoting data in accordance with a portion of the depoting methodology of the present invention and discussed below in conjunction with FIG. 3, in addition to providing other functionality as discussed in the aforesaid U.S. Patent Application entitled “A Method and Apparatus for Distributing and Collecting Bulk Data Between Data Processing Systems.” 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     For a more complete understanding of the present invention, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which: 
     FIG. 1 illustrates, in block diagram form, a data processing network in accordance with one embodiment of the present invention; 
     FIG. 2 illustrates, in block diagram form, a data processing system implemented in accordance with an embodiment of the present invention; and 
     FIGS. 3A and 3B illustrate, in flowchart form, a methodology implemented to depot data in a local repository. 
    
    
     DETAILED DESCRIPTION 
     When large distributions are sent through a network, network inefficiencies will occur when all network assets are not available to the distribution. These inefficiencies cause network slowdowns due to increased data traffic. The present invention allows the distributions to be depoted on repeaters near the intended endpoints. Depoting allows the system to more efficiently utilize network assets by determining the availability of endpoints for the transferring of the data. 
     In the following description, numerous specific details are set forth to provide a thorough understanding of the present invention. However, it will be obvious to those skilled in the art that the present invention may be practiced without such specific details. In other instances, well-known circuits have been shown in block diagram form in order not to obscure the present invention in unnecessary detail. For the most part, details concerning timing considerations and the like have been omitted inasmuch as such details are not necessary to obtain a complete understanding of the present invention and are within the skills of persons of ordinary skill in the relevant art. 
     FIG. 1 illustrates a communications network  100 . The subsequent discussion and description of FIG. 1 are provided to illustrate an exemplary environment used by the present invention. 
     The network system  100  includes server  101 , one or more fan out/collector nodes referred to as repeaters  110 ,  111 ,  118 ,  119 , and a plurality of resources (which may also be referred to as clients or endpoints)  112 - 117 . Additionally, certain repeaters, such as  118  and  119 , are directly connected to one or more endpoints, in the exemplary embodiment of FIG. 1, endpoints  112 - 114  or  115 - 117 , respectively, and may be referred to as “gateway” repeaters (or, simply, “gateways”). 
     Server  101  provides distribution services with respect to resources  112 - 117 . Repeater  110  may be the primary repeater through which resources  112 - 114  receive their data transfers, and repeater  111 , likewise, may primarily service endpoints  115 - 117 . Additionally, any report-back of successful transfers will be transmitted primarily via the endpoints primary domain except as explained below. It would be understood by an artisan of ordinary skill that additional repeaters may be inserted into the network and may be arranged in a multi-level hierarchy according to the demands imposed by the network size. Gateway repeaters  118  and  119  are such repeaters in the exemplary embodiment of FIG.  1 . 
     However, in its preferred embodiment, network system  100  provides cross connections in order to provide redundant, parallel communication paths should the primary communication path to the endpoint become unavailable. For example, in FIG. 1, endpoint  114  has a primary pathway to server  101  through repeaters  118  and  110 . Should repeater  110  become unavailable, server  101  can transfer bulk data to endpoint  114  via an alternative pathway through repeaters  118  and  111 . Additionally, should repeater  118  become unavailable, endpoint  114  may receive data via repeaters  111  and  119 . 
     Referring next to FIG. 2, an example is shown of a data processing system  200  which may be used to implement servers such as  101 , repeaters, such as repeaters  110  and  111 , or endpoints, such as endpoints  112 - 117 , executing the methodology of the present invention. The system has a central processing unit (CPU)  210 , which is coupled to various other components by system bus  212 . Read only memory (“ROM”)  216  is coupled to the system bus  212  and includes a basic input/output system (“BIOS”) that controls certain basic functions of the data processing system  200 . Random access memory (“RAM”)  214 , I/O adapter  218 , and communications adapter  234  are also coupled to the system bus  212 . I/O adapter  218  may be a small computer system interface (“SCSI”) adapter that communicates with a disk storage device  220 . Communications adapter  234  interconnects bus  212  with an outside network enabling the data processing system to communicate with other such systems. Input/Output devices are also connected to system bus  212  via user interface adapter  222  and display adapter  236 . Keyboard  224 , track ball  232 , mouse  226  and speaker  228  are all interconnected to bus  212  via user interface adapter  222 . Display monitor  238  is connected to system bus  212  by display adapter  236 . In this manner, a user is capable of inputting to the system throughout the keyboard  224 , trackball  232  or mouse  226  and receiving output from the system via speaker  228  and display  238 . 
     Preferred implementations of the invention include implementations as a computer system programmed to execute the method or methods described herein, and as a computer program product. According to the computer system implementation, sets of instructions for executing the method or methods are resident in the random access memory  214  of one or more computer systems configured generally as described above. Until required by the computer system, the set of instructions may be stored as a computer program product in another computer memory, for example, in disk drive  220  (which may include a removable memory such as an optical disk or floppy disk for eventual use in the disk drive  220 ). Further, the computer program product can also be stored at another computer and transmitted when desired to the user&#39;s work station by a network or by an external network such as the Internet. One skilled in the art would appreciate that the physical storage of the sets of instructions physically changes the medium upon which it is stored so that the medium carries computer readable information. The change may be electrical, magnetic, chemical, biological, or some other physical change. While it is convenient to describe the invention in terms of instructions, symbols, characters, or the like, the reader should remember that all of these and similar terms should be associated with the appropriate physical elements. 
     Note that the invention may describe terms such as comparing, validating, selecting, identifying, or other terms that could be associated with a human operator. However, for at least a number of the operations described herein which form part of at least one of the embodiments, no action by a human operator is desirable. The operations described are, in large part, machine operations processing electrical signals to generate other electrical signals. 
     Refer now to FIGS. 3A and 3B in which is illustrated a flow chart of methodology  300  for depoting distribution data. Data may be selectively depoted permanently or, alternatively, temporarily in a local repository. A system administrator may select a distribution for depoting and so initiate an administrative command to depot the transmission. Additionally, an application can identify data for depoting and mark those distribution segments as storable as described in the aforesaid commonly owned co-pending U.S. Patent Application entitled “A Method and Apparatus for Distributing and Collecting Bulk Data Between Data Processing Systems.” 
     When a bulk transmission occurs, the system may initiate a command set to permanently store the bulk data in a local repository. (For the purposes of the present invention, permanently storing, or depoting, distribution data means that the data is stored until deleted in response to an explicit predetermined event, such as command from a system administrator or expiration of a preselected lifetime.) For example, referring to FIG. 1, if bulk data is to be transferred to endpoints  112 - 114 , the system might establish gateway repeater  118  as the local repository of the bulk data in step  302 . Likewise, if  115 - 117  were to also receive the same data, the administrator might establish gateway repeater  119  as the local repository of the bulk data in step  302 . It is important to note that any repeater may be designated as the depot for the data, and it is not limited to gateway repeaters to provide this function and each repeater receiving a distribution may temporarily store the data, in accordance with step  365 , FIG. 3B, below. 
     Referring again to methodology  300 , a distribution request is received from a client or from the administrator in step  301 . The processing of distribution requests described in the commonly owned, co-pending U.S. Patent Application entitled “A Method and Apparatus for Distributing and Collecting Bulk Data Between Data Processing Systems, incorporated herein by reference. In step  302  a control block is generated. A control block may be a data structure including a pair of string data values containing an identifier and version number uniquely identifying the distribution, and a third data value for tagging the associated distribution as storable. A control block  360  that may be used with the present invention is shown in FIG.  3 A. Control Block  360  includes identifier (ID) field  361 , version field  363 , storable tag field  365 , and target ID field  367  which includes a list of repeater and endpoints receiving the distribution. Methodology  300  then determines if an administrative command has been entered to permanently depot the data on an assigned depoting repeater, step  305 . If it does, then the data value tagging the distribution as storable control block generated in step  302  will be prepared with the predetermined logic state marking the distribution for permanently depoting the data transmission, step  315 . Similarly, in step  310 , it is determined if the application has requested the distribution to be storable. If so, the predetermined data value marking the distribution as storable is entered into the control block, for example by setting field  365  in an embodiment of the present invention using data block  360 . If in step  310  the application has not requested the distribution be storable, step  315  is bypassed. In step  320  the depoting repeater&#39;s identifier is stored into the target ID list in the control block. In step  325  the filename and version of the data transmission is stored into the control block for later use, as described further below. In an embodiment of the present invention using a control block such as control block  360 , the identifier and version number may be stored in fields  361  and  363 , respectively. Each repeater mediating the distribution is operable for controlling depoting of the distribution in response to information in the control block. 
     Referring to FIG. 3B, once the transmission commences, the destination repeater receives the control block in step  340 . As described above, although not a requirement of the present invention, the depoting repeater may be a gateway repeater such as repeater  118 . Once repeater  118  receives the control block, it is determined whether the current version of the distribution is already depoted, step  345 . If so, then in step  347 , a report is sent back to the distribution source system canceling the distribution, and hence preventing an unnecessary transmission. If the depoting repeater does not have the current version of the transmission then, in step  350 , a report is sent back to the distribution source system requesting that the transmission proceed. 
     When the source system receives the request, the distribution transmission occurs in step  355 . The source system will also transmit to the depoting repeater information concerning the allocation of the data, for example the target endpoint machines, and the life span of the data, i.e., how long the transmission should be depoted on the depoting repeater before removing the data from the repeater. A data structure that may be used to pass this information may have the form of a sequence of key/value pairs. A set life span may apply to data that is to be either permanently and temporarily stored. For example, an application update may be transferred to a gateway repeater so that the associated endpoints can receive the update. This may be depoted temporarily until all the endpoint machines can receive the update. However, a preselected time interval, or life span, of the distribution may be specified by the system administrator or requesting application after which the distribution expires. Thus, if one or more of the machines are unavailable during that period, the repeater and thus the network will not be burdened with attempting to transmit data to an unavailable endpoint and the distribution data will be removed from the repeater&#39;s storage. Further, permanently stored data may be deleted after the lifetime expires. 
     Once received, the depoting repeater will allocate the data to its system memory based on whether the data was marked for permanent or temporary storage. This is determined in step  360 . If the data is not marked for permanent storage, then the depot will store the data in its temporary storage in step  365 . It would be understood by an artisan of ordinary skill that such temporarily stored data may be tagged as deletable by resetting a “locked” bit in a corresponding directory entry, or similar such mechanism, that are known in the data processing art. Temporary distributions may be marked as “temporary” and permanent distributions may, likewise, be marked as “permanent.” As described further below, temporary distributions may have an associated reference count that indicates how many repeaters and endpoints require the distribution. When the distribution has been sent to all repeaters and endpoints requiring it, the reference count goes to zero and the distribution is deleted from the depots. Permanent distribution may only be deleted by a request from an administrator. 
     In step  370 , the data is distributed to one of the target endpoints identified in step  355 . If the transmission to the endpoint was not successful, as determined in step  372 , then the repeater will continue to attempt to transmit the distribution data. Those skilled in the art will understand that the data can be transmitted asynchronously such that parallel transmissions may occur to different machines as described in the commonly owned co-pending U.S. Patent Application entitled “A Service for Distributing and Collecting Bulk Data Between Data Processing Systems,” Ser. No. 09/438,437. In an embodiment of the present invention, multicast connections may be used so that transmissions are not delayed due to an unsuccessful transmission to one machine. 
     If the transmission to an endpoint or repeater, as appropriate depending on the stage of the distribution, is successfully completed, then a report back to the sender will be sent in step  374 . Additionally, a reference counter is maintained that holds a count of the number of endpoints and repeaters requiring the distribution. In step  376 , the reference counter is decremented in response to the successful transfer to the target, step  372 . 
     In step  378 , it is determined if the reference counter is zero, and the distribution is not tagged for permanent storage. If so, then all endpoints using the distribution have successfully received the data, and in step  384 , the temporarily stored data is deleted from the depot, and methodology  300  terminates. If, however, in step  378 , all the targets requiring the distribution have not received the distribution, then the reference count is not zero. Then step  378  proceeds by the “Yes” branch to step  380  and it is determined if the distribution is temporary and the distribution life span has expired. If the life span of the data transmission times out as determined in step  380 , methodology proceeds to step  376  to decrement the reference counter. The reference counter is decremented to zero. The distribution is then deleted is step  384  as step  378  takes the “Yes” branch. Additionally, in step  384  a report is sent back to the source system including a list of the endpoints that did not receive the data. 
     Returning to step  380 , if otherwise, it is determined that the distribution has not timed out, it is then determined if a delete request has been received, step  385 . In an embodiment of the present invention, a delete request may be initiated by a system administrator via a graphical user interface (GUI) provided by the depoting repeater. If a delete request has been received, the data is deleted in step  384  as previously described. Otherwise, methodology  300  returns to step  370  via the “No” branch of step  385  and continues to send data to the targets until either all target repeaters/endpoints have received the distribution or the life span expires. 
     Returning to step  360 , when the distribution received by the depoting repeater is marked for permanent storage as discussed above in conjunction with step  315 , the data is loaded into permanent storage in step  392 . Methodology  300  then proceeds to step  370 , and distribution data transferred to the target endpoints as previously described. 
     In this way, distribution of bulk data may be held in repositories nearing, within a network architecture, to the using endpoints. Additionally, mobile endpoints, which are transiently connected to the network, may advantageously use depoting. When a mobile endpoint is connected to the network, it can query its local repository designated for that endpoint for any data transfer that is pending for it. For example, when a mobile endpoint is connected to a gateway such as repeater  118  in FIG. 1, it may query the gateway&#39;s depoted data bank to determine if an update is due. By depoting the data on a repeater, mobile endpoints can obtain the data therefrom (possibly in several steps as the mobile system connects and disconnects) without having to remain connected to the network to receive the distribution in its entirety. Furthermore, in conjunction with checkpoint restart, the sending repeater queries the target for the starting point of the distribution, and, if the target has all of the data, no transmission will be necessary. Checkpoint restart is described in detail in the commonly owned, co-pending U.S. Patent Application entitled “Apparatus for Restarting Interrupted Data Transfer and Method Therefor,” incorporated herein by reference. The using application can then process the data distribution while disconnected from the network. 
     The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter which form the subject of the claims of the invention. Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.