Method and system for archiving and compacting data in a data storage array

Embodiments of the present invention are directed to a method and system for archiving and compacting data in a data storage array. In one embodiment, a plurality of archivable files are identified, archived, and removed from a data storage array. A first configuration of the data storage array is then determined in which the free space on each of a plurality of data storage devices in the data storage array is calculated. A process is performed for moving a plurality of files from a first of the data storage devices to other data storage devices in the data storage array. The moving process is repeated to create a second configuration of the data storage array. In the second configuration, the first data storage device has a greater amount of free space than in the first configuration.

FIELD OF THE INVENTION

Embodiments of the present invention relate to managing data in data storage arrays. More specifically, the present invention pertains to a method and system for archiving and compacting data in a data storage array.

BACKGROUND OF THE INVENTION

For many organizations, managing data storage resources is a time consuming process which is frequently prone to error. Data storage systems can quickly fill with redundant or outdated files which are occasionally purged from the system. Frequently, it is difficult to determine the ownership of the files being purged or whether those files are actually still needed. This results in mistakenly removing needed files or allowing outdated or redundant files to remain on the data storage array. Furthermore, it is a time consuming process for a person to examine these files and try to determine which files should be purged and which files should remain on the storage array.

For example, software development is frequently performed by teams of developers working in geographically separated locations. These development teams will use a centralized data storage area for storing the particular software component they are working on. Typically, these components resemble directories having hundreds of subdirectories and files. Other software development teams can access this data to ensure their components work together, to import dependencies, as well as in the final integration of the software components.

During the development cycle, there are frequent updates to the components, as well as patches of previous component versions, and/or multiple component versions to allow for the dependencies of other software components. Thus, in a short amount of time the centralized data storage area can fill with redundant or outdated files which must be purged. Occasionally, older files may be extremely stable components and are still considered the most current version. This complicates manually selecting and purging files because the age of the file is not an accurate indicator of whether the file should be purged. Additionally, this complicates identifying files which may be candidates for archival.

Often a list is distributed of the files which are to be purged. This allows developers to identify files which should remain in the centralized data storage area. However, it is a time consuming task for the developers to search the data component hierarchy to determine which files should be purged. This is increasingly true as directory structures become larger and more complex.

Thus, prior art methods for managing centralized data storage systems are time consuming and prone to error. Specifically, there is no method for automatically identifying files which are candidates for archival or removal from the data storage array.

Additionally, there is no method for consolidating files across multiple storage devices (e.g., multiple disk drives in a data storage array). Currently, compacting data in data storage arrays involves compacting each data storage device in the data storage array as a separate resulting in haphazard distribution of available free space among the separate devices. This results in less than optimal utilization of the free space in the data storage array. For example, collectively the free space in the data storage array may be large enough to accommodate large files and directories. However, prior art methods for compacting data result in multiple smaller areas of free space which, individually, may not be large enough to accommodate the larger files.

SUMMARY OF THE INVENTION

Accordingly, a need exists for a method and system for automatically identifying files which can be archived. While meeting the above stated need, it would be advantageous to provide a system which can consolidate free in data storage arrays comprising multiple data storage devices.

Embodiments of the present invention provide a method and system for archiving and compacting data in a data storage array. Embodiments of the present invention facilitate automatically identifying files which are candidates for archiving. According to one embodiment of the present invention, files are assigned a time period for maintaining the files in a data storage array. When the time period expires, users can be notified of an impending archival process in order to identify files which should be exempted from the list of archival candidate files. Files which are not identified as exemptions are automatically archived and removed from the data storage array.

Additionally, embodiments of the present invention facilitate compacting the remaining files across multiple data storage devices in the data storage array in order to consolidate free space into fewer, larger areas. In one embodiment, the free space on each of the storage devices in the data storage array is determined. A drive list is created in which the storage devices are sorted in descending order based upon their amount of free space. The storage device having the largest amount of free space is set aside to support the compacting process. The files on the largest remaining storage device are then sorted in descending order based upon their size. The largest file is compared to a minimum size threshold and, if it exceeds the minimum size threshold, is moved to a second storage device in the data storage array. This process continues until there are no more files remaining on the storage device larger than the minimum size threshold. The process is then repeated upon the next storage device in the drive list, etc.

Embodiments of the present invention are directed to a method and system for archiving and compacting data in a data storage array. In one embodiment, a plurality of archivable files are identified, archived, and removed from a data storage array. A first configuration of the data storage array is then determined in which the free space on each of a plurality of share storage devices in the data storage array is calculated. A process is performed for moving a plurality of file entities from a first of the shared data storage devices to other shared data storage devices in the data storage array. The moving process is repeated to create a second configuration of the data storage array. In the second configuration, the first shared data storage device has a greater amount of free space than in the first configuration.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Notation and Nomenclature

It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout the present invention, discussions utilizing terms such as “archiving,” “determining,” “calculating,” “performing,” “recognizing,” “creating,” “identifying,” “removing,” “comparing,” “adding,” “selecting,” “moving,” “sorting,” “repeating,” “excluding,” “designating” or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

With reference toFIG. 1, portions of the present invention are comprised of computer-readable and computer-executable instructions that reside, for example, in computer system100which is used as a part of a general purpose computer network (not shown). It is appreciated that computer system100ofFIG. 1is exemplary only and that the present invention can operate within a number of different computer systems including general-purpose computer systems, embedded computer systems, and stand-alone computer systems.

In the present embodiment, computer system100includes an address/data bus101for conveying digital information between the various components, a central processor unit (CPU)102for processing the digital information and instructions, a volatile main memory103comprised of volatile random access memory (RAM) for storing the digital information and instructions, and a non-volatile read only memory (ROM)104for storing information and instructions of a more permanent nature. In addition, computer system100also includes a data storage array105for storing vast amounts of data. There are a variety of implementations of data storage array105upon which embodiments of the present invention may be utilized including: Storage Area Network (SAN) systems, Network Attached Storage (SAN) systems, Redundant Arrays of Multiple Disks (RAID), etc. It should be noted that the software program for archiving and compacting data of the present invention can be stored either in volatile memory103, data storage array105, or in an external storage device (not shown).

Devices which are optionally coupled to computer system100include a display device106for displaying information to a computer user, an alpha-numeric input device107(e.g., a keyboard), and a cursor control device108(e.g., mouse, trackball, light pen, etc.) for inputting data, selections, updates, etc. Computer system100can also include a mechanism for emitting an audible signal (not shown).

Furthermore, computer system100can include an input/output (I/O) signal unit (e.g., interface)109for interfacing with a peripheral device110(e.g., a computer network, modem, mass storage device, etc.). Accordingly, computer system100may be coupled in a network, such as a client/server environment, whereby a number of clients (e.g., personal computers, workstations, portable computers, minicomputers, terminals, etc.) are used to run processes for performing desired tasks. In particular, computer system100can be coupled in a system for archiving and compacting data in a data storage array.

FIG. 2is a data flow and block diagram showing components used in the compacting of data in accordance with embodiments of the present invention.FIG. 2will be referenced in conjunction with the following discussions to clearly describe embodiments of the present invention.

FIG. 3is a flowchart of a computer implemented method300for archiving and compacting data in a centralized data storage array in accordance with embodiments of the present invention. In step310ofFIG. 3, a plurality of files are archived. In embodiments of the present invention, all entities on data storage array105, whether referring to a single file or group of related files (e.g., a directory), are generally managed in a similar manner. In one embodiment of the present invention, when a file is initially stored in data storage array105, a time period for maintaining the file in the data storage array is assigned to the file. In one embodiment of the present invention, the archival process is initiated automatically at a predefined time interval.

Files which are candidates for archival are automatically identified when the time period for maintaining the files in the data storage array has expired. In one embodiment, a list of archival candidates is distributed to allow users to create ad-hoc exemptions to the archival rules. For example, if access to a file is still needed beyond the defined time period, the time period for maintaining the file on data storage array105can be extended. In one embodiment of the present invention, archiving comprises creating a copy of the selected files for storage at a remote location. The archived files, as well as files which are no longer needed, can then be removed from data storage array105. The method for automatically archiving data is described in greater detail inFIG. 4.

In step320ofFIG. 3, a first configuration of data storage array105is determined. After removing archived files and/or other unneeded files, it is likely that the individual data storage devices in data storage array105are fragmented. In embodiments of the present invention, free space in data storage array105is consolidated by moving files between the data storage devices of data storage array105to create larger areas of free space on some of the data storage devices. This allows users to store larger files on data storage array105after the compacting process has been performed. A first step in moving the files is to determine the first configuration of data storage array105. In one embodiment of the present invention, the amount of free space and the total capacity of each of the data storage devices is determined.

In step330ofFIG. 3, a process is performed for moving a plurality of files from a first data storage device in data storage array105to a second data storage device in data storage array to create a second configuration of the data storage array. Once the configuration of data storage array105has been determined, a process is then performed which moves files between the data storage devices. Again, in one embodiment of the present invention, all file entities (e.g., a single file or a group of related files such as a directory) are moved in a similar manner. Files are moved from a first data storage device to a second data storage device to create second configuration of data storage array105in which free space consolidated in fewer, larger areas. In one embodiment of the present invention, the data storage devices are sorted into a list (e.g., drive list210ofFIG. 2) and the process for moving files is performed sequentially upon the list of data storage devices. For example, all of the files (e.g., file list230ofFIG. 2) on the first data storage device which are within assigned parameters are moved from the first data storage device to a second data storage device within data storage array105. The process for moving files between data storage devices is discussed in greater detail inFIG. 5.

In one embodiment of the present invention, the process for moving files is performed upon the half of the data storage devices having the greatest amount of free space. For example, if there are 30 data storage devices in the data storage array, the process for moving files is repeated until files have been moved from the 15 data storage devices having the greatest amount of free space. This expedites the moving process because at some point the benefit derived from moving files between the various data storage devices is out-weighed by the amount of time used to perform the process.

In so doing, the present invention provides a method and system for archiving and compacting data in a data storage array. Files are automatically identified which are candidates for consolidation. This reduces the workload associated with manually identifying files which can be archived. Additionally, the possibility of error in identifying files which are candidates for archival is greatly reduced. The present invention also provides a method and system for compacting files in a data storage array which consolidates available free space in the data storage array as a whole rather than a collection of separate data storage devices.

FIG. 4is a flowchart of a computer implemented method310for archiving data in accordance with one embodiment of the present invention. Process310ofFIG. 4corresponds to step310ofFIG. 3.

In step410ofFIG. 4, an assigned logical date associated with a file is recognized. In accordance with embodiments of the present invention, the logical date defines the beginning of a time period for maintaining a file in data storage array105. For example, in a software development environment the logical date can be a milestone in the development cycle defining when a software component should be available in data storage array105for access by other development teams. In accordance with embodiments of the present invention, the logical date can be assigned by a user when they initially store the file in data storage array105. Alternatively, a logical date (e.g., a system timestamp) can be automatically assigned to the file if a user does not assign a logical date to the file.

In step420ofFIG. 4, an assigned category associated with a file is recognized. In accordance with embodiments of the present invention, the assigned category defines a length of time for maintaining the file in data storage array105. For example, a “category A” file is kept in data storage array105for 3 weeks after the logical date of step410, while a “category B” file is kept for 2 weeks after the logical date, etc. Thus, the logical date in combination with the assigned category define a specific time period for maintaining a given file in data storage array105. Additionally, the category can be used for implementing other rules or policies on specific files. This is discussed in greater detail in the discussion of step430.

In step430ofFIG. 4, the file is added to a list of candidate files. In one embodiment, a query is performed in which the length of time defined by the assigned category of step420is subtracted from the current date to define an archival date. If the logical date of step410precedes the archival date, the file is added to the list of candidate files. In other words, if the time window for maintaining the file in data storage array105has elapsed, the file is automatically added to a list of candidate files. The list of candidate files identifies files which are eligible for archival but have not yet been selected for archival.

For example, if the current date is January 31, and a file has a predetermined time period of 3 weeks, the archival date is January 10. In one embodiment, any file with a logical date which precedes the archival date is automatically added to the candidate list. Therefore, if a file has a logical date of January 9 or before, it is automatically added to the list of candidate files.

Additionally, in one embodiment of the present invention, files lacking either an assigned logical date or an assigned category are automatically added to the list of candidate files. This facilitates archival of files which were, for example, stored in data storage array105prior to the implementation of the present invention.

In accordance with embodiments of the present invention, files can be identified as exceptions to the archival process according to the category which has been assigned to them. Files preserved by policies or business rules are queried out of data storage array105and filtered out of the candidate list. For example, a policy can be defined as:if a file has a logical date D earlier than the archival date, and a real timestamp later than the archival date, then all the files in the same category with logical timestamps later than D will also remain in data storage array105.

In step440ofFIG. 4, an archivable file is identified from the list of candidate files and a copy of the archivable file is created. In one embodiment, a list of the candidate files is distributed to allow users to see what files are on the archivable file list. Additionally, users can then identify ad-hoc exceptions to the archival process. For example, if a particular file is needed for a longer period than was originally anticipated, a user can identify that file as an ad-hoc exception to the archival process to extend the time period for maintaining that particular file in data storage array105. Files which are identified by users as ad-hoc exceptions are removed from the candidate entities list. A copy of each of the files on the archivable file list is made for storage at a remote location.

In step450ofFIG. 4, the archivable file is removed from data storage array105. After a copy has been made of the archivable files, the archivable entities are removed from data storage array105. Additionally, files which are no longer needed (e.g., files not archived and not identified as exceptions to the archival process) are removed from data storage array105. In one embodiment of the present invention, information is maintained to facilitate future recovery of the file if necessary. For example, a repository file number, symbolic link, real path, archive date etc., is maintained to facilitate retrieving the archived file becomes necessary.

Thus, embodiments of the present invention provide a method for automatically identifying files which are candidates for archival. This reduces the possibility of human error in identifying archivable files while reducing the workload for users. Additionally, the method of the present can be adapted to the unique requirements and policies of the user.

FIG. 5is a flowchart of a computer implemented process330for moving a plurality of files between a plurality of data storage devices in a data storage array in accordance with embodiments of the present invention. Process330ofFIG. 5corresponds to step330ofFIG. 3. In step510ofFIG. 5, a data storage device in data storage array105is selected.FIG. 6is a flowchart showing in greater detail one embodiment of process510for selecting a data storage device.

Referring now to step610ofFIG. 6, the plurality of data storage devices is sorted in descending order. In one embodiment of the present invention, the descending order is based upon the amount of free space remaining on each of the data storage devices as determined in step320ofFIG. 3. Thus, a list of the data storage devices (e.g., drive list210ofFIG. 2) is created wherein the device with the greatest amount of free space (e.g., device211ofFIG. 2) is at the top of the list and the device with the least amount of free space (e.g., device215ofFIG. 2) is at the bottom of the list.

In step620, ofFIG. 6, a data storage device in data storage array105is selected as a support device (e.g., support device240ofFIG. 2). In one embodiment of the present invention, the data storage with the greatest amount of free space device (e.g., device211ofFIG. 2) is set aside for use as a support device240. In accordance with embodiments of the present invention, support device240can be used as temporary storage space when moving files between the various data storage devices in data storage array105. Support device240is removed from drive list210to prevent including it in subsequent steps for compacting data.

In step630ofFIG. 6, a data storage device in the shared storage array is selected as the first data storage device to be compacted. In one embodiment of the present invention, the remaining data storage device having the greatest amount of free space (e.g., device212ofFIG. 2) is selected as the first data storage device to be compacted. Therefore, the first data storage device has less free space than support device240, but has a greater amount of free space than any other data storage device in data storage array105. Thus, a data storage device is selected for compacting from data storage array105in accordance with embodiments of the present invention.

Referring now to step520ofFIG. 5, a file from the first data storage device is selected.FIG. 7is a flowchart showing in greater detail process520for selecting a file in accordance with embodiments of the present invention. In step710ofFIG. 7, the size of each file on data storage device212is calculated.

In step720ofFIG. 7, the files are sorted. In one embodiment of the present invention, a file list220is created which lists the files on device212sorted in descending order based upon the size of the file. For example, the largest file on file list220(e.g., file221ofFIG. 2) is at the top of the list220followed by successively smaller files with the smallest file (e.g., file224ofFIG. 2) at the bottom of the list220.

In step730ofFIG. 7, a file is selected and compared to a size threshold. In one embodiment of the present invention, the largest file (e.g., file221ofFIG. 2) is selected as the first file for moving. In accordance with embodiments of the present invention, the each file being moved is compared with a size threshold. In one embodiment, the size threshold defines the minimum size of a file being moved. While the present embodiment recites the size threshold defining the minimum size of a file being moved, the present invention is well suited utilizing the size threshold to define a variety of parameters.

Referring now to step530ofFIG. 5, the selected file (e.g., file221ofFIG. 2) is then moved from the first data storage device (e.g., device212ofFIG. 2) to a second data storage device in data storage array105(e.g., device214ofFIG. 2).FIG. 8is a flowchart of a process530for moving a file from a first data storage device to a second data storage device in accordance with one embodiment of the present invention. Process530ofFIG. 8corresponds to step530ofFIG. 5.

In step810ofFIG. 8, each data storage is compared with a predetermined receiving threshold. For example, each of the data storage devices which is not the support device or the first data storage device being compacted can be added to a list of candidate receiving devices (e.g., candidate receiving list230ofFIG. 2). In one embodiment of the present invention, the predetermined receiving threshold is a percentage of the amount of free space compared to the total capacity of the data storage device and is unique to each of the candidate receiving devices. This receiving threshold can be used to identify storage devices which may not have sufficient capacity for receiving additional files. For example, the receiving threshold can be set so that a data storage device with free space less than 20% of its total capacity is not eligible for receiving additional files. In one embodiment of the present invention, devices not having enough free space are removed from the list of candidate receiving devices230.

In step820ofFIG. 8, one of the data storage devices (e.g., device214ofFIG. 2) is selected to receive the file from the first data storage device. There are a variety of rules for selecting the receiving data storage device. For example, in one embodiment, the first data storage device on the list of candidate receiving devices having enough free space to accommodate the file is as the receiving data storage device. In another embodiment, the data storage device having an amount of free space which most closely matches the size of the file being moved is selected as the receiving device.

In step830ofFIG. 8, the file is moved from the first data storage device to the receiving data storage device. The selected file is moved from the data storage device upon which it resides to a second data storage device in data storage array105. For example, file221is moved from data storage device212to data storage device224if device224exceeds its receiving threshold and has sufficient space for receiving file221.

In step840ofFIG. 8, the amount of free space on the receiving data storage device is recalculated. After receiving the file from the first data storage device, the amount of free space on the receiving data storage device214is recalculated and compared with its total capacity to determine whether device224is eligible for receiving additional files. If receiving data storage device214is still within the receiving threshold, it will remain on the list of candidate receiving devices230, otherwise, it will be removed from the list of candidate receiving devices230. In accordance with embodiments of the present invention, step530is repeated until there are no longer any files remaining on the data storage device that are larger than the minimum size threshold. When this occurs, the data storage device being compacted is removed from the list of data storage devices and the compacting process is repeated. For example, the amount of free space on the remaining data storage devices is determined, the devices are sorted according to the amount of free space on each device, and the process for moving files is initiated upon the device (e.g., device213ofFIG. 2) having the greatest amount of free space.

An example of the high-level logic describing process530can be exemplified in the following pseudo-code where “driveList” describes a list of data storage devices in data storage array105and fileList describes a list of the files on a given data storage device:for a file in fileList {if (size (file)>FILE_SIZE_THRESHOLD) {s<- find slot on a Drfrom (D3. . . Dx) using first fit rule:(or best-fit rule, or other rule defined for the specificsituation)if (slot s found) {move file to s:decrease free space on Drby size(file);if (free space (Dr)/space(Dr)<threshold) {driveList <_driveList-Dr:}}}}

In accordance with embodiments of the present invention, the number of times this process is performed can be set as a parameter. For example, if the parameter is set to only perform one “loop” of process330, files will only be moved from the first data storage device in data storage array105(e.g., device211ofFIG. 2). If the parameter is set to perform two loops, only two data storage devices (e.g., devices211and212ofFIG. 2) will be compacted.

In one embodiment of the present invention, another parameter allows a user to determine which devices in data storage array105will be compacted. For example, in a data storage array having 36 data storage devices, the data compacting process could be limited to devices D4–D14.

In so doing, a process for archiving and compacting data in a data storage array is performed in accordance with embodiments of the present invention. The process for compacting data in the present invention consolidates free space in the data storage array as a whole rather than consolidating free space in each data storage device separately. This results in fewer, larger areas of free space in the data storage array which can accommodate larger files.

FIGS. 9A and 9Bare a flowchart of a computer implemented method900for archiving data stored in a centralized data storage array (e.g., data storage array105ofFIG. 1) in accordance with embodiments of the present invention. Method900is used to automatically identify files which are candidates for archiving and for archiving and removing the files from a data storage array.

In step905ofFIG. 9A, a logic operation is performed to determine whether a logical date has been assigned to a selected file (e.g., file221ofFIG. 2). If a logical date is recognized, the selected file proceeds to step910ofFIG. 9A, otherwise, the selected file is automatically added to a list of candidate files (e.g., candidate list935ofFIG. 9B). The logical date defines a beginning of a time period for maintaining the selected file in data storage array105.

In step910ofFIG. 9A, a logic operation is performed to determine whether a category has been assigned to a selected file. If a category has been assigned to the selected file, the selected file proceeds to step920ofFIG. 9A, otherwise, the selected file is automatically added to a list of candidate files (e.g., candidate list935ofFIG. 9B). The category is used to define the length of the time period for maintaining the selected file in data storage array105.

In step915ofFIG. 9A, the current date is determined.

In step920ofFIG. 9A, the time period defined by the category of step910is subtracted from the current date. For example, if the time period for maintaining the selected file in data storage array105is 3 weeks, and the current date is January 31, an archival date (e.g., archival date925ofFIG. 9A) of January 10 is defined.

In step930ofFIG. 9A, a logic operation930is to determine whether the time period for maintaining the selected file in data storage array has elapsed. For example, in one embodiment of the present invention, if the logical date precedes the archival date, the time period for maintaining the selected file has elapsed and the selected file will automatically be added to candidate list935ofFIG. 9B. In the example cited above, it the logical date is January 4, and the archival date is January 10, the selected file will automatically be added to candidate list935. If the time period for maintaining the selected file on data storage array has not elapsed, the file will remain on data storage array105(e.g., step931ofFIG. 9A).

Referring now to step940ofFIG. 9B, a logic operation is performed to determine whether any of the files on candidate list935are exceptions to the archival process. In one embodiment, category910ofFIG. 9Acan also include policies or rules which will identify a selected file as an exception to the archival process. Files preserved by policies or business rules are queried out of data storage array105and filtered out of the candidate list. For example, a policy can be defined as:if a file has a logical date D earlier than the archival date, and a real timestamp later than the archival date, then all the files in the same category with logical timestamps later than D will also remain in data storage array105.

In accordance with embodiments of the present invention, a list can be distributed to allow users to identify files which are ad-hoc exceptions to the archival process. Ad-hoc exceptions to the archival process are files which are not exempted from archival based upon their category, but which should be maintained in data storage array105. For example, if access to a file is still necessary after the time period for maintaining the file on data storage array105has expired, a user can identify that file as an ad-hoc exception to the archival process. The file will then be removed from candidate list935and the selected file will remain on data storage array105(e.g., step941ofFIG. 9B).

In step945ofFIG. 9B, an archivable file list is created. In one embodiment of the present invention, archivable file list945comprises the files from candidate list935which were not identified as exceptions to the archival process. In one embodiment of the present invention, users can identify files which are no longer needed on data storage array105and which can be deleted. A copy of each of the files on archivable file list945is made for storage at a remote location. In one embodiment of the present invention, information is kept to facilitate future recovery of the file if necessary. For example, a repository file number, symbolic link, real path, archive date etc., is kept to facilitate retrieving the archived file becomes necessary.

In step950ofFIG. 9B, files on the archivable file list are copied for storage at a remote location. In step955ofFIG. 9B, the archived files are removed from data storage array105. Once a copy of one of the archivable files is created, the file is removed from data storage array105.

FIGS. 10A,10B, and10C are a flowchart of a computer implemented method1000for compacting data stored in a data storage array in accordance with embodiments of the present invention.

In step1001ofFIG. 10A, the free space on each of the data storage devices comprising data storage array105is determined. In embodiments of the present invention, the total capacity of each of the data storage devices is determined as well.

In step1002ofFIG. 10A, the data storage devices are sorted in descending order. In embodiments of the present invention, the data storage devices are sorted in descending order based upon the amount of free space as determined in step1001. Referring toFIG. 2, drive list210lists the data storage devices comprising data storage array105. The data storage devices are listed in descending order (e.g., device211–device215) based upon the amount of free space remaining on each drive.

In step1003ofFIG. 10A, a support device is selected. In one embodiment, the data storage device with the largest amount of free space is selected as the support device. The support device can be used to facilitate moving files between the data storage devices during the compacting process. Referring toFIG. 2, device211is selected as support device240because it has a greatest amount of free space than any other data storage device in data storage array105.

In step1004ofFIG. 10A, a data storage device is selected for compacting. In one embodiment of the present invention, the data storage device with the largest amount of free space of the remaining data storage devices on drive list210(e.g., device212ofFIG. 2) is selected as the first data storage device to be compacted.

In step1005ofFIG. 10A, the file sizes on the selected data storage device are calculated. In embodiments of the present invention, the files residing on the selected data storage device are sorted in descending order based upon their size. Referring toFIG. 2, file list220lists files212–224in descending order based upon the size of each file.

In step1006ofFIG. 10A, the largest file on the selected data storage device (e.g., file221ofFIG. 2) is selected. In accordance with embodiments of the present invention, the largest file on a data storage device selected for compacting is the first file to be moved.

Referring now toFIG. 10B, in step1007a logic operation is performed in which the file from step1006is compared with a predetermined size threshold. In embodiments of the present invention, the size threshold defines the minimum size a file must be in order to be moved to a different data storage device. In one embodiment, the size threshold is set per system (e.g., computer system100ofFIG. 1).

If the selected file (e.g., file221ofFIG. 2) is larger than the predetermined size threshold, process1000proceeds to1011ofFIG. 10C. If the selected file is smaller than the predetermined size threshold, there are no files on the selected data storage device which are large enough to move. Process1000then proceeds to step1008.

In step1008ofFIG. 10B, a logic operation is performed to determine whether another data storage device of data storage array105should be compacted.

In embodiments of the present invention, parameters can be set to limits the number of data storage devices being compacted. For example, a parameter can be set limiting the compacting process to a specific number of devices such as the first five devices in data storage array105. In another embodiment, a parameter can be set limiting the compacting process to a specific set of data storage devices in data storage array105. For example, if data storage array105is comprised of 30 data storage devices, the parameter can be set to only compact drives5–12.

In another embodiment of the present invention, the compacting process is performed upon no more than one half of the total number of data storage devices in data storage array105. Thus, if data storage array is comprised of 36 data storage devices, the compacting process is performed upon the first 18 data storage devices on drive list210.

Referring again to step1008ofFIG. 10B, if there are no more data storage devices in data storage array105to be compacted (e.g., no more data storage devices on drive list210ofFIG. 2), process1000ends. If there are other data storage devices remaining on drive list210, process1000continues at step1018.

In step1018ofFIG. 10B, the previously selected data storage device is excluded from process1000. For example, at some point there are no files remaining on device212for moving to other data storage devices. Data storage device212is then removed from drive list212as it will not be included in process1000.

In step1020ofFIG. 10B, the free space on the data storage devices remaining on drive list210is recalculated. At this point, process1000returns to step1004ofFIG. 10A.

Referring now to step1011ofFIG. 10C, a receiving data storage device is (e.g., device215ofFIG. 2) selected. A second data storage device (e.g., device215ofFIG. 2) is selected from drive list210to receive the selected file from data storage device212.

In step1012ofFIG. 10C, a logic operation is performed in which the receiving threshold of the selected receiving device is compared with the amount of available free space. In embodiments of the present invention, the amount of free space is compared to the total capacity of the selected data storage device, and if the amount of free space exceeds the receiving threshold, the selected device can receive files from other data storage devices. For example, if the receiving threshold is set at 20%, at least 20% of the total capacity of the selected data storage device should be free space.

If the amount of free space is larger than the receiving threshold, process1000continues to step1013. If the amount of free space on the selected receiving data storage device is less than the receiving threshold, process1000continues to step1016.

In step1016ofFIG. 10C, the selected receiving device is removed from the list of data storage devices. Because the amount of free space does not exceed the receiving threshold, the selected data storage device is not a candidate for receiving files during process1000and is removed from drive list210. At this point, a new receiving data storage device is selected and process1000proceeds from step1011.

Referring now to step1013ofFIG. 10C, a logic operation is performed in which the size of the selected file is compared with the available free space on the receiving data storage device. If the selected file is larger than the amount of free space on the selected data storage device, process1000continues to step1017.

In step1017ofFIG. 10C, a different receiving device is selected from drive list210. If the selected file is smaller than the amount of available free space on the selected receiving device, process1000proceeds to step1014. In step1014ofFIG. 10C, the selected file is moved to the selected receiving data storage device. In step1015ofFIG. 10C, the amount of free space remaining on the receiving device is recalculated. This is to determine whether the data storage device has enough available free space after receiving the file to exceed the receiving threshold and receive additional files.

At this point, process1000returns to step1006and selects the largest file remaining on file list220for moving to another data storage device.

In so doing, a method for compacting data in a data storage array is performed in accordance with one embodiment of the present invention. The present invention consolidates free space in data storage arrays by compacting the data storage array as a whole rather than a collection of separate data storage devices.

The preferred embodiment of the present invention, a method and system for archiving and compacting data in a data storage array, is thus described. While the present invention has been described in particular embodiments, it should be appreciated that the present invention should not be construed as limited by such embodiments, but rather construed according to the following claims.