Source: http://www.google.com/patents/US20040039891?dq=patent:5881444
Timestamp: 2015-08-02 11:20:21
Document Index: 113103379

Matched Legal Cases: ['art 400', 'art 400', 'art 500', 'art 500', 'art 600', 'art 600', 'art 700', 'art 700', 'art 800', 'art 800']

Patent US20040039891 - Optimizing storage capacity utilization based upon data storage costs - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsTechniques for optimizing capacity utilization among multiple storage units based upon costs associated with storing data on the storage units. Embodiments of the present invention automatically determine when data movement is needed to optimization storage utilization for a group of storage units. According...http://www.google.com/patents/US20040039891?utm_source=gb-gplus-sharePatent US20040039891 - Optimizing storage capacity utilization based upon data storage costsAdvanced Patent SearchPublication numberUS20040039891 A1Publication typeApplicationApplication numberUS 10/650,360Publication dateFeb 26, 2004Filing dateAug 27, 2003Priority dateAug 31, 2001Publication number10650360, 650360, US 2004/0039891 A1, US 2004/039891 A1, US 20040039891 A1, US 20040039891A1, US 2004039891 A1, US 2004039891A1, US-A1-20040039891, US-A1-2004039891, US2004/0039891A1, US2004/039891A1, US20040039891 A1, US20040039891A1, US2004039891 A1, US2004039891A1InventorsAlbert Leung, Giovanni Paliska, Bruce Greenblatt, Claudia ChandraOriginal AssigneeArkivio, Inc.Export CitationBiBTeX, EndNote, RefManPatent Citations (5), Referenced by (90), Classifications (8), Legal Events (1) External Links: USPTO, USPTO Assignment, EspacenetOptimizing storage capacity utilization based upon data storage costs
DETAILED DESCRIPTION OF THE INVENTION [0026] In the following description, for the purposes of explanation, specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent that the invention may be practiced without these specific details. [0027] For purposes of this application, migration of a file involves moving the file (or a data portion of the file) from its original storage location on a source storage unit to a target storage unit. A stub or tag file may be stored on the source storage unit in place of the migrated file. The stub file occupies less storage space than the migrated file and generally comprises metadata related to the migrated file. The stub file may also comprise information that can be used to determine the target storage location of the migrated file. When a user or application accesses a stub on a source storage unit, a recall operation is performed. The recall transparently restores the migrated (or remigrated) file to its original storage location on the source storage unit for the user or application to access. [0028] For purposes of this application, remigration of a file involves moving a previously migrated file from its present storage location to another storage location. The stub file information or information stored in a database corresponding to the remigrated file may be updated to reflect the storage location to which the file is remigrated. [0029] For purposes of this application, unless specified otherwise, moving a file from a source storage unit to a target storage unit is intended to include migrating the file from the source storage unit to the target storage unit, or remigrating a file from the source storage unit to the target storage unit, or simply changing the location of a file from one storage location to another storage location. Movement of a file may have varying levels of impact on the end user. For example, in case of migration and remigration operations, the movement of a file is transparent to the end user. The use of techniques such as symbolic links in UNIX, Windows shortcuts may make the move somewhat transparent to the end user. The move may also be accomplished without leaving any links, shortcuts, or stub/tag files, which may impact the way the end user accesses the file. [0030]FIG. 1 is a simplified block diagram of a storage environment 100 that may incorporate an embodiment of the present invention. Storage environment 100 depicted in FIG. 1 is merely illustrative of an embodiment incorporating the present invention and does not limit the scope of the invention as recited in the claims. One of ordinary skill in the art would recognize other variations, modifications, and alternatives. [0031] As depicted in FIG. 1, storage environment 100 comprises a plurality of physical storage devices 102 for storing data. Physical storage devices 102 may include disk drives, tapes, hard drives, optical disks, RAID storage structures, solid state storage devices, SAN storage devices, NAS storage devices, and other types of devices and storage media capable of storing data. The term “physical storage unit” is intended to refer to any physical device, system, etc. that is capable of storing information or data. [0032] Physical storage units 102 may be organized into one or more logical storage units (or logical devices) 104 that provide a logical view of underlying disks provided by physical storage units 102. Each logical storage unit (e.g., a volume) is generally identifiable by a unique identifier (e.g., a number, name, etc.) that may be specified by the administrator. A single physical storage unit may be divided into several separately identifiable logical storage units. A single logical storage unit may span storage space provided by multiple physical storage units 102. A logical storage unit may reside on non-contiguous physical partitions. By using logical storage units, the physical storage units and the distribution of data across the physical storage units becomes transparent to servers and applications. For purposes of description and as depicted in FIG. 1, logical storage units 104 are considered to be in the form of volumes. However, other types of storage units including physical storage units and logical storage units are also within the scope of the present invention. [0033] Storage environment 100 also comprises several servers 106. Server 106 may be data processing systems that are configured to provide a service. Each server 106 may be assigned one or more volumes from logical storage units 104. For example, as depicted in FIG. 1, volumes V1 and V2 are assigned to server 106-1, volume V3 is assigned to server 106-2, and volumes V4 and V5 are assigned to server 106-3. A server 106 provides an access point for the one or more volumes assigned to that server. Servers 106 may be coupled to a communication network 108. [0034] In FIG. 1, a storage management system/server (SMS) 110 is coupled to server 106 via communication network 108. Communication network 108 provides a mechanism for allowing communication between SMS 110 and servers 106. Communication network 108 may be a local area network (LAN), a wide area network (WAN), a wireless network, an Intranet, the Internet, a private network, a public network, a switched network, or any other suitable communication network. Communication network 108 may comprise many interconnected computer systems and communication links. The communication links may be hardwire links, optical links, satellite or other wireless communications links, wave propagation links, or any other mechanisms for communication of information. Various communication protocols may be used to facilitate communication of information via the communication links, including TCP/IP, HTTP protocols, extensible markup language (XML), wireless application protocol (WAP), Fiber Channel protocols, protocols under development by industry standard organizations, vendor-specific protocols, customized protocols, and others. [0035] SMS 110 is configured to provide storage management services for storage environment 100 according to an embodiment of the present invention. These management services include performing automated capacity management and data movement between the various storage units in the storage environment 100. The term “storage unit” is intended to refer to a physical storage unit (e.g., a disk) or a logical storage unit (e.g., a volume). According to an embodiment of the present invention, SMS 110 is configured to monitor and gather information related to the capacity usage of the storage units in the storage environment and to perform capacity management (including managing capacity based upon data storage costs) and data movement based upon the gathered information. SMS 110 may perform monitoring in the background to determine the instantaneous state of each of the storage units in the storage environment. SMS 110 may also monitor the file system in order to collect information about the files such as file size information, access time information, file type information, etc. The monitoring may also be performed using agents installed on the various servers 106 for monitoring the storage units assigned to the servers and the file system. The information collected by the agents may be forwarded to SMS 110 for processing according to the teachings of the present invention. [0036] The information collected by SMS 110 may be stored in a memory or disk location accessible to SMS 110. For example, as depicted in FIG. 1, the information may be stored in a database 112 accessible to SMS 110. The information stored in database 112 may include information 114 related to storage policies and rules configured for the storage environment, information 116 related to the various monitored storage units, information 118 related to the files stored in the storage environment, and other types of information 120. Various formats may be used for storing the information. As described below, the stored information may be used to perform capacity management based upon data storage costs according to an embodiment of the present invention. [0037] Information 116 related to the storage units may include information related to the cost of storing data on the storage units. For purposes of this application, for a storage unit the cost of storing data on that storage unit will be referred to as the “data storage cost” associated with the storage unit. The data storage cost for a storage unit may be provided by the manufacturer of the storage unit. The data storage cost for a storage unit may also be assigned by an administrator of the storage environment or by a user of the storage environment. [0038] The data storage cost for a storage unit may be expressed in various forms. According to one form, the storage cost may be expressed as a monetary value of storing data per unit of storage, for example, dollars-per-Gigabyte of storage. For example, the data storage cost for a first storage unit may be $1-per-GB, for a second storage unit may be $2-per-GB, for a third storage unit may be $5-per-GB, and the like. The data storage cost for an storage unit may also be expressed in the form of a label or category or classification, such as “low cost”, “high cost”, “medium cost”, “expensive”, “cheap”, etc. These labels/classifications/categories are generally assigned by a system administrator. According to the teachings of the present invention, the data storage costs associated with storage units may be used to classify the storage units into one or more groups. [0039]FIG. 2 is a simplified block diagram of SMS 110 according to an embodiment of the present invention. As shown in FIG. 2, SMS 110 includes a processor 202 that communicates with a number of peripheral devices via a bus subsystem 204. These peripheral devices may include a storage subsystem 206, comprising a memory subsystem 208 and a file storage subsystem 210, user interface input devices 212, user interface output devices 214, and a network interface subsystem 216. The input and output devices allow a user, such as the administrator, to interact with SMS 110. [0040] Network interface subsystem 216 provides an interface to other computer systems, networks, servers, and storage units. Network interface subsystem 216 serves as an interface for receiving data from other sources and for transmitting data to other sources from SMS 110. Embodiments of network interface subsystem 216 include an Ethernet card, a modem (telephone, satellite, cable, ISDN, etc.), (asynchronous) digital subscriber line (DSL) units, and the like. [0041] User interface input devices 212 may include a keyboard, pointing devices such as a mouse, trackball, touchpad, or graphics tablet, a scanner, a barcode scanner, a touchscreen incorporated into the display, audio input devices such as voice recognition systems, microphones, and other types of input devices. In general, use of the term “input device” is intended to include all possible types of devices and mechanisms for inputting information to SMS 110. [0042] User interface output devices 214 may include a display subsystem, a printer, a fax machine, or non-visual displays such as audio output devices, etc. The display subsystem may be a cathode ray tube (CRT), a flat-panel device such as a liquid crystal display (LCD), or a projection device. In general, use of the term “output device” is intended to include all possible types of devices and mechanisms for outputting information from SMS 110. [0043] Storage subsystem 206 may be configured to store the basic programming and data constructs that provide the functionality of the present invention. For example, according to an embodiment of the present invention, software code modules implementing the functionality of the present invention may be stored in storage subsystem 206. These software modules may be executed by processor(s) 202. Storage subsystem 206 may also provide a repository for storing data used in accordance with the present invention. For example, the information gathered by SMS 110 may be stored in storage subsystem 206. Storage subsystem 206 may also be used as a migration repository to store data that is moved from another storage unit. Storage subsystem 206 may also be used to store data that is moved from another storage unit. Storage subsystem 206 may comprise memory subsystem 208 and file/disk storage subsystem 210. [0044] Memory subsystem 208 may include a number of memories including a main random access memory (RAM) 218 for storage of instructions and data during program execution and a read only memory (ROM) 220 in which fixed instructions are stored. File storage subsystem 210 provides persistent (non-volatile) storage for program and data files, and may include a hard disk drive, a floppy disk drive along with associated removable media, a Compact Disk Read Only Memory (CD-ROM) drive, an optical drive, removable media cartridges, and other like storage media. [0045] Bus subsystem 204 provides a mechanism for letting the various components and subsystems of SMS 110 communicate with each other as intended. Although bus subsystem 204 is shown schematically as a single bus, alternative embodiments of the bus subsystem may utilize multiple busses. [0046] SMS 110 can be of various types including a personal computer, a portable computer, a workstation, a network computer, a mainframe, a kiosk, or any other data processing system. Due to the ever-changing nature of computers and networks, the description of SMS 110 depicted in FIG. 2 is intended only as a specific example for purposes of illustrating the preferred embodiment of the computer system. Many other configurations having more or fewer components than the system depicted in FIG. 2 are possible. [0047] Embodiments of the present invention perform automated capacity management and data movement between multiple storage units based upon costs associated with storing data on the storage units. The operation generally involves moving one or more files from a storage unit (referred to as the “source storage unit”) to one or more other storage units (referred to as “target storage units”). As described above in the “Background” section, in conventional HSM-type applications, in order to perform data movement, the administrator has to explicitly specify the file(s) to be moved, the source storage unit, and the target storage unit to which the files are to be moved. According to embodiments of the present invention, the administrator does not have to explicitly specify the file to be moved, the source storage unit, or the target storage unit. The administrator may only specify the data storage costs associated with the storage units and data movement is automatically performed between the storage units such that total utilized storage costs are minimized. The administrator may only specify groups of storage units to be managed (referred to as the “managed groups”) and the data storage costs associated with each managed group of storage units. Embodiments of the present invention are then able to automatically move data between the managed groups such that overall utilized storage costs are minimized. Embodiments of the present invention are also able to automatically determine when data movement is to be performed, determine a source storage unit, files to be moved, and one or more target storage units to which the selected file(s) are to be moved. [0048] According to an embodiment of the present invention, each managed group can include one or more storage units. The storage units in a managed group may be assigned or coupled to one server or to multiple servers. A particular storage unit can be a part of multiple managed groups. Multiple managed groups may be defined for a storage environment. [0049]FIG. 3 depicts three managed groups according to an embodiment of the present invention. The first managed group 301 includes four volumes, namely, V1, V2, V3, and V4. Volumes V1 and V2 are assigned to server S1 and volumes V3 and V4 are assigned to server S2. Accordingly, managed group 301 comprises volumes assigned to multiple servers. The second managed group 302 includes three volumes, namely, V4 and V5 assigned to server S2, and V6 assigned to server S3. Volume V4 is part of managed groups 301 and 302. Managed group 303 includes volumes V7 and V8 assigned to server S4. Various other managed groups may also be specified. [0050] According to an embodiment of the present invention, storage units are assigned or allocated to one or more managed groups based upon data storage costs associated with the storage units. As previously described, information identifying data storage costs for the storage units in a storage environment may be stored (e.g., stored as part of storage unit information 116 depicted in FIG. 1). In one embodiment, this cost information is analyzed and managed groups are automatically formed based upon the analysis. In this embodiment, storage units with data storage costs that fall within a certain cost range may be classified into one managed group, storage units with data storage costs that fall within another range may be classified into another managed group, and the like. Alternatively, all storage units having data storage costs above a user-configurable threshold value may be organized into one managed group and the other storage units may be organized into another managed group. For example, storage units in a storage environment may be classified into two managed groups: a “high cost” managed group comprising storage units whose data storage cost is above a user-configurable threshold value, and a “low cost” managed group comprising storage units whose data storage cost is below the user-configurable threshold value. For example, the user-configurable threshold may set at $4 per GB. [0051] The storage environment administrator may also pick and select storage units to be included in a managed group and assign a data storage cost for the managed group. For example, a user interface may be displayed on SMS 100 that displays a list of storage units in the storage environment that are available for selection and the data storage costs associated with the storage units. A user may then form managed groups by selecting one or more of the displayed storage units and assign data storage value to the managed groups. [0052] Managed groups based upon storage costs may also be automatically formed based upon storage data cost-related criteria specified by the administrator. According to this technique, an administrator may define cost criteria for a managed group and a storage unit is included in the managed group if it satisfies the cost criteria specified for that managed group. [0053] Multiple managed groups, each comprising one or more storage units, may thus be defined for a storage environment based upon data storage costs associated with the storage units. A data storage cost may be associated with each managed group based upon the cost criteria used for forming the group. The data storage cost for a managed group may be expressed as a dollar-per-GB, a category/label/classification (e.g., “high cost” group, “low cost” group, etc.), etc. [0054] The managed groups in a storage environment may be ranked relative to each other based upon the data storage costs associated with groups. For example, if two managed groups have been defined based upon data storage costs, one group may be classified as the “high cost” group (or “greater than $4-per-GB” group) while the other group may be classified as the “low cost group” (or “less than $4-per-GB” group). If three groups have been configured, a first group may be classified as the “high cost” group, a second group may be classified as the “medium cost” group, and a third group may be classified as the “low cost group”. Given a particular managed group, the ranking information is useful for determining groups that have greater data storage costs than the particular managed group and groups that have lower data storage costs than the particular managed group. [0055] It should be noted that in addition to data storage cost related criteria, other criteria related to other attributes of the storage units may also be used for forming managed groups. The other criteria may include a criterion related to volume capacity, a criterion related to the manufacturer of the storage device, a criterion related to device type (e.g., SCSI, Fibre Channel, IDE, NAS, etc.), and the like. However, for purposes of this application the managed groups refer to groups that are formed based upon data storage costs associated with the storage units and possibly other criteria. Accordingly, a storage unit is included in a particular managed group if the storage unit matches the cost criteria (and other specified criteria) specified for that particular managed group. A managed group based upon data storage costs may also include one or more other managed groups configured using other criteria. [0056] For each managed group, embodiments of the present invention automatically perform storage optimization for the storage units in the managed groups based upon the data storage costs associated with the storage units. FIG. 4 is a simplified high-level flowchart 400 depicting a method of optimizing storage capacity utilization and data storage costs according to an embodiment of the present invention. The method depicted in FIG. 4 may be performed by software modules executed by a processor, hardware modules, or combinations thereof. According to an embodiment of the present invention, the processing is performed by a policy management engine (PME) executing on SMS 110. Flowchart 400 depicted in FIG. 4 is merely illustrative of an embodiment of the present invention and is not intended to limit the scope of the present invention. Other variations, modifications, and alternatives are also within the scope of the present invention. For sake of description, the processing depicted in FIG. 4 assumes that the storage units are in the form of volumes. It should be apparent that the processing can also be applied to other types of storage units. [0057] As depicted in FIG. 4, processing is initiated upon detecting that used storage capacity for a volume in the storage environment has exceeded a user-configured threshold value (or alternatively, the available storage capacity of a volume in the storage environment has fallen below a user-configured threshold value) (step 402). The used storage capacity is the amount of the storage unit that is used or occupied. The available storage capacity is the portion of a storage unit that is available for storing data. As previously indicated, according to an embodiment of the present invention depicted in FIG. 1, SMS 110 is configured to monitor and gather information related to the utilization of the storage units in the storage environment. SMS 110 may perform the monitoring in the background to determine the instantaneous state of each of the storage units in the storage environment. The monitoring may also be performed using agents installed on the various servers 106 for monitoring the storage units assigned to the servers and the file system. Accordingly, the condition that is detected in step 402 may be detected by SMS 110. The condition may also be detected by other systems, devices, or application programs. The volume that is experiencing the condition detected in step 402 is referred to as the “source volume” or “source storage unit” as it represents a volume or storage unit from which data is to be moved in order to resolve the detected overcapacity condition. [0058] The managed group to which the volume experiencing the condition detected in step 402 belongs is then determined (step 404). A “target” managed group is then determined that has a lower data storage cost associated with it than the managed group determined in step 404 (step 406). As indicated above, the managed groups may be ranked relative to each other based upon the storage data costs information associated with the groups. This ranking information may be used to determine the managed group in step 406. For example, if a “high cost” managed group and a “low cost” managed group have been defined for a storage environment, and it is determined in step 404 that the volume experiencing an overcapacity condition belongs to the “high cost” managed group, then in step 406 the “low cost” managed group is selected. As another example, if a “high cost” managed group, a “medium cost” managed group, and a “low cost” managed group have been defined for a storage environment, and it is determined in step 404 that the volume experiencing an overcapacity condition belongs to the “high cost” managed group, then in step 406 either the “low cost” managed group or the “medium cost” managed group may be selected. [0059] A check is then made to determine if a target managed group was selected in step 406 (step 408). If no group was selected, it indicates that there is no other managed group in the storage environment with a data storage cost that is lower than the data storage cost associated with the managed group determined in step 404. In this case the processing is terminated. After termination, the managed groups of volumes continue to be monitored for the next condition that triggers the processing depicted in FIG. 4. If it is determined in step 408 that a managed group with a lower data storage cost associated with it was identified in step 406, then processing continues with step 410. [0060] A file is then selected to be moved from the volume experiencing the condition detected in step 402 (step 410). Various techniques may be used for selecting the file to be moved from the source volume. According to one technique, the largest file stored on the source volume is selected. According to another technique, the least recently accessed file may be selected to be moved. Other file attributes such as age of the file, type of the file, etc. may also be used to select a file to be moved. [0061] According to an embodiment of the present invention, the techniques described in U.S. patent application Ser. No. 10/232,875 filed Aug. 30, 2002 (Attorney Docket No. 21154-000210US) and techniques described below may be used to select the file to be moved from the source volume. According to these techniques, a data value score (DVS) is generated for the files stored on the source volume, and the file with the highest DVS is selected in step 410 for the move operation. Further description related to the use of DVSs for selecting files to be moved is discussed below. [0062] A volume to which the file selected in step 410 is to be moved is then selected from the target managed group of volumes determined in step 406 or step 416 (step 412). The volume (or storage unit in general) identified in step 412 is referred to as a “target volume” or “target storage unit” as it represents a storage unit to which data will be moved. The target volume selected in step 412 and the source volume may be assigned to the same or different servers. [0063] Various techniques may be used for selecting the target volume in step 412. According to one embodiment, the least full volume from the managed group of volumes determined in step 406 (or 416) is selected as the target volume. According to another embodiment of the present invention, the administrator may specify criteria for selecting a target, and a volume that satisfies the criteria is selected as the target volume. According to yet another embodiment, techniques described in U.S. patent application Ser. No. 10/232,875 filed Aug. 30, 2002 (Attorney Docket No. 21154-000210US), and techniques described below may be used to select a target volume in step 410. In this embodiment, a storage value score (SVS) (also referred to as the “relative storage valued score” or RSVS) is generated for the various volumes included in the managed group of volumes determined in step 406 or 416. A volume with the highest SVS is then selected as the target volume from among the volumes in the managed group. Further details related to generation of SVSs and uses of the SVSs to select a target volume are given below. [0064] A check is then made to determine if a volume was selected in step 412 (step 414). If no volume could be determined in step 412, then another previously unselected target managed group that has less data storage costs associated with it than the managed group of the source volume (i.e., the managed group determined in step 404) is selected (step 416). A check is then made to determine if a target managed group was selected in step 416 (step 418). If no group was selected it implies that there is no other target managed group with a data storage cost associated with it that is lower than the data storage cost of the managed group determined in step 404. In this case the processing depicted in FIG. 4 is terminated. Upon termination, the managed groups of volumes continue to be monitored for the next condition that triggers the processing depicted in FIG. 4. [0065] If it is determined in step 414 that a target managed group with a lower data storage cost associated with it was identified in step 412, then processing continues with step 422. The file selected in step 410 is then moved from the source volume to the target volume selected in step 412 (step 420). A check is then made to determine if the move operation was successful (step 422). If the move operation was unsuccessful, then the file selected in step 410 is restored back to it original location on the source volume (step 424). Processing then continues with step 410 and another file from the source volume is selected to be moved. [0066] If the move operation in step 420 was successful, then information identifying the new location of the file on the target volume is stored and/or updated (step 426). According to an embodiment of the present invention, if there is any stub file associated with the moved file, then the stub file information (or information stored in a database) may be updated to reflect the new location of the file on the target volume. In an alternative embodiment, other information may be left in the original location in the form of UNIX symbolic links, Window shortcuts, etc., or the administrator may need to inform users if the operation is to simply move (not migrate) the file. The information may also be stored or updated in a storage location (e.g., a database) accessible to SMS 110. [0067] The used storage capacity information for the source volume and the target volume to which the file is moved is updated to reflect the file move (step 428). [0068] A check is then made to see if the condition detected in step 402 that triggered the processing depicted in FIG. 4 has been resolved (step 430). For example, if the condition in step 402 was an overcapacity condition, a check is made in step 430 to determine if the overcapacity condition for the source volume has been resolved. If it is determined in step 430 that the condition has been resolved, then processing terminates for the condition detected in step 402. The volumes in the storage environment then continue to be monitored for the next condition that triggers the processing depicted in FIG. 4. [0069] If it is determined in step 430 that the condition detected in step 402 has not been resolved, then processing continues with step 410 wherein another file is selected to be moved from the source volume. Alternatively, processing may continue to select another source volume from the managed group determined in step 404. During the processing, the target volume selected in step 412 may be the same as or different from the previously selected target volume. The steps depicted in FIG. 4 are then repeated as described above. [0070] As described above, embodiments of the present invention provide the ability to automatically detect when an overcapacity condition (e.g., when the used storage capacity for a volume exceeds a user-configured threshold value) has been reached for a volume. A target volume is then automatically and dynamically determined for receiving files from the source volume to resolve the overcapacity condition of the source volume. The target volume is selected from a managed group that has a lower data storage cost associated with it than the managed group of the source volume. Accordingly, data is moved from a source volume to a target volume that has a lower storage data cost associated with it. [0071]FIG. 5 depicts another flowchart 500 depicting another method of optimizing capacity utilization based upon data storage costs associated with storage units according to an embodiment of the present invention. Flowchart 500 depicted in FIG. 5 is merely illustrative of an embodiment of the present invention and is not intended to limit the scope of the present invention. Other variations, modifications, and alternatives are also within the scope of the present invention. For sake of description, the processing depicted in FIG. 5 assumes that the storage units are in the form of volumes. It should be apparent that the processing can also be applied to other types of storage units. [0072] As depicted in FIG. 5, processing is initiated upon detecting that used storage capacity for a volume has exceeded a user-configured threshold value (or alternatively, the available storage capacity of a volume in the storage environment has fallen below a user-configured threshold value) (step 502). The condition may be detected using any of the techniques described above. The volume that is experiencing the condition detected in step 502 is referred to as the “source volume” or “source storage unit” as it represents a volume or storage unit from which data is to be moved in order to resolve the detected overcapacity condition. [0073] As part of step 502, the extent of the overcapacity for the source volume may also be determined. This may be determined by calculating the difference between the used storage capacity of the source volume and the user-configured threshold capacity value (e.g., extent of overcapacity=(used storage capacity of source volume)−(user-configured capacity threshold)). [0074] Volumes in the storage environment that have an associated data storage cost that is lower than the data storage cost associated with the volume experiencing the overcapacity condition detected in step 502 and that are available for storing data are then determined (step 504). [0075] A file to be moved from the source volume is then selected (step 506). Various techniques may be used for selecting the file to be moved from the source volume. According to one technique, the largest file stored on the source volume is selected. According to another technique, the least recently accessed file may be selected to be moved. Other file attributes such as age of the file, type of the file, etc. may also be used to select a file to be moved. [0076] According to an embodiment of the present invention, the techniques described in U.S. patent application Ser. No. 10/232,875 filed Aug. 30, 2002 (Attorney Docket No. 21154-000210US), and described below, may be used to select the file to be moved from the source volume. According to these techniques, a data value score (DVS) score is generated for the files stored on the source volume, and the file with the highest DVS is selected in step 506 for the move operation. Further description related to the use of DVSs for selecting files to be moved is discussed below. [0077] From the volumes determined in step 504, a volume is selected for storing the file selected in step 506 (step 508). The volume (or storage unit in general) identified in step 508 is referred to as a “target volume” or “target storage unit” as it represents a storage unit to which data will be moved. The target volume selected in step 508 and the source volume may be assigned to the same or different servers. [0078] Various techniques may be used for selecting the target volume in step 508. According to one embodiment, the least full volume from the volumes determined in step 504 is selected as the target volume in step 508. According to another embodiment, the volume with the lowest data storage cost associated with it is selected as the target volume in step 508. According to another embodiment, the administrator may specify criteria for selecting the target volume, and a volume from the volumes determined in step 504 that satisfies the criteria is selected as the target volume. [0079] According to yet another embodiment, techniques described in U.S. patent application Ser. No. 10/232,875 filed Aug. 30, 2002 (Attorney Docket No. 21154-000210US), and described below, may be used to select a target volume in step 410. In this embodiment, a storage value score (SVS) (also referred to as the “relative storage valued score” or RSVS) is generated for the various volumes determined in step 504. A volume with the highest SVS is then selected as the target volume. Further details related generation of SVSs and use of SVSs to select a target volume are given below. [0080] The file selected in step 506 is then moved from the source volume to the target volume selected in step 508 (step 510). A check is then made to determine if the move operation was successful (step 512). If the move operation was unsuccessful, then the file selected in step 506 is restored back to it original location on the source volume (step 514). Processing then continues with step 506 wherein another file from the source volume is selected to be moved. [0081] If the move operation in step 510 was successful, then information identifying the new location of the file on the target volume is stored and/or updated (step 516). According to an embodiment of the present invention, if there is any stub file associated with the moved file, then the stub file information (or information stored in a database) may be updated to reflect the new location of the file on the target volume. In an alternative embodiment, other information may be left in the original location in the form of UNIX symbolic links or Window shortcuts, or the administrator may have to inform the user of the new location if the operation is to move (and not migrate) the data. The information may also be stored or updated in a storage location (e.g., a database) accessible to SMS 110. [0082] The used storage capacity information for the source volume from which the file is moved and the target volume to which the file is moved is updated to reflect the file move (step 518). [0083] A check is then made to see if the overcapacity condition detected in step 502 that triggered the processing depicted in FIG. 5 has been resolved (step 520). The processing depicted in FIG. 5 terminates if it is determined that the condition detected in step 502 has been resolved. The volumes in the storage environment continue to be monitored for the next condition that triggers the processing depicted in FIG. 5. [0084] If it is determined in step 520 that the condition detected in step 502 has not been resolved, then processing continues with step 506 wherein another file from the source volume is selected to be moved. The steps in FIG. 5 are then repeated as described above. For each pass through the flowchart, the target volume selected in step 508 may be the same as or different from the previously selected target volume. [0085] As described above, embodiments of the present invention provide the ability to automatically detect when an overcapacity condition (e.g., when the used storage capacity for a volume exceeds a user-configured threshold value) has been reached for a volume. A target volume that has a lower storage data cost than the source volume is then automatically and dynamically determined for moving files from the source volume to resolve the overcapacity condition of the source volume. In this manner, by moving data to storage units with cheaper data storage costs, the cost of storing data in the storage environment is reduced or minimized. [0086] As indicated above, according to an embodiment of the present invention, DVSs may be used to select a file to be moved from the source volume to a target volume. FIG. 6 is a simplified flowchart 600 depicting a method of selecting a file for a move or migration operation according to an embodiment of the present invention. The processing depicted in FIG. 6 may be performed in step 410 depicted in FIG. 4 and/or step 506 depicted in FIG. 5. The processing in FIG. 6 may be performed by software modules executed by a processor, hardware modules, or combinations thereof. According to an embodiment of the present invention, the processing is performed by a policy management engine (PME) executing on SMS 110. Flowchart 600 depicted in FIG. 6 is merely illustrative of an embodiment of the present invention and is not intended to limit the scope of the present invention. Other variations, modifications, and alternatives are also within the scope of the present invention. [0087] As depicted in FIG. 6, a placement rule specified for the storage environment is determined (step 602). Examples of placement rules according to an embodiment of the present invention are provided in U.S. patent application Ser. No. 10/232,875 filed Aug. 30, 2002 (Attorney Docket No. 21154-000210US), and described below. For sake of simplicity of description, it is assumed for the processing depicted in FIG. 6 that a single placement rule is defined for the storage environment. [0088] Given the placement rule determined in step 602, data value scores (DVSs) are then calculated for the files stored on the source volume (step 604). The file with the highest DVS is then selected for the move operation (step 606). According to an embodiment of the present invention, the processing depicted in FIG. 6 is performed the first time that a file is to be selected. During this first pass, the files may be ranked based upon their DVSs calculated in step 606. The ranked list of files is then available for subsequent selections of the files during subsequent passes of the flowcharts depicted in FIGS. 4 and 5. The highest ranked and previously unselected file is then selected during each pass. [0089] According to an embodiment of the present invention, files that contain migrated data are selected for the move operation before files that contain original data (i.e., files that have not been migrated). A migrated file comprises data that has been migrated or remigrated from its original storage location by applications such as HSM applications. Generally, a stub or tag file is left in the original storage location of the migrated file identifying the migrated location of the file. An original file represents a file that has not been migrated or remigrated. [0090] Thus, according to an embodiment of the present invention, migrated files are moved before original files. In this embodiment, in step 606, two separate ranked lists are created based upon the DVSs associated with the files: one list comprising migrated files ranked based upon their DVSs, and the other comprising original files ranked based upon their DVSs. When a file is to be selected for a move operation in order to resolve an overcapacity condition associated with a volume, files from the ranked migrated files list are selected before selection of files from the ranked original files list (i.e., files from the original files list are not selected until the files on the migrated files list have been selected and moved). [0091] According to an embodiment of the present invention, file groups may be configured for the storage environment. A file is included in a file group if the file satisfies criteria specified for the file group. The file group criteria may be specified by the administrator or some other user. For example, an administrator may create file groups based upon a business value associated with the files. The administrator may group files that are deemed important or critical for the business into one file group (a “more important” file group) and the other files may be grouped into a second group (a “less important” file group). Other criteria may also be used for defining file groups including file size, file type, file owner or group of owners, last modified time of the file, last access time of a file, etc. The file groups may be created by the administrator or automatically by a storage policy engine. The file groups may also be prioritized relative to each other depending upon the files included in the file groups. Based upon the priorities associated with the file groups, files from a certain file group may be selected for the move operation in step 606 before files from another group. For example, the move operation may be configured such that files from the “less important” file group are moved before files from the “more important” file group. Accordingly, in step 606, files from the “less important” file group are selected for the move operation before files from the “more important” file group. Within a particular file group, the DVSs associated with the files may determine the order in which the files are selected for the move operation. [0092] In FIG. 6 it was assumed that only one placement rule was configured for the storage environment. However, in other embodiments, multiple placement rules may be configured for a storage environment. FIG. 7 is a simplified flowchart 700 depicting a method of selecting a file for a move or migration operation according to an embodiment of the present invention wherein multiple placement rules are configured. The processing depicted in FIG. 7 may be performed in step 410 depicted in FIG. 4 and/or step 506 depicted in FIG. 5. The processing in FIG. 7 may be performed by software modules executed by a processor, hardware modules, or combinations thereof. According to an embodiment of the present invention, the processing is performed by a policy management engine (PME) executing on SMS 110. Flowchart 700 depicted in FIG. 7 is merely illustrative of an embodiment of the present invention and is not intended to limit the scope of the present invention. Other variations, modifications, and alternatives are also within the scope of the present invention. [0093] As depicted in FIG. 7, the multiple placement rules configured for the storage environment are determined (step 702). Examples of placement rules according to an embodiment of the present invention are provided in U.S. patent application Ser. No. 10/232,875 filed Aug. 30, 2002 (Attorney Docket No. 21154-000210US), and described below. [0094] A set of placement rules that do not impose any constraints on moving data from a source volume are then determined from the rules determined in step 702 (step 704). For each file stored on the source volume, a DVS is calculated for the file for each placement rule in the set of placement rules identified in step 704 (step 706). For each file, the highest DVS calculated for the file, from the DVSs generated for the file in step 704, is then selected as the DVS for that file (step 708). In this manner, a DVS is associated with each file. The files are then ranked based upon their DVSs (step 710). From the ranked list, the file with the highest DVS is then selected for the move operation (step 712). [0095] According to an embodiment of the present invention, the processing depicted in FIG. 7 is performed the first time that a file is to be selected during the first pass of the flowcharts depicted in FIGS. 4 and 5. During this first pass, the files may be ranked based upon their DVSs in step 710. The ranked list of files is then available for subsequent selections of the files during subsequent passes of the flowcharts depicted in FIGS. 4 and 5. The highest ranked and previously unselected file is then selected during each subsequent pass. [0096] According to an embodiment of the present invention, files that contain migrated data are selected for the move operation before files that contain original data (i.e., files that have not been migrated). A migrated file comprises data that has been migrated (or remigrated) from its original storage location by applications such as HSM applications. Generally, a stub or tag file is left in the original storage location of the migrated file identifying the migrated location of the file. An original file represents a file that has not been migrated or remigrated. [0097] Thus, according to an embodiment of the present invention, migrated files are moved before original files. In this embodiment, in step 712, two separate ranked lists are created based upon the DVS scores associated with the files: one list comprising migrated files, and the other comprising original files. When a file is to be selected for a move operation, files from the ranked migrated files list are selected before selection of files from the ranked original files list (i.e., files from the original files list are not selected until the files on the migrated files list have been selected and moved). [0098] As indicated above, according to an embodiment of the present invention, a target volume may be selected from multiple volumes based upon SVSs. FIG. 8 is a simplified flowchart 800 depicting a method of selecting a target volume from a set of volumes according to an embodiment of the present invention. The processing depicted in FIG. 8 may be performed in step 412 depicted in FIG. 4 and/or step 508 depicted in FIG. 5. The processing in FIG. 8 may be performed by software modules executed by a processor, hardware modules, or combinations thereof. According to an embodiment of the present invention, the processing is performed by a policy management engine (PME) executing on SMS 110. Flowchart 800 depicted in FIG. 8 is merely illustrative of an embodiment of the present invention and is not intended to limit the scope of the present invention. Other variations, modifications, and alternatives are also within the scope of the present invention. [0099] As depicted in FIG. 8, a placement rule to be used for determining a target volume from a set of volumes is determined (step 802). In an embodiment where a single placement rule is configured for the storage environment, that single placement rule is selected in step 802. In embodiments where multiple placement rules are configured for the storage environment, the placement rule selected in step 802 corresponds to the placement rule that that was used to calculate the DVS associated with the selected file. [0100] Using the placement rule determined in step 802, a storage value score (SVS) (or “relative storage value score” RSVS) is generated for each volume in the set of volumes (e.g., volumes in the selected target managed group) (step 804). The SVS for a volume indicates the degree of suitability of storing the selected file on that volume. Various techniques may be used for calculating the SVSs. According to an embodiment of the present invention, the SVSs may be calculated using techniques described in U.S. patent application Ser. No. 10/232,875 filed Aug. 30, 2002 (Attorney Docket No.21154-000210US), and described below. The SVSs are referred to as relative storage value scores (RSVSs) in U.S. patent application Ser. No. 10/232,875. The volume with the highest SVS score is then selected as the target volume (step 806). [0101] In the flowcharts depicted in FIGS. 4 and 5, the SVSs are calculated every time that a target volume is to be determined (for example, in step 412 in FIG. 4 and in step 508 in FIG. 5) for storing the selected file, as the SVS for a particular volume may change based upon the conditions associated with the volume. Accordingly, different volumes may be selected as target volumes during successive passes of the flowchart depicted in FIG. 8. Embodiments of the present invention thus provide the ability to automatically and dynamically select a volume for moving data based upon the dynamic conditions associated with the volumes. [0102]FIG. 9 is a simplified block diagram showing modules that may be used to implement an embodiment of the present invention. The modules depicted in FIG. 9 may be implemented in software, hardware, or combinations thereof. As shown in FIG. 9, the modules include a user interface module 902, a policy management engine (PME) module 804, a storage monitor module 906, and a file I/O driver module 908. It should be understood that the modules depicted in FIG. 9 are merely illustrative of an embodiment of the present invention and are not meant to limit the scope of the invention. One of ordinary skill in the art would recognize other variations, modifications, and alternatives. [0103] User interface module 902 allows a user (e.g., an administrator) to interact with the storage management system. An administrator may provide rules/policy information for managing storage environment 912, information identifying the managed groups of storage units, thresholds information, selection criteria, cost criteria, etc., via user interface module 902. The information provided by the user may be stored in memory and disk storage 910. Information related to storage environment 912 may be output to the user via user interface module 902. The information related to the storage environment that is output may include status information about the capacity of the various storage units in the storage environment, the status of capacity utilization balancing operations, data storage costs information, error conditions, and other information related to the storage system. User interface module 902 may also provide interfaces that allow a user to define the managed groups of storage units using one or more techniques described above. [0104] User interface module 902 may be implemented in various forms. For example, user interface 902 may be in the form of a browser-based user interface, a graphical user interface, text-based command line interface, or any other application that allows a user to specify information for managing a storage environment and that enables a user to receive feedback, statistics, reports, status, and other information related to the storage environment. [0105] The information received via user interface module 902 may be stored in a memory and disk storage 910 and/or forwarded to PME module 904. The information may be stored in the form of configuration files, Windows Registry, a directory service (e.g., Microsoft Active Directory, Novell eDirectory, OpenLDAP, etc), databases, and the like. PME module 804 is also configured to read the information from memory and disk storage 910. [0106] Policy management module 904 is configured to perform the processing to optimize capacity utilization and move data between storage units based upon data storage costs according to an embodiment of the present invention. Policy management module 904 uses information received from user interface module 902 (or stored in memory and disk storage 910) and information related to storage environment 912 received from storage monitor module 906 to automatically perform the capacity utilization balancing task. Information specifying costs for storing data on the various storage units is also used for the capacity utilization balancing. According to an embodiment of the present invention, PME module 904 is configured to perform the processing depicted in FIGS. 4, 5, 6, 7, and 8. [0107] Storage monitor module 906 is configured to monitor storage environment 912. The monitoring may be done on a continuous basis or on a periodic basis. As described above, the monitoring may include monitoring attributes of the storage units such as usage information, capacity utilization, types of storage devices, etc. Monitoring also includes monitoring attributes of the files in storage environment 912 such as file size information, file access time information, file type information, etc. The monitoring may also be performed using agents installed on the various servers coupled to the storage units or may be done remotely from agents running on other systems. The information gathered from the monitoring activities may be stored in memory and disk storage 910 or forwarded to PME module 904. [0108] Various formats may used for storing the information in memory and disk storage 910. For example, the storage capacity usage for a storage unit may be expressed as a percentage of the total storage capacity of the storage unit. For example, if the total storage capacity of a storage unit is 100 Mbytes, and if 40 Mbytes are free for storage (i.e., 60 Mbytes are already used), then the used storage capacity of the storage unit may be expressed as 60% (or alternatively, 40% available capacity). The value may also be expressed as the amount of free storage capacity (e.g., in MB, GB, etc.) or used storage. [0109] PME module 904 may use the information gathered from the monitoring to detect presence of conditions that trigger a storage capacity optimization operation. For example, PME module 904 may use the gathered information to determine if a storage unit in storage environment 912 is experiencing an overcapacity condition. [0110] File I/O driver module 908 is configured to intercept file system calls received from consumers of data stored by storage environment 912. For example, file I/O driver module 908 is configured to intercept any file open call (which can take different forms in different operating systems) received from an application, user, or any data consumer. When file I/O driver module 908 determines that a requested file has been migrated from its original location to a different location, it may suspend the file open call and perform the following operations: (1) File I/O driver 908 may determine the actual location of the requested data file in storage environment 912. This can be done by looking up from the file header or stub file that is stored in the original location. Alternatively, if the file location information is stored in a persistent storage location (e.g., a database managed by PME module 904), file I/O driver 908 may determine the actual remote location of the file from that persistent location; [0111] (2) File I/O driver 908 may then restore the file content from the remote storage unit location; (3) File I/O driver 908 then resumes the file open call so that the application can resume with the restored data. [0112] Techniques for Generating DVSs and SVSs Using Placement Rules [0113] As described above, an embodiment of the present invention can automatically determine files to be moved and target storage units for storing the files using DVSs and SVSs calculated using one or more placement rules. According to an embodiment of the present invention, each placement rule comprises: (1) data-related criteria and (2) device-related criteria. The data-related criteria comprises criteria associated with the data to be stored and is used to select the file to move. According to an embodiment, the data-related criteria comprise (a) data usage criteria information, and (b) file selection criteria information. [0114] The device-related criteria comprises criteria related to storage units. In one embodiment, the device related criteria is also referred to as location constraint criteria information. [0115]FIG. 10 depicts examples of placement rules according to an embodiment of the present invention. In FIG. 10, each row 1008 of table 1000 specifies a placement rule. Column 1002 of table 1000 identifies the file selection criteria information for each rule, column 1004 of table 1000 identifies the data usage criteria information for each placement rule, and column 1006 of table 1000 identifies the location constraint criteria information for each rule. [0116] The “file selection criteria information” specifies information identifying conditions related to files. According to an embodiment of the present invention, the selection criteria information for a placement rules specifies one or more clauses (or conditions) related to an attribute of a file such as file type, relevance score of file, file owner, etc. Each clause may be expressed as an absolute value (e.g., File type is “Office files”) or as an inequality (e.g., Relevance score of file>=0.5). Multiple clauses may be connected by Boolean connectors (e.g., File type is “Email files” AND File owner is “John Doe”) to form a Boolean expression. The file selection criteria information may also be left empty (i.e., not configured or set to NULL value), e.g., file selection criteria for placement rules 1008-6 and 1008-7 depicted in FIG. 10. According to an embodiment of the present invention, the file selection criteria information defaults to a NULL value. An empty or NULL file selection criterion is valid and indicates that all files are selected or are eligible for the placement rule. [0117] The “data usage criteria information” specifies criteria related to file access information associated with a file. For example, for a particular placement rule, this information may specify condition related to when the file was last accessed, created, last modified, and the like. The criteria may be specified using one or more clauses or conditions connected using Boolean connectors. The data usage criteria clauses may be specified as equality conditions or inequality conditions. For example, “file last accessed between 7 days to 30 days ago” (corresponding to placement rule 1008-2 depicted in FIG. 10). These criteria may be set by an administrator. [0118] The “location constraint information” for a particular placement rule specifies one or more constraints associated with storing information on a storage unit based upon the particular placement rule. Location constraint information generally specifies parameters associated with a storage unit that need to be satisfied for storing information on the storage unit. The location constraint information may be left empty or may be set to NULL to indicate that no constraints are applicable for the placement rule. For example, no constraints have been specified for placement rule 1008-3 depicted in FIG. 10. [0119] According to an embodiment of the present invention, the constraint information may be set to LOCAL (e.g., location constraint information for placement rules 1008-1 and 1008-6). This that the file is to be stored on a local storage unit that is local to the device used to create the file and is not to be moved or migrated to another storage unit. According to an embodiment of the present invention, a placement rule is not eligible for selection if the constraint information is set to LOCAL, and a DVS of 0 (zero) is assigned for that specific placement rule. A specific storage unit group, or a specific device may be specified in the location constraint information for storing the data file. A minimum bandwidth requirement (e.g., Bandwidth>=10 MB/s) may be specified indicating that the data can only be stored on a storage unit satisfying the constraint. Various other constraints or requirements may also be specified (e.g., constraints related to file size, availability, etc.). The constraints specified by the location constraint information are generally hard constraints implying that a file cannot be stored on a storage unit that does not satisfy the location constraints. [0120] As stated above, a numerical score (referred to as the Data Value Score or DVS) can be generated for a file for each placement rule. For each placement rule, the DVS generated for the file and the placement rule indicates the level of suitability or applicability of the placement rule for that file. The value of the DVS calculated for a particular file using a particular placement rule is based upon the characteristics of the particular file. For example, according to an embodiment of the present invention, for a particular file, higher scores are generated for placement rules that are deemed more suitable or relevant to the particular file. [0121] Several different techniques may be used for generating a DVS for a file using a placement rule. According to one embodiment, the DVS for a file using a placement rule is a simple product of a “file_selection_score” and a “data_usage_score”,
[0151] As described above, the desired_threshold_% for a storage device is usually set by a system administrator. The current_usage_% value is monitored by embodiments of the present invention. The “cost” value may be set by the system administrator. [0152] It should be understood that the formula for calculating SVS shown above is representative of one embodiment of the present invention and is not meant to reduce the scope of the present invention. Various other factors may be used for calculating the SVS in alternative embodiments of the present invention. For example, the availability of a storage unit may also be used to determine the SVS for the device. According to an embodiment of the present invention, availability of a storage unit indicates the amount of time that the storage unit is available during those time periods when it is expected to be available. Availability may be measured as a percentage of an elapsed year in certain embodiments. For example, 99.95% availability equates to 4.38 hours of downtime in a year (0.0005*365 *24=4.38) for a storage unit that is expected to be available all the time. According to an embodiment of the present invention, the value of SVS for a storage unit is directly proportional to the availability of the storage unit. [0153] STEP 3: Various adjustments may be made to the SVS calculated according to the above steps. For example, in some storage environments, the administrator may want to group “similar” files together in one storage unit. In other environments, the administrator may want to distribute files among different storage units. The SVS may be adjusted to accommodate the policy adopted by the administrator. Performance characteristics associated with a network that is used to transfer data from the storage devices may also be used to adjust the SVSs for the storage units. For example, the access time (i.e., the time required to provide data stored on a storage unit to a user) of a storage unit may be used to adjust the SVS for the storage unit. The throughput of a storage unit may also be used to adjust the SVS value for the storage unit. Accordingly, parameters such as the location of the storage unit, location of the data source, and other network related parameters might also be used to generate SVSs. According to an embodiment of the present invention, the SVS value is calculated such that it is directly proportional to the desirability of the storage unit for storing the file. [0154] According to an embodiment of the present invention, a higher SVS value represents a more desirable storage unit for storing a file. As indicated, the SVS value is directly proportional to the available capacity percentage. Accordingly, a storage unit with higher available capacity is more desirable for storing a file. The SVS value is inversely proportional to the cost of storing data on the storage unit. Accordingly, a storage unit with lower storage costs is more desirable for storing a file. The SVS value is directly proportional to the bandwidth requirement. Accordingly, a storage unit supporting a higher bandwidth is more desirable for storing the file. SVS is zero if the bandwidth requirements are not satisfied. Accordingly, the SVS formula for a particular storage unit combines the various storage unit characteristics to generate a score that represents the degree of desirability of storing data on the particular storage unit. [0155] According to the above formula, SVS is zero (0) if the value of Bandwidth_factor is zero. As described above, Bandwidth_factor is set to zero if the bandwidth supported by the storage unit is less than the bandwidth requirement, if any, specified in the location constraints criteria information specified for the selected placement rule. Accordingly, if the value of SVS for a particular storage unit is zero (0) it implies that bandwidth supported by the storage unit is less than the bandwidth required by the placement rule, or the storage unit is already at or exceeds the desired capacity threshold. Alternatively, SVS is zero (0) if the desired_threshold_% is equal to the current_usage_%. [0156] If the SVS for a storage unit is positive, it indicates that the storage unit meets both the bandwidth requirements (i.e., Bandwidth_factor is non zero) and also has enough capacity for storing the file (i.e., desired_threshold_% is greater than the current_usage_%). The higher the SVS value, the more suitable (or desirable) the storage unit is for storing a file. For storage units with positive SVSs, the storage unit with the highest positive RSVS is the most desirable candidate for storing the file. The SVS for a particular storage unit thus provides a measure for determining the degree of desirability for storing data on the particular storage unit relative to other storage unit for a particular placement rule being processed. Accordingly, the SVS is also referred to as the relative storage value score (RSVS). The SVS in conjunction with the placement rules and their rankings is used to determine an optimal storage location for storing the data to be moved from the source storage unit. [0157] The SVS for a particular storage unit may be negative if the storage unit meets the bandwidth requirements but the storage unit's usage is above the intended threshold (i.e., current_usage_% is greater than the desired_threshold_%). The relative magnitude of the negative value indicates the degree of over-capacity of the storage unit. Among storage units with negative SVSs, the closer the SVS is to zero (0) and the storage unit has capacity for storing the data, the more desirable the storage unit is for storing the data file. For example, the over-capacity of a storage unit having SVS of −0.9 is more than the over-capacity of a second storage unit having RSVS −0.1. Accordingly, the second storage unit is a more attractive candidate for storing the data file as compared to the first storage unit. Accordingly, the SVS, even if negative, can be used in ranking the storage units relative to each other for purposes of storing data. [0158] The SVS for a particular storage unit thus serves as a measure for determining the degree of desirability or suitability of the particular storage unit for storing data relative to other storage devices. A storage unit having a positive SVS value is a better candidate for storing the data file than a storage unit with a negative SVS value, since a positive value indicates that the storage unit meets the bandwidth requirements for the data file and also possesses sufficient capacity for storing the data file. Among storage units with positive SVS values, a storage unit with a higher positive SVS is a more desirable candidate for storing the data file than a storage unit with a lower SVS value, i.e., the storage unit having the highest positive SVS value is the most desirable storage unit for storing the data file. [0159] If a storage unit with a positive SVS value is not available, then storage units with negative SVS values are more desirable than devices with an SVS value of zero (0). The rationale here is that it is better to select a storage unit that satisfies the bandwidth requirements (even though the storage unit is over capacity) than a storage unit that does not meet the bandwidth requirements (i.e., has a SVS of zero). Among storage units with negative SVS values, a storage unit with a higher SVS value (i.e., SVS closer to 0) is a more desirable candidate for storing the data file than a storage unit with a lesser SVS value. Accordingly, among storage units with negative SVS values, the storage unit with the highest SVS value (i.e., SVS closest to 0) is the most desirable candidate for storing the data file. [0160] Although specific embodiments of the invention have been described, various modifications, alterations, alternative constructions, and equivalents are also encompassed within the scope of the invention. The described invention is not restricted to operation within certain specific data processing environments, but is free to operate within a plurality of data processing environments. Additionally, although the present invention has been described using a particular series of transactions and steps, it should be apparent to those skilled in the art that the scope of the present invention is not limited to the described series of transactions and steps. It should be understood that the equations described above are only illustrative of an embodiment of the present invention and can vary in alternative embodiments of the present invention. [0161] Further, while the present invention has been described using a particular combination of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of the present invention. The present invention may be implemented only in hardware, or only in software, or using combinations thereof. [0162] The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that additions, subtractions, deletions, and other modifications and changes may be made thereunto without departing from the broader spirit and scope of the invention as set forth in the claims. Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS2151733May 4, 1936Mar 28, 1939American Box Board CoContainerCH283612A * Title not availableFR1392029A * Title not availableFR2166276A1 * Title not availableGB533718A Title not available* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS7092977Aug 30, 2002Aug 15, 2006Arkivio, Inc.Techniques for storing data based upon storage policiesUS7103740Dec 31, 2003Sep 5, 2006Veritas Operating CorporationBackup mechanism for a multi-class file systemUS7177883Jul 15, 2004Feb 13, 2007Hitachi, Ltd.Method and apparatus for hierarchical storage management based on data value and user interestUS7197520Apr 14, 2004Mar 27, 2007Veritas Operating CorporationTwo-tier backup mechanismUS7225211 *Dec 31, 2003May 29, 2007Veritas Operating CorporationMulti-class storage mechanismUS7269701 *Oct 10, 2003Sep 11, 2007International Business Machines CorporationSelf-configuration of source-to-target mappingUS7293133Dec 31, 2003Nov 6, 2007Veritas Operating CorporationPerforming operations without requiring split mirrors in a multi-class file systemUS7383292 *Jul 14, 2005Jun 3, 2008Microsoft CorporationMoving data from file on storage volume to alternate location to free spaceUS7392362 *Aug 5, 2005Jun 24, 2008Hitachi, Ltd.System and method for volume managementUS7426619 *Jun 16, 2005Sep 16, 2008Hitachi, Ltd.Inter-volume migration system, inter-volume relocation method, and program thereforUS7441096 *Jul 7, 2004Oct 21, 2008Hitachi, Ltd.Hierarchical storage management systemUS7480734 *Jan 19, 2006Jan 20, 2009Sprint Communications Company L.P.Class of service analysis for data flows in a data storage infrastructure for a communication networkUS7505986Jul 14, 2005Mar 17, 2009Microsoft CorporationMoving data from file on storage volume to alternate location to free spaceUS7506003Jul 14, 2005Mar 17, 2009Microsoft CorporationMoving data from file on storage volume to alternate location to free spaceUS7506004Jul 14, 2005Mar 17, 2009Microsoft CorporationMoving data from file on storage volume to alternate location to free spaceUS7506005Jul 14, 2005Mar 17, 2009Microsoft CorporationMoving data from file on storage volume to alternate location to free spaceUS7509316Jun 24, 2004Mar 24, 2009Rocket Software, Inc.Techniques for performing policy automated operationsUS7512135Dec 10, 2004Mar 31, 2009International Business Machines CorporationMethod for transferring data among a logical layer, physical layer, and storage deviceUS7590671 *Nov 25, 2005Sep 15, 2009Hitachi, Ltd.Storage system, file migration method and computer program productUS7590777Dec 10, 2004Sep 15, 2009International Business Machines CorporationTransferring data between system and storage in a shared bufferUS7661012Dec 1, 2005Feb 9, 2010International Business Machines CorporationSpare device managementUS7716440Jan 27, 2006May 11, 2010Hitachi, Ltd.Storage system and management method thereofUS7752437 *Jan 19, 2006Jul 6, 2010Sprint Communications Company L.P.Classification of data in data flows in a data storage infrastructure for a communication networkUS7788302Jan 19, 2006Aug 31, 2010Sprint Communications Company L.P.Interactive display of a data storage infrastructure for a communication networkUS7797395Jan 19, 2006Sep 14, 2010Sprint Communications Company L.P.Assignment of data flows to storage systems in a data storage infrastructure for a communication networkUS7801973Jan 19, 2006Sep 21, 2010Sprint Communications Company L.P.Classification of information in data flows in a data storage infrastructure for a communication networkUS7853667 *Aug 5, 2005Dec 14, 2010Network Appliance, Inc.Emulation of transparent recall in a hierarchical storage management systemUS7873681 *Jul 14, 2005Jan 18, 2011Microsoft CorporationMoving data from file on storage volume to alternate location to free spaceUS7895295Jan 19, 2006Feb 22, 2011Sprint Communications Company L.P.Scoring data flow characteristics to assign data flows to storage systems in a data storage infrastructure for a communication networkUS7925851Jun 9, 2008Apr 12, 2011Hitachi, Ltd.Storage deviceUS7930496Jun 29, 2006Apr 19, 2011International Business Machines CorporationProcessing a read request to a logical volume while relocating a logical volume from a first storage location to a second storage location using a copy relationshipUS7987206 *Jan 11, 2008Jul 26, 2011Hitachi Ltd.File-sharing system and method of using file-sharing system to generate single logical directory structureUS7991835May 18, 2009Aug 2, 2011Cisco Technology, Inc.Distributed client services based on execution of service attributes and data attributes by multiple nodes in resource groupsUS8024064 *Jun 30, 2008Sep 20, 2011Amazon Technologies, Inc.Placement of inventory in a materials handling facilityUS8037072 *Jun 30, 2004Oct 11, 2011Emc CorporationSystem and method for generating charge information for users of a data storage systemUS8051170 *Feb 10, 2005Nov 1, 2011Cisco Technology, Inc.Distributed computing based on multiple nodes with determined capacity selectively joining resource groups having resource requirementsUS8091087Apr 20, 2007Jan 3, 2012Microsoft CorporationScheduling of new job within a start time range based on calculated current load and predicted load value of the new job on media resourcesUS8127095Dec 31, 2003Feb 28, 2012Symantec Operating CorporationRestore mechanism for a multi-class file systemUS8127113Dec 1, 2006Feb 28, 2012Synopsys, Inc.Generating hardware accelerators and processor offloadsUS8140785 *Jun 29, 2006Mar 20, 2012International Business Machines CorporationUpdating metadata in a logical volume associated with a storage controller for data units indicated in a data structureUS8195742Jun 20, 2011Jun 5, 2012Cisco Technology, Inc.Distributed client services based on execution of service attributes and data attributes by multiple nodes in resource groupsUS8230194Mar 18, 2011Jul 24, 2012Hitachi, Ltd.Storage deviceUS8239540Aug 30, 2011Aug 7, 2012Cisco Technology, Inc.Distributed computing based on multiple nodes with determined capacity selectively joining resource groups having resource requirementsUS8280853 *Mar 31, 2009Oct 2, 2012Symantec Operating CorporationDynamic storage mechanismUS8289966Dec 1, 2006Oct 16, 2012Synopsys, Inc.Packet ingress/egress block and system and method for receiving, transmitting, and managing packetized dataUS8407445Mar 31, 2010Mar 26, 2013Emc CorporationSystems, methods, and computer readable media for triggering and coordinating pool storage reclamationUS8443163Jun 28, 2010May 14, 2013Emc CorporationMethods, systems, and computer readable medium for tier-based data storage resource allocation and data relocation in a data storage arrayUS8443369Jun 30, 2008May 14, 2013Emc CorporationMethod and system for dynamically selecting a best resource from each resource collection based on resources dependencies, prior selections and statistics to implement an allocation policyUS8510429Jan 19, 2006Aug 13, 2013Sprint Communications Company L.P.Inventory modeling in a data storage infrastructure for a communication networkUS8639816 *Jul 3, 2012Jan 28, 2014Cisco Technology, Inc.Distributed computing based on multiple nodes with determined capacity selectively joining resource groups having resource requirementsUS8706987 *Dec 1, 2006Apr 22, 2014Synopsys, Inc.Structured block transfer module, system architecture, and method for transferringUS8712971Jul 13, 2012Apr 29, 2014Symantec CorporationRestore software with aggregated view of content databasesUS8738585Jul 13, 2012May 27, 2014Symantec CorporationRestore software with aggregated view of site collectionsUS8745327Jun 24, 2011Jun 3, 2014Emc CorporationMethods, systems, and computer readable medium for controlling prioritization of tiering and spin down features in a data storage systemUS8762995 *Feb 28, 2008Jun 24, 2014Hitachi, Ltd.Computing system, method of controlling the same, and system management unit which plan a data migration according to a computation job execution scheduleUS8825591 *Dec 31, 2003Sep 2, 2014Symantec Operating CorporationDynamic storage mechanismUS8843721Mar 14, 2013Sep 23, 2014International Business Machines CorporationData storage using bitmapsUS8862848Aug 25, 2010Oct 14, 2014International Business Machines CorporationData storage using bitmapsUS8886909Apr 10, 2008Nov 11, 2014Emc CorporationMethods, systems, and computer readable medium for allocating portions of physical storage in a storage array based on current or anticipated utilization of storage array resourcesUS8918677 *Sep 30, 2004Dec 23, 2014Emc CorporationMethods and apparatus for performing data validation in a network management applicationUS8924681 *Mar 31, 2010Dec 30, 2014Emc CorporationSystems, methods, and computer readable media for an adaptative block allocation mechanismUS8935501 *Jul 11, 2012Jan 13, 2015Fujitsu LimitedApparatus, method, and recording medium storing a program for selecting a destination storage based on a buffer sizeUS8984027 *Jul 28, 2011Mar 17, 2015Symantec CorporationSystems and methods for migrating files to tiered storage systemsUS9032146 *Nov 30, 2010May 12, 2015Lenovo Enterprise Solutions (Singapore) Pte. Ltd.Dynamic use of raid levels responsive to workload requirementsUS9037794 *Jul 3, 2012May 19, 2015Lenovo Enterprise Solutions (Singapore) Pte. Ltd.Dynamic use of raid levels responsive to workload requirementsUS20040083202 *Aug 30, 2002Apr 29, 2004Arkivio, Inc.Techniques to control recalls in storage management applicationsUS20040153481 *Jan 21, 2004Aug 5, 2004Srikrishna TalluriMethod and system for effective utilization of data storage capacityUS20040163029 *Dec 1, 2003Aug 19, 2004Arkivio, Inc.Data recovery techniques in storage systemsUS20040193760 *Feb 10, 2004Sep 30, 2004Hitachi, Ltd.Storage deviceUS20050015409 *May 28, 2004Jan 20, 2005Arkivio, Inc.Techniques for performing operations on migrated files without recalling dataUS20050021566 *May 28, 2004Jan 27, 2005Arkivio, Inc.Techniques for facilitating backup and restore of migrated filesUS20050033757 *Jun 24, 2004Feb 10, 2005Arkivio, Inc.Techniques for performing policy automated operationsUS20050081006 *Oct 10, 2003Apr 14, 2005International Business Machines CorporationSelf-configuration of source-to-target mappingUS20050120057 *Nov 10, 2004Jun 2, 2005Hirobumi HashimotoInformation processing device for managing data storage and method of data managementUS20070233868 *Mar 31, 2006Oct 4, 2007Tyrrell John CSystem and method for intelligent provisioning of storage across a plurality of storage systemsUS20100262774 *Oct 14, 2010Fujitsu LimitedStorage control apparatus and storage systemUS20110072253 *Mar 24, 2011International Business Machines CorporationMethod, system and program product for determining an optimal configuration and operational costs for implementing a capacity management serviceUS20110125742 *Sep 30, 2008May 26, 2011Haines Matthew DAuto-Publishing Photo Albums On A Home ServerUS20110276772 *Nov 10, 2011Hitachi, Ltd.Management apparatus and management methodUS20120072257 *Sep 16, 2010Mar 22, 2012International Business Machines CorporationOptimizing Facility Utilization For Service DeliveryUS20120137066 *Nov 30, 2010May 31, 2012International Business Machines CorporationDynamic use of raid levels responsive to workload requirementsUS20120272001 *Oct 25, 2012International Business Machines CorporationDynamic use of raid levels responsive to workload requirementsUS20130080699 *Mar 28, 2013Fujitsu LimitedInformation processing apparatus control method, computer-readable recording medium, and information processing apparatusUS20130227085 *Feb 18, 2013Aug 29, 2013Pantech Co., Ltd.Terminal and method for using cloud servicesEP1902393A2 *Jul 10, 2006Mar 26, 2008Microsoft CorporationMoving data from file on storage volume to alternate location to free spaceEP1902394A2 *Jul 10, 2006Mar 26, 2008Microsoft CorporationMoving data from file on storage volume to alternate location to free spaceWO2006036808A2 *Sep 22, 2005Apr 6, 2006Xyratex Tech LtdMethod and system for classifying networked devicesWO2007011585A2Jul 10, 2006Jan 25, 2007Microsoft CorpThe ghost fileWO2012035574A1 *Sep 14, 2010Mar 22, 2012Hitachi, Ltd.Server apparatus and control method of the same for migrating file based on user quota and file quotaWO2015047689A1 *Sep 5, 2014Apr 2, 2015Symantec CorporationImproving backup system performance* Cited by examinerClassifications U.S. Classification711/165, 711/114, 707/E17.005, 707/E17.01International ClassificationG06F17/30Cooperative ClassificationG06F17/30067European ClassificationG06F17/30S, G06F17/30FLegal EventsDateCodeEventDescriptionJan 8, 2004ASAssignmentOwner name: ARKIVIO, INC., CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LEUNG, ALBERT;PALISKA, GIOVANNI;GREENBLATT, BRUCE;AND OTHERS;REEL/FRAME:014244/0446Effective date: 20031205RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services