Automated information life-cycle management with thin provisioning

A system for managing data includes providing at least one logical device having a table of information that maps sections of the logical device to sections of at least two storage areas. Characteristics of data associated with at least one section of the logical device may be evaluated. The at least one section of the data may moved between the at least two storage areas according to a policy and based on the characteristics of the data. The table of information is updated according to the movement of data between the at least two storage areas.

TECHNICAL FIELD

This application relates to computer storage devices and, more particularly, to the field of managing data stored on a computer storage device.

BACKGROUND OF THE INVENTION

Host processor systems may store and retrieve data using storage devices containing a plurality of host interface units (host adapters), disk drives, and disk interface units (disk adapters). Such storage devices are provided, for example, by EMC Corporation of Hopkinton, Mass. and disclosed in U.S. Pat. No. 5,206,939 to Yanai et al., U.S. Pat. No. 5,778,394 to Galtzur et al., U.S. Pat. No. 5,845,147 to Vishlitzky et al., and U.S. Pat. No. 5,857,208 to Ofek, which are incorporated herein by reference. The host systems access the storage device through a plurality of channels provided therewith. Host systems provide data and access control information through the channels of the storage device and the storage device provides data to the host systems also through the channels. The host systems do not address the disk drives of the storage device directly, but rather, access what appears to the host systems as a plurality of logical volumes. Different sections of the logical volumes may or may not correspond to the actual disk drives.

Information Lifecycle Management (ILM) concerns the management of data throughout the data's lifecycle. The value of data may change over time and, accordingly, the needs for the storage and accessibility of the data may change during the lifecycle of the data. For example, data that is initially accessed often may, over time, become less valuable and the need to access that data become more infrequent. It may not be efficient for such data infrequently accessed to be stored on a fast and expensive storage device. On the other hand, older data may suddenly become more valuable and, where once accessed infrequently, become more frequently accessed. In this case, it may not be efficient for such data to be stored on a slower storage system when data access frequency increases.

Accordingly, it would be desirable to provide a system that allows for the efficient automatic management of data in a storage system throughout the data's lifecycle on each section of the user logical volume based on a user defined policy.

SUMMARY OF THE INVENTION

According to the system described herein, a method for managing data includes providing at least one logical device having a table of information that maps sections of the logical device to sections of at least two storage areas. Characteristics of data associated with at least one section of the logical device may be evaluated. At least one section may be moved between the at least two storage areas according to a policy and based on the characteristics of the data. The table of information is updated according to the movement of data between the at least two storage areas. In response to a request for accessing data of the at least one logical device, a location of the data is determined on the at least two storage areas and the data is accessed on the particular one of the storage areas using the table of information. In response to a request for writing new data to the logical device, an available location for the new data may be determined on at least one of the at least two storage areas and the new data written to at least one of the storage areas.

The characteristics of the data may include at least one of: frequency of use of the data, a time of last use of the data, and user information associated with the data. Characteristics of the data may be updated after data access. Moving data between the at least two storage areas may be performed automatically based on at least one of: periodically and after a trigger event and/or may be initiated by a user or administrator. The trigger event may include at least one of: a log in by a user, a log out by a user, and an assessment of at least one of the two storage areas. The policy may include criteria for managing data according to at least one of: frequency of use of the data, a time of last use of the data, and user information associated with the data. The logical device may include a thin device and each of the at least two storage areas may include a data device. Further, another logical device may be provided that is served by a same pool of data devices as the at least one logical device.

According further to the system described herein, a computer program product, stored in a computer-readable medium, for managing data includes executable code that provides at least one logical device having a table of information that maps sections of the logical device to sections of at least two storage areas. Executable code may be provided that evaluates characteristics of data associated with at least one section of the logical device. Executable code may be provided that moves the at least one section between the at least two storage areas according to a policy and based on the characteristics of the data. Executable code may be provided that updates the table of information according to the movement of data between the at least two storage areas. Executable code may be provided that, in response to a request for accessing data of the at least one logical device, determines a location of the data on the at least two storage areas and accesses the data on the particular one of the storage areas using the table of information. Executable code may be provided that, in response to a request for writing new data to the at least one logical device, determines an available location for the new data on at least one of the at least two storage areas and writes the new data to at least one of the storage areas. Further, executable code may be provided that provides another logical device that is linked to the at least one logical device.

According further to the system described herein, a computer storage device includes a plurality of interconnected directors, wherein at least some of the directors handle reading and writing data for the computer storage device. A plurality of disk drives, coupled to at least some of the directors, store data for the computer storage device. Computer software, provided on a computer readable medium of at least one of the directors, includes executable code that provides at least one logical device having a table of information that maps sections of the logical device to sections of at least two storage areas associated with the disk drives; executable code that evaluates characteristics of data associated with the at least one section of the logical device; executable code that moves the at least one section between the at least two storage areas according to a policy and based on the characteristics of the data; and executable code that updates the table of information according to the movement of data between the at least two storage areas.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

Referring now to the figures of the drawings, the figures comprise a part of this specification and illustrate exemplary embodiments of the described system. It is to be understood that in some instances various aspects of the system may be shown schematically or may be shown exaggerated or altered to facilitate an understanding of the system.

FIG. 1is a schematic illustration showing a storage device30that includes a plurality of host adapters (HA)32-34, a plurality of disk adapters (DA)36-38and a plurality of disk drives42-44. Each of the disk drives42-44is coupled to a corresponding one of the DA's36-38. AlthoughFIG. 1shows a one-for-one correspondence between the DA's36-38and the disk drives36-38, it should be understood that it is possible to couple more than one disk drive to a DA and/or to couple more than one DA to a disk drive. The storage device30also includes a global memory46that may be accessed by the HA's32-34and the DA's36-38. The storage device30also includes an RDF adapter (RA)48that may also access the global memory46. The RA48may communicate with one or more additional remote storage devices (not shown) and/or one or more other remote devices (not shown) via a data link22. The HA's32-34, the DA's36-38, the global memory46and the RA48may be coupled to a bus24that is provided to facilitate communication therebetween. In various embodiments, additional RA's may be incorporated into the storage device30.

Each of the HA's32-34may be coupled to one or more host computers (not shown) that access the storage device30. The host computers (hosts) access data on the disk drives42-44through the HA's32-34and the DA's36-38. The global memory46contains a cache memory that holds tracks of data read from and/or to be written to the disk drives42-44as well as storage for tables that may be accessed by the HA's32-34, the DA's36-38and the RA48. Note that, for the discussion herein, blocks of data are described as being a track or tracks of data. However, it will be appreciated by one of ordinary skill in the art that the system described herein may work with any appropriate incremental amount, or section, of data, including possibly variable incremental amounts of data and/or fixed incremental amounts of data.

FIG. 2is schematic diagram showing an embodiment of the storage device30where each of a plurality of directors52a-52care coupled to the memory46. Each of the directors52a-52cmay represent one or more of the HA's32-34, the DA's36-38and/or RA48. In an embodiment disclosed herein, there may be up to sixty-four directors coupled to the memory46. Of course, for other embodiments, there may be a higher or lower maximum number of directors that may be used.

The diagram50also shows an optional communication module (CM)54that provides an alternative communication path between the directors52a-52c. Each of the directors52a-52cmay be coupled to the CM54so that any one of the directors52a-52cmay send a message and/or data to any other one of the directors52a-52cwithout needing to go through the memory46. The CM54may be implemented using conventional MUX/router technology where a sending one of the directors52a-52cprovides an appropriate address to cause a message and/or data to be received by an intended receiving one of the directors52a-52c. Some or all of the functionality of the CM54may be implemented using one or more of the directors52a-52cso that, for example, the directors52a-52cmay be interconnected directly with the interconnection functionality being provided on each of the directors52a-52c. In addition, a sending one of the directors52a-52cmay be able to broadcast a message to all of the other directors52a-52cat the same time.

In some embodiments, one or more of the directors52a-52cmay have multiple processor systems thereon and thus may be able to perform functions for multiple directors. In some embodiments, at least one of the directors52a-52chaving multiple processor systems thereon may simultaneously perform the functions of at least two different types of directors (e.g., an HA and a DA). Furthermore, in some embodiments, at least one of the directors52a-52chaving multiple processor systems thereon may simultaneously perform the functions of different types of director and perform other processing with the other processing system.

The system described herein is suitable for use with the technique of thin provisioning. Thin provisioning allows for the creation of logical volumes of storage space where allocation of physical storage space occurs only when space is actually needed (e.g., when data is written in the first time to the storage space). Logical storage space may be identified to a user as being available even though no physical storage space has been committed, at least initially. When data is written to the logical storage space, physical storage space is drawn for use from a pool of physical storage space, as further described elsewhere herein. In addition, as described in more detail elsewhere herein, stored data may be moved between physical locations using the storage infrastructure described herein.

FIG. 3is a schematic diagram showing the storage device30as including a plurality of data devices61-68. Data devices61-67may be implemented as logical devices like standard logical devices provided in a Symmetrix data storage device. In some embodiments, the data devices61-67may not be directly useable (visible) to hosts coupled to the storage device30. Each of the data devices61-67may correspond to a portion (including a whole portion) of one or more of the disk drives42-44. Thus, for example, the data device61may correspond to the disk drive42, may correspond to a portion of the disk drive42, or may correspond to a portion of the disk drive42and a portion of the disk drive43. The data devices61-67may be designated as corresponding to different classes, so that different ones of the data devices61-67correspond to different physical storage having different relative access speeds (or some other relevant distinguishing characteristic or combination of characteristics), as further discussed elsewhere herein. The data device68is illustrated with broken lines to indicate that the data device68that does not yet exist but may be created, as further discussed elsewhere herein.

The storage device30may also include one or more thin devices71-74. Each of the thin devices71-74may appear to a host coupled to the storage device30as a logical volume (logical device) containing a contiguous block of data storage. Each of the thin devices71-74may contain tables that point to some or all of the data devices61-67(or portions thereof), as further discussed elsewhere herein. In some instances, the thin devices71-74may be concatenated to form a metavolume of thin devices. In some embodiments, only one thin device may be associated with the same data device while, in other embodiments, multiple thin devices may be associated with the same data device.

In some embodiments, it may be possible to implement the system described herein using storage areas, instead of storage devices. Thus, for example, the thin devices71-74may be thin storage areas, the data devices61-67may be standard logical areas, and so forth. In some instances, such an implementation may allow for hybrid logical devices where a single logical device has portions that behave as a data device and/or portions that behave as a thin device. Accordingly, it should be understood that, in appropriate instances, references to devices in the discussion herein may also apply to storage areas that may or may not correspond directly with a storage device.

FIG. 4is a diagram100illustrating tables that are used to keep track of device information. A first table102corresponds to all of the devices used by a storage device or by an element of a storage device, such as an HA and/or a DA. The table102includes a plurality of logical device (logical volume) entries106-108that correspond to all the logical devices used by the storage device (or portion of the storage device). The entries in the table102may include information for thin devices, for data devices, for standard logical devices, for virtual devices, for BCV devices, and/or any or all other types of logical devices used in connection with the system described herein.

Each of the entries106-108of the table102correspond to another table that may contain information for one or more logical volumes, such as thin device logical volumes. For example, the entry107may correspond to a thin device table112. The thin device table112may include a header114that contains overhead information, such as information identifying the corresponding thin device, information concerning the last used data device and/or other information including counter information, such as a counter that keeps track of used group entries (described below). The header information, or portions thereof, may be available globally to the storage device30.

The thin device table112may include one or more group elements116-118, that contain information corresponding to a group of tracks on the data device. A group of tracks may include one or more tracks, the number of which may be configured as appropriate. In an embodiment herein, each group has sixteen tracks, although this number may be configurable.

One of the group elements116-118(for example, the group element116) of the thin device table112may identify a particular one of the data devices61-67having a track table122that contains further information, such as a header124having overhead information and a plurality of entries126-128corresponding to each of the tracks of the particular one of the data devices61-67. The information in each of the entries126-128may include a pointer (either direct or indirect) to the physical address on one of the disk drives42-44of the storage device30(or a remote storage device if the system is so configured) that maps to the logical address(es) of the particular one of the data devices61-67. Thus, the track table122may be used in connection with mapping logical addresses of the logical devices corresponding to the tables102,112,122to physical addresses on the disk drives42-44of the storage device30.

The tables102,112,122may be stored in the global memory46of the storage device30. In addition, the tables corresponding to particular logical devices accessed by a particular host may be stored (cached) in local memory of the corresponding one of the HA's32-36. In addition, the RA48and/or the DA's36-38may also use and locally store (cache) portions of the tables102,112,122.

FIG. 5is a schematic diagram illustrating a group element116of the thin device table112in connection with an embodiment of the system described herein. The group element116may includes a plurality of entries116a-116f. The entry116amay provide group information, such as a group type that indicates whether there has been physical address space allocated for the group. The entry116bmay include information identifying one (or more) of the data devices61-67that correspond to the group (i.e., the one of the data devices61-67that contains pointers for physical data for the group). The entry116cmay include other identifying information for the one of the data devices61-67, including a speed indicator that identifies, for example, if the data device is associated with a relatively fast access physical storage (disk drive) or a relatively slow access physical storage (disk drive). Other types of designations of data devices are possible (e.g., relatively expensive or inexpensive). The entry116dmay be a pointer to a head of the first allocated track for the one of the data devices61-67indicated by the data device ID entry116b. Alternatively, the entry116dmay point to header information of the data device track table122immediately prior to the first allocated track. The entry116emay identify a cylinder of a first allocated track for the one the data devices61-67indicated by the data device ID entry116b. The entry116fmay contain other information corresponding to the group element116and/or the corresponding thin device. In other embodiments, entries of the group table116may identify a range of cylinders of the thin device and a corresponding mapping to map cylinder/track identifiers for the thin device to tracks/cylinders of a corresponding data device. In an embodiment, the size of table element116may be eight bytes.

Accordingly, a thin device presents a logical storage space to one or more applications running on a host where different portions of the logical storage space may or may not have corresponding physical storage space associated therewith. However, the thin device is not mapped directly to physical storage space. Instead, portions of the thin storage device for which physical storage space exists are mapped to data devices, which are logical devices that map logical storage space of the data device to physical storage space on the disk drives42-44. Thus, an access of the logical storage space of the thin device results in either a null pointer (or equivalent) indicating that no corresponding physical storage space has yet been allocated, or results in a reference to a data device which in turn references the underlying physical storage space.

FIG. 6is a flow diagram200illustrating processing for handling a read of one or more logical tracks of one of the thin devices71-74in an embodiment of the system described herein. In a step202, an appropriate one of the host adapters32-34reads the group table112of the one of the thin devices71-74corresponding to the logical tracks being read. After the step202, at a test step204, it is determined whether the logical tracks identified from the group table112corresponds to any of the data devices61-67(i.e., if there is physical data corresponding to the logical tracks being read). If there is no corresponding physical data (meaning that no logical track(s) were ever written), then processing proceeds to a step206where an error processing is performed, such as returning a NULL value to the host. Other appropriate error processing may be performed at the step206. After the step206, processing is complete.

If it is determined at the step204that there is physical data corresponding to the logical tracks being read, then processing proceeds to a step208where one or more of the data devices61-67associated with the logical tracks being read are identified from the group table112. After the step208, processing proceeds to a step210where the track table122is read from the identified one or more of the data devices61-67and the corresponding location of the physical data (i.e., cylinder and track) is determined. As further discussed elsewhere herein, physical storage space may be provided in connection with one data device and/or by a concatenation of multiple data devices or portions thereof. Logical storage space of the physical devices maps to logical storage space. After the step210, processing proceeds to a step212where a request may be sent to one or more disk adapters36-38corresponding to disk drives42-44that provide the physical storage space associated with the identified one of the data devices61-67and corresponding location information. After the step212, processing proceeds to a step214where the physical data is read. Note that the data may be stored in a cache or other memory (for example, the memory46) in connection with being read. After the step214, processing proceeds to a step216where the data may be received by an appropriate one of the host adapters32-34(e.g., by reading the memory46). After the step216, processing is complete.

FIG. 7is a flow diagram300illustrating processing for handling a write of logical track(s) to one or more of the thin device(s)71-74in connection with the system described herein. At a step302, an appropriate one of the host adapters32-34reads the group table112of the one of the thin device(s)71-74corresponding to the logical tracks being written.

Following the step302is a test step304where it is determined whether physical space had been previously allocated (i.e., in a prior write operation) for the logical tracks being written. If so, then processing proceeds to a step306where the data device that includes the logical tracks is identified. After the step306, is a step308where the track table122is read from the identified one or more of the data devices61-67and the corresponding location of the physical data (i.e., cylinder and track) is determined. As further discussed elsewhere herein, physical storage space may be provided in connection with one data device and/or by a concatenation of multiple data devices or portions thereof. Logical storage space of the physical devices maps to logical storage space. Following the step308processing proceeds to a step312where the data being written is directed to the appropriate physical storage space. The incoming data overwrites the appropriate portion of the data where directed. After the step312, processing is complete.

If it is determined at the step304that there is no physical storage that has been allocated for the logical track(s) being written, then control transfers from the step304to a step314, where a next available data device identifier (i.e., the data device68) is determined. This information may be obtained from the header114of the device table112. In an embodiment herein, data device identifiers are provided by numbers so that a next available data device identifier is simply one more than a last allocated data device.

After the step314, processing proceeds to a step316where available physical storage space on the disk drives42-44is determined. In an embodiment herein, available physical storage space is allocated sequentially from one or more of the disk drives42-44. Following the step316is a step318where a request may be sent to a disk adapter36-38to allocate the physical storage space for the write. Also at the step318, header info is updated to reflect the newly allocated data device and physical tracks. After the step318, processing proceeds to the step312, discussed above, where the data being written is directed to the one or more data devices. After the step312, processing is complete.

After the read and write processes illustrated inFIGS. 6 and 7, information concerning access of the data, such as access frequency, time of last access or use and/or other characteristics and statistics, may be updated and stored by the system described herein. The updated data access information or other characteristic information of the data and/or any portion of the data may, for example, be stored as an entry in a group element of the thin device table112(for example, the entry116fof the group element116as shown inFIG. 5). Alternatively, the data characteristic information may be stored in a memory, such as the global memory46of the storage device30, and a pointer to this information stored in the group element116. Other implementations for storing and access of the data characteristic information are possible.

The allocation of the physical storage space for a thin device at the time of writing the data may be transparent to a user. For example, a user's inquiry into how much storage space is available on a particular thin device may indicate a maximum amount of physical storage space that could be allocated for a thin storage device (provisioned storage space) even though the corresponding physical storage space had not yet been allocated. In some embodiments, used physical storage space does not exceed 30% of the provisioned storage space.

In an embodiment herein, different of the physical data may be automatically moved between different physical disk drives or other storage devices with different characteristics according to one or more policies. For example, data may be initially allocated to a particular fast disk drive, but a portion of the data that has not been used over a period of time (for example, three weeks) may be automatically moved, according to the system described herein, to a slower (and perhaps less expensive) disk drive. The physical data may then be automatically moved back to the faster disk drive if the data is subsequently used and/or accessed according to a policy or other criteria (for example, accessed twice in any given week), as further described herein. Thus, the system described herein may operate to automatically move data between disk drives or other storage devices within the same machine according to the one or more policies.

A policy may be configured by an administrator on a system-wide level or may be specific to a particular user on a specific logical device. The system described herein allows for the remapping of physical data based on policy criteria or other statistics. For example, the policy may be based on the last time data was used and/or accessed. Alternatively, the policy may be based on anticipated use of data over specific times and/or dates. For example, data that is expected to be used at a particular time may be stored on relatively fast disk drives and then moved to relatively slow disk drives when it is expected that the data will not be used. Moreover, different policies and/or criteria may be implemented corresponding to different users and/or different levels of importance or security of data. For example, it may be known that user A accesses particular data more frequently than user B and, accordingly, the policy for moving physical data according to the system described herein may be to leave more data associated with user A on the relatively fast disk drive as compared with the data associated with user B. Alternatively, user A may access data that is generally of a higher level of importance or requires higher security than that of user B and, accordingly, the system described herein may maintain and/or move more data associated with user A on a disk drive that is relatively more reliable and/or secure as compared with the data associated with user B.

In an embodiment herein, data may be moved between physical disk drives (or other physical storage) having different characteristics, such as speed, cost, reliability, security and/or other characteristics. As discussed elsewhere herein, logical data devices may be established having different classes corresponding to characteristics of the physical disk drives to which the data devices are mapped. Further, it should be noted that any section of the logical device may be moved according to the system described herein based on the characteristics of the data.

FIG. 8is a flow diagram400illustrating processing for copying and remapping physical data according to the system described herein. In a step402, a group of tracks is allocated on a data device having a second class than a data device of a first class, where the first and second classes are different. For example, the data device having a different class may be mapped to a physical disk drive that is slower than that of the data device of the first class, as further discussed elsewhere herein. After the step402, processing proceeds to a step404where data associated with the data device of the first class is copied to a location corresponding to the data device of the second class. After the step404, processing proceeds to a step406where the group table of the thin device is updated in accordance with the remapping. After the step406, processing proceeds to a step408where the group of tracks associated with the data device of the first class, from which the data was copied, is deallocated, freeing the locations for future use.

FIG. 9is a flow diagram500illustrating implementation of a policy for data storage and management in connection with an embodiment of the system described herein. In a step502, certain characteristics of stored data are identified (for example, from the group element116, as discussed elsewhere herein). In various embodiments, the characteristics may include usage information such as when the stored data was last accessed and/or how often the stored data has been accessed over a specific time period (for example, hours, days, weeks, etc. . . . ). As further discussed elsewhere herein, the characteristics may also include particular user information corresponding to the stored data. After the step502, processing proceeds to a step504where policy information is accessed. The policy information provides the specific criteria used for data storage and management.

After the step504, processing proceeds to a step506where the policy is applied to the stored data. The policy may include criteria used for managing stored data such as criteria concerning frequency of use of data and/or criteria with respect to specific users and/or other criteria. The policy may be applied to identify data for lifecycle management according to characteristics of entire data volumes or any portions thereof.

After the step506, processing proceeds to a step508where the data for which characteristics have been determined is managed according to the policy and based on the characteristics of the data. For example, data that is frequently used may be moved to a relatively fast storage device whereas data that has not been used over a certain period of time may be moved to a relatively slow storage device according to the data processing as discussed elsewhere herein. As noted herein, the data that is moved may be entire data volumes or portions thereof.

After the step508, processing proceeds to a test step510where it is determined if another policy with other criteria should be applied to the stored data being managed. If an additional policy is to be applied, then processing proceeds to the step506. If no further policies are to be applied then processing proceeds to a test step512where it is determined whether there is more data to be managed according to the system described herein. If there is further stored data to manage, then processing proceeds back to the step502. If no further stored data is to be managed, then after the test step512, processing is complete.

The above-noted steps may be performed automatically by the system described herein. For example, the above-noted steps may be performed periodically, at designated times, and/or after particular trigger events, such as access by a particular user to the system (log in and/or log out) and/or after assessment of space usage on the disk drives (for example, space usage on the fast disk drive). Alternatively, the above-noted steps may be activated manually by a user and/or a system administrator. In an embodiment, the system described herein may include a system having at least one processor that performs any of the above-noted steps. Further, computer software, stored in a computer-readable medium, may be provided according to the system described herein including executable code for carrying out any of the above-noted steps and processes.