Dynamic scratch pool management on a virtual tape system

A method for managing volumes in a scratch pool of a virtual tape system is disclosed. In one embodiment, such a method provides a scratch pool containing volumes for use in a virtual tape system. The method further enables a user to predefine an external pool of volumes residing outside of the scratch pool. This external pool may be hidden to a host system accessing the virtual tape system. The method monitors current and/or past usage of the volumes in the scratch pool and, based on the usage, predicts a future need for volumes in the scratch pool. The method automatically moves volumes between the external pool and the scratch pool in accordance with the future need. A corresponding system and computer program product are also disclosed.

BACKGROUND

Field of the Invention

This invention relates to systems and methods for managing scratch pools in virtual tape systems.

Background of the Invention

As data storage needs continue to increase at a rapid rate, magnetic tape continues to offer some significant advantages over other data storage technologies. At an average cost on the order of $0.01 per gigabyte, tape storage is typically the most affordable option for storing massive quantities of data. Recent technological advances have also increased the speed that data can be written to and/or retrieved from tape, with some tape drives having the ability to read and/or write data at speeds of over 1 terabyte per hour. Other advantages of magnetic tape include reduced energy costs associated with storing data, portability, greater reliability and longevity, and the ability to easily scale tape storage as storage needs increase. For the reasons provided above, tape storage often plays a significant role in an organization's data storage infrastructure.

A virtual tape system (VTS) is a storage solution that combines a high-speed disk cache with tape automation, tape drives, and intelligent storage management software running on a server. The disk cache associated with the VTS acts as a buffer to the tape drives, providing near-instantaneous performance for multiple, simultaneous scratch-mount requests and for specific mount requests for tape volumes that reside in the disk cache. A VTS breaks the one-to-one connection between a logical tape drive and a physical tape drive, enabling logical access to significantly more tape drives than are physically installed. In addition, a VTS breaks the one-to-one connection between a tape cartridge and a tape volume. One key reason tapes are significantly underutilized is that a single application may own a particular drive and the associated tapes. If that application does not fully utilize the associated tape capacity, it may be wasted.

In a VTS, a user typically must have at least one volume available in a scratch pool in order to satisfy a request to mount a volume to write new files to tape. It is common for a scratch pool to run out of volumes, which can cause disruption to batch and online processing. When this occurs, an administrator must typically intervene to free up additional volumes to be placed in the scratch pool. This can be a time-consuming process that may undesirably cause delays to production cycles. To avoid such delays, an administrator may need to decide how many scratch volumes are needed in the pool and monitor the number of volumes to ensure production cycles are not negatively impacted by running out of scratch volumes.

In view of the foregoing, what are needed are systems and methods to monitor scratch pools and automatically add scratch volumes to the scratch pools on an as-need basis. Ideally, such systems and methods will minimize impacts to production activities such as batch and online processing.

SUMMARY

The invention has been developed in response to the present state of the art and, in particular, in response to the problems and needs in the art that have not yet been fully solved by currently available systems and methods. Accordingly, systems and methods are disclosed for managing scratch pool volumes in a virtual tape system. The features and advantages of the invention will become more fully apparent from the following description and appended claims, or may be learned by practice of the invention as set forth hereinafter.

Consistent with the foregoing, a method for managing volumes in a scratch pool of a virtual tape system is disclosed. In one embodiment, such a method provides a scratch pool containing volumes for use in a virtual tape system. The method further enables a user to predefine an external pool of volumes residing outside of the scratch pool. This external pool may be hidden to a host system accessing the virtual tape system. The method monitors current and/or past usage of the volumes in the scratch pool and, based on the usage, predicts a future need for volumes in the scratch pool. The method automatically moves volumes between the external pool and the scratch pool in accordance with the future need.

A corresponding system and computer program product are also disclosed and claimed herein.

DETAILED DESCRIPTION

The present invention may be embodied as a system, method, and/or computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

The computer readable program instructions may execute entirely on a user's computer, partly on a user's computer, as a stand-alone software package, partly on a user's computer and partly on a remote computer, or entirely on a remote computer or server. In the latter scenario, a remote computer may be connected to a user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

Referring toFIG. 1, one example of a network environment100is illustrated. The network environment100is presented to show one example of an environment where systems and methods in accordance with the invention may be implemented. The network environment100is presented by way of example and not limitation. Indeed, the systems and methods disclosed herein may be applicable to a wide variety of network environments, in addition to the network environment100shown.

As shown, the network environment100includes one or more computers102,106interconnected by a network104. The network104may include, for example, a local-area-network (LAN)104, a wide-area-network (WAN)104, the Internet104, an intranet104, or the like. In certain embodiments, the computers102,106may include both client computers102and server computers106(also referred to herein as “host systems”106). In general, the client computers102initiate communication sessions, whereas the server computers106wait for requests from the client computers102. In certain embodiments, the computers102and/or servers106may connect to one or more internal or external direct-attached storage systems112(e.g., arrays of hard-disk drives, solid-state drives, tape drives, tape libraries, virtual tape libraries etc.). These computers102,106and direct-attached storage systems112may communicate using protocols such as ATA, SATA, SCSI, SAS, Fibre Channel, or the like.

The network environment100may, in certain embodiments, include a storage network108behind the servers106, such as a storage-area-network (SAN)108or a LAN108(e.g., when using network-attached storage). This network108may connect the servers106to one or more storage systems110, such as arrays110aof hard-disk drives or solid-state drives, tape libraries110bor virtual tape libraries110b, individual hard-disk drives110cor solid-state drives110c, tape drives110dor virtual tape drives110d, CD-ROM libraries, or the like. To access a storage system110, a host system106may communicate over physical connections from one or more ports on the host106to one or more ports on the storage system110. A connection may be through a switch, fabric, direct connection, or the like. In certain embodiments, the servers106and storage systems110may communicate using a networking standard such as Fibre Channel (FC).

Referring toFIG. 2, in certain embodiments, a virtual tape library110b(also referred to herein as a virtual tape system110b), may utilize a scratch pool200of virtual tape volumes202to accommodate requests to mount volumes to store files thereon. When a volume202is full of data, the volume202may be inventoried in an archive204until the data is no longer needed or has expired. At this point, the volume202may be returned to the scratch pool200so it can be reused for future mount requests.

As previously mentioned, a virtual tape system110bmust typically have at least one volume202available in its scratch pool200in order to satisfy a request to mount a volume202to write new files to tape. It is common for a scratch pool200to run out of volumes202, which can cause disruption to production activities such as batch and online processing. When this occurs, an administrator may need to intervene to free up additional volumes202for placement in the scratch pool200. This can be a time-consuming process that may undesirably cause delays to production cycles. To avoid such delays, an administrator may need to decide how many scratch volumes202are needed in the scratch pool200and monitor the level of available scratch volumes202to make sure production cycles are not adversely impacted by running out of scratch volumes202.

Referring toFIG. 3, in certain embodiments in accordance with the invention, an external pool300of volumes202may be established to increase a number of scratch volumes202in the scratch pool200. In certain embodiments, the volumes202in the external pool300may be predefined in advance. For example, the volumes202in the external pool300may be assigned a range of volume serial numbers (i.e., volsers), default constructs, and media types. The default constructs may establish whether the volumes202support encryption and/or compression and, if so, what types of encryption/compression the volumes202support. The default constructs may also establish the storage capacities of the volumes202, the recording technologies use to record data on the volumes202, and the like. The designated media types may indicate the type of magnetic tape the volumes202are configured to emulate. Because defining the characteristics of the volumes202may take significant time, predefining the volumes202in the external pool300before they are actually needed may reduce delay and enable the volumes202to be dynamically added to the scratch pool200on an as-need basis.

Although the virtual tape system110binFIG. 3is shown with a single scratch pool200and external pool300, in reality the virtual tape system110bmay include multiple scratch pools200and external pools300. Each scratch pool200and external pool300may have its own volume serial numbers, default constructs, and media types. The volumes202in an external pool300may be predefined with volume serial numbers, default constructs, and media types that are consistent with the scratch pool200to which it is assigned. This enables volumes202to be dynamically added to the scratch pool200while assuring that the volumes202in the external pool300have the characteristics required of volumes202in the scratch pool200. In certain cases, the volumes202in the external pool300may be dynamically and automatically moved to the scratch pool200to accommodate temporary or unanticipated spikes in workload processing.

Referring toFIG. 4, in order to provide various features and functions in association with the external pool300, a scratch pool management module400may be included in the virtual tape system110b. The scratch pool management module400may be implemented in software, hardware, firmware, or a combination thereof. In general, the scratch pool management module400may be used to manage volumes202in the scratch pool200and dynamically add volumes202to the scratch pool200on an as-need basis and/or to anticipate a future need.

As shown, the scratch pool management module400may include various sub-modules to provide different features and functions. The sub-modules may include one or more of a predefinition module402, monitoring module404, prediction module406, threshold module408, movement module410, grace period module412, reporting module414, broadcast module416, substitution module418, and notification module420. These sub-modules are presented by way of example and not limitation. More of fewer modules may be provided in different embodiments. For example, the functionality of some sub-modules may be combined into a single or smaller number of sub-modules, or the functionality of a single sub-module may, in certain embodiments, be distributed across several sub-modules.

The predefinition module402may enable a user to predefine an external pool300for a scratch pool200, as well as volumes202within the external pool300. For example, the predefinition module402may enable a user to define a range of volume serial numbers (i.e., volsers) for the external pool300, as well as default constructs, default media types, thresholds, and the like, for volumes202within the external pool300. This may be performed before the volumes202in the external pool300are actually needed within the scratch pool200.

The monitoring module404may monitor usage of volumes202in the scratch pool200. This may include monitoring past and present usage as well as the number of volumes202that are available in the scratch pool200during these time periods. The monitoring module404may also monitor peak usage times or spikes in usage that may consume additional volumes202in the scratch pool200, or times or periods when the scratch pool200ran out of volumes202. In other cases, the monitoring module404may monitor the growth rate of volumes202in the scratch pool200and, in certain embodiments, whether this growth rate is outside of normal or an indicator of some type of problem or error. In addition to monitoring usage of volumes202in the scratch pool200(i.e., volumes202changing from scratch to private), the monitoring module404may also monitor the return of volumes202from the archive204to the scratch pool200(i.e., volumes202changing from private to scratch). The monitoring module404may, in certain embodiments, record observed numbers in a log and keep a rolling average (e.g., a 30 day rolling average) in order to track trends in scratch volume202consumption and return to the scratch pool200.

Based on the usage, growth rates, trends, etc. monitored by the monitoring module404, the prediction module406may predict future need for volumes202in the scratch pool200. Alternatively, or in addition, a threshold module408may detect whether a number of volumes202in the scratch pool200has fallen below a threshold associated with the scratch pool200. This threshold may, in certain embodiments, be based on future need determined by the prediction module406. For example, the threshold may be set at ninety percent of the designated need. If the number of volumes202in the scratch pool200falls below this threshold, the movement module410may move volumes202from the external pool300to the scratch pool200to more closely align the number of volumes202in the scratch pool200with the designated need for volumes202.

In certain embodiments, the scratch pool management module400carefully manages the number of volumes202in the scratch pool200to avoid exceeding the designated need and thereby placing additional processing and book-keeping overhead on host systems106using the virtual tape system110b. In certain embodiments, the movement module410is configured to move volumes202from the scratch pool200to the external pool300when the scratch pool200contains more volumes202than are needed or anticipated to be needed.

When volumes202are returned to the scratch pool200from the archive204, such as when data has expired or is no longer needed, the grace period module412may, in certain embodiments, provide a host system106a certain amount of time (i.e., a “grace period”) to still be able to access data on the returned volumes202. Thus, in certain cases, volumes202that have been returned to the scratch pool200may not be available for reuse by the virtual tape system110buntil the grace period has expired. In certain embodiments, the monitoring module404may take this grace period into account when determining how many volumes202are not just present in the scratch pool200, but are actually available for reuse. This, in turn, may affect how many volumes202are moved from the external pool300to the scratch pool200to accommodate the designated need.

Using data gathered by the monitoring module404, the reporting module414may create reports showing which host systems106, programs, job names, and data set naming patterns are consuming volumes202in the scratch pool200. This may help an administrator determine if the behavior is expected or if configurations changes are needed in the workload. Thus, in certain embodiments, data that is collected by the monitoring module404and used by the prediction module406to forecast need may also be provided to an administrator in reports so that the administrator may use the data to make decisions or configuration changes in the virtual tape system110b. This data may help an administrator understand what is consuming volumes202in the scratch pool200and/or why the volumes202are being consumed faster or at a different rate than expected.

When volumes202are moved from the external pool300to the scratch pool200, or vice versa, the broadcast module416may communicate these changes to connected host systems106. In certain embodiments, software (e.g., tape management system software) on a host system106may receive the broadcast and update its internal records, such as a tape configuration database and/or tape management system database.

When a user attempts to insert/define new volumes202in the scratch pool200, the substitution module418may check whether volumes202already exist in the external pool300. If volumes202are present in the external pool300, the substitution module418may substitute the volumes202in the external pool300for those the user is attempting to insert/define. These volumes202may then be moved from the external pool300to the scratch pool200in lieu of the attempted insertion. If the volumes202the user is attempting to insert are not in the external pool300, the volumes202may be inserted into the scratch pool200in the conventional manner using conventional insert processing.

In the event the external pool300is anticipated to run out of volumes202, the notification module420may notify a user via, for example, a console message, that additional volumes202(i.e., an additional volser range) need to be added to the external pool300. This may performed well in advance of the external pool300actually running out of volumes202. This enables the user to define additional volumes202in the external pool300so that the volumes202are available to the scratch pool200but without actually placing additional volumes202in the scratch pool200until they are needed. This will prevent over-allocation of volumes202to the scratch pool200and the attendant negative effects to host performance.

Referring toFIG. 5, one embodiment of a method500for predefining an external pool300of volumes202is illustrated. This method500may, in certain embodiments, be executed by the predefinition module402previously discussed. As shown, the method500may enable a user to assign502a range of volume serial numbers (volsers) to an external pool300, as well as specify default constructs, default media types (e.g., what type of magnetic tape to emulate), and the like. The user may also specify thresholds for each scratch pool200of the virtual tape system110b. For example, the user may configure the virtual tape system110bto move volumes202from the external pool300to the scratch pool200if the number of volumes202in the scratch pool200falls below a certain specified number or percentage.

Using the configuration settings established at step502, the virtual tape system110bmay automatically create504logical volumes202in the specified volser range with the specified constructs and media types. These logical volumes202may be placed in the external pool300so they are available for movement to the scratch pool200if and when they are needed.

Referring toFIG. 6, a method600for utilizing the external pool300to increase a number of volumes202in the scratch pool200is illustrated. This method600may, in certain embodiments, be executed by the threshold module408and movement module410previously discussed. This method600may be executed after the scratch pools200and associated external pools300have been established. As shown, the method600initially examines602a first scratch pool200of the set of scratch pools200. The method600then determines604the number of logical volumes202in the scratch pool200. If, at step606, the number of logical volumes202is below the threshold established for the scratch pool200, the method600may move608logical volumes202from the associated external pool300to the scratch pool200. The number of logical volumes202that are moved may, in certain embodiments, depend on the amount that the threshold is exceeded.

The method600may then notify610any connected host systems106of the logical volumes202that have been moved into the scratch pool200. This may allow the host systems106to update their internal catalogs and/or databases. For example, the notification may prompt tape management software on the host systems106to update their tape configuration databases and/or tape management system databases. The method600may then be repeated612for each scratch pool200and associated external pool300in the virtual tape system110b. The method600may be executed periodically to maintain a needed number of volumes202in the scratch pools200.

FIG. 7shows embodiment of a method700for substituting external pool volumes202for those that are manually inserted by a user. This method700may, in certain embodiments, be executed by the substitution module418previously discussed. As shown, the method700initially detects702when a user is attempting to insert volumes202into a scratch pool200. When this occurs, the method700determines704whether the volumes202that are being inserted are already defined in an external pool300associated with the scratch pool200. If the volumes202are already defined in the external pool300, the method700substitutes706the volumes202from the external pool300for those being inserted by the user (particularly if the volumes202in the external pool300have the same volsers as those being inserted by the user). If the volumes202are not already defined in the external pool300, the method700may continue708with insert processing in the conventional manner. The method700completes710the insert processing by inserting the volumes202into the scratch pool. The method700may also notify712any connected host systems106and/or administrator consoles that volumes202were added to the scratch pool200so that the host systems106and/or administrator consoles may updates their internal records/databases. In certain embodiments, the method700also notifies the host systems106and/or administrator consoles that volumes202from the external pool300were substituted for those that were attempted to be manually inserted.