Data transfer

A virtual tape library including a module which is operable to issue input and output commands to a tape drive connected to the virtual tape library, wherein an input command is a command to write data to a tape connected to the tape drive and an output command is a command to read data from a tape connected to the tape drive.

BACKGROUND OF THE INVENTION

Data held on a first data storage device may be copied to a second data storage device remote from the first data storage device for data security purposes in a backing-up procedure. Should data on the first data storage device be deleted or damaged, the data can be recovered by accessing the second data storage device over a communication network and, potentially, copying the data back to the first data storage device. The original data on the first data storage device may, itself, be a copy of data held on a primary data storage device.

The communication network has a capacity and much of the capacity of the network may be absorbed in activities other than the copying data between the first and second data storage devices. As such, the capacity of the communication network may be in high demand. Backing-up data is, therefore, usually limited to periods of low network demand. During certain periods backing-up procedures may not be possible due to the level of network usage.

SUMMARY OF INVENTION

An embodiment of the present invention provides a virtual tape library apparatus, comprising: a virtual tape library; a tape drive connected to the virtual tape library, the tape drive being operable to receive a tape, wherein read and write operations to and from the tape drive are controlled by the virtual tape library and, the virtual tape library comprises: one or more library media; one or more virtual data tapes stored on the one or more library media; and a control module coupled to the tape drive and one or more library media, the control module being operable to control the copying of a data between the virtual tape library and a tape received in the tape drive.

In an embodiment, the tape drive is attached to the virtual tape library.

In an embodiment, the tape drive is directly connected to the virtual tape library.

An embodiment of the invention provides a virtual tape library system, comprising: a rack including a processing unit and two or more device receiving slots; a first virtual tape library storage device attached to the rack at a first device receiving slot, the first virtual tape drive storage device having one or more virtual tapes stored thereon; and a first tape drive attached to the rack at a second device receiving slot and coupled to the first virtual tape library storage device, wherein the processing unit is coupled to the first virtual tape library storage device and the first tape drive and controls the movement of tape data between the first virtual tape library storage device and the first tape drive.

An embodiment of the present invention provides a virtual tape library system comprising: one or more library media with one or more virtual tapes stored thereon; and a firmware module which is operable to issue input and output commands to a tape drive connected to the virtual tape library system, wherein an input command is a command to write data to a tape connected to the tape drive and an output command is a command to read data from a tape connected to the tape drive.

In an embodiment, the input and output commands form part of a command set for the tape drive and at least part of the command set is stored on the virtual tape library system.

An embodiment of the invention provides a method of writing data to a virtual tape library system from a tape comprising the steps of: receiving an instruction to write data to a virtual tape library system from a tape; reading data from the tape; writing the read data to the virtual tape library; reading auxiliary data from an auxiliary memory of the tape; and writing at least part of the auxiliary data to the virtual tape library.

In an embodiment, the at least part of the auxiliary data comprises an identifier for that tape.

DETAIL DESCRIPTION

FIG. 1shows a schematic representation of an apparatus1in accordance with an embodiment of the invention. The apparatus1comprises a first device2which may be connected to a data storage device3over a first communication network4. The data storage device3comprises a virtual tape library3and may store one or more data sets or partial data sets. Each data set or partial data set may represent a virtual tape.

The one or more data sets may include one or more filemarks (for example, to mark the end of a file) and/or one or more block separators (for example, to mark the end of a block). A data set may include one or more data blocks. A data set may include one or more data files.

A virtual tape may be a volume of data which is stored in a format which corresponds with a format that the volume of data would be stored in if the data volume was stored on a tape. In an embodiment, the virtual tape format is a compacted version of the corresponding tape format. As such, the virtual tape library may include data chunks which represent sections of data.

The virtual tape library3is, in an embodiment, operable to communicate with other devices as if it was a tape library; the inputs and outputs of the virtual tape library3to the other devices mimic those of a tape library.

In an embodiment of the invention, the virtual tape library3is connected to one or more library media5(seeFIG. 3). In an embodiment, the virtual tape library3is attached to one or more library media5. In an embodiment, the one or more library media5comprise one or more library data storage devices5. The one or more library media5may constitute the storage medium for the virtual tape library3on which on or more virtual tapes may be stored. In an embodiment, the virtual data library3comprises a rack19(seeFIG. 5) with one or more library media receiving members to which respective library media5may be attached and connected. The library media5may include: a magnetic storage device (such as a hard disk), an optical storage device (such as digital versatile disk), or an electronic storage device (such as flash memory). In an embodiment, the one or more library media5may comprise an array of magnetic storage devices. The array may be a RAID array. In an embodiment, the rack19has two or more device receiving slots20and at least one of the receiving slots20is operable to receive a library medium5and at least one of the receiving slots20is operable to receive a tape drive11. The rack19may include a processing unit21.

The data storage device3may be connected to a second communication network6.

The first4and second6communication networks may comprise a local area network or a wide area network. The networks4,6are operated using respective network protocols which may be the same network protocol for each network4,6. One or more further devices7may be connected to the first4and/or second6communication networks. The first communication network4may comprise a direct connection between the first device2and the data storage device3to which other devices7are not connected.

The data storage device3may maintain one or more indices8which record the format and/or shape of data within one or more data sets which are stored on the library media5. An index8may form part of a data set. For example, the index8may record the sizes of blocks of data stored in the data set, the location or locations of the first data in the data set and/or a block of the data set, the location or locations of the last data in the data set and/or block of the data set, and/or the location of one or more filemarks within the data set. The location information may be provided as a logical address based on a known location in the data set (for example, with respect to the first data item or byte of the data set). A unique or substantially unique identifier for a block or set of data may be stored in the one or more indices8in order to all identification of the one or more data sets.

In an embodiment, a single index8may be maintained for a single data set. In an embodiment, a single index8may be maintained for multiple data sets.

The one or more indices8may be updated following input or output functions of the data storage device3which act on the one or more data sets and, therefore, result in a change of one or more of the records in the one or more indices8.

The data storage device3may comprise a compacting virtual tape library.

The first device2may comprise a computer or processing apparatus. In an embodiment, the first device2includes a user interface9which allows a user to control the movement of data to and from the library media5of the data storage device3.

The data storage device3includes a control module10(seeFIG. 2) which is operable to communicate with the first device2in order to effect user commands initiated on the first device. The control module10may include firmware16. The data storage device3may include one or more tape drives11each suitable to receive or to be connected a tape12(seeFIG. 3). In an embodiment, the tape12is located in a library of tapes12. In an embodiment, the data storage device3is operable to pass input and output commands to one or more tape drives11connected to the data storage device3. In an embodiment, the input and output commands comprise write and read instructions respectively. In an embodiment, the input and output commands are made in accordance with a command set18of the one or tape drives11. In an embodiment, the command set18is part of a driver17for the one or more tape drives11. In an embodiment, the driver17is located on firmware16of the data storage device3(seeFIG. 4).

In an embodiment of the invention, at least one of the one or more tape drives11is only able to receive a single tape12at a given time. In an embodiment of the invention, the one or more tape drives11may be connected and attached to the virtual tape library3. In an embodiment of the invention, the one or more tape drives11are directly connected to the virtual tape library3; in this embodiment, the one or more tape drives11are not connected to the virtual tape library3through a server; in an embodiment, the one or more tape drives11are directly connected to the virtual tape library3through a single data controller13which links the or each tape drive11to a data bus of the virtual tape library3without the need for another data controller. In an embodiment, the one or more tape drives11are directly attached to the virtual tape library3. The data controller may form part of the control module10.

In an embodiment the or each tape drive11is connected to the virtual tape library3by a SCSI, IDE, Fibre Channel, ANSI FC-SB-3 (Single-Byte Command Code Sets-3 Mapping Protocol), ESCON, parallel port, USB, IEEE 1394 or other interface.

In an embodiment of the invention, the or each tape drive11is a tape drive which is connected to the virtual tape library3through a data bus. In an embodiment of the invention, each tape drive11is a tape drive which is not connected to the virtual tape library3using an Ethernet connection. In an embodiment of the invention, each tape drive11is controlled by an operating system of the virtual tape library3. In an embodiment of the invention, each tape drive11is controlled by firmware in the virtual tape library3. In an embodiment of the invention, each tape drive11is controlled by an operating system of the virtual tape library3in combination with firmware located in the virtual tape library3.

In an embodiment, each tape drive11comprises a tape drive which is fitted to the virtual tape library rack19.

In an embodiment, each tape drive11is under the direct control of a processing unit of the data storage system3. In other words, the tape drive11is not under the control of, for example, a server processor which is, itself, under the control of the processing unit of the data storage system3.

The control module10may, in some embodiments, comprise a processing unit.

The control module10may include a table14of characteristics for transfer operations for tapes12which may be connected to the one or more tape drives11. The characteristics may include, in an embodiment, an identifier for the type of tape12, and a transfer threshold for the tape12. In an embodiment, the characteristics include a tape capacity, maximum data write rate, and/or maximum data read rate. The transfer threshold may be a minimum size for a volume of data which should transferred to the tape12in a single transaction.

In operation, if the size of a data volume which is waiting to be transferred to the tape12is below the threshold, then the data of the data volume may be buffered until additional data has been added to the data volume to bring the total data volume size above the threshold. Transferring data to the tape12in this manner allows data to be effectively streamed to the tape12. The streaming of data is often a more efficient method of transferring data to a tape12. In an embodiment of the invention, a transfer threshold may be dependent upon the tape drive11and/or the data storage device3.

The first device2connected to the data storage device3is operable to issue one or more instructions to the data storage device3; these instructions are issued over the first communication network4. The one or more instructions may include instructions to move a tape12into a tape drive11or to eject a tape12from a tape drive11. The one or more instructions may include instructions to copy one or more data sets or a portion thereof from the one or more library media5of the data storage device3to one or more tapes12located in one or the one or more tape drives11of the data storage device3(ie. an off-loading process). The one or more instructions may include instructions to copy one or more data sets or a portion thereof from one or more tapes12located in the one or more tape drives11of the data storage device3to the one or more library media5of the data storage device3(ie. an in-loading process).

A graphical user interface may be presented at the first device2to allow a user to view a visual representation of the data storage device3and other parts and components of embodiments of the present invention. The first device2may accept “drag-and-drop” commands input by a user and output the associated one or more instructions to instruct the data storage device3to carry out the user's commands.

The one or more tape drives11of the data storage device3are not connected to the data storage device3across the first4or second6communication networks; therefore, the communication networks4,6need not be utilized to copy data from the data storage device3to a tape12or vice versa (ie. to in-load or off-load data). As such, an embodiment of the present invention may result in a decreased use of both or one of the communication networks4,6; as the capacity of the networks4,6may be in high demand, this may constitute an advantage over prior systems.

Moreover, it is not necessary to connect a server to the first4or second6communication network, the server including connections to one or more data tapes which may be located in tape drives of the server, to achieve in-loading and off-loading processes; instead there is a direct connection between the tape drives11and the virtual tape library3. Thus, embodiments of the present invention may provide a simpler network configuration for the connection of tapes12to a virtual tape library3for the transfer of data between the tapes12and the virtual tape library3.

When the control module10receives an instruction to copy data from the library media5to one or more tapes12connected to the one or more tape drives11of the virtual tape library3(ie. an out-loading transaction) then, in an embodiment, the control module10checks whether or not one or more appropriate tapes12are connected to the one or more tape slots11.

An appropriate tape12may be a tape12of a specified identity or a tape12of one or more specified types (for example, tape12sufficiently large to store the data which the virtual tape library3has been instructed to copy to the tape12and/or a tape12which can accept data in the format in which it is stored on the library media5—eg. with regard to block size).

If the data storage device3is connected to a tape library, then the control module10may instruct the library to locate an appropriate tape12and to connect the tape12to one of the tape drives11.

If an appropriate tape12cannot be connected to one or more tape drives11, then the control module10issues an error signal. In an embodiment, the control module10may ignore or delete an out-loading transaction if no appropriate tape12can be or is connected to the one or more tape drives11. The error signal may be transmitted to the first device2.

By way of example, an appropriate tape12will be assumed to have been located and connected to a tape drive11.

The control module10then mounts the tape12and determines whether or not the tape12is blank. In an embodiment the control module10determines whether or not the tape12is write-protected. If the tape12is not blank then an error signal is issued and, in an embodiment, a user may be prompted to instruct the control module10to locate one or more other appropriate tapes12. A user may be prompted, as a result of the error signal, to instruct the control module10to overwrite the tape12. If the tape12is write-protected then an error signal is issued and, in an embodiment, a user may be prompted to instruct the control module10to locate one or more other appropriate tapes12. The first device2or the control module10may be provided with default instructions which are processed should the user not respond to the error signal, or automatically in the event of an error signal.

The control module10may interrogate the table14of tape transfer characteristics in order to determine the transfer threshold for each particular transaction. In an embodiment, the table14is interrogated to determine characteristics of the transfer other than the transfer threshold.

The control module10receives the data associated with the off-loading transaction from the one or more library media5of the data storage device3. In an embodiment, the data may include an index or indices8(as described above). If an index or indices8are present, then the control module10may read the index or indices8to determine a mode in which the data or part of the data is written to the tape12. For example, the control module10may calculate the maximum multiple of the data blocks (using information recorded in the index or indices8) that has a data volume size greater than the threshold but smaller than the maximum data volume size which the tape12can receive (this maximum may be set by characteristics of the tape12, the tape drive11and/or the virtual tape library3—for example the size of a data buffer or cache in which the data must be stored before it is transferred the tape12).

If the data is a filemark (the location or locations of which may be stored in the index or indices8) then the data may be directly transferred to the tape12irrespective of the threshold thereof. This may be known as an immediate writing operation.

The first device2may issue an abort signal after issuance of the off-loading instructions. The abort signal may be received by the control module10and, in response, the control module10may abort the off-loading transaction. The abortion of the off-loading transaction may comprise the deletion of any data already copied to the tape12(a long erase), the deletion of any index for the data copied to the tape12(a short erase), the resetting of the tape12into its original state before the transaction commenced, the ejection or disconnection of the tape12from the tape drive11, and/or issuance of a signal to the first device2to confirm abortion of the transaction.

During the transaction, statistical information regarding the transaction may be recorded and/or transmitted to the first device2. This statistical information may include the transfer rate, the quantity of data copied in the transaction, the portion of the total transaction which has been completed, and/or the expected time of completion for the transaction. In an embodiment, statistical information is sent to the first device2on request; in an embodiment, statistical information is sent to the first device2at predetermined intervals.

During a transaction the transaction may fail due to an error in the data storage device3, the tape drive11or the tape12. If an error occurs, then the transaction may be aborted as described above. In an embodiment, an error may occur if any data fails to be successfully copied as part of the transaction.

The data which is the subject of the transaction may be a data set; the data may be a virtual tape.

On the completion of a successful off-loading transaction, the virtual tape library3may, in an embodiment, delete the data which was the subject of the transaction from the library media5. In an embodiment, the virtual tape library3may mark the data which was the subject of the transaction as read-only. An embodiment of the invention ensures that there is only one “active” copy of a volume of data; the active copy (which may be written to and read from) may be stored on the virtual tape library3or a tape12. Thus, in this embodiment, when a copy of data is made one copy of the data is marked as an inactive copy and the other copy is marked as the active copy. If the copy of the data is a copy of inactive data, then the copy is also marked as inactive data. In an embodiment of the invention, inactive data is read-only. In an embodiment of the invention, inactive data is deleted. In an embodiment of the invention inactive data may be activated—for example, if the active data becomes lost or damaged.

In an in-loading transaction an analogous process occurs. A tape12containing data is connected to a tape drive11. The connection may be as a result of instructions issued by the first device2. Instructions are issued by the first device2to the control module10to initiate an in-loading transaction.

In an embodiment of the invention, an in-loading transaction begins with the control module10confirming that there is sufficient storage space on the one or more library media5to store a copy of the data on the tape12. If there is insufficient capacity, then the control module10issues an error signal which may be handled in the same manner as the out-loading error signals. If there is sufficient capacity, then the data is copied from the tape12to the virtual tape library3.

The in-loading process may be similar to the off-loading process with regard to, for example, error signals, statistics, active/inactive copies and/or transfer thresholds. As the data is being copied to the library media5, an index or indices8of the data (as described above) are created and stored in or with the data. The index or indices8may contain data block sizes which are derived from the data copied from the tape12.

The data may be a data set. The data which is copied to the virtual tape library3may form a virtual tape.

In an embodiment of the invention, as data is being copied to the data storage device3the control module10may generate an error checking code for all or part of the data being copied. The error checking code may be calculated as data is being copied or after the data has been copied.

Possible error checking codes which may be used with embodiments of the invention include cyclic redundancy checks.

In an embodiment of the invention, during a transaction, data may be written or copied to an auxiliary memory15of the tape12. The data may include, for example, a virtual barcode, an identification code, a maximum configured data set size, and/or a read/right access code.

In an embodiment of the invention a record is maintained of data which has been written to the auxiliary memory15of the tape12and a description of the content of the tape. The description of the content of the tape12may comprise a data identifier. When new data is written to the tape12or to the auxiliary memory15of the tape, then the record is updated. The record may be maintained on the data storage device3. The record may be maintained on the first device2. The record may be maintained on a further device7.

In an embodiment of the invention, when data is written to the data storage device3from a tape12, the content (or part of the content) of the auxiliary memory15of the tape12is copied to the data storage device3and may be associated with the data which has been copied from the tape12(seeFIG. 6). Thus, if the data is later written to a tape12, the auxiliary memory15of that tape12can be provided with at least some of the information which was stored on the auxiliary memory15of the original tape12. In an embodiment, the data which may be written to the auxiliary memory of a tape and which is associated with data on the data storage device3is provided by the first device2.

It will be appreciated that embodiments of the invention may be utilized to copy many data sets or virtual tapes to and from one or more tapes12and the virtual tape library3.

In embodiments of the invention, one or more further devices7and/or further data storage devices7may be connected to the first4and/or second6communication networks.

It will be understood that embodiments of the present invention may reduce network demand and that transactions can occur during periods of high network demand where they were previously not possible or not allowed.

It will be understood that embodiments of the present invention may be implemented without increasing network demand. Thus, data may be transferred or copied to and from a tape without causing a substantial reduction in network availability for other processes.

Embodiments of the present invention may also aid in the reduction of network complexity.

For example, if the one or more tape drives11of an embodiment of the invention were attached to a server which was, in turn attached to the data storage device3, then a software application may request the copying of data to a tape drive onto the data storage device3. The request would be made over a communication network. The data would be read from the data storage device3to the server over a communication network. The data would then be written to a tape12in the tape drive. This comprises three instructions which must be passed through one or more communication networks. In addition, a large amount of data must be passed though a communication network from the data storage device3to the server.

In an embodiment of the present invention, an application on the first device2may request the copying of data from the data storage device3to a tape drive11. The request is passed to the data storage device3which simply needs to write the data to a tape12in a tape drive11connected to the data storage device. This comprises just one two instructions and negates the need to pass the data itself through a communication network.

In an embodiment of the invention, the control module10comprises a computer program stored and run on the data storage device3.