Supporting non-disruptive movement of a logical volume of non-volatile data storage between storage appliances

A primary copy and one or more shadow copies of a logical volume are created and discovered by a host rescan performed when the logical volume is initially created. Data storage resources are allocated to the primary copy, but not to the shadow copy. The initial path state of the logical volume describes the path to the primary copy as active, and the path to the shadow copy as unavailable for accessing the logical volume. Movement of the logical volume to the storage appliance providing the shadow copy can be performed without an additional host rescan, by making the shadow copy the new primary copy, making the primary copy a new shadow copy, and updating the path state of the logical volume to indicate i) that the path to the new primary copy is active, and ii) that the path to the new shadow copy is unavailable.

TECHNICAL FIELD

The present disclosure relates generally to techniques for providing logical volumes of non-volatile data storage to host computer systems, and more specifically to technology for supporting non-disruptive movement of a logical volume between storage appliances.

BACKGROUND

Data storage systems are arrangements of hardware and software that typically provide non-volatile data storage from non-volatile data storage devices that they contain or are communicably connected to, such as magnetic disk drives, electronic flash drives, and/or optical drives. Data storage systems service host I/O operations (e.g. I/O reads, writes, etc.) that they receive from host computers. The received host I/O operations specify one or more data storage objects (e.g. logical volumes, sometimes also referred to as logical units or “LUNs”), and indicate host I/O data that is to be written to or read from the storage objects. Data storage systems include specialized hardware and execute specialized software that process incoming host I/O operations and perform various data storage tasks to organize and secure the host I/O data that is received from the host computers and store the received host I/O data on the non-volatile data storage devices of the data storage system. Data storage systems may sometimes include or consist of a cluster of data storage appliances.

Under various types of circumstances, it may be desirable to change the location of a logical volume, e.g. from a first data storage appliance to a second data storage appliance within a cluster of data storage appliances. Examples of such circumstances include without limitation resource imbalances that may arise between different data storage appliances, such as an inadequate amount of resources (e.g. storage, processing, and/or network resources) being available to support the logical volume on a first data storage appliance, and a sufficient or more sufficient amount of resources being available to support the logical volume on a second data storage appliance.

SUMMARY

Previous technologies for moving the location of a logical volume have exhibited significant technical shortcomings. For example, some previous technologies have operated such that when a logical volume was moved, a rescan operation had to be performed at the time of the move on each host computer that used the logical volume in order for the hosts to start accessing the logical volume from the new location. In some previous systems, such rescan operations performed at the time of a logical volume move had to be done manually on each host computer, e.g. by a storage administrator user, and also sometimes required that the storage administrator determine the current set of host computers that use the logical volume. The time consumed to perform rescan operations at each host computer that uses a logical volume at the time the logical volume is moved may cause a significant disruption in the flow of I/O operations between host computers and the logical volume, during which critical data on the logical volume may become unavailable to applications executing on the host computer, potentially causing such applications to fail.

To address the above described and/or other shortcomings of previous technologies, new technology is disclosed herein for supporting movement of a logical volume (e.g. a block based logical volume of non-volatile data storage sometimes also referred to as a logical unit or “LUN”) between storage appliances. In the disclosed technology, a logical volume is created at least in part by i) creating a primary copy of the logical volume in a first storage appliance in a cluster of storage appliances, and ii) creating a shadow copy of the logical volume in a second storage appliance in the cluster of storage appliances. Creating the primary copy of the logical volume in the first storage appliance includes allocating units of non-volatile data storage from data storage devices of the first storage appliance to the primary copy of the logical volume to store host data written to the logical volume. Creating the shadow copy of the logical volume in the second storage appliance includes allocating no units of non-volatile data storage from data storage devices of the second storage appliance to the shadow copy of the logical volume to store host data.

In response to creation of the logical volume, an initial path state of the logical volume is also created at the time the logical volume is created. The initial path state is created by i) initially setting a path state of a data path between the host computer and the first storage appliance to an active state, and ii) initially setting a path state of a data path between the host computer and the second storage appliance to unavailable. The active state of the data path between the host computer and the first storage appliance indicates to the host computer that the host computer is permitted to send I/O operations (e.g. I/O read and/or I/O write operations) directed to the logical volume over the data path between the host computer and the first storage appliance. The unavailable state of the data path between the host computer and the second storage appliance indicates to the host computer that the host computer is not permitted to send I/O operations directed to the logical volume over the data path between the host computer and the second storage appliance.

Further in response to creation of the logical volume, both the primary copy of the logical volume and the shadow copy of the logical volume are made visible to the host computer as the logical volume at the time the logical volume is created. For example, the primary copy of the logical volume and the shadow copy of the logical may be made visible to the host computer when a rescan operation is performed on the host computer at the time the logical volume is created. The initial path state of the logical volume is also provided to the host computer at the time the logical volume is created, e.g. in response to host inquiries performed at the time of creation of the logical volume. The initial path state of the logical volume provided to the host computer causes the host computer to send host I/O operations directed to the logical volume only over the data path between the host computer and the first storage appliance.

In some embodiments, the logical volume has a unique name, and making both the primary copy of the logical volume and the shadow copy of the logical volume visible to the host computer as the logical volume also includes providing the unique name of the logical volume to the host computer as a name for both the primary copy of the logical volume and the shadow copy of the logical volume, e.g. in response to status inquiries or the like received from the host computer. The unique name of the logical volume may be a Small Computer System Interface (SCSI) World Wide Name (WWN) of the logical volume, and providing the unique name of the logical volume to the host computer as a name for the primary copy of the logical volume may, for example, be performed by sending the WWN of the logical volume to the host computer as the name of the primary copy of the logical volume as part of a Vital Product Data (VPD) page that is related to the primary copy of the logical volume, and that is sent to the host computer in response to a SCSI inquiry VPD page command sent from the host computer and directed to the primary copy of the logical volume. Providing the unique name of the logical volume to the host computer as a name for the shadow copy of the logical volume may similarly be performed by sending the WWN of the logical volume to the host computer as the name of the shadow copy of the logical volume as part of a VPD page that is related to the shadow copy of the logical volume, and that is sent to the host computer in response to a SCSI inquiry VPD page command sent from the host computer and directed to the shadow copy of the logical volume.

In some embodiments, providing the initial path state of the logical volume to the host computer may include providing the initial path state of the logical volume to the host computer in response to both i) a status inquiry sent from the host computer and directed to the primary copy of the logical volume, and ii) a status inquiry sent from the host computer and directed to the shadow copy of the logical volume. In some embodiments, the status inquiry sent from the host computer and directed to the primary copy of the logical volume may be an RTPG command sent from the host computer and directed to the primary copy of the logical volume, and the status inquiry sent from the host computer and directed to the shadow copy of the logical volume may be an RTPG command sent from the host computer and directed to the shadow copy of the logical volume. Providing the initial path state of the logical volume to the host computer may be accomplished by sending the initial path state of the logical volume to the host computer both i) as part of an ALUA state related to the primary copy of the logical volume and sent to the host computer in response to the RTPG command sent from the host computer and directed to the primary copy of the logical volume, and ii) as part of an ALUA state related to the shadow copy of the logical volume and sent to the host computer in response to the RTPG command sent from the host computer and directed to the shadow copy of the logical volume.

In some embodiments, the data path between the host computer and the first storage appliance may extend at least in part between an initiator port on the host computer and a target port on the first storage appliance. The data path between the host computer and the second storage appliance may similarly extend at least in part between an initiator port on the host computer and a target port on the second storage appliance.

In some embodiments, host access to the logical volume may be moved from the first storage appliance to the second storage appliance during a relocation operation performed in response to receipt of a relocation command. The relocation operation is performed at least in part by creating an updated path state for the logical volume. Creating the updated path state for the logical volume may include i) updating the path state of the data path between the host computer and the first storage appliance to unavailable, the unavailable state of the data path between the host computer and the first storage appliance indicating to the host computer that the host computer is not permitted to send I/O operations directed to the logical volume over the data path between the host computer and the first storage appliance, and ii) updating the path state of the data path between the host computer and the second storage appliance to an active state, the active state of the data path between the host computer and the second storage appliance indicating to the host computer that the host computer is permitted to send I/O operations directed to the logical volume over the data path between the host computer and the second storage appliance. In response to creation of the updated path state for the logical volume, the primary copy of the logical volume may be changed to a new shadow copy of the logical volume, and the shadow copy of the logical volume may be changed to a new primary copy of the logical volume. The updated path state of the logical volume may then be provided to the host computer. The updated path state of the logical volume provided to the host computer causes the host computer to subsequently send I/O operations directed to the logical volume only over the data path between the host computer and the second storage appliance.

In some embodiments, moving host access to the logical volume from the first storage appliance to the second storage appliance may include allocating units of non-volatile data storage from data storage devices of the second storage appliance to the new primary copy of the logical volume to store host data written to the logical volume.

In some embodiments, moving host access to the logical volume from the first storage appliance to the second storage appliance may further include moving host data stored in the units of non-volatile data storage allocated from data storage devices of the first storage appliance to the primary copy of the logical volume to the units of non-volatile data storage allocated from the data storage devices of the second storage appliance to the new primary copy of the logical volume.

The disclosed technology may be embodied to provide significant advantages over previous technologies. For example, because both the primary and shadow copies of the logical volume are visible to the host through a rescan operation performed on the host computer at the time that the logical volume is created, any subsequent movement of the logical volume between the storage appliance providing the primary copy and the storage appliance providing the shadow copy is anticipated. In the event that the logical volume is eventually moved between to the storage appliance providing the shadow copy of the logical volume, there is no need to notify the host computer, or the host computer's system administrator, in order for another rescan to be performed on the host to discover the new primary copy of the logical volume on the storage appliance to which the logical volume was moved, since the host computer already has knowledge of both the primary and shadow copy of the logical volume from the rescan performed when the logical volume was initially created. The storage administrator does not need to manually perform a rescan operation from each host computer that accesses the logical volume to find the new location of the logical volume at the time host access to the logical volume is moved. The storage administrator accordingly need not determine which host computers are accessing the logical volume at that time. The movement of the logical volume may in fact be fully automated between the storage appliances, transparently with regard to the host computer and/or host computer system administrator. The disclosed technology eliminates the significant delays that could occur in previous systems as a result of performing rescan operations at each host computer at the time the logical volume is moved. The disclosed technology accordingly eliminates the disruption in the flow of I/O operations between host computers and the logical volume caused by such rescans, during which data on the logical volume may become unavailable to applications executing on the host computer. The disclosed technology accordingly eliminates the potential for those applications to fail as a result of such data unavailability.

DETAILED DESCRIPTION

Embodiments will now be described with reference to the figures. Such embodiments are provided only by way of example and for purposes of illustration. The scope of the claims is not limited to the examples of specific embodiments shown in the figures and/or otherwise described herein.

The individual features of the particular embodiments, examples, and implementations described herein can be combined in any manner that makes technological sense. Such features are hereby combined to form all possible combinations, permutations and/or variations except to the extent that such combinations, permutations and/or variations have been expressly excluded herein and/or are technically impractical. The description in this document is intended to provide support for all such combinations, permutations and/or variations.

As described herein, a logical volume is created by i) creating a primary copy of the logical volume in a first storage appliance in a cluster of storage appliances, and ii) creating a shadow copy of the logical volume in a second storage appliance in the cluster of storage appliances. Creating the primary copy of the logical volume in the first storage appliance includes allocating units of non-volatile data storage from data storage devices of the first storage appliance to the primary copy of the logical volume to store host data written to the logical volume. Creating the shadow copy of the logical volume in the second storage appliance includes allocating no units of non-volatile data storage from data storage devices of the second storage appliance to the shadow copy of the logical volume to store host data.

At the time the logical volume is created, in response to creation of the logical volume, an initial path state of the logical volume is also created. The initial path state is created by i) initially setting a path state of a data path between the host computer and the first storage appliance to an active state, and ii) initially setting a path state of a data path between the host computer and the second storage appliance to unavailable. The active state of the data path between the host computer and the first storage appliance indicates to the host computer that the host computer is permitted to send I/O operations (e.g. read I/O operations and/or write I/O operations) that are directed to the logical volume over the data path between the host computer and the first storage appliance. The unavailable state of the data path between the host computer and the second storage appliance indicates to the host computer that the host computer is not permitted to send I/O operations directed to the logical volume over the data path between the host computer and the second storage appliance.

Also at the time the logical volume is created, in response to creation of the logical volume, both the primary copy of the logical volume and the shadow copy of the logical volume are made visible to the host computer as the logical volume. For example, the primary copy of the logical volume and the shadow copy of the logical may be made visible to the host computer when a SCSI rescan operation is performed by the host computer at the time the logical volume is created. The initial path state of the logical volume is also provided to the host computer at the time the logical volume is created, e.g. in response to one or more inquiries received from the host computer creation of the logical volume. The initial path state for the logical volume provided to the host computer causes the host computer to send I/O operations that are directed to the logical volume only over the data path between the host computer and the first storage appliance.

FIG. 1is a block diagram showing an example of components in an operational environment including an embodiment of the disclosed technology. As shown inFIG. 1, a Storage Appliance Cluster105includes multiple storage appliances, shown for purposes of illustration by Storage Appliance1100, Storage Appliance2102, and Storage Appliance3104. While in the example ofFIG. 1Storage Appliance Cluster105includes three storage appliances, the technology disclosed herein is not limited to storage appliance clusters made up of that specific number of storage appliances, and may be embodied in storage appliance clusters that include other specific numbers of storage appliances.

Each one of the storage appliances in the Storage Appliance Cluster105contains and/or is communicably coupled to one or more non-volatile data storage devices, such as one or more magnetic disk drives, one or more electronic flash drives, and/or one or more optical drives. In the example ofFIG. 1, Storage Appliance1100is shown including Storage Devices112, Storage Appliance2102is shown including Storage Devices142, and Storage Appliance3104is shown including Storage Devices172.

Each one of the storage appliances in the Storage Appliance Cluster105also includes communication circuitry operable to connect to, and to transmit and receive data signals over, a communication network connecting the storage appliances to Host Computer106. In the example ofFIG. 1, Storage Appliance1100is shown including Communication Interfaces110, Storage Appliance2102is shown including Communication Interfaces140, and Storage Appliance3104is shown including Communication Interfaces170. Communication Interfaces110, Communication Interfaces140, and Communication Interfaces170may each include or consist of SCSI target adapters and/or network interface adapters or the like for converting electronic and/or optical signals received over the network connecting each one of the storage appliances to Host Computer106into electronic form for use by the respective storage appliance.

For example, communications between the storage appliances in Storage Appliance Cluster105may be performed using the Small Computer System Interface (SCSI) protocol, through paths in a communication network connecting the Host Computer106and the storage appliances. The paths may include or consist of paths between SCSI initiator ports in the Host Computer106and SCSI target ports in the communication interfaces of the storage appliances. For example, Host Computer106may include one or more SCSI host adapters, having some number of initiator ports. In the example ofFIG. 1, for purposes of illustration, Host Computer106is shown including Initiator Port184, Initiator Port186, and Initiator Port188. The communication interfaces of each storage appliance may include SCSI target adapters having some number of target ports. In the example ofFIG. 1, for purposes of illustration, Storage Appliance1100is shown including Target Port122, Storage Appliance2is shown including Target Port152, and Storage Appliance3104is shown including Target Port182.

Each one of the storage appliances in the Storage Appliance Cluster105includes processing circuitry for executing program code. In the example ofFIG. 1, Storage Appliance1100is shown including Processing Circuitry108, Storage Appliance2102is shown including Processing Circuitry138, and Storage Appliance3104is shown including Processing Circuitry168. Processing Circuitry108, Processing Circuitry138, and Processing Circuitry168may each include or consist of one or more central processing units (CPUs) and associated electronic circuitry.

Each one of the storage appliances in the Storage Appliance Cluster105also includes a memory operable to store program code and/or associated data structures operable when executed by the processing circuitry to cause the processing circuitry to perform various functions and provide various features of the disclosed technology. In the example ofFIG. 1, Storage Appliance1100is shown including Memory114, Storage Appliance2102is shown including Memory144, and Storage Appliance3104is shown including Memory174. Memory114, Memory144, and Memory174may each include or consist of volatile memory (e.g., RAM), and/or non-volatile memory, such as one or more ROMs, disk drives, solid state drives, and the like.

The memory in each storage appliance stores various specific program code that is executable by the processing circuitry of the storage appliance, and associated data structures used during the execution of the program code. For purposes of illustration, program code that is executable on storage appliance processing circuitry to cause the processing circuitry to perform the operations and functions of the technology described herein with regard to each storage appliance is shown by logical volume management program logic stored in the memory of each one of the storage appliances. For example, LV Management Logic123in Storage Appliance1100is operable when executed to cause Processing Circuitry108to perform the operations and functions of the disclosed technology in Storage Appliance1100, LV Management Logic153in Storage Appliance2102is operable when executed to cause Processing Circuitry138to perform the operations and functions of the disclosed technology in Storage Appliance2102, and LV Management Logic183is operable when executed to cause Processing Circuitry168to perform the operations and functions of the disclosed technology in Storage Appliance3104. Although certain program code and data structures are specifically shown in Memory114, Memory144, and Memory174, each memory may additionally include various other program code and/or other software constructs that are not shown but are additional to those shown, and that are operable in whole or in part to perform specific functions and/or operations described herein. Such additional program logic and other software constructs may include without limitation an operating system, various applications, and/or other processes and/or data structures.

Each storage appliance in Storage Appliance Cluster105also includes storage mappings and allocations that store indications of units of non-volatile data storage space that are allocated from the storage devices in that storage appliance to various logical volumes and/or other data storage objects that are provided by that storage appliance. The units of storage space allocated to a given logical volume may be mapped to respective portions of that logical volume, and may be used to store host data directed to the logical volume in I/O operations (e.g. write I/O operations) that are received from Host Computer106. A “slice” is one example of the units of storage space (e.g. 256 megabytes or 1 gigabytes in size) that may be allocated from a storage device to a storage object such as a logical volume. In the example ofFIG. 1, Storage Mappings/Allocations116in Storage Appliance1100store indications of units of storage space that are allocated from Storage Device(s)112to individual logical volumes provided by Storage Appliance1100to Host Computer106, and/or the mappings of such units of storage space to respective portions the logical volumes to which they are allocated. Storage Mappings/Allocations146in Storage Appliance2102store indications of units of storage space that are allocated from Storage Device(s)142to individual logical volumes provided by Storage Appliance2102to Host Computer106, and/or the mappings of such units of storage space to respective portions the logical volumes to which they are allocated. Storage Mappings/Allocations176in Storage Appliance3104store indications of units of storage space that are allocated from Storage Device(s)172to individual logical volumes provided by Storage Appliance3104to Host Computer106, and/or the mappings of such units of storage space to respective portions the logical volumes to which they are allocated.

The storage appliances in Storage Appliance Cluster105provide storage services that are consumed by Host Computer106(e.g. by one or more applications executing on Host Computer106). Each one of the storage appliances in Storage Appliance Cluster105exposes a set of logical volumes (also sometimes referred to as logical units or “LUNS”) to the Host Computer106. In some embodiments, the storage services provided by the storage appliances in Storage Appliance Cluster100include one or more block-based storage services that provide Host Computer106with blocks of non-volatile storage from the logical volumes. Such block-based storage services may, for example, employ the Small Computer System Interface (SCSI) protocol, the Internet Small Computer System Interface (iSCSI) protocol, and/or Fibre Channel (FC) network technology to communicate between the Host Computer106and the storage appliances in the Storage Appliance Cluster105.

While in the example ofFIG. 1, Host Computer106is shown external to the Storage Appliance Cluster105, the techniques described herein are not limited to such embodiments. Alternatively, Host Computer106may be located in whole or in part together with the storage appliances in Storage Appliance Cluster105, as in the case of a hyper-converged storage array that hosts both storage and compute resources.

During operation of the components shown inFIG. 1, e.g. at least in part through execution of the logical volume management logic LV Management Logic123, LV Management Logic153, and/or LV Management Logic183, a logical volume LV-1is created. The logical volume LV-1is created by i) creating a primary copy of LV-1in one of the storage appliances, e.g. Storage Appliance1100, and ii) creating a shadow copy of LV-1in each of one or more other storage appliances in the Storage Appliance Cluster105, e.g. in Storage Appliance2102and Storage Appliance3104. In the example ofFIG. 1, LV-1is created by creating LV-1Primary Copy120in Storage Appliance1100, LV-1Shadow Copy150in Storage Appliance2102, and LV-1Shadow Copy180in Storage Appliance3104. While in the example ofFIG. 1Storage Appliance Cluster105includes three storage appliances, and accordingly shadow copies of LV-1are created on two storage appliances, in examples in which greater numbers of storage appliances are contained in the Storage Appliance Cluster105, those storage appliances not providing the primary copy of LV-1would all have a shadow copy of LV-1created thereon. Accordingly, in the case where Storage Appliance Cluster105includes four storage appliances, shadow copies of LV-1would be created on three of the storage appliances, in the case where Storage Appliance Cluster105includes five storage appliances, shadow copies of LV-1would be created on four of the storage appliances, and so on.

Creating the primary copy of the logical volume LV-1in Storage Appliance1100includes allocating units of non-volatile data storage from Storage Device(s)112to the primary copy of the logical volume to store host data written to the logical volume by I/O operations received from Host Computer106. In the example ofFIG. 1, creating LV-1Primary Copy120includes allocating some number of units of non-volatile data storage from Storage Device(s)112to LV-1Primary Copy120, in order to store host data received in write I/O operations received by Storage Appliance1100from Host Computer106over Path190that are directed to the logical volume LV-1. Indications of the specific units of non-volatile storage allocated to LV-1Primary Copy120, and of the mappings between those units of non-volatile storage and corresponding locations within LV-1Primary Copy120, are stored in Storage Mappings/Allocations116, and shown for purposes of illustration inFIG. 1by Mappings/Allocations128.

In contrast, creating each one of the shadow copies of LV-1on Storage Appliance2102and Storage Appliance3104includes allocating no units of non-volatile data storage from either Storage Device(s)142or Storage Device(s)172to the shadow copies. No units of non-volatile data storage in Storage Device(s)142or Storage Device(s)172are allocated to LV-1Shadow Copy150or LV-1Shadow Copy180to store host data received in write I/O operations directed to LV-1. Accordingly, there are no indications of units of non-volatile storage allocated from Storage Device(s)142to LV-1Shadow Copy150, or any indications of mappings between units of non-volatile storage in Storage Device(s)142to LV-1Shadow Copy150, that are added to Storage Mappings/Allocations146when LV-1Shadow Copy150is created. Similarly, there are no indications of units of non-volatile storage allocated from Storage Device(s)172to LV-1Shadow Copy180, or any indications of mappings between units of non-volatile storage in Storage Device(s)172to LV-1Shadow Copy180, that are added to Storage Mappings/Allocations176when LV-1Shadow Copy180is created.

Also at the time logical volume LV-1is created, in response to creation of the logical volume LV-1, an initial path state for the logical volume is also created, shown for purposes of illustration inFIG. 1by LV-1Initial Path State124. Initial Path State124is created by i) initially setting a path state of the data path between the Host Computer106and the storage appliance that contains the primary copy of LV-1to an active state (e.g. Active-Optimized or Active-Non-Optimized), and ii) initially setting a path state of a data path between the host computer and the second storage appliance to unavailable. In the example ofFIG. 1, setting the path state of the data path between Host Computer106and the storage appliance that contains the primary copy of LV-1, e.g. Storage Appliance1100, to an active state, includes setting the state of the Path190between Initiator Port184and Target Port122to an Active-Optimized or Active-Non-Optimized state in LV-1Initial Path State124. As also shown in the example ofFIG. 1, setting the path state of the data path between Host Computer106and each storage appliance that contains a shadow copy of LV-1, e.g. Storage Appliance2102and Storage Appliance3104, to unavailable, includes setting the state of the Path192between Initiator Port186and Target Port152, and the state of the Path194between Initiator Port188and Target Port182to unavailable in LV-1Initial Path State124.

The active state of Path190indicates to Host Computer106(e.g. to a multi-path driver associated with LV-1and executing in Host Computer106) that Host Computer106is permitted to send I/O operations (e.g. I/O read and/or I/O write operations) that are directed to LV-1over Path190. The unavailable state of Path192indicates to Host Computer106(e.g. to the multi-path driver associated with LV-1and executing in Host Computer106) that Host Computer106is not permitted to send I/O operations directed to LV-1over Path192. Similarly, the unavailable state of Path194indicates to Host Computer106(e.g. to the multi-path driver associated with LV-1and executing in Host Computer106) that Host Computer106is not permitted to send I/O operations directed to LV-1over Path104.

LV-1Initial Path State124may be a portion (e.g. a subset) of the Asymmetric Logical Unit Access (ALUA) state that is relevant to LV-1, and that is present in each one of the storage appliances that contains either the primary copy or a shadow copy of LV-1, as shown in the example ofFIG. 1by LV-1ALUA State118. ALUA is an industry standard protocol described in the T10 SCSI-3 specification SPC-3. LV-1ALUA State118may be part of the ALUA state that is associated with and that may be obtained by Host Computer106with regard to each target port group that contains a target port through which either the primary or a shadow copy of LV-1can be accessed. Accordingly, LV-1ALUA State118may be the same in each of Storage Appliance1100, Storage Appliance2102, and Storage Appliance3104, and may be obtained by Host Computer106by issuing a SCSI Report Target Port Group (RTPG) command to a target port group that contains Target Port122, by issuing an RTPG command to a target port group that contains Target Port152, and/or by issuing an RTPG command to a target port group that contains Target Port182.

Also at the time of and/or in response to the creation of LV-1, LV-1Primary Copy120, LV-1Shadow Copy150, and LV-1Shadow Copy180are all made visible in Storage Appliance Cluster105to Host Computer106as LV-1. For example, LV-1Primary Copy120, LV-1Shadow Copy150, and LV-1Shadow Copy180may be made visible by the storage appliances in Storage Appliance Cluster105to Host Computer106as LV-1as least in part when Host Computer106performs a SCSI rescan operation across all the storage appliances in Storage Appliance Cluster105at the time LV-1is created.

Also at the time of and/or in response to the creation of LV-1, LV-1Initial Path State124may be provided to Host Computer106. For example, Initial Path State124may be provided by one or more of the storage appliances in Storage Appliance Cluster105to Host Computer106at the time LV-1is created in response to one or more host inquiries performed by Host Computer106at the time LV-1is created. Because LV-1Initial Path State124indicates an active state for Path190, but indicates the unavailable state for Path192and Path194, Initial Path State124causes Host Computer106to send host I/O operations (e.g. read I/O operations and write I/O operations) that are directed to LV-1only over Path190.

In some embodiments, LV-1has a unique name, e.g. that is unique within the Storage Appliance Cluster105, and making LV-1Primary Copy120, LV-1Shadow Copy150, and LV-1Shadow Copy180all visible to Host Computer106as the logical volume also includes providing the unique name of LV-1to Host Computer106as the name of each one of LV-1Primary Copy120, LV-1Shadow Copy150, and LV-1Shadow Copy180, e.g. in response to one or more status inquiries or the like received by the storage appliances in Storage Appliance Cluster105from Host Computer106.

The unique name of LV-1may, for example, be a SCSI World Wide Name (WWN) of LV-1. Providing the WWN of LV-1to Host Computer106as the name of LV-1Primary Copy120may, for example, be performed by sending the WWN of LV-1to Host Computer106as the name of LV-1Primary Copy120as part of a Vital Product Data (VPD) page sent to the host computer in response to a SCSI inquiry VPD page command sent from Host Computer106to Storage Appliance1100, and directed to LV-1Primary Copy120. For example, LV-1WWN126may be returned in a VPD page that is returned from Storage Appliance1100.

Providing the WWN of LV-1to Host Computer106as the name of LV-1Shadow Copy150may, for example, be performed by sending the WWN of LV-1to Host Computer106as the name of LV-1Shadow Copy150as part of a VPD page sent to the host computer in response to a SCSI inquiry VPD page command sent from Host Computer106to Storage Appliance2102, and directed to LV-1Shadow Copy150. For example, LV-1WWN126may be returned in a VPD page that is returned from Storage Appliance2102.

Providing the WWN of LV-1to Host Computer106as the name of LV-1Shadow Copy180may, for example, be performed by sending the WWN of LV-1to Host Computer106as the name of LV-1Shadow Copy180as part of a VPD page sent to the host computer in response to a SCSI inquiry VPD page command sent from Host Computer106to Storage Appliance3104, and directed to LV-1Shadow Copy180. For example, LV-1WWN126may be returned in a VPD page returned from Storage Appliance3104.

While the unavailable state for Path192in LV-1Initial Path State118prevents Host Computer106from sending I/O read and I/O write operations to LV-1Shadow Copy150on Storage Appliance2102over Path192, it does not prevent Host Computer106from sending an RTPG command to Storage Appliance2102over Path192, and that is directed to LV-1Shadow Copy150(e.g. that is directed to the target port group containing Target Port152). Similarly, while the unavailable state for Path194in LV-1Initial Path State118prevents Host Computer106from sending I/O read and I/O write operations to LV-1Shadow Copy180on Storage Appliance3104over Path194, it does not prevent Host Computer106from sending an RTPG command to Storage Appliance3104over Path194, and that is directed to LV-1Shadow Copy180(e.g. that is directed to the target port group containing Target Port182).

In some embodiments, providing the LV-1Initial Path State124to Host Computer106may include providing LV-1Initial Path State124to Host Computer106in response to both i) a status inquiry sent from Host Computer106directed to LV-1Primary Copy120on Storage Appliance1100, and/or ii) a status inquiry sent from Host Computer106directed to LV-1Shadow Copy150on Storage Appliance2102and/or LV-1Shadow Copy180on Storage Appliance3104.

In some embodiments, the status inquiry sent from Host Computer106and directed to LV-1Primary Copy120may be an RTPG command sent from Host Computer106over Path190and directed to LV-1Primary Copy120(e.g. directed to the target port group containing Target Port122).

In some embodiments, the status inquiry sent from Host Computer106and directed to LV-1Shadow Copy150may be an RTPG command sent from Host Computer106over Path192and directed to LV-1Shadow Copy150(e.g. directed to the target port group containing Target Port152).

In some embodiments, the status inquiry sent from Host Computer106and directed to LV-1Shadow Copy180may be an RTPG command sent from Host Computer106over Path194and directed to LV-1Shadow Copy180(e.g. directed to the target port group containing Target Port182).

Providing LV-1Initial Path State124to Host Computer106may be accomplished by sending LV-1Initial Path State124to Host Computer106i) as part of LV-1ALUA State118sent to Host Computer106in response to an RTPG command sent from Host Computer106over Path190to Storage Appliance1100and directed to LV-1Primary Copy120(e.g. directed to the target port group containing Target Port122), and ii) as part of LV-1ALUA State118sent to Host Computer106in response to an RTPG command sent from Host Computer106over Path192to Storage Appliance2102and directed to LV-1Shadow Copy150(e.g. directed to the target port group containing Target Port152), and iii) as part of LV-1ALUA State118sent to Host Computer106in response to an RTPG command sent from Host Computer106over Path194to Storage Appliance3104and directed to LV-1Shadow Copy180(e.g. to the target port group containing Target Port182).

FIG. 2is a block diagram showing an example of an operational environment including an embodiment of the disclosed technology, at a time subsequent to creation of the logical volume, when a logical volume relocation command is generated and received. As shown inFIG. 2, in some embodiments, host access to LV-1may be moved from Storage Appliance1100to one of the storage appliances providing one of the shadow copies of LV-1, e.g. to Storage Appliance2102, during a relocation operation performed in response to receipt of a relocation command by one or more of the storage appliances. For example, Storage Appliance Cluster105may include a Resource Balancer202. Resource Balancer202may include or consist of program code executing on one or more of the storage appliances in Storage Appliance Cluster105, and/or program code executing on a separate computer located within or external to Storage Appliance Cluster105. Resource Balancer202may automatically detect circumstances that make it desirable to change the host access to LV-1from Storage Appliance1100to Storage Appliance2102. Examples of such circumstances include without limitation resource imbalances that may arise while providing host access to LV-1on Storage Appliance1100, such as an inadequate amount of resources (e.g. storage, processing, and/or network resources) being available to support providing host access to LV-1on Storage Appliance1100, and a sufficient or more sufficient amount of resources being available to support providing host access to LV-1on Storage Appliance2102. Upon detection of such circumstances, Resource Balancer202may automatically convey a Logical Volume Relocation Command204to each of the storage appliances in Storage Appliance Cluster105, indicating that host access to LV-1is to be automatically moved from Storage Appliance1100to Storage Appliance2102.

FIG. 3is a block diagram showing an example of an operational environment including an embodiment of the disclosed technology, at a time subsequent to the generation and receipt of the logical volume relocation command, when host access to the logical volume is automatically moved from Storage Appliance1100to Storage Appliance2102in response to the Logical Volume Relocation Command204shown inFIG. 2. In response to the Logical Volume Relocation Command204, a relocation operation may automatically be performed at least in part by execution of LV Management Logic123, LV Management Logic153, and LV Management Logic183. The relocation operation includes creating an updated path state for LV-1, as shown by LV-1Updated Path State304. Creation of LV-1Updated Path State304for LV-1during the relocation operation may include i) updating the path state of the data path between Host Computer106and the storage appliance providing the primary copy of LV-1, e.g. Path190to Storage Appliance1100, to unavailable, and ii) updating the path state of a data path between Host Computer106and one of the storage appliances providing a shadow copy of LV-1, e.g. Path192to Storage Appliance2102, to an active state (e.g. Active-Optimized or Active-Non-Optimized). Updating the path state of Path190to unavailable indicates to Host Computer106that Host Computer106is no longer permitted to send I/O operations (e.g. read I/O and write I/O operations) that are directed to LV-1over Path190. Updating the path state of Path192to an active state indicates to Host Computer106that Host Computer106is subsequently permitted to send I/O operations (e.g. read I/O and write I/O operations) that are directed to LV-1over Path192.

In response to creation of LV-1Updated Path State304, the previous primary copy of LV-1, e.g. LV-1Primary Copy120in Storage Appliance1100(FIG. 1), may be changed to a shadow copy of LV-1, e.g. of the logical volume may be changed to a new shadow copy of the logical volume, e.g. LV-1New Shadow Copy302. Further in response to creation of LV-1Updated Path State304, the shadow copy of LV-1in Storage Appliance2102, e.g. LV-1Shadow Copy150(FIG. 1), may be changed to the new primary copy of the logical volume, e.g. LV-1New Primary Copy300.

LV-1Updated Path State304may then be provided to Host Computer106, e.g. from each one of the storage appliances in Storage Appliance Cluster105using the same techniques described herein for providing LV-1Initial Path State124to Host Computer106, albeit performed at the time host access to LV-1is changed from Storage Appliance1100to Storage Appliance2102. Providing LV-1Updated Path State304to Host Computer106causes Host Computer106(e.g. the multi-path driver associated with LV-1and executing in Host Computer106) to subsequently send I/O operations (e.g. read I/O and write I/O operations) that are directed to LV-1only to Storage Appliance2102over Path192.

It should be noted that LV-1WWN126is not changed when host access to LV-1is changed from Storage Appliance1100to Storage Appliance2102, and LV-1Updated Path Stage304is created and provided to Host Computer106, since even after host access to LV-1is changed from Storage Appliance1100to Storage Appliance2102, the primary copy of LV-1and all the shadow copies of LV-1will provide LV-1WWN126to Host Computer106as their name.

In some embodiments, moving host access to LV-1from Storage Appliance1100to Storage Appliance2102may include allocating units of non-volatile data storage from Storage Device(s)142to LV-1New Primary Copy300to store host data written by Host Computer106to LV-1, e.g. in write I/O operations directed to LV-1and passed from Host Computer106to Storage Appliance2102over Path192. In such embodiments, indications of the specific units of non-volatile storage allocated to LV-1New Primary Copy300, and indications of the mappings between those units of non-volatile storage and corresponding locations within LV-1New Primary Copy300, may be stored in Storage Mappings/Allocations146, and shown for purposes of illustration inFIG. 1by Mappings/Allocations306.

In some embodiments, moving host access to LV-1from Storage Appliance1100to Storage Appliance2102may further include moving host data stored in the units of non-volatile data storage allocated from Storage Device(s)112to LV-1Primary Copy120to the units of non-volatile data storage allocated from Storage Device(s)142to LV-1New Primary Copy300. Movement of such host data may be performed in whole or in part by pushing the host data from Storage Appliance1100to Storage Appliance2102in whole or in part prior to changing the host access to LV-1from Storage Appliance1100to Storage Appliance2102, and/or by pulling the host data to Storage Appliance2102from Storage Appliance1100in whole or in part after changing the host access to LV-1from Storage Appliance1100to Storage Appliance2102. In either case, moving the host data may include copying one or more snapshots (point in time copies) of LV-1Primary Copy120to Storage Appliance2102for storage by Storage Appliance2102into units of Storage Device(s)142allocated to LV-1New Primary Copy300, and/or synchronously mirroring I/O operations that are directed to LV-1to both Storage Appliance1100and Storage Appliance2102for some period of time, such that the I/O operations directed to LV-1and received over one of Path190or Path192are synchronously performed on both Storage Appliance1100and Storage Appliance2102, until all host data stored in the units of non-volatile data storage allocated from Storage Device(s)112to LV-1Primary Copy120has been copied to the units of non-volatile data storage allocated from Storage Device(s)142to LV-1New Primary Copy300, and the contents of LV-1New Primary Copy300is the same as the contents of LV-1Primary120. Such host data movement between Storage Appliance1100and Storage Appliance2102may be performed automatically in the background, e.g. through one or more communication paths external to Path190, Path192, and/or Path194, so that the data movement is performed transparently with regard to Host Computer106, and such that there is no interference with or interruption to the data storage services provided from the storage appliances in Storage Appliance Cluster105to Host Computer106.

In some embodiments, host access to LV-1may be moved to Storage Appliance2102without allocating any units of non-volatile storage from Storage Device(s)142to store host data written to LV-1in write I/O operations received by Storage Appliance2102from Host Computer106over Path192. In such embodiments, I/O operations directed to LV-1and received by Storage Appliance2102over Path192may, for example, be processed using units of non-volatile storage allocated to LV-1New Primary Copy300from Storage Device(s)112in Storage Device1100, or alternatively using units of non-volatile storage allocated to LV-1Primary Copy120.

FIG. 4is a block diagram showing an example of an operational environment and illustrating movement of logical volume data from LV-1Primary Copy120in Storage Appliance1100to Storage Appliance2102in some embodiments. InFIG. 4, LV-1Data450represents data that may be moved between LV-1Primary Copy120in Storage Appliance1100and Storage Appliance2102when host access to LV-1is changed from Storage Appliance1100to Storage Appliance2102. As shown inFIG. 4, LV-1Data450may be passed from a Transit Logic402in Storage Appliance1100to a Replica Front End404in Storage Appliance2102, and then stored by Replica Front End404into a Replica Logical Volume406in Storage Appliance2102. Transit Logic402and Replica Front End404are operable to pass data between in Storage Appliance1100and Storage Appliance2102, and are logically and/or physically separate from Target Port122and Target Port152and/or any other target ports in Storage Appliance1100and/or Storage Appliance2102that provide communications with Host Computer106. In this way, Transit Logic402and Replica Front End404enable Storage Appliance1100to communicate with Storage Appliance2102independently with regard to any data paths between Host Computer106and Storage Appliance1100and/or Storage Appliance2102.

In some embodiments, LV-1Data450may include one or more snapshots of LV-1Primary Copy120that are point in time copies of the contents of LV-1Primary Copy120.

In some embodiments, LV-1Data450may further include a “delta” of host data written to LV-1Primary Copy120and collected in Storage Appliance1100after a point in time at which the last snapshot of LV-1Primary Copy120that was sent to Storage Appliance2102was captured.

In some embodiments, after the “delta” of host data captured in Storage Appliance1100is collected and/or sent to Storage Appliance2102as part of LV-1Data450, write I/O operations directed to LV-1that are received by Storage Appliance1100from Host Computer106over Path190may be synchronously mirrored to the Replica Logical Volume406. For example, Mirroring Logic400in Storage Appliance1100may synchronously mirror write I/O operations directed LV-1that are received by Storage Appliance1100over Path190, such that the write I/O operations directed to LV-1are both i) applied to LV-1Primary Copy120, and also ii) passed to Transit Logic402. Transit Logic402then conveys the mirrored write I/O operations to Replica Front End404in Storage Appliance2102and applied by Replica Front End404to Replica Logical Volume406. The synchronously mirrored write I/O operations are only acknowledged as completed to Host Computer106after they have been successfully applied to both LV-1Primary Copy120and Replica Logical Volume406.

Replica Front End404writes LV-1Data450until a point in time at which LV-1Primary Copy120and Replica Logical Volume406are completely synchronized, such that all data stored in LV-Primary Copy120is also stored in Replica Logical Volume406, and write I/O operations being received over Path190are being synchronously mirrored to both LV-Primary Copy120and Replica Logical Volume406. At that time, host access to LV-1may be disabled, e.g. access to LV-1may be disabled with regard to all paths to storage appliances in Storage Appliance Cluster105. All pending I/O operations directed to LV-1that are contained in Storage Appliance1100and/or Storage Appliance2102at the time host access to LV-1is disabled are then completed. While host access to LV-1is disabled, LV-1Updated Path State304may be created and stored into LV-1ALUA State118in each one of the storage appliances in Storage Appliance Cluster105. Also while host access to LV-1is disabled, Replica Logical Volume406may be changed to LV-1New Primary Copy300. Since the unique name of LV-1has not changed, there is no need to modify LV-1WWN126in any of the storage volumes.

Subsequently, when host access to LV-1is re-enabled, Host Computer106is provided with LV-1Updated Path State304in the LV-1ALUA State118that is provided from any one of the storage appliances in the Storage Appliance Cluster105as described above. LV-1Updated Path State304indicates that LV-1is to be accessed by Host Computer106only on Storage Appliance1102over Path192, based on the active state of Path192in LV-1Updated Path State304, and based also on the unavailable state of Path190and Path194in LV-1Updated Path State304.

FIG. 5is a flow chart showing an example of steps performed in some embodiments. At step500, a logical volume is initially created by i) creating a primary copy of the logical volume in a first storage appliance in a cluster of storage appliances, and ii) creating a shadow copy of the logical volume in at least a second storage appliance in the cluster of storage appliances. Creating the primary copy of the logical volume in the first storage appliance includes allocating units of non-volatile data storage from data storage devices of the first storage appliance to the primary copy of the logical volume to store host data written to the logical volume. Creating the shadow copy of the logical volume in the second storage appliance and/or any other storage appliance in the cluster includes allocating no units of non-volatile data storage from data storage devices in those storage devices to any shadow copy of the logical volume to store host data.

In step502, in response to creation of the logical volume, and at the time the logical volume is created, steps504,506, and508are performed.

At step504, an initial path state of the logical volume is created. Creating the initial path state of the logical volume includes initially setting a path state of a data path between the host computer and the first storage appliance to active. The active state of the data path between the host computer and the first storage appliance indicates to the host computer that the host computer is permitted to send I/O operations (e.g. read I/O and/or write I/O operations) that are directed to the logical volume over the data path between the host computer and the first storage appliance. Creating the initial path state of the logical volume also includes initially setting a path state of a data path between the host computer and the second storage appliance to unavailable. The unavailable state of the data path between the host computer and the second storage appliance indicates to the host computer that the host computer is not permitted to send I/O operations (e.g. read I/O and/or write I/O operations) that are directed to the logical volume over the data path between the host computer and the second storage appliance.

At step506, both the primary copy of the logical volume and the shadow copy of the logical volume are made visible to the host computer as the logical volume.

At step508, the initial path state of the logical volume is provided to the host computer. The initial path state of the logical volume causes the host computer to send I/O operations directed to the logical volume only over the data path between the host computer and the first storage appliance.

While the above description provides examples of embodiments using various specific terms to indicate specific systems, devices, and/or components, such terms are illustrative only, and are used only for purposes of convenience and concise explanation. The disclosed system is not limited to embodiments including or involving systems, devices and/or components identified by the terms used above.

As will be appreciated by one skilled in the art, aspects of the technologies disclosed herein may be embodied as a system, method or computer program product. Accordingly, each specific aspect of the present disclosure may be embodied using hardware, software (including firmware, resident software, micro-code, etc.) or a combination of software and hardware. Furthermore, aspects of the technologies disclosed herein may take the form of a computer program product embodied in one or more non-transitory computer readable storage medium(s) having computer readable program code stored thereon for causing a processor and/or computer system to carry out those aspects of the present disclosure.

Any combination of one or more computer readable storage medium(s) may be utilized. The computer readable storage medium may be, for example, but not limited to, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any non-transitory tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The figures include block diagram and flowchart illustrations of methods, apparatus(s) and computer program products according to one or more embodiments of the invention. It will be understood that each block in such figures, and combinations of these blocks, can be implemented by computer program instructions. These computer program instructions may be executed on processing circuitry to form specialized hardware. These computer program instructions may further be loaded onto a computer or other programmable data processing apparatus to produce a machine, such that the instructions which execute on the computer or other programmable data processing apparatus create means for implementing the functions specified in the block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the block or blocks. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the block or blocks.

Those skilled in the art should also readily appreciate that programs defining the functions of the present invention can be delivered to a computer in many forms; including, but not limited to: (a) information permanently stored on non-writable storage media (e.g. read only memory devices within a computer such as ROM or CD-ROM disks readable by a computer I/O attachment); or (b) information alterably stored on writable storage media (e.g. floppy disks and hard drives).

While the invention is described through the above exemplary embodiments, it will be understood by those of ordinary skill in the art that modification to and variation of the illustrated embodiments may be made without departing from the inventive concepts herein disclosed.