Abstract:
An apparatus, system, and method for replicating a snapshot volume in a first storage system to a second storage system includes mapping information corresponding to data in the first storage system that is transferred from the first storage system to the second storage system so that a file system in the second storage system can mount the data after replication. Replication of the snapshot volume can be accomplished using a remote copy mechanism. The snapshot volume can be obtained from a primary source volume P-VOL and a differential source volume D-VOL. If the corresponding destination volumes are not known, a search is conducted to locate appropriate volumes in the second storage system. Mapping information regarding these destination volumes is utilized to enable the file system in the second storage system to mount the replicated snapshot volume.

Description:
[0001]    This is a continuation application of U.S. Ser. No. 11/107,904, filed Apr. 18, 2005. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    1. Field of the Invention 
         [0003]    The invention relates generally to storage systems. More specifically, the invention relates to making a replica of a snapshot image in a remote network attached storage (NAS) system, and to enabling a remote NAS system to mount the snapshot image after making the replica of the snapshot image. 
         [0004]    2. Description of the Related Art 
         [0005]    Generally speaking, there are two major storage system types: SAN (Storage Area Network) and NAS (Network Attached Storage). SAN employs block level access to data in its storage system, while file level access is used in NAS. 
         [0006]    “Snapshot” is a mechanism for taking a point in time (PIT) copy of a file system in the NAS system. There are various ways of implementing the snapshot mechanism. Copy on write (COW) is one of the most popular implementations in the NAS system. 
         [0007]    “Remote Copy” is a mechanism for creating a volume pair between two separate storage systems. One main purpose of remote copy is to prepare for catastrophic events or disasters, and to enable disaster recovery (DR). Although the NAS system is a file level storage system, it can collaborate with the remote copy mechanism in a disk array storage system residing in the backend of the NAS system. 
         [0008]    It is possible to use both the snapshot mechanism and the remote copy mechanism at the same time. More specifically, the snapshot image can be copied by using remote copy operations. However, in the prior art, it is not possible for a local file system of a remote NAS system to mount the copied snapshot image because the snapshot module in the remote NAS system does not have sufficient information regarding logical units corresponding the snapshot image. 
         [0009]    Examples of prior art patents include: “Method and System for Providing a Static Snapshot of Data Stored on a Mass Storage System”, Richard S. Ohrann et al, U.S. Pat. No. 5,649,152; “Storage System Assuring Data Integrity and A Synchronous Remote Data Duplexing”, Akira Yamamoto, U.S. Pat. No. 6,408,370; and “Heterogeneous Computer System, Heterogeneous Input Output System and Data Backup Method for the Systems”, Yasuko Fukuzawa et al, U.S. Pat. No. 6,529,976. The entire disclosures of all of these patents are hereby incorporated by reference. 
       BRIEF SUMMARY OF THE INVENTION 
       [0010]    An object of the invention is to create a replica of a snapshot image in a remote NAS system, and also to enable the remote NAS system to mount the snapshot image. To realize this object of the present invention, according to one embodiment, snapshot replication commands are prepared. Furthermore, the logical unit information for the snapshot image can be provided to the remote NAS system. 
         [0011]    The method of replicating a snapshot volume in a first storage system to a second storage system, includes the steps of: providing a primary volume (P-VOL), which represents data existing in a first storage system; providing a differential volume (D-VOL) containing information regarding a change in data of the primary volume, the differential volume being used for recreating the data of the primary volume at random or predetermined points in time; storing mapping information of P-VOL and D-VOL in a first snapshot volume management table (SVMT); designating a target volume in the second storage system; storing transfer information in a table in the first storage system, the transfer information including the mapping information of P-VOL and D-VOL and information of a target volume in the second storage system in a table; commencing replication of the P-VOL and D-VOL from the first storage system to the second storage system; and sending the transfer information to the second storage system. 
         [0012]    The method further includes the steps of: creating a snapshot image logical volume (V-VOL) from P-VOL and D-VOL; and after replicating P-VOL and D-VOL to the second storage system, creating a replicated snapshot image logical volume (V′-VOL), wherein V-VOL is accessible by a first host and V′-VOL is accessible by a second host. 
         [0013]    These and other features and advantages of the present invention will become apparent to those of ordinary skill in the art in view of the following detailed description of the preferred embodiments. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0014]    The accompanying drawings, in conjunction with the general description given above, and the detailed description of the preferred embodiments given below, serve to illustrate and explain the principles of the preferred embodiments of the best mode of the invention presently contemplated. 
           [0015]      FIG. 1  illustrates an example of hardware configuration of the invention according to one embodiment. 
           [0016]      FIG. 2  illustrates an example of a software configuration in which the method and apparatus of this invention are applied. 
           [0017]      FIG. 3  illustrates an example of snapshot operations used according to the present invention. 
           [0018]      FIG. 4  illustrates an example of remote copy operations used according to the present invention. 
           [0019]      FIG. 5  illustrates a conceptual diagram of a current snapshot remote replication system. 
           [0020]      FIG. 6  illustrates a conceptual diagram of a snapshot remote replication system according to the present invention. 
           [0021]      FIG. 7  illustrates processing details of the snapshot remote replication system according to the present invention. 
           [0022]      FIG. 8  illustrates a flow of snapshot remote replication of a snapshot module according to an embodiment of the present invention. 
           [0023]      FIG. 9  illustrates a conceptual diagram of a mount operation of the replicated snapshot image. 
           [0024]      FIG. 10  illustrates detailed processes of the inventive snapshot remote replication system. 
           [0025]      FIG. 11  illustrates a flow of the inventive snapshot remote replication of the snapshot module. 
           [0026]      FIG. 12  illustrates a flowchart for finding unused logical units in a remote snapshot module. 
           [0027]      FIG. 13  illustrates a typical construction of a logical unit management table (LUMT). 
           [0028]      FIG. 14  illustrates detailed processes of the inventive snapshot remote replication system. 
           [0029]      FIG. 15  illustrates a flow of the inventive snapshot remote replication. 
           [0030]      FIG. 16  illustrates detailed processes of the inventive snapshot remote replication system. 
           [0031]      FIG. 17  illustrates a flow of the inventive snapshot remote replication. 
           [0032]      FIG. 18  illustrates a flow of the inventive snapshot remote replication, continued from  FIG. 17 . 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0033]    In the following detailed description of the invention, reference is made to the accompanying drawings which form a part of the disclosure, and, in which are shown by way of illustration, and not of limitation, specific embodiments by which the invention may be practiced. In the drawings, like numerals describe substantially similar components throughout the several views. 
       1st Embodiment 
       [0034]      FIG. 1  shows an example of a hardware configuration in which the method and apparatus of this invention are applied according to a first embodiment. The system includes an Application Host  1000 , a Storage Management Host  1100 , and Storage Systems  2000  and  3000 . 
         [0035]    Application Host  1000  includes a memory  1002  and a CPU  1011  for running an Application System Software (AP)  1011  (illustrated in  FIG. 2 ). The Application System  1011  on the Application Host  1000  issues input/output (I/O) operations to the Storage System  2000 , as will be described in more detail below. The Application Host  1000  is connected to the Storage System  2000  via a network interface connection (NIC)  1003 . Additional connections to the Storage System  2000  may exist. Moreover, the Application Host  1000  may also be connected to Storage System  3000 . 
         [0036]    Storage Management Host  1100  includes a memory  1102  and a CPU  1101  for running a Storage Management Software  1021  (illustrated in  FIG. 2 ). The Storage Management Software  1021  issues I/O operations to the Storage System  2000 , as will be described in more detail below. The Storage Management Host is connected to the Storage System  2000  via NIC  1103  and a network  2700 . Additional connections to the Storage System  2000  may exist. Moreover, Storage Management Host  1100  may also be connected to the Storage System  3000  via network  2700 . 
         [0037]    Storage System  2000  includes two primary portions: a NAS Head  2100  and a Storage Controller  2200  (hereinafter referred to as controller  2200 ). NAS Head  2100  and controller  2200  can be connected via a network interface and can exist in a single storage unit. In such a case, they are interconnected via a system bus. On the other hand, the NAS Head  2100  and controller  2200  can be physically separated, in which case, they are interconnected via network connections such as Fibre Channel, Ethernet, etc. 
         [0038]    The NAS Head  2100  includes a CPU  2001 , a memory  2002 , a cache memory  2003 , a network interface (NIC)  2004 , a disk interface (I/F)  2005 , and a management interface (NIC)  2007 . NAS Head  2100  processes I/O requests from the Application Host  1000  and Storage Management Host  1100 . A program for processing I/O requests or other operations is stored in memory  2002 , and CPU  2001  executes this program. Cache memory  2003  stores the write data from host computer  1000  temporarily before the data is forwarded into the controller  2200 . Cache memory  2003  may also store the read data that are requested by the host computer  1000 . Cache memory  2003  may also be realized by a non-volatile memory backed-up by a battery. In another implementation, memory  2002  and cache memory  2003  are commonly combined as a single memory. 
         [0039]    A host interface (NIC)  2004  is used to connect between host  1000  and NAS Head  2100 . Ethernet is a typical example of the connection, but other suitable connection types may also be used. Storage connection interface  2005  is used to connect between NAS head  2100  and storage controller  2200 . Fibre Channel (FC) and Ethernet are typical examples of the connection, but other connections types may be used. In the case of an internal connection between NAS head  2100  and controller  2200 , a system bus is a typical example of the connection. Management interface (NIC)  2007  is used to connect between a storage management computer service processor (SVP)  2006  and NAS Head  2100 . Service Processor (SVP)  2006  is connected with the storage system  2000 , and is used to set/change the configuration of the storage system  2000 . Ethernet is again a typical example of such connection. 
         [0040]    The controller  2200  includes a CPU  2011 , a memory  2012 , a cache memory  2013 , a host interface (HBA)  2014 , a storage connection interface (HBA)  2015 , a disk interface (DKA)  2016 , and a management interface (NIC)  2017 . Controller  2200  processes I/O requests from the NAS Head  2100 . A program to process I/O requests or other operations is stored in the memory  2012 , and CPU  2011  executes the program. Cache memory  2013  stores the write data from the NAS Head  2100  temporally before the data is stored into storage devices  2300 . Cache memory  2013  may also store the read data that are requested by the NAS Head  2100 . Cache  2013  may be a battery backed-up non-volatile memory. In another implementation, memory  2012  and cache memory  2013  are commonly combined as a single memory. 
         [0041]    Host interface (HBA)  2014  is provided between NAS Head  2100  and controller  2200 . Storage connection interface (HBA)  2015  is provided between storage controllers  2200  and  3200 . Controller  3200  includes a connection interface (HBA)  3015  for connecting controller  3200  to HBA  2015  at controller  2200 . Fibre Channel (FC) and Ethernet are typical examples of the network  2800  for these connections. Disk interface (DKA)  2016  is used to connect storage devices  2300  and the disk controller  2200 . Management interface (NIC)  2017  is used to connect storage management computer SVP  2006  and storage controller  2200 . Ethernet is a typical example of such connection. Each of storage devices  2300  process I/O requests in accordance with SCSI Device protocol commands. Other appropriate hardware architecture can also be used with the present invention. 
         [0042]      FIG. 2  shows an example of application of a software configuration for the method and apparatus of the present invention. The system includes Application Host  1000 , Management Host  1100 , NAS Head  2100  and Storage Controller  2200 , as described above. 
         [0043]    Application Host  1000  includes Application Software (AP)  1011  which generates I/O operations. The I/O operations are sent to the NAS Head by a network file sharing protocol such as NFS/CFIS  1012 . Storage Management Software (Storage Manager)  1021  resides on Management Host  1100 . The storage administrator issues storage management operations such as snapshot operations and remote copy operations using the Storage Manager  1021   
         [0044]    As mentioned above, NAS Head  2100  is a part of storage system  2000 . File related operations are processed in this module. Network file sharing protocol server  2021  resides on NAS Head  2100  in order to communicate with its client host  1000 . A number of service programs run on NAS Head  2100  such as service daemons  2023  and a NAS Management server  2022 . The local file system  2024  processes file I/O operations to the storage system  2000 , and drivers  2029  of the storage system translate the file I/O operations to block level operations, and communicate with storage controller  2200  via SCSI commands. A Logical Volume Manager (LVM)  2026  can exist between storage controller  2200  and local file system  2024 . LVM  2026  virtualizes physical volumes provided by the storage controller  2200  into logical volumes. Multiple physical volumes can be combined into a logical volume. Moreover, logical volumes can be expanded dynamically. A snapshot module  2025  takes point in time (PIT) copies of files. When it is based on the LVM  2026 , the logical volumes are used for snapshot operations. A NAS OS kernel  2027  is included for controlling operations in NAS Head  2100 , and a RAID Manager  2028  is used by the storage management software  1021  to control operations served by the storage controller  2200 . 
         [0045]    Additionally, a Remote Copy Module  2221  provides remote copy operations of storage controller  2200 . An I/O operation module (not shown) and other modules can also exist on the storage controller  2200 . 
         [0046]      FIG. 3  shows an example of snapshot operations used in the following explanation. The snapshot module is based on LVM  2026 , which virtualizes physical volumes provided by the storage controller  2200  into logical volumes. The snapshot can be taken for a file system. In the example, a file system is associated with a logical volume. Then, the mapping information between a file system and one or more physical volumes (logical unit) is managed in the Snapshot Volume Management Table (SVMT)  2030  on the snapshot module  2025 . Actually, a file system can consist of more than two logical volumes, but only two are shown here for convenience and explanation. Moreover, multiple file systems can exist on a logical volume. In these cases, another mapping table  2030  of the file systems and logical volumes is needed. 
         [0047]    It is possible to take a snapshot in the Local FS layer  2024  without using LVM  2026 . In this case, the mapping information of file system and logical unit is managed by the Local FS  2024 . The following explanations use the first case, which is the case of using LVM  2026 . 
         [0048]    As illustrated in  FIG. 3 , snapshot operations are issued by storage administrators on storage management host  1100  or application software  1011  on AP Host  1000 . The first operation of snapshot is “start snapshot”. After the last point of snapshot, each file or block on a primary volume (P-VOL)  2301  to which new data is written is copied onto a differential volume (D-VOL)  2302 . When application software  1011  reads a snapshot image, a snapshot volume (V-VOL)  2303  is created from the P-VOL and D-VOL, and then the local file system Local FS  2024  mounts the V-VOL  2303  and exports it (not shown) to the application software  1011 . 
         [0049]    The above description is an example of a Copy On Write (COW) snapshot mechanism. P-VOL mirroring can also be used for the snapshot mechanism. In this case, multiple mirrors of P-VOL are prepared. Each time a snapshot command is issued, one of the mirror volumes is split from the P-VOL, and becomes a snapshot image. The mapping information of snapshot image (volume) and point of snapshots (generations) are managed by the RAID Manager  2028  (not depicted in  FIG. 3 ) on NAS Head  2100  which operates the mirror command. In any case, the snapshot related information is managed in some module on NAS head  2100 . 
         [0050]      FIG. 4  shows an example of remote copy operations. RAID Manager Module  2028  is called by storage manager  1021  on storage management host  1100  to invoke pair creation of a remote copy volume or other operations. Then, RAID Manager  2028  calls a Remote Copy module  2221  on the storage controller  2200 . The Remote Copy module  2221  stores the pair information in a Copy Pair Management Table (CPMT)  2220 . The CPMT  2220  typically consists of source LU number, destination LU number, destination node name, and pair status. In  FIG. 4 , a source volume which is LU 1  (P  2301 ) on a storage system  2000  is replicated to a destination volume which is LU 10  (P′ 3301 ) on a storage system  3000 . 
         [0051]    The operations to the RAID Manager  2028  can be through LVM  2026 . In this case, the operation parameters are designated by logical volumes, instead of logical units, and the LVM  2026  manages the mapping information of logical volumes and logical units. 
         [0052]    Other appropriate remote copy mechanisms can be applied to the present invention. For example, an external storage system with local replication can be used. In this case, an external volume is mapped onto the target volume of the local copy of P-VOL. The remote replication operations to the RAID Manager  2028  are replaced by local replication operations, and a local copy module (not depicted in  FIG. 4 ) is called instead of remote copy module  2221  on controller  2200 . 
         [0053]      FIG. 5  represents a conceptual diagram of a snapshot remote replication system according to one embodiment of the present invention. Storage system  3000  has modules in a NAS Head  3100  similar to those described above with reference to NAS Head  2100 , including a Local FS  3024 , a Snapshot module  3025 , an LVM  3026 , and a RAID manager  3028 . Also, storage controller  3200  may include a Remote Copy module  3221 . To create a snapshot image (V-VOL)  2303 , both P-VOL  2301  and D-VOL  2302  are needed. Both P-VOL  2301  and D-VOL  2303  are replicated to remote storage system  3000  using remote copy, as described above. However, it is impossible for Local FS  3024  on the remote NAS head  3100  to mount a snapshot image because snapshot module  3025  does not have any information regarding the logical units copied as P-VOL (P′-VOL  3301 ) and D-VOL (D′-VOL  3302 ). 
         [0054]      FIG. 6  shows a conceptual diagram of snapshot remote replication system according to the present invention. The logical unit mapping information is sent to the remote snapshot module  3025 . Then, the local FS  3024  can find the relation of the remote copies of P-VOL (P′-VOL)  3301  and D-VOL (D′-VOL)  3302 , and mount a remote snapshot image (V′-VOL)  3303 . 
         [0055]      FIG. 7  shows detailed processes of the snapshot remote replication system according to the present invention, and  FIG. 8  is flowchart of the steps followed by the snapshot module  2025 . It is assumed that snapshot operations have already been started and that mapping information of P-VOL and D-VOL are stored in SVMT  2030 . In SVMT  2030  in  FIG. 7 , a logical unit number of P-VOL for file system A is LU 1 . A logical unit number of D-VOL for file system A is LU 2 . 
         [0056]    At first, a storage management software  1021  on a storage management host  1100  calls a snapshot copy command such as “SnapCopy” of snapshot module  2025  with a file system name (A), destination P-VOL logical unit number (LU 10 ), destination D-VOL logical unit number (LU 11 ), and target host name (Str 2 ) designating Storage System  3000  (Step  5000 ). The command could be as follows: SnapCopy (FS name, dest P-VOL LUN, dest D-VOL LUN, target host). Next, the snapshot module  2025  checks if the designated file system name is in SVMT  2030  (Step  5001 ). If the file system does not exist, the snapshot module  2025  sends back an error (Step  5005 ). If the file system exists, the snapshot module  2025  finds its P-VOL logical unit number (LU 1 ) and D-VOL logical unit number (LU 2 ) in SVMT  2030 . Snapshot module  2025  then calls RAID Manager  2028  with a target host name (Str 2 ), the logical unit number of both source P-VOL (LU 1 ) and destination P′-VOL (LU 10 ), and also the logical unit number of both source D-VOL (LU 2 ) and destination D′-VOL (LU 11 ) (Step  5002 ). The RAID manager module  2028  calls Remote Copy module  2221  on storage controller  2200  with the same parameters as it receives from the snapshot module  2025 . The Remote Copy module  2221  stores the received information into a Copy Pair Management Table (CPMT)  2220  and the replication of P-VOL and D-VOL is started. When the snapshot module  2025  receives an indication of the successful start of replication, it generates and sends mapping information of remote P′-VOL (LU 10 ) and remote D′-VOL (LU 11 ) for the replicated file system snapshot to the destination node&#39;s snapshot module  3025  via network  2700  between NAS head  2100  and NAS head  3100  (Step  5003 ). Finally, the snapshot module  2025  sends back a result of the snapshot replication operation to the storage management software  1021  (Step  5004 ). 
         [0057]      FIG. 9  shows a conceptual diagram of the mount operation of the replicated snapshot image according to the present invention. A storage management software (not depicted in  FIG. 9 ) calls snapshot module  3025  to create a snapshot image (V′-VOL) of a file system A. A snapshot image logical volume (V′-VOL)  3303  can be generated from the data of both replicated primary volume (P′-VOL)  3301  and replicated differential volume (D′-VOL)  3302  by the snapshot module  3025 . The logical unit information for both P′-VOL  3301  and D′VOL  3302  is stored in SVMT  3030 . Thus, the snapshot module  3025  can find both P′-VOL and D′-VOL correctly. When the remote snapshot image (V′-VOL) is created, the remote replication pair can be split. A local FS  3024  on a NAS head  3100  can mount the remote snapshot volume (V′-VOL)  3303 . It can then export the virtual volume, which is a snapshot image of file system A. Then, application software  4011  on a second application host  4000  can mount the snapshot volume (V′-VOL)  3303  and accesses the file system A through a network file sharing protocol. 
       2nd Embodiment 
       [0058]      FIG. 10  shows detailed processes of a second embodiment of the snapshot remote replication system according to the present invention.  FIG. 11  is a flow chart illustrating steps performed by snapshot module  2025 . Assuming that snapshot operations have already been started, mapping information of P-VOL  2301  and D-VOL  2302  are stored in SVMT  2030 . In  FIG. 10 , a logical unit number of P-VOL  2301  for file system A is LU 1 . A logical number of D-VOL  2302  for file system A is LU 2 . First, storage manager  1021  on storage management host  1100  calls a snapshot copy command like “SnapCopy” of snapshot module  2025  with a file system name (A) and target host name (Str 2 ) designating storage system  3000  as the target host (Step  5100 ). However, in this embodiment, logical unit numbers of both destination P-VOL (P′-VOL)  3301  and destination D-VOL (D′-VOL)  3302  are not designated by the storage management software  1021  (e.g., SnapCopy (FS name, target host)). The snapshot module  2025  checks if the designated file system name (A) is in the SVMT  2030  (Step  5101 ). If the file system does not exist, the snapshot module  2025  sends back an error (Step  5106 ). If the file system exists, the snapshot module  2025  calls remote snapshot module  3025  in order to find unused logical units for target of replication of P-VOL  2301  and D-VOL  2302  using information of volume size and type for both P-VOL  2301  and D-VOL  2302  (Step  5102 ). A flow of the steps followed by remote snapshot module  3025  is illustrated in  FIG. 12 , and explained later. If there are not two or more unused appropriate logical units in the remote storage system  3000 , the snapshot module  2025  sends back an error (Step  5106 ). After receiving designations for unused logical units (LU 10  and LU 11 ) from the remote snapshot module  3025 , the snapshot module  2025  finds its P-VOL logical unit number (LU 1 ) and D-VOL logical unit number (LU 2 ) in the SVMT  2030  for the designated file system A. Then, snapshot module  2025  calls RAID Manager  2028  with a target host name (Str 2 ), the logical unit number of both source P-VOL (LU 1 )  2301  and destination P′-VOL (LU 10 )  3301 , and also the logical unit number of both source D-VOL (LU 2 )  2302  and destination D′-VOL (LU 11 )  3302  (Step  5103 ). The RAID manager module  2028  calls Remote Copy module  2221  on a storage controller  2200  with the same parameters that were received from the snapshot module  2025 . The Remote Copy module  2221  stores the received information into a Copy Pair Management Table (CPMT)  2220  and the replication of P-VOL  2301  and D-VOL  2302  is started. When the snapshot module  2025  receives an indication that successful replication has started, it generates and sends mapping information of remote P′-VOL (LU 10 )  3301  and remote D′-VOL (LU 11 )  3302  for the replicated file system snapshot to the destination node&#39;s snapshot module  3025  via network  2700  between NAS head  2100  and NAS head  3100  (Step  5104 ). Finally, the snapshot module  2025  sends back a result of the snapshot replication operation to the storage manager  1021  (Step  5105 ). 
         [0059]      FIG. 12  is a flowchart illustrating the steps of finding unused logical units in the remote snapshot module  3025 . When the remote snapshot module  3025  receives an unused logical unit search request (Step  5200 ), it searches the Logical Unit Management Table (LUMT)  3031  managed by LVM  3026  to determine whether there are unused logical units (Step  5201 ). 
         [0060]      FIG. 13  shows a typical construction of LUMT  3031 . LUMT  3031  includes information such as Logical Unit Number (LUN), unit type, size, and whether or not the unit is being used. Snapshot module  3025  checks whether there are at least two unused volumes available (Step  5202 ). If snapshot module  3025  finds fewer than two unused volumes, it sends back an error (Step  5205 ). However, if snapshot module  3025  can find two or more unused logical units, it checks whether the type of the unused logical units is same as the source logical unit, the information of which is passed from the source snapshot module  2025  (Step  5203 ). If snapshot module  3025  cannot find two or more logical units of same type, it sends back an error (Step  5205 ). If snapshot module  3025  finds two or more logical units of same type, it checks whether they are of the correct size (Step  5204 ). If snapshot module  3025  cannot find two or more logical units of the correct size, it sends back an error (Step  5205 ). If snapshot module  3025  finds two or more logical units of correct size, it picks two of the logical units (Step  5206 ). There are various ways of picking two logical units, any of which can be employed. For example, the first two appropriate logical units could be picked from their entry in the LUMT  3031 . Other replication target volume properties can be checked here in addition to type and size. Such additional specification information would also be passed from the source snapshot module  2025 . Finally, the snapshot module  3025  returns designations for two unused logical units for replicated P-VOL (P′-VOL)  3301  and D-VOL (D′-VOL)  3302 , or an error to the source snapshot module  2025  (Step  5207 ). The mount operations of the replicated snapshot are the same as described above in the previous embodiment. 
         [0061]    In the second embodiment above, the destination primary volume P′-VOL  3301  and destination differential volume D′-VOL  3302  are located and their information is returned to snapshot module  2025 . However, it is possible to transfer information regarding P-VOL  2301  and D-VOL  2302  to snapshot module  3025  and allow snapshot module  3025  to find suitable unused destination volumes and call RAID manager module  2028  which in turn calls a Remote Copy module  2221  to start replication as mentioned above. 
       3rd Embodiment 
       [0062]    The above embodiments extend snapshot operations. In other words, the snapshot module receives a request of snapshot remote replication from the storage management software. According to the third embodiment, the snapshot operation is not extended, but in instead so called “PairCreate” operations to a RAID Manager are extended. This means that the RAID Manager module  2028  receives a request of snapshot remote replication from the storage manager.  FIGS. 14 and 15  are directed to this third embodiment of the present invention. 
         [0063]    Assuming that snapshot operations have already been started, the mapping information of P-VOL  2301  and D-VOL  2302  are stored in SVMT  2030 . In  FIG. 14 , a logical unit number of P-VOL  2301  for file system A is LU 1 . A logical number of D-VOL  2302  for file system A is LU 2 . Storage Manager  1021  on storage management host  1100  calls a remote replication pair creation command in RAID Manager module  2028  with both source primary volume (P-VOL  2301 ) and destination primary volume (P′-VOL  3301 ) logical unit numbers (LU 1 , LU 10 ), both source differential volume (D-VOL  2302 ) and destination differential volume (D′-VOL  3302 ) logical unit numbers (LU 2 , LU 11 ), and target host name (Str 2 ) (Step  5300 ). An example of the command is as follows: SnapPairCreate (src P-VOL LUN, dest P-VOL LUN, src D-VOL LUN, dest D-VOL LUN, target host). The RAID Manager module  2028  asks if the designated P-VOL LU and D-VOL LU have already made a pair to Remote Copy module  2221  in a storage controller  2200 , and passes source and destination primary volume LUNs, source and destination differential volume LUNs, and destination host name (Step  5301 ). The Remote Copy module  2221  searches the CPMT  2220 . If either LUN has already been made into a pair, the RAID Manager module  2028  sends back an error (Step  5308 ). If both LUNs have not made into a pair yet, the Remote Copy module  2221  makes them a pair and registers the designated pair information in the CPMT  2220 , and also sends back an indication of the successful pair create operation (Step  5302 ). The RAID Manager module  2028  calls the snapshot module  2025  with both source and destination primary volume LUN (LU 1 , LU 10 ), both source and destination differential volume LUN (LU 2 , LU 11 ), and target host name (Str 2 ) in order to send a P-VOL and D-VOL relation map to remote snapshot module  3025  (Step  5303 ). The snapshot module  2025  checks whether the designated source P-VOL logical unit number (LU 1 ) and D-VOL logical unit number (LU 2 ) are in the SVMT  2030  (Step  5304 ). If either LUN does not exist, the snapshot module  2025  returns an error. After receiving the error, the RAID Manager module  2028  asks the Remote Copy module  2221  to stop the pair operation and changes the pair status (Step  5307 ). If both LUNs exist, the snapshot module  2025  generates mapping information of remote P′VOL (LU 10 ) and remote D′-VOL (LU 11 ) for the replicated file system snapshot (Step  5305 ). Then, snapshot module  2025  sends the mapping information to the destination node&#39;s snapshot module  3025  via network  2700  between NAS head  2100  and NAS head  3100 . ( 5306 ) Finally, the RAID Manager module  2028  sends back a result of the snapshot replication operation to the storage management software  1021  (Step  5309 ). The mount operations of replicated snapshot are the same as the previous embodiment. 
       4th Embodiment 
       [0064]    A fourth embodiment of the present invention extends the PairCreate operation on the RAID Manager module.  FIGS. 16-18  are directed to this embodiment. Assuming that snapshot operations have already been started, mapping information of P-VOL  2301  and D-VOL  2302  are stored in SVMT  2030 . A logical unit number of P-VOL for file system A is LU 1 . A logical number of D-VOL for file system A is LU 2 . A storage Manager  1021  on storage management host  1100  calls a remote replication pair creation command in RAID Manager module  2028  with both source and destination primary volume logical unit number (LU 1 , LU 10 ), and target host name (Str 2 ) (Step  5400 ). Both source and destination D-VOL can be designated instead of both source and destination P-VOL as an alternative, but under the present scenario the following command can be used: SnapPairCreate (src P-VOL LUN, dest P-VOL LUN, target host). The RAID Manager module  2028  asks if the designated P-VOL LUN has already been made a pair to Remote Copy module  2221  in a storage controller  2200  and transmits source and destination P-VOL LUNs and the destination host name (Step  5401 ). The Remote Copy module searches the CPMT  2220 . If the P-VOL  2301  source LUN and P′-VOL  3301  destination LUN have already been made a pair, the RAID Manager module  2028  sends back an error (Step  5407 ). If the P-VOL  2301  source LUN and P′-VOL  3301  destination LUN have not been made a pair yet, the Remote Copy module makes a pair, registers the designated pair information in the CPMT  2220  and also sends back an indication of the successful pair create operation (Step  5402 ). The RAID Manager module  2028  calls the snapshot module  2025  with both source P-VOL and destination P′-VOL logical unit numbers (LU 1 , LU 10 ), and target host name (Str 2 ) in order to send a P-VOL and D-VOL relation map to a remote snapshot module  3025  (Step  5403 ). The snapshot module  2025  checks if the designated P-VOL LUN (LU 1 ) is in the SVMT  2030  (Step  5404 ). If the LUN does not exist, the snapshot module  2025  returns an error. After receiving the error, the RAID Manager module  2028  asks the Remote Copy module  2221  to stop the pair operation and changes the pair status (Step  5406 ). If the LUN does exist, the snapshot module  2025  searches the SVMT  2030  to find the source D-VOL (LU 2 ) that is associated with source P-VOL (Step  5501 ). Then, the snapshot module  2025  calls remote snapshot module  3025  in order to find unused logical units for target of replication of D-VOL  2302  (Step  5502 ). Steps performed by the remote snapshot module  3025  will be explained later. If there are no unused appropriate logical units in the remote storage system  3000 , the snapshot module  2025  sends back an error (Step  5507 ). After receiving the unused logical unit (LU 11 ) from the remote snapshot module  3025 , the snapshot module  2025  calls RAID Manager  2028  with a target host name (Str 2 ), and the logical unit number of both source D-VOL  2302  (LU 2 ) and destination D′-VOL  3302  (LU 11 ). The RAID Manager module  2028  asks if the designated D-VOL, D′-VOL LUNs have already been made a pair to Remote Copy module  2221  in a storage controller  2200  and passes source D-VOL and destination D′-VOL LUNs, and destination host name (Step  5503 ). If the source D-VOL LUN and destination D′-VOL LUN have already been made into a pair, the RAID Manager module  2028  sends back an error (Step  5507 ). If the LUNs have not been made into a pair yet, the Remote Copy module  2221  makes a pair, registers the designated pair information in the CPMT  2220 , and sends back an indication of the successful pair create operation (Step  5504 ). The snapshot module  2025  generates mapping information of remote destination P′-VOL  3301  (LU 10 ) and remote destination D′-VOL  3302  (LU 11 ) for the replicated file system snapshot (Step  5505 ). Then, snapshot module  2025  sends this to the destination node&#39;s snapshot module  3025  via network  2700  between NAS head  2100  and NAS head  3100  (Step  5506 ). Finally, RAID Manager module  2028  sends back a result of the snapshot replication operation to the storage manager  1021  (Step  5408 ). 
         [0065]    The method of finding two or more unused logical units was described above with respect to  FIG. 12 . In the present embodiment, one or more logical units are searched for in the Logical Unit Management Table (LUMT)  3031 . When the remote snapshot module  3025  receives an unused logical unit search request, snapshot module  3025  searches LUMT  3031 , which is managed by LVM  3026 . Snapshot module  3025  checks whether there is at least one unused volume. If snapshot module  3025  finds no unused logical unit, it sends back an error. If the snapshot module  3025  can find one or more unused logical units, it checks if the type of the unused logical units is the same as the source logical unit, which is information passed from the source snapshot module  2025 . If snapshot module  3025  can find no logical units of same type, it sends back an error. If the snapshot module  3025  finds one or more logical units of same type, it checks whether the size of the logical units is sufficient. If snapshot module  3025  cannot find any logical units of the correct size, it sends back an error. If snapshot module  3025  finds one or more logical units of the correct size, it picks one of them from among a variety of ways. For example, it is possible to just pick the first appropriate LUNs listed in the LUMT. Additional replication target volume properties other than type and size can be specified here, and the information regarding these properties is passed from the source snapshot module  2025  to the destination snapshot module  3025 . Finally, the snapshot module  3025  returns an unused logical unit designation for replicated D′-VOL  3302  or an error to the source snapshot module  2025 . The mount operations of replicated snapshot are the same as described in the previous embodiments. 
         [0066]    While specific embodiments have been illustrated and described in this specification, those of ordinary skill in the art appreciate that any arrangement that is calculated to achieve the same purpose may be substituted for the specific embodiments disclosed. This disclosure is intended to cover any and all adaptations or variations of the present invention, and it is to be understood that the above description has been made in an illustrative fashion, and not a restrictive one. Accordingly, the scope of the invention should properly be determined with reference to the appended claims, along with the full range of equivalents to which such claims are entitled.