Abstract:
In one aspect, a method to upgrade software on nodes in a clustered environment, includes terminating processes on a first node before upgrading the software on the first node, upgrading the software to a first version from a second version on the first node, running the processes on the first node after upgrading the software on the first node to the first version, determining whether a second node is about to upgrade to the first version of software, allowing transfer of site control from the second node to the first node, if the second node is about to upgrade to the first version of software and upgrading the software on the second node to the first version of software after the transferring of site control from the second node to the first node.

Description:
BACKGROUND 
     Computer data is vital to today&#39;s organizations, and a significant part of protection against disasters is focused on data protection. As solid-state memory has advanced to the point where cost of memory has become a relatively insignificant factor, organizations can afford to operate with systems that store and process terabytes of data. 
     Conventional data protection systems include tape backup drives, for storing organizational production site data on a periodic basis. Such systems suffer from several drawbacks. First, they require a system shutdown during backup, since the data being backed up cannot be used during the backup operation. Second, they limit the points in time to which the production site can recover. For example, if data is backed up on a daily basis, there may be several hours of lost data in the event of a disaster. Third, the data recovery process itself takes a long time. 
     Another conventional data protection system uses data replication, by creating a copy of the organization&#39;s production site data on a secondary backup storage system, and updating the backup with changes. The backup storage system may be situated in the same physical location as the production storage system, or in a physically remote location. Data replication systems generally operate either at the application level, at the file system level, or at the data block level. 
     Current data protection systems try to provide continuous data protection, which enable the organization to roll back to any specified point in time within a recent history. Continuous data protection systems aim to satisfy two conflicting objectives, as best as possible; namely, (i) minimize the down time, in which the organization production site data is unavailable, during a recovery, and (ii) enable recovery as close as possible to any specified point in time within a recent history. 
     SUMMARY 
     In one aspect, a method to upgrade software on nodes in a clustered environment, includes terminating processes on a first node before upgrading the software on the first node, upgrading the software to a first version from a second version on the first node, running the processes on the first node after upgrading the software on the first node to the first version, determining whether a second node is about to upgrade to the first version of software, allowing transfer of site control from the second node to the first node, if the second node is about to upgrade to the first version of software and upgrading the software on the second node to the first version of software after the transferring of site control from the second node to the first node. 
     In another aspect, an article includes a non-transitory machine-readable medium that stores executable instructions to upgrade software on nodes in a clustered environment. The instructions causing a machine to terminate processes on a first node before upgrading the software on the first node, upgrade the software to a first version from a second version on the first node, run the processes on the first node after upgrading the software on the first node to the first version, determine whether a second node is about to upgrade to the first version of software, transfer site control from the second node to the first node, if the second node is about to upgrade to the first version of software and upgrade the software on the second node to the first version of software after the transfer of site control from the second node to the first node. 
     In a further aspect, a first node includes circuitry configured to terminate processes on a first node before upgrading the software on the first node, upgrade the software to a first version from a second version on the first node, run the processes on the first node after upgrading the software on the first node to the first version, determine whether a second node is about to upgrade to the first version of software and be allowed to receive site control, if the second node is about to upgrade to the first version of software. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram of an example of a data protection system. 
         FIG. 2  is a flowchart of an example of a process to upgrade software on a data protection appliance (DPA). 
         FIG. 3  is an example of a data protection appliance on which the process of  FIG. 2  may be implemented. 
     
    
    
     DETAILED DESCRIPTION 
     Described herein is an approach to upgrade software on nodes. In particular, the methods and techniques described herein allow for two nodes to use the same software upgrade process independently from each other while allowing only one of the nodes to have site control at a time. In one example, a node can crash in the middle of an upgrade, recover and continue upgrading seamlessly. While the description describes one particular pair of nodes as being data protection appliances (DPAs), the nodes may be any nodes in a clustered computing environment where one of the nodes is determined as site control. As used herein, site control is a determination of one and only one of the nodes as the controller of all other nodes in a system (e.g., a site). While the manner of selecting the site control and the responsibilities assumed by it are determined by a leader election protocol not described herein, more than one site control is strictly prohibited; however, the system can function for brief periods of time without a site control. 
     Referring to  FIG. 1 , a data protection system  100  includes a client  12 , data protection appliances (DPAs) (e.g., a DPA  14   a  and a DPA  14   b ) and a storage volume  16   a  at a production site and data protection appliances (DPAs) (e.g., a DPA  14   c  and a DPA  14   d ) and a storage volume  16   b  at a replication site. The DPAs  14   a - 14   d  and the storage volumes  16   a - 16   b  are coupled together through the network  18 . In one example, the network  18  may be a network such as a wide area network (WAN). 
     The DPAs  14   a ,  14   b  are redundant in case of failure so that one of the DPAs  14   a ,  14   b  controls the production site (i.e., has site control) at a time. Likewise, the DPAs  14   c ,  14   d  are redundant in case of failure so that one of the DPAs  14   c ,  14   d  controls the replication site at a time. The DPA  14   c ,  14   d  are also redundant to the DPAs  14   a - 14   b , in the event the production site fails. 
     Each of the DPAs  14   a - 14   d  includes an upgrade status object. For example, the DPA  14   a  includes an upgrade status object  22   a , the DPA  14   b  includes an upgrade status object  22   b , the DPA  14   c  includes an upgrade status object  22   c  and the DPA  14   d  includes an upgrade status object  22   d . The upgrade status objects  22   a - 22   d  indicate whether a software update is in progress for the respective DPA  14   a ,  14   b . In one example, the upgrade status object  22   a ,  22   b  is a persistent bit that is not removed or copied over during a software upgrade of the respective DPA  14   a - 14   d . In one particular example, if the upgrade status object  22   a - 22   d  is set to “True” no processes on the respective DPA  14   a - 14   d  will run when a script is executed to run all processes on the DPA. For example, the DPA  14   a - 14   d  will not be automatically or manually run whether it be processes/services/daemons/webservers and so forth. 
     In one example, the upgrade status objects  22   a - 22   d  may be used in the event of a crash during upgrade so that their respective DPA  14   a - 14   d  can recognize its previous state prior to the crash. 
     The storage volume  16   a  includes a DPA upgrade object  26   a  and a DPA upgrade object  26   b . The DPA upgrade object  26   a  is updated by the DPA  14   a  and the DPA upgrade object  26   b  is updated by the DPA  14   b . The DPA  14   a  can read or access the DPA upgrade object  26   b  but it cannot write to it. Likewise, the DPA  14   b  can read or access the DPA upgrade object  26   a  but it cannot write to it. 
     Similarly, the storage volume  16   b  includes a DPA upgrade object  26   c  and a DPA upgrade object  26   d . The DPA upgrade object  26   c  is updated by the DPA  14   c  and the DPA upgrade object  26   d  is updated by the DPA  14   d . The DPA  14   c  can read or access the DPA upgrade object  26   d  but it cannot write to it. Likewise, the DPA  14   d  can read or access the DPA upgrade object  26   c  but it cannot write to it. 
     The DPA upgrade objects  26   a - 26   d  include two fields. For example, the DPA upgrade object  26   a  includes a site control field  32   a  and a version field  36   a , the DPA upgrade object  26   b  includes a site control field  32   b  and a version field  36   b , the DPA upgrade object  26   c  includes a site control field  32   c  and a version field  36   c  and the DPA upgrade object  26   d  includes a site control field  32   d  and a version field  36   d.    
     The site control field  32   a - 32   d  indicates whether the respective DPA  14   a - 14   d  can take over site control. In one example, a “True” in the site control field  32   a - 32   d  indicates that the DPA can take over site control while a “False” in the site control field  32   a - 32   d  indicates that the DPA cannot take over site control. The version field  36   a - 36   d  indicates what version of software is on or is about to be on the respective DPA  14   a - 14   d.    
     Referring to  FIG. 2 , an example of a process to upgrade software on data protection appliance (DPAs) is a process  200 . In particular, each DPA  14   a ,  14   b  can execute the process  200  independently of each other to upgrade software from a version “n” to a version “n+1”. While the example below is a description of the process  200  executing on the production site DPAs  14   a - 14   b , the process  200  can also be executed on the replication site DPAs  14   c - 14   d . While the examples herein describe going from version “n” to “n+1”, other examples may include going from version “n” to “n−1” in situations where the “n” version is not working properly and reverting back to an older version is required. Still further examples can include upgrade of software to any different version of software. 
     The following is an example of process  200  executing on the DPA  14   a  first before executing on the DPA  14   b . Process  200  receives notification that a new software version (n+1) is available ( 202 ) and terminates the processes running on the DPA ( 204 ) and sets the upgrade status object  22   a  to “True” ( 208 ). Process  200  determines if the software version for the DPA  14   a  is the same as the software version for the DPA  14   b  ( 214 ). For example, the DPA  14   a  reads the version field  36   b  in the DPA upgrade object  26   b  to determine the version of software on or is about to be put on the DPA  14   b . If the software version for the DPA  14   a  is the same as the software version for the DPA  14   b , the process  200  sleeps for a predetermined time ( 220 ). For example, the DPA  14   a  sleeps for 30 seconds. By allowing the DPA  14   a  to sleep, the DPA  14   b  can take over site control from the DPA  14   a  if the DPA  14   a  has site control. In other examples, alternatively to sleep, if the current node is running site control it can electively relinquish control to another node if such a mechanism exists (e.g., using a push mechanism instead of pull mechanism). Using a sleep mechanism is one example to minimize the time in which the system is without Site Control. System  10  can function for a while without site control; however, this is costly so that reducing the time that there is no site control is desired. 
     Process  200  sets the fields in the DPA upgrade object  26   a  ( 228 ). For example, the DPA  14   a  sets the site control field to False and the version field from “n” to “n+1.” 
     Process  200  upgrades the software on the DPA  14   a  ( 234 ) and sets the upgrade status object  22   a  to False from True ( 238 ). Process  200  runs the processes on the DPA  14   a  ( 244 ). For example, the DPA  14   a  runs a script that runs all the processes on the DPA  14   a . The script is allowed to execute if the upgrade status object  22   a  is set to False. 
     Process  200  determines if the DPA upgrade object  26   b  is not corrupted ( 252 ) and determines whether the software versions between the DPAs  14   a ,  14   b  are the same ( 254 ). If the DPA upgrade object  26  is not corrupted and the software versions are the same, process  200  sets the site control field  32   a  to True from False and allows transfer of site control ( 264 ). Thus, the DPA  14   a  waits until just before the DPA  14   b  starts its upgrade (i.e., when DPA  14   b  executes processing block  228  and changes version filed  36   b  from “n” to “n+1.”) to allow transfer of the site control at the production site thereby ensuring that only one DPA  14   a ,  14   b  is in control of the production site at a time. The transfer of site control is determined by a leader election protocol such as described, for example, in U.S. Pat. No. 7,840,662. 
     The process  200  persistently updates fields in the DPA upgrade object  26   a  ( 270 ). For example, the site control field is updated with a “True” and the version field is updated with “n+1.” 
     Referring to  FIG. 3 , an example of part of the DPAs  14   a - 14   d  is a DPA  14 ′. The DPA  14 ′ includes a processor  302 , a volatile memory  304 , a non-volatile memory  306  (e.g., hard disk) and a user interface (UI)  308  (e.g., a mouse, a keyboard, a display, touch screen and so forth). The non-volatile memory  306  stores computer instructions  314 , an operating system  316  and data  318 . In one example, the computer instructions  314  are executed by the processor  302  out of volatile memory  304  to perform all or part of the processes described herein (e.g., process  200 ). 
     The processes described herein (e.g., process  200 ) are not limited to use with the hardware and software of  FIG. 3 ; they may find applicability in any computing or processing environment and with any type of machine or set of machines that is capable of running a computer program. The processes described herein may be implemented in hardware, software, or a combination of the two. The processes described herein may be implemented in computer programs executed on programmable computers/machines that each includes a processor, a storage medium or other article of manufacture that is readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and one or more output devices. Program code may be applied to data entered using an input device to perform any of the processes described herein and to generate output information. 
     The system may be implemented, at least in part, via a computer program product, (e.g., in a machine-readable storage device), for execution by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers)). Each such program may be implemented in a high level procedural or object-oriented programming language to communicate with a computer system. However, the programs may be implemented in assembly or machine language. The language may be a compiled or an interpreted language and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network. A computer program may be stored on a storage medium or device (e.g., CD-ROM, hard disk, or magnetic diskette) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer to perform the processes described herein. The processes described herein may also be implemented as a machine-readable storage medium, configured with a computer program, where upon execution, instructions in the computer program cause the computer to operate in accordance with the processes. 
     The processes described herein are not limited to the specific examples described. For example, the process  200  is not limited to the specific processing order of  FIG. 2 . Rather, any of the processing blocks of  FIG. 2  may be re-ordered, combined or removed, performed in parallel or in serial, as necessary, to achieve the results set forth above. 
     The processing blocks (for example, process  200 ) associated with implementing the system may be performed by one or more programmable processors executing one or more computer programs to perform the functions of the system. All or part of the system may be implemented as, special purpose logic circuitry (e.g., an FPGA (field-programmable gate array) and/or an ASIC (application-specific integrated circuit)). 
     Elements of different embodiments described herein may be combined to form other embodiments not specifically set forth above. Other embodiments not specifically described herein are also within the scope of the following claims.