Enhanced recovery of highly available computing systems

Exemplary embodiments disclose a method and system for detecting a failure and resuming processing in a computing system encompassing at least two sites, a primary site and a secondary site. In a module, an exemplary embodiment generates a record of a logically consistent state and data of system components of the primary site periodically and transfers the record of a logically consistent state and data of system components of the primary site to the secondary site. In another module, an exemplary embodiment detects a failure in the primary site, halts the generation of the record of a logically consistent state and data of system components of the primary site periodically with a data freeze function, and resumes a processing of the primary site on the secondary site with secondary site components updated with a most recent logically consistent state and data of system components of the primary site.

FIELD OF THE INVENTION

The present invention relates generally to the recovery of cooperating computer systems after failure, and more particularly to the recovery of mirrored computer systems that enhance availability by replicating cooperating system components.

BACKGROUND OF THE INVENTION

Computing systems host software that runs modern economies and societies—affecting the real time day-to-day safety and welfare of millions of people. The continuous availability of many computer systems is essential to a society's functioning, health, and security. Sporadic unavailability, even for minutes, of some systems is dangerous and disruptive: air traffic control and financial systems are examples. Therefore, much thought and effort has been applied to enhancing the availability of some systems—and to decreasing downtime in the event of a failure.

The decreasing cost and increasing performance of computer system hardware over time has channeled efforts to increase system availability toward a model in which system components are replicated and dispersed geographically to avoid natural disasters and to avoid any one failure from causing a total system failure. In this model, software running on replicated controllers synchronizes and coordinates data movement and data replication among the replicated components to avoid total failure scenarios and to facilitate expeditious system recovery.

SUMMARY

Exemplary embodiments of the present invention disclose a method and system for detecting a failure and resuming processing in a computing system encompassing at least two sites, a primary site and a secondary site. In a module, an exemplary embodiment generates a record of a logically consistent state and data of system components of the primary site periodically. In another module, an exemplary embodiment transfers the record of a logically consistent state and data of system components of the primary site to the secondary site. In another module, an exemplary embodiment updates a state and data of system components of the secondary site with the contents of the record of a logically consistent state and data of system components of the primary site. In another module, an exemplary embodiment detects a failure in the primary site. In another module, an exemplary embodiment halts the generation of a record of a logically consistent state and data of system components of the primary site periodically with a data freeze function upon detecting a failure. In another module, an exemplary embodiment resumes a processing of the primary site on the secondary site with secondary site components updated with a most recent logically consistent state and data of system components of the primary site.

DETAILED DESCRIPTION

The Geographically Dispersed Parallel Sysplex (GDPS)® of IBM is a highly available system architecture based on such a model. A GDPS system incorporates redundant components and mechanisms within a site and at geographically dispersed sites whose activity can be synchronized periodically to a continuous sequence of known states. Doing so prepares the system to quickly replace failed components and to continue processing from a last known state. Exemplary embodiments of the present invention disclose a method and system for detecting a failure and resuming processing in a computing system encompassing at least two sites, a primary site and a secondary site.

To manage redundancy and synchronization in a GDPS system, users may form groups of system components (related, cooperating hardware and software components) that are managed as a consistent entity by system software to perform a function or run an application, e.g., a database. These groups are called consistency groups. A consistency group that is in a state from which the consistency group may be stopped and restarted without a risk of a logical fault (e.g., all writes previous to a specified completed write have completed and no following writes have, and logs and tables reflect activity to a specified processing point and not beyond) is deemed to be in a consistent state. A self-describing record that specifies a consistent state for a consistency group is a consistency group record. System software in a primary site of a GDPS system that employs consistency groups continuously generates a sequence of consistency group records which are sent as they are formed to a secondary site that can use them to establish a consistent state from which to continue the processing of the primary site in the event of a failure.

For example, in a GDPS formed by two dispersed sites, a primary site A, and a secondary site B, a consistency group record is formed periodically in site A to encapsulate updates to site A's system data and state at a particular point in time and is sent to site B, to update its components to the known consistent state A had when the record was composed. The consistency group record describes a consistent state and the operations to be performed (e.g., disk writes or updates to log files) to update a previous consistent state to a current consistent state. These records are sent sequentially and frequently so that site B can assume the processing of site A quickly and near to the state A was in when a failure was detected.

A monitoring facility detects system failures, and in the event of a failure, implements a freeze function that stops the generation of consistency group records. The last consistency group record formed before the freeze now reflects the last known consistent state of a system before the failure and is used to recover from the failure. The quick resumption of processing in the event of a failure is advantageous in many situations.

FIG. 1depicts a highly available computer system100comprised of two computer systems at two geographically dispersed sites, a primary computer system101at a primary site and a secondary computer system102at a secondary site. The primary computer system includes at least one processor103, at least one coupling facility104, and at least one storage system105. The storage system includes at least one disk system106and at least one tape system107. The secondary computer system102inFIG. 1system includes at least one processor108, at least one coupling facility109, and at least one storage system110. The storage system is comprised of at least one disk system111and at least one tape system112. Primary computer system101uses a communications link113between primary computer system101and secondary computer system102to keep secondary computer system102updated with the state and data of the primary computer system101.

An example of a highly available computer system100is an IBM Geographically Dispersed Parallel Sysplex® (GDPS) system implementing Peer-to-peer Remote Copy (PPRC). High availability provided by a GDPS/PPRC system is often desired when executing finance-oriented database applications, applications in which human safety is at risk, and many other applications. Depending on system configuration and software and hardware implemented to achieve redundancy, communications link113between sites participating in a GDPS/PPRC system may be over fiber-optic, microwave, and satellite communications links. Many site configurations and inter-site communication configurations are possible in a dispersed system and are tailored to meet to a customer's requirements for response time, throughput, (given a customer's workload) and desired level of availability.

Exemplary embodiments of coupling facility104and/or109provide services to operating systems and middle ware so that system software can manage where applications run, manage resources the applications use, synchronize operations between processors, and to temporarily cache information that enables an efficient use of system resources.

In a computer system200implementing an exemplary embodiment of the present invention, shown inFIG. 2, a record generator204in primary computing system201periodically generates a record203. Exemplary embodiments of record203include a logically consistent state information and data that contains the information necessary to update the system components of secondary computer system202to a recent logically consistent state and data of primary computer system201prior to the record203being generated. A computer system that is in a state from which the computer system may be stopped and restarted without a risk of a logical fault (e.g., all writes previous to a specified completed write have completed and no following writes have, and logs and tables reflect activity to a common specified processing point and not beyond) is in a logically consistent state. Record203is transferred to secondary computer system202over a communications link206.

Exemplary embodiments of record203may include permanent data or temporary data. Permanent data is defined as data that is written and stored in a disk subsystem, tape subsystem, etc. Temporary data is defined as cached data. Some examples of permanent data may include, but are not limited to: network information, configuration files, etc. Some examples of temporary data may include, but are not limited to: emails, documents, recent hypertext markup language documents, etc.

A failure monitor205monitors primary computer system201for failures. In exemplary embodiments, failure monitor205monitors alerts from the disk subsystem, tape subsystem, host software messages, etc. for failures. If a failure occurs, failure monitor205implements a quiescing function that halts a generation and transmission of record to secondary computer system202. In exemplary embodiments, the quiescing function is a data freeze function that quiesces all primary computing system201functions and halts a generation and transmission of records to secondary computer system202. In other exemplary embodiments, the quiescing function may be a signal from primary computer system201to secondary computer system202, or a lack of a heartbeat signal, etc.

The modules of the method for detecting a failure and resuming processing in a computing system encompassing at least two sites, a primary site and a secondary site are shown inFIG. 3. In exemplary embodiments the modules of the method for detecting a failure and resuming processing in a computing system encompassing at least two sites initiates during start-up of primary computer system201. In another embodiment, the method initiates when primary secondary computer system202. In module301, record generator204generates a record203of a logically consistent state and data of primary computer system201. In module302, record203of a logically consistent state and data is transferred to secondary computer system202. In module303, components of secondary computer system202are updated with a content of record203of a logically consistent state and data of primary computing system201. In module304, a failure in primary computer system201is detected by failure monitor205. In module305, a Data Freeze function is generated by failure monitor205in response to a failure which quiesces primary computing system201and halts a generation and transmission of records to secondary computing system202. In module306, secondary computing system202resumes a processing of primary computing system201at a point described in a last received record of a logically consistent state and data of primary computing system201.

The forgoing description is an example only, and those skilled in the art understand that there are many ways to generate a logically consistent record of state and data of various computer system components and groups of computer system components to achieve a desired level of confidence in system availability and computer system restoration delay after specific types of failures and combinations of failures.

Communications unit410, in these examples, provides for communications with other data processing systems or devices, including resources of computer system100. In these examples, communications unit410includes one or more network interface cards. Communications unit410may provide communications through the use of either or both physical and wireless communications links. Record generator204and failure monitor205may be downloaded to persistent storage408through communications unit410.

I/O interface(s)412allows for input and output of data with other devices that may be connected to computer systems101,102,201and202. For example, I/O interface412may provide a connection to external devices418such as a keyboard, keypad, a touch screen, and/or some other suitable input device. External devices418can also include portable computer-readable storage media such as, for example, thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to practice embodiments of the present invention, e.g., record generator204and failure monitor205can be stored on such portable computer-readable storage media and can be loaded onto persistent storage408via I/O interface(s)412. I/O interface(s)412also connect to a display420.