Patent Publication Number: US-9405644-B2

Title: Redundant automation system

Description:
FIELD OF INVENTION 
     The invention relates to a redundant automation system having a plurality of automation devices that are connected to one another, wherein each of the plurality of automation devices processes a control program in order to control a technical process. 
     DESCRIPTION OF THE RELATED ART 
     In the automation environment, there is an increasing demand for highly available solutions (e.g., H systems) which are suitable for minimizing possible downtimes of the installation. The development of such highly available solutions is very cost-intensive with an H system usually used in the automation environment being distinguished by the fact that two or more subsystems in the form of automation devices or computer systems are coupled to one another via a synchronization connection. In principle, both subsystems can have read and/or write access to the peripheral units connected to this H system. One of the two subsystems leads with respect to the peripherals connected to the system. This means that outputs to peripheral units or output information for these peripheral units is/are effected only by one of the two subsystems which operates as a master or has assumed the master function. For both subsystems to run in a synchronous manner, the subsystems are synchronized at regular intervals via the synchronization connection. With respect to the frequency and extent of synchronization, different forms (warm standby, hot standby) may be used. 
     An H system often requires a smooth “failover” if one of the subsystems fails and it is necessary to change over to the other subsystem. This means that, despite this unplanned changeover or this unplanned change from one subsystem to the other, this changeover or change does not have a disruptive effect on the technical process to be controlled. In this case, it is permissible for a (short) dead time, during which the outputs remain at their last valid process output values, to occur at the outputs of the connected peripherals. However, a jump (surge) in the values at these outputs on account of the changeover is undesirable and should therefore be avoided. Therefore, “smooth” should also be understood to mean the continuity of the curve shape of the process output values. 
     In order to achieve this, the two subsystems must have the same system state at the time of the failure. This is ensured using the suitable synchronization method. If both subsystems are processing the input information (inputs) of the process, both systems are in the same system state when they change their respective “thread global” data (shared data of programs, in particular programs with different priorities) in the same manner given the same process input data or process input information. In order to achieve this, the synchronization method ensures that the individual threads of the two subsystems are interrupted or executed in the same manner. This results in an identical “thread mountain”. 
     The Siemens catalog ST 70, chapter 6, 2013 edition, discloses a redundant automation system which comprises two subsystems (a master PLC and a slave PLC) and is intended to increase the availability of an installation to be controlled. For this purpose, the automation system is provided with device/apparatus that decides, on the basis of an event, which program must be started in order to suitably react to the event. If, for example during the execution of a program, an event in the form of a pending alarm for the technical process to be controlled is applied to a signaling input of the automation system, the running program is usually stopped at a waiting point and a program which is intended to analyze the alarm and initiate measures which eliminate the cause of the alarm is started. This automation system is regularly synchronized and it is ensured that the failure of one of these subsystems does not have a disruptive effect on a process to be controlled because the other subsystem can continue the execution or processing of the corresponding part of its respective control program or the execution or processing of the corresponding parts of this control program. 
     If, for example, an event which has occurred in a first subsystem is not synchronized with a second subsystem of an automation system comprising two subsystems and, after the event has been processed by the first subsystem, this subsystem fails, the course of a technical process to be controlled may be disrupted. This is because the second subsystem—without knowledge of the event—runs through a different program path, representing the execution order of the programs, from the program path which would be run through by the second subsystem with knowledge of the event and which would also be necessary in order to avoid disrupting the course of the technical process to be controlled. 
     It is pointed out that a program is understood to mean both a program and a subroutine, a part of a program, a task, a thread, an organizational module, a functional module or another suitable program code for implementing an automation function. The programs of an automation system usually is categorized into priority classes and processed or executed according to their associated priority. 
     A slave automation device (backup PLC) is usually respectively provided for each master automation device (master PLC) in a redundant automation system as a redundancy partner, i.e., 2n slave automation devices are required for n master automation devices, which means a very large amount of hardware. Such an automation system is cost-intensive and the planning of such an automation system is also very complex. 
     SUMMARY OF THE INVENTION 
     Therefore, the invention is based on an object of providing an automation system of the type mentioned at the outset which can be used to achieve redundancy with a reduced amount of hardware. This object is achieved by the features of described/recited in the claimed invention. 
     It is advantageous that (hot standby) redundancy can also be subsequently achieved in a simple manner. Redundancy need not necessarily comprise all masters (master automation devices, master PLCs) of the automation system. It is possible to select only a particular number of master PLCs which are operated in a redundant manner using a backup PLC. On account of the small amount of hardware, the failure rate (MTTF) of the automation system is reduced with regard to a 2n solution in comparison with the prior art. If one of the master PLCs fails, the backup PLC assumes the function of this master PLC. The slave control program assigned to this master PLC in the backup PLC becomes the master control program and carries out the control tasks of this failed master PLC. This failover is effected smoothly for any potentially defective master PLC, in which case the failure of all master PLCs is safeguarded by the backup PLC. 
     The master PLCs do not have to (actively) wait for a response from the slave or the backup automation device (backup PLC) in order to continue their program processing: all relevant information is transmitted from the masters to the slave in a temporally asynchronous manner. As a result, the processing performance of the masters is decoupled from the communication bandwidth available for event synchronization, which is important, in particular, with regard to the increasing imbalance between the increase in the processing performance of the processors, on the one hand, and the increase in the communication performance, on the other hand. This is because the communication performance usually cannot keep up with the increasing processing performance. 
     After events have occurred, each master is synchronized with the slave such that both the masters and the slave run through the same program paths on account of the events, the runs being effected in a temporally asynchronous manner. This means that the masters temporally lead the slave or the slave temporally trails the masters with regard to the program processing. In this context, “trailing” or “leading” of the slave with respect to one of the masters is understood as meaning the time difference between the beginning of the processing of the processing sections by this master and the beginning of the processing of the processing sections by the slave. 
     A corresponding situation also applies if different HW platforms with different performance are provided for the masters and the slave (reserve). On account of the asynchronous coupling of the masters and the slave, runtime differences can be compensated for more easily, in particular when a powerful backup PLC, for example, effects the reserve functionality of a plurality of master PLCs in a time-slice method. In this case, the temporal trailing of a reserve program (slave program) becomes temporarily greater and is then reduced again. 
     One refinement of the invention provides for the respective master to be used to also transmit process input values to the slave at the time when the current releases are transmitted. The information relevant to the slave is combined or collected and is finally transmitted to the slave. In contrast with known temporal synchronization methods during which relevant information must be immediately transmitted to the slave, this means a considerably reduced amount of “management effort” both for the automation devices provided as master and for the slave or the backup automation device. 
     Another refinement of the invention provides for the slave to acknowledge the respective release to the respective master after the respective processing sections have been processed. The number of unacknowledged releases makes the respective master aware of the current trailing of the slave, and as a result the respective master can take suitable measures in order to prevent the temporal trailing from becoming too great. 
     A multi-core-based unit is preferably provided as the backup PLC, each core of the backup PLC being able to be allocated to a master PLC as a redundancy partner. The backup PLC can also be in the form of a powerful industrial PC with an accordingly configured main memory and processing performance. Further advantageous refinements and features emerge from the claimed invention as well. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The invention, its refinements and advantages are explained in more detail below using the drawing which illustrates an exemplary embodiment of the invention and in which: 
         FIGS. 1-2 and 6  show redundant automation systems in accordance with some embodiments of the present invention; and 
         FIGS. 3-5  show sequences of temporally asynchronous coupling of a master and a slave in accordance with some embodiments of the present invention. 
     
    
    
     The same parts in  FIGS. 1-6  are provided with the same reference symbols. 
     DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED EMBODIMENTS 
     Reference is made to  FIG. 6  which illustrates a redundant automation system that is known in the art and comprises two subsystems. A first subsystem Ta is in the form of a master automation device (master PLC) and a second subsystem Tb is in the form of a slave automation device (backup PLC), and the two subsystems are connected to a peripheral unit Pe via a field bus Fb. In this case, the field bus Fb complies with the PROFIBUS-DP specification, for example. In principle, other bus systems, for example Ethernet, Fieldbus, Modbus, or else parallel bus systems, are also suitable. The peripheral unit Pe receives, via input lines Es, signals from transducers or measuring transducers, which are used to detect a process state, outputs via output lines As, and signals to actuators, which are used to influence the process. The process as well as the transducers, measuring transducers and actuators are not illustrated in  FIG. 6  for the sake of clarity. The two subsystems Ta, Tb of the known automation system execute the same control program in a cyclical and synchronous manner. A synchronization connection Sv is provided in order to synchronize the subsystems, and the redundancy and monitoring functions are implemented via this synchronization connection Sv. 
     Reference is made below to  FIGS. 1-2  which illustrate redundant automation systems. In one embodiment, these automation systems have three master automation devices PLC 1 , PLC 2 , PLC 3 , four peripheral units Pe 1 , Pe 2 , Pe 3 , Pe 4  and a slave automation device backup PLC. The automation devices PLC 1 , PLC 2 , PLC 3 , the backup PLC and the peripheral units Pe 1 , Pe 2 , Pe 3 , Pe 4  are connected to one another. In this embodiment, a ring structure Rs, which, e.g., uses the PROFINET standard, is provided as the connection. It goes without saying that other network structures or network topologies, such as a bus or star topology as well as a mixed form of these topologies, can be provided in order to implement redundancy. 
     In order to achieve more powerful coupling of the master automation devices PLC 1 , PLC 2 , PLC 3  to the slave automation device backup PLC, the automation system according to  FIG. 2  has a double-ring structure Rss, and as a result the data required for synchronization can be transmitted separately from the peripheral data and the changeover times in the event of a failover can be shortened, in particular. Each of the three master automation devices PLC 1 , PLC 2 , PLC 3  processes a control program Pc 1 , Pc 2 , Pc 3 , and the slave automation device backup PLC is configured to process a corresponding slave control program Pb 1 , Pb 2 , Pb 3  for each master control program Pc 1 , Pc 2 , Pc 3 . This ensures that, if the master automation device PLC 1  fails, for example, the slave automation device backup PLC can assume the function of this failed master PLC 1 . This is because the slave automation device backup PLC processes the slave control program Pb 1  corresponding to the master control program Pc 1  of the failed master automation device PLC 1 . 
     On account of the fact that the master automation device PLC 1  has failed, the connection Rs has been interrupted and the slave automation device backup PLC interchanges information with the master automation devices PLC 2 , PLC 3  and the peripheral units Pe 2 , Pe 3 , Pe 4  via a first section R 1  of the connection Rs, but interchanges information with the peripheral unit Pe 1  via a second section R 2  of the connection Rs. 
     In order to explain event-synchronous processing of the control programs, reference is made below to  FIGS. 3-5  which illustrate sequences of temporally asynchronous coupling of one of the master automation devices PLC 1 , PLC 2 , PLC 3 —referred to as the master M below—and the backup automation device backup PLC—referred to as the slave S below. In this context, “event-synchronous processing” means that both the master and the slave run through the same program paths of the respective control program on account of an event, and the runs are effected in a temporally asynchronous manner. 
     In the embodiments according to  FIGS. 3-5 , the master M leads with respect to the control of a technical process and undertakes process control, and the master reads the process input information or process input values from one or more of the peripheral units and makes it/them available to the slave S in a temporally asynchronous manner. The slave S assumes the master function or the role of the master only if the master M fails on account of a fault. 
     The master M processes a program P 1  for controlling the technical process, and the slave S also processes a program P 2  corresponding to this control program P 1 . Both control programs P 1 , P 2  have a multiplicity of processing sections (Va) of different duration, and the control programs P 1 , P 2  are interrupted at the respective beginning and the respective end of each processing section Va. The beginning and end of each processing section Va, which usually comprises a multiplicity of program codes, therefore, represent interruptible program points or breakpoints  0 ,  1 ,  2 , . . . y. If necessary, the respective control program P 1 , P 2  can be interrupted at these points  0 ,  1 ,  2 , . . . y using the master M and the slave S in order to initiate suitable reactions after an event or a process alarm has occurred. 
     Furthermore, the respective control program P 1 , P 2  can be interrupted at these breakpoints  0 ,  1 ,  2 , . . . y so that the master M and the slave S can interchange releases, acknowledgements or other information via the field bus Fb or via the synchronization connection Sv ( FIG. 4 ). After a respective predefinable or predefined interval of time Zi, where i=1, 2, . . . has expired and at the respective time when a breakpoint following the expiry of the respective interval of time Zi occurs, preferably the first breakpoint following the respective interval of time Zi, the master M transmits a release or release signal to the slave S, wherein the release or the release signal indicates to the slave S the processing section Va up to which the slave S can process the control program P 2 . 
     These processing sections Va of the control program P 2  correspond to those which have already been processed by the master M during the processing of the control program P 1 . In one embodiment, it is assumed that, after an interval of time Z 1  has expired, the master M transmits a release F 1  to the slave S at a time t 1  and at a time t 2  when a first breakpoint P 1 _ 6  (breakpoint  6 ) follows the interval of time Z 1 . This release F 1  comprises the information for the slave S indicating that the latter can process its control program P 2  to be processed up to a breakpoint P 2 _ 6  (breakpoint  6 ), wherein the breakpoint P 2 _ 6  of the control program P 2  corresponds to the breakpoint P 1 _ 6  of the control program P 1 . This means that, on the basis of the release, the slave S can process those processing sections Va of the control program P 2  that correspond to the processing sections Va of the control program P 1  up to the time at which the release or the release signal is generated. In this case, it is assumed in the example for the sake of simplicity that the time at which the release is generated corresponds to the time at which the release is transmitted to the slave S. 
     These processing sections Va are, therefore, processed using the slave S in a temporally asynchronous manner with respect to the processing of the corresponding processing sections Va using the master M, and the slave S processes further processing sections Va, after the processing sections Va of the control program P 2  have been processed by the slave S, only when the master M transmits a further release to the slave S. The time at which this breakpoint P 1 _ 6 , P 2 _ 6  (breakpoint  6 ) occurs represents the beginning of an interval of time Z 2  following the interval of time Z 1 . 
     The further temporally asynchronous processing of the control programs P 1 , P 2  is carried out in the described manner. At a time t 3  when a first breakpoint P 1 _A occurs after the expiry of the interval of time Z 2 , the master M transmits a further release F 2  to the slave S, which release indicates to the slave S that the latter can process further processing sections Va up to the breakpoint P 2 _A. These processing sections Va again correspond to those which have already been processed by the master M from the time t 2  to the time t 3 , i.e., up to the breakpoint P 1 _A. This means that the slave S processes the processing sections Va from the time t 2  of the previous release F 1  to the time t 3  of the current release F 2 . The time t 3  when the first breakpoint P 1 _A has occurred after the expiry of the interval of time Z 2  is the beginning of an interval of time Z 3  following the interval of time Z 2 . 
     An event, for example an event in the form of a process alarm, may now occur during an interval of time. In the exemplary embodiment, E is used to denote such an event to which the master M must react in a suitable manner during the interval of time Z 3  at a time t 4  in accordance with the control program P 1 . In this case, the master M does not transmit a release F 3  to the slave S at a time when a breakpoint following the interval of time Z 3  occurs after the interval of time Z 3  but rather at a time t 5  when a breakpoint P 1 _C (breakpoint C) following the occurrence of the event E occurs. This means that the interval of time Z 3  is shortened on account of the event E, and the time t 5  is the beginning of a following interval of time Z 4 . On the basis of the release F 3  transmitted to the slave S, the slave S processes those processing sections Va of the control program P 2  that correspond to those processing sections Va of the control program P 1  that have already been processed by the master M between the times t 3  and t 5 . 
     On account of the event E, the master M processes higher-priority processing sections Va during the interval of time Z 4 . For example, the master M carries out a thread change at the time t 5 , and, after the interval of time Z 4  has expired at the time t 6 , again transmits a release F 4  at a time t 7  when a first breakpoint P 1 _ 12  (breakpoint  12 ) following the interval of time Z 4  occurs. On the basis of this release, the slave S likewise processes processing sections Va up to a breakpoint P 2 _ 12  (breakpoint  12 ) in the control program P 2 , wherein these processing sections Va correspond to the processing sections Va of the control program P 1  between the times t 5  and t 7 , and the slave S likewise carries out a thread change. 
     As explained, the releases from the master M make it possible for the slave S to run through the same “thread mountain” as the master M. This means that the slave S carries out a “thread change” at a point in the control program P 2  corresponding to the point in the control program P 1 . The slave S continues its processing only when requested to do so by the master M using a release. With regard to the processing of the processing sections, the master M processes them in real time like in stand-alone operation or in non-redundant operation and issues releases for corresponding processing sections to be processed by the slave S at regular intervals of time and after the occurrence of events, the master M continuing to process its control program P 1  and not actively waiting for a response from the slave S. With regard to the processing of the corresponding processing sections, the slave S trails the master M and processes the sections on the basis of the issued master releases. 
     Reference is made below to  FIG. 4  which illustrates a transition of the role of master from the master M to the slave S. The master M transmits releases F 5 , F 6 , F 7  to the slave S in the described manner, in which case it is assumed that the master M fails at a time t 8 . 
     On the basis of the releases F 5  to F 7 , the slave S processes the processing sections Va of a control program P 4  up to a breakpoint P 4 _B (breakpoint B), wherein these processing sections Va correspond to those processing sections Va of a control program P 3  that have been processed using the master M up to the breakpoint P 3 _B (breakpoint B). 
     At times te 1 , te 2 , the master M has read access to the peripheral unit Pe within the scope of the processing of the control program P 3 . This means that the master M reads in process input values Ew 1 , Ew 2 , processes them in accordance with the control program P 3  and generates process output values Aw 1 , Aw 2 , which are transmitted to the peripheral unit Pe at times ta 1 , ta 2  by the master M. The master M transmits the process input values Ew 1 , Ew 2  to the slave S (this is indicated in the drawing by means of curved lines L 1 , L 2 ). The transmission is effected together with the releases F 5 , F 7  in order to avoid increasing the communication load between the master M and the slave S while processing the processing sections Va up to these releases F 5 , F 7 . 
     The slave S likewise processes these process input values Ew 1 , Ew 2  in accordance with the control program P 4  and likewise generates the process output values Aw 1 , Aw 2 , which are transmitted to the peripheral unit Pe by the slave S. In this case, it is assumed that the peripheral unit Pe is a “switched” peripheral unit having a primary connection and a secondary connection. The primary connection is intended to receive the process output values from the master M and the secondary connection is intended to receive the process output values from the slave S, wherein the slave S changes over the peripheral unit from the primary connection to the secondary connection if the slave S detects that the master M has failed. 
     As explained, it is assumed that the master M fails at a time t 8 . The slave S detects the failure, for example by virtue of the fact that the master M has not transmitted any sign of life to the slave S via the synchronization connection Sv or the field bus Fb ( FIG. 4 ) during a predefined duration. After the slave S has detected the failure, for example at a time t 9 , the slave S does not immediately assume the role of master. This is because the system state of the slave S differs from that of the master M at time t 9  and a smooth change or transition is, therefore, impossible. 
     At this time t 9 , the slave S has only processed the processing sections Va up to a breakpoint P 4 _ 6  (breakpoint  6 ) and the corresponding processing sections Va of the master M up to a breakpoint P 3 _ 6  (breakpoint  6 ) and, therefore, “lie” in the past. Only after a transition, i.e., after the slave S has processed the processing sections Va released using the release F 7  up to the breakpoint P 4 _B at a time t 10 , does the slave S assume the role of master and thus the control of the technical process, wherein the slave S changes over the peripheral unit from the primary connection to the secondary connection at time t 10 . 
     During this transition, the (previous) slave S still runs through the same thread mountain with path synchronization and processes the same process input values as those processed by the (previous) master M before its failure, wherein the (previous) slave S determines the same process output values as the (previous) master M on the basis of these input values. The transition is ended when the aim of the last release—the processing of the processing sections Va up to the breakpoint P 4 _B in the present example—has been achieved. 
     The duration of the transition substantially corresponds to the duration of the temporal trailing at the “failover” time. In order to keep the temporal trailing at a tolerable degree, every release F 8  to F 12  ( FIG. 3 ) from the master M is then acknowledged by the slave S in an asynchronous manner using respective acknowledgements Q 8  to Q 12  if the slave S has concluded the respective processing. The master M evaluates the number of unacknowledged releases and determines the current trailing of the slave S therefrom. If the temporal trailing is too high or too long, which may result in a loss of redundancy for example, the master M takes suitable measures in order to reduce the temporal trailing or to avoid it becoming too large. 
     For example, the master M can suspend or delay the processing of low-priority threads in response to excessive trailing, wherein the processing of the higher-priority threads requires considerably less than 100% of the computation time. The master M, therefore, has fewer processing sections to run through and generates fewer releases, with the result that the slave S can “catch up”. 
     Although the present invention has been described above with reference to presently preferred embodiments, it is not limited thereto but rather can be modified in a wide variety of ways. In particular, the invention can be altered or modified in multifarious ways without departing from the essence of the invention.