Abstract:
An information processing system includes sets of multiple processors performing processing synchronously. The system includes: a ROM storing a firmware program activating the processors to a synchronized state; a RAM defined by one address map; a firmware copying section copying the firmware program in the ROM to the RAM, on system boot; and a RAM address register storing an address of the RAM and of a copy destination of the firmware program. The system further includes: a RAM address storing section storing the address of the RAM and of the copy destination of the firmware program; a loss-of-synchronism detection section detecting loss of synchronism of the processors; and an address replacing section referring to the RAM address register upon detection of the loss of synchronism, thereby replacing an address for reading the stored firmware program, with the address of the RAM and of the copy destination of the firmware program.

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
       [0001]    This is a continuation application of PCT/JP2009/054305, filed on Mar. 6, 2009. 
     
    
     FIELD 
       [0002]    Embodiments discussed herein are directed to an information processing system, a resynchronization method, and a storage medium storing a firmware program. 
       BACKGROUND 
       [0003]    In an information processing system such as a mission critical server system or the like desired to perform continuous operation, a system failure causes a large effect and thus, there is a demand for high reliability to the extent that the system hardly stops. There is a method of causing two CPUs (processors) to perform synchronous dual operation, in order to improve reliability. In the case of this synchronous dual CPU system, system operation may continue even when a failure occurs in one of the pair of CPUs during synchronous dual operation. Further, it is desirable to improve reliability by restoring the synchronous operation (resynchronization) of the CPU, thereby increasing the time during which the synchronous dual operation is performed. At the time of the resynchronization, downtime is long if the system is rebooted and therefore, it is desirable to carry out the resynchronization without performing a system reboot. 
         [0004]      FIG. 1  is a block diagram that illustrates an example of a configuration of an information processing system. 
         [0005]    An information processing system  10  illustrated in this  FIG. 1  includes three system boards  20 _ 1 ,  20 _ 2  and  20 _ 3 . The system boards  20 _ 1 ,  20 _ 2  and  20 _ 3  include two CPUs  21 _A and  21 _B, two CPUs  21 _C and  21 _D, and two CPUs  21 _E and  21 _F, respectively. Further, the system boards  20 _ 1 ,  20 _ 2  and  20 _ 3  include: main storage RAMs (volatile memories)  22 _ 1 ,  22 _ 2  and  22 _ 3 ; firmware ROMs (non-volatile memories)  23 _ 1 ,  23 _ 2  and  23 _ 3 ; and system control circuits  24 _ 1 , 24 _ 2  and  24 _ 3 , respectively. 
         [0006]    The two CPUs;  21 _A and  21 _B,  21 _C and  21 _D, and  21 _E and  21 _F of the respective system boards  20 _ 1 ,  20 _ 2  and  20 _ 3  are synchronous dual CPUs that perform the same processing in synchronization with each other. 
         [0007]    The main storage RAMs  22 _ 1 ,  22 _ 2  and  22 _ 3  are random-access memories used as working areas in the processing at the CPUs;  21 _A and  21 _B,  21 _C and  21 _D, and  21 _E and  21 _F. These main storage RAMs  22 _ 1 ,  22 _ 2  and  22 _ 3  are defined by a single address map for all the main storage RAMs  22 _ 1 ,  22 _ 2  and  22 _ 3 , to avoid the respective addresses from overlapping one another. This allows any of the system boards  20 _ 1 ,  20 _ 2  and  20 _ 3  to refer to the contents of the main storage RAM in other system board. Therefore, data may be exchanged between the system boards  20 _ 1 ,  20 _ 2  and  20 _ 3 . 
         [0008]    Furthermore, a firmware program for activating the synchronous dual CPUs to bring the CPUs to a synchronous state is stored in the firmware ROMs  23 _ 1 ,  23 _ 2  and  23 _ 3 . 
         [0009]    Incidentally,  FIG. 1  illustrates the three system boards  20 _ 1 ,  20 _ 2  and  20 _ 3 , but the number of the system boards is not limited to three. 
         [0010]    Further, the information processing system  10  illustrated in  FIG. 1  includes three IO control circuits  30 _ 1 ,  30 _ 2 ,  30 _ 3 , and an interconnect  40 . Here, what kind of IO each of these three IO control circuits  30 _ 1 ,  30 _ 2 ,  30 _ 3  controls does not matter. Moreover, the number of the IO control circuits of one information processing system  10  is not limited to three, and may not agree with the number of the system boards. Furthermore, the interconnect  40  transfers signals between the system boards  20 _ 1 ,  20 _ 2 ,  20 _ 3  and the IO control circuits  30 _ 1 ,  30 _ 2 ,  30 _ 3 . 
         [0011]    This information processing unit IO further includes a system management device  50 . This system management device  50  manages this entire information processing system  10 . 
         [0012]    There will be described below a method of performing resynchronization without carrying out a system reboot, in the information processing system configured as in  FIG. 1 . Here, the description will be provided assuming that loss of synchronism has occurred in the CPU  21 _A that is one of the two CPUs  21 _A and  21 _B mounted on the system board  20 _ 1 . 
         [0013]    When a redundancy (loss of synchronism) caused by a failure in the CPU  21 _A is detected in the system control circuit  24 _ 1 , this abnormal CPU  21 _A is separated. The normal CPU  21 _B of the synchronous pair is notified of a halt on the CPU  21 _A by an interrupt notice. Upon receipt of this interrupt notice, the CPUs  21 _A and  21 _B are reset for resynchronization. Here, the CPUs  21 _A and  21 _B in the course of resetting are not allowed to respond to a request such as an interrupt from other CPUs  21 _C,  21 _D,  21 _E and  21 _F, and the IO control circuits  31 _ 1 ,  30 _ 2  and  30 _ 3 . For this reason, an interrupt or the like from any of other CPUs  21 _C,  21 _D,  21 _E and  21 _F, and the IO control circuits  30 _ 1 ,  30 _ 2  and  30 _ 3  to the CPUs  21 _A and  21 _B that are about to be resynchronized is stopped. At this moment, an OS (Operating System) is temporarily suspended. 
         [0014]    The normal CPU  21 _B saves minimum CPU internal information to be used at the time of resynchronization into the main storage RAM  22 _ 1 , and also saves a cache of the CPU into the main storage RAM  22 _ 1 . 
         [0015]    At the time when this processing is completed, the CPUs  21 _A and  21 _B are reset at the same time, and the CPU synchronous operation is resumed. The CPUs  21 _A and  21 _B after reset read firmware from the firmware ROM  23 _ 1 , and after starting the firmware, restore the information saved into the main storage RAM  22 _ 1  to the CPUs  21 _A and  21 _B. Lastly, the halt on the interrupt or the like for the CPUs  21 _A and  21 _B to be resynchronized is released, and the OS is caused to return. 
         [0016]      FIG. 2  is a diagram that illustrates a time sequence in the resynchronization method described above. 
         [0017]    Here, the CPU  21 _A, CPU  21 _B, and other CPUs  21 _C,  21 _D,  21 _E, and  21 _F are referred to as “CPU A”, “CPU B”, and “other CPUs”, respectively. 
         [0018]    When loss of synchronism occurs in the CPU A, firmware processing, namely, prohibition of interrupts, saving of the CPU cache into the main storage RAM, and the like, is performed in the CPU B, and other CPUs are stopped. 
         [0019]    In the CPU A and the CPU B, reset and reading out of firmware are performed and further, the firmware processing such as restoration of the information saved into the main storage RAM and release of the prohibition of interrupts is performed. Subsequently, the CPU A, the CPU B, the other CPUs are all returned to normal operation. 
         [0020]    Here, in particular, reading the firmware out of the firmware ROM consumes the time and thus, it takes a long time to complete the resynchronization. In particular, when a flash ROM is employed as the firmware ROM, since the flash ROM typically operates at a slow-speed frequency (around a few tens of MHz) and has a small bus width, it takes a long time to read the firmware from the flash ROM to start the firmware. 
         [0021]    During the resynchronization, the OS halts and thus, work of a system user is suspended. Further, since a packet in the system is stopped, there arises such a problem that a large value is desired to set timeout of each module. In other words, in a case where a general-purpose module is used, there is a possibility that this timeout may become a value larger than expected and the resynchronization method described above may not be adopted. 
         [0022]    As a way of reducing warm-up time in the resynchronization, there is such a suggestion that the firmware program is moved from the ROM to the RAM on starting, and the firmware program is read from the RAM on restarting. In this suggestion, switching between the RAM and the ROM is performed by an end selector. 
         [0023]    However, in the case of an ordinary synchronous dual CPU configuration, the firmware ROM is provided for each CPU or each CPU group, whereas the main storage RAM is defined by the single address map to avoid overlap among addresses in the system as a whole, as described above. In such a configuration, if an attempt is made to adopt the conventionally proposed way in which the firmware program is moved to the RAM, it is desirable to prepare a dedicated RAM for each ROM separately, increasing the cost. Further, there is a case where the firmware ROM is used not only for reading out, but also for writing to save error information or retain configuration information. The error information and the like may not be saved into a volatile RAM. Therefore, when switching between the ROM and the RAM is performed in an end part as in the conventional proposal, exclusive control between CPUs is desired, making the control complicated. 
         [0024]    Furthermore, conventionally, there have been proposed: to cancel redundancy when one of synchronous dual CPUs fails, and perform operation only with the other CPU; and to carry out a transfer of processing within a short time by copying modified data in a system currently in use to a standby system. However, keeping the operation with the other CPU alone may not avoid a deterioration in reliability, and the proposal of copying the modified data in the system currently in use to the standby system is not directly related to the loss of synchronism. 
         [0025]    For example, refer to Japanese Laid-open Patent Publications No. 63-268030, No. 8-235125, No. 7-200334, and No. 2008-140080 for reference. 
         [0026]    A challenge in an information processing system, a resynchronization method and a firmware program of Japanese Laid-open Patent Publication No. 2008-140080 is to shorten the timeout at the time of occurrence of loss of synchronism and perform restoration to a state with high reliability, in the information processing system mounted with two or more pairs of dual CPUs operating synchronously. 
       SUMMARY 
       [0027]    According to an aspect of the invention, an information processing system includes a plurality of sets of two or more multiple CPUs that perform processing in synchronization with each other. The information processing system further includes a ROM, a RAM, a firmware copying section, a RAM address register, a RAM address storing section, a loss-of-synchronism detection section, and an address replacing section. The ROM stores a firmware program activating the multiple CPUs to a state in which the multiple CPUs are synchronized with each other. The RAM is defined by one address map as a whole. The firmware copying section copies the firmware program stored in the ROM to the RAM, on system boot. In the RAM address register, an address of the RAM and of a copy destination to which the firmware program is copied is stored. The RAM address storing section stores the address of the RAM and of the copy destination to which the firmware program is copied by the firmware copying section, in the RAM address register. The loss-of-synchronism detection section detects loss of synchronism of the multiple CPUs. The address replacing section refers to the RAM address register in response to the loss of synchronism being detected by the loss-of-synchronism detection section, thereby replacing an address for reading the firmware program stored in the ROM, with the address of the RAM and of the copy destination of the firmware program. 
         [0028]    The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims. 
         [0029]    It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0030]      FIG. 1  is a block diagram that illustrates an example of a configuration of an information processing system; 
           [0031]      FIG. 2  is a diagram that illustrates a time sequence in the resynchronization method described above; 
           [0032]      FIG. 3  is a block diagram that illustrates a configuration of an information processing system in the first embodiment of the present case; 
           [0033]      FIGS. 4(A) and 4(B)  are a block diagram that illustrates a configuration of an information processing system according to a second embodiment of the present case; 
           [0034]      FIGS. 5(A) and 5(B)  are a diagram that illustrates an operating sequence of the firmware and the circuit in the second embodiment illustrated in  FIG. 4 ; 
           [0035]      FIG. 6  is a block diagram that illustrates a configuration of an information processing system according to the third embodiment of the present case; 
           [0036]      FIG. 7  is a block diagram that illustrates a configuration of an information processing system according to the fourth embodiment of the present case; 
           [0037]      FIG. 8  is a diagram sequentially illustrating operations when loss of synchronism occurs in the information processing system of the fourth embodiment illustrated in  FIG. 7 ; 
           [0038]      FIG. 9  is a diagram sequentially illustrating operations when loss of synchronism occurs in the information processing system of the fourth embodiment illustrated in  FIG. 7 ; 
           [0039]      FIG. 10  is a diagram sequentially illustrating operations when loss of synchronism occurs in the information processing system of the fourth embodiment illustrated in  FIG. 7 ; 
           [0040]      FIG. 11  is a diagram sequentially illustrating operations when loss of synchronism occurs in the information processing system of the fourth embodiment illustrated in  FIG. 7 ; 
           [0041]      FIG. 12  is a diagram sequentially illustrating operations when loss of synchronism occurs in the information processing system of the fourth embodiment illustrated in  FIG. 7 ; and 
           [0042]      FIG. 13  is a diagram sequentially illustrating an operation sequence of each section in the information processing system of the fourth embodiment illustrated in  FIGS. 8-12 . 
       
    
    
     DESCRIPTION OF EMBODIMENTS 
       [0043]    Embodiments of the present case will be described below. Incidentally, for a first embodiment to be described below,  FIG. 1  will be used as an overall block diagram. However, the internal configurations of the system control circuits  24 _ 1 ,  24 _ 2  and  24 _ 3  are slightly different. 
         [0044]      FIG. 3  is a block diagram that illustrates a configuration of an information processing system in the first embodiment of the present case. However, in order to avoid complication of illustration, this  FIG. 3  illustrates two of the three system boards illustrated in  FIG. 1 . Further, as to the two system control circuits of these two system boards, only elements used for the resynchronization are illustrated. Furthermore, here, illustration of the interconnect  40  depicted in  FIG. 1  is omitted, and slave request processing circuits included in the respective two system control circuits  24 _ 1  and  24 _ 2  are indicated collectively by one block. 
         [0045]    In this  FIG. 3 , dual processing circuits  241 _ 1  and  241 _ 2  are illustrated as elements of the system control circuits  24 _ 1  and  24 _ 2  of the system boards  20 _ 1  and  20 _ 2  each illustrated as one block in  FIG. 1 , respectively. Further, ROM-address detecting circuits  242 _ 1  and  242 _ 2  and RAM address registers  243 _ 1  and  243 _ 2  are also illustrated as elements of the system control circuits  24 _ 1  and  24 _ 2 , respectively. Furthermore, as the elements, conversion permitting flag registers  244 _ 1  and  244 _ 2 , gate circuits  345 _ 1  and  345 _ 2  and selection circuits  246 _ 1  and  246 _ 2  are also illustrated. In addition, a slave request processing circuit  247  illustrated as one integral block for the two system control circuits  24 _ 1  and  24 _ 2  is also illustrated. 
         [0046]    The dual processing circuits  241 _ 1  and  241 _ 2  perform operation for dual synchronous processing of the CPUs  21 _A and  21 _B, and  21 _C and  21 _D, respectively. In other words, these dual processing circuits  241 _ 1  and  241 _ 2  serve as a switch to select an address from one CPU of addresses output from two CPU bus interfaces and the two CPUs. Moreover, these dual processing circuits  241 _ 1  and  241 _ 2  perform processing such as detection of loss of synchronism in the two CPUs, respectively. 
         [0047]    Further, the ROM-address detecting circuits  242 _ 1  and  242 _ 2  are circuits that detect whether the addresses output from the dual processing circuits  241 _ 1  and  241 _ 2  agree with firmware program storage addresses of the firmware ROMs  23 _ 1  and  23 _ 2 . 
         [0048]    Furthermore, the RAM address registers  243 _ 1  and  243 _ 2  are registers in which when the firmware programs in the firmware ROMs  23 _ 1  and  23 _ 2  are copied to the main storage RAMs  22 _ 1  and  22 _ 2 , the addresses of the copy destinations are stored. The details will be described later. 
         [0049]    Further, in each of the conversion permitting flag registers  244 _ 1  and  244 _ 2 , a conversion permitting flag to allow conversion of the address of the firmware ROM into the address of the main storage RAM is stored. Each of these conversion permitting flag registers  244 _ 1  and  244 _ 2  is equivalent to an example of the copy flag register of the present case. 
         [0050]    When satisfying the following two conditions (a) and (b) at the same time, the gate circuits  245 _ 1  and  245 _ 2  output RAM address selection signals for the conversion into the addresses of the main storage RAMs  22 _ 1  and  22 _ 2 . 
         [0051]    (a) The conversion permitting flags are stored in the conversion permitting flag registers  244 _ 1  and  244 _ 2 . 
         [0052]    (b) The storage addresses of the firmware programs in the firmware ROMs  23 _ 1  and  23 _ 2  are detected by the ROM-address detecting circuits  242 _ 1  and  242 _ 2 . 
         [0053]    Normally, the selection circuits  246 _ 1  and  246 _ 2  directly output the addresses received from the dual processing circuits  241 _ 1  and  241 _ 2 . However, upon receipt of the RAM address selection signals from the gate circuits  245 _ 1  and  245 _ 2 , the selection circuits  246 _ 1  and  246 _ 2  output the addresses of the main storage RAMs  22 _ 1  and  22 _ 2  stored in the RAM address registers  243 _ 1  and  243 _ 2 . 
         [0054]    Here, at the time of starting to the first initial state in which this information processing system is powered on, the conversion permitting flag is reset without being stored in each of the conversion permitting flag registers  244 _ 1  and  244 _ 2 . For this reason, even when the firmware program storage addresses of the firmware ROMs  23 _ 1  and  23 _ 2  are detected by the ROM-address detecting circuits  242 _ 1  and  242 _ 2 , the RAM address selection signal is not output from each of the gate circuits  245 _ 1  and  245 _ 2 . The identical firmware programs are stored in the firmware ROMs  23 _ 1  and  23 _ 2 . Therefore, upon power-on, the firmware program is read from either one of the firmware ROMs. Here, the firmware program is assumed to be read from the firmware ROM  23 _ 1 . When the address of the firmware ROM  23 _ 1  is output from the dual processing circuit  241 _ 1 , the address of the firmware ROM  23 _ 1  is directly output from the selection circuit  246 _ 1 , and input into the firmware ROM  23 _ 1  via the slave request processing circuit  247 . As a result, the firmware program is read from the firmware ROM  23 _ 1 . This firmware program performs initialization including the synchronization, in the two CPUs  21 _A and  21 _B and the two CPUs  21 _C and  21 _D. In this initialization, the firmware program read from the firmware ROM  23 _ 1  is copied to the main storage RAM  22 _ 1  by the operation of the firmware program. In addition, the RAM address of the copy destination of the main storage RAM  22 _ 1  is stored in each of the RAM address registers  243 _ 1  and  2432 . Further, the conversion permitting flag is set to each of the conversion permitting flag registers  244 _ 1  and  244 _ 2 . 
         [0055]    It is to be noted that as described above, the same firmware programs are stored in the firmware ROMs  23 _ 1  and  23 _ 2  and thus, reading the firmware program from either one of the firmware ROMs is sufficient. Further, even when loss of synchronism occurs in any of the system boards, the firmware program may be read from the RAM that is the copy destination, in the resynchronization, and making any one of the RAMs to serve as the copy destination is sufficient. 
         [0056]    However, the RAM address of the copy destination is stored in all the RAM address registers  243 _ 1  and  243 _ 2 , and the conversion permitting flag also is set in all the conversion permitting flag registers  244 _ 1  and  244 _ 2 . 
         [0057]    After such initialization is performed, various kinds of processing are performed by the dual operation in each of the dual CPUs. 
         [0058]    Suppose loss of synchronism has occurred in the CPU  21 _A during execution of the processing. Then, the loss of synchronism is detected by the dual processing circuit  241 _ 1 . In this case, as described above with reference to  FIG. 2 , the resynchronization processing is executed by the main operation of the other CPU  21 _B. In this resynchronization processing, the address of a firmware program storage area of the firmware ROM  23 _ 1  is output from the CPU  21 _B to read the firmware program from the firmware ROM  23 _ 1 , and the address output from the CPU  21 _B is output in the dual processing circuit  241 _ 1 . At this moment, the firmware program storage address of the firmware ROM  23 _ 1  which is output from the dual processing circuit  241 _ 1  is detected in the ROM-address detecting circuit  242 _ 1 . Further, the conversion permitting flag is set in the conversion permitting flag register  244 _ 1 . For this reason, a RAM address selection signal is output from the gate circuit  245 _ 1 . Upon receipt of the RAM address selection signal, the selection circuit  246 _ 1  outputs the address of the main storage RAM  22 _ 1  stored in the RAM address register  243 _ 1 , in place of the address of the firmware ROM  23 _ 1  output from the dual processing circuit  241 _ 1 . In other words, the CPU  21 _B outputs the address of the firmware ROM  23 _ 1 , which is replaced with the address of the main storage RAM  22 _ 1  in the selection circuit  246 _ 1 , and this address of the main storage RAM  22 _ 1  is output. For this reason, the firmware program copied to the main storage RAM  22 _ 1  is read out. In this way, in the CPUs  21 _A and  21 _B, the resynchronization processing is performed by the firmware program read from the main storage RAM  22 _ 1 . 
         [0059]    Normally, the access speed of the main storage RAM  22 _ 1  is much higher than that of the firmware ROM  23 _ 1  and therefore, the time for the “firmware readout” illustrated in  FIG. 2  is greatly reduced. For this reason, high-speed resynchronization may be carried out, allowing short-time returning to the state with high reliability. 
         [0060]    Further, in the case of the configuration illustrated in this  FIG. 3 , a large increase in cost such as providing ROMs and RAMs separately in a one-to-one relationship may be avoided and thus, high-speed resynchronization is obtained by merely making a slight modification to a conventional circuit configuration. 
         [0061]      FIGS. 4(A) and 4(B)  coupled with each other by connecting the same references ((a), (b), . . . , (f)) respectively are a block diagram that illustrates a configuration of an information processing system according to a second embodiment of the present case. This second embodiment also is the same as  FIG. 1  in terms of overall configuration, but  FIG. 4  illustrates only a configuration of one system board  20 _ 1  to avoid complication of illustration. A system control circuit  24 _ 1  of the system board  20 _ 1  illustrated in  FIG. 4  includes two CPU bus interfaces  241   a  and  241   b  corresponding to two CPUs  21 _A and  21 _B, respectively. Further, here, two bus error detectors  241   c  and  241   d , and an error management section  241   e , and a switch  241   f  are provided. As for the CPU bus interfaces  241   a  and  241   b , the bus error detectors  241   c  and  241   d , the error management section  241   e , and the switch  241   f  combined correspond to each of the dual processing circuits  241 _ 1  and  241 _ 2  illustrated in  FIG. 3 . The bus error detectors  241   c  and  241   d  detect an error in address or data, namely, loss of synchronism, which is output from each of the CPUs  21 _A and  21 _B via the CPU bus interfaces  241   a  and  241   b . A detection result obtained by each of the bus error detectors  241   c  and  241   d  is reported to the error management section  241   e . When the two CPUs  21 _A and  21 _B operate synchronously, the error management section  241   e  changes the switch  241   f  so that the address and data from either one of these two CPUs  21 _A and  21 _B (for example, the CPU  21 _A) is output. 
         [0062]    Here, when loss of synchronism is detected, the error management section  241   e  changes the switch  241   f  so that the address and data are output from the other CPU (for example, the CPU  21 _B) which is not the CPU (for example, the CPU  21 _A) in which the loss of synchronism has occurred. 
         [0063]    The address output from the switch  241   f  is set in an address queue  251  configured of a FIFO (first-in, first-out) register in which address or data (here, address) arriving first is output first. Subsequently, via the interconnect  40 , the address is input to a slave request processing circuit  247 _ 1 , when the address is the address of the main storage RAM  22 _ 1 , the firmware ROM  23 _ 1 , or the register managed by this system board  20 _ 1 . In the slave request processing circuit  247 _ 1 , it is determined whether the input address is the address of the main storage RAM  22 _ 1 , the address of the firmware ROM  23 _ 1 , or the address of the register. When the input address is the address of the main storage RAM  22 _ 1 , the address is stored in a buffer  247   b  or a buffer  247   a  each configured by FIFO, depending on whether the address is a command for writing data to the main storage RAM  22 _ 1  or a command for readout from the main storage RAM  22 _ 1 . Alternatively, when it is determined that the address is the address of the firmware ROM  23 _ 1  in the slave request processing circuit  247 _ 1 , the address is stored in a buffer  247   c  or a buffer  247   d , depending on whether the address is a command for data writing or a command for data readout. The firmware ROM  23 _ 1  is not read-only, in which a log at the time of occurrence of an error, system information and the like are written and thus, the firmware ROM  23 _ 1  also has a configuration for writing. 
         [0064]    Further, when the address is the address indicating the register, the address is stored in a buffer  247   f  for writing or a buffer  247   e  for reading, depending on whether the address is a command for writing or a command for reading. 
         [0065]    Furthermore, when the data for writing is output from the switch  241   f , the data is temporarily stored in a write data buffer  252  configured by FIFO. Subsequently, when the data is to be written in the main storage RAM  22 _ 1 , the data is stored in the buffer  247   b  via the interconnect  40 . Similarly, when the data is to be written in the firmware ROM  23 _ 1 , the data is stored in the buffer  247   c , and when the data is to be written in the register, the data is stored in the buffer  247   e.    
         [0066]    When the data and the address are both present in the buffer  247   b , a RAM controller  261  writes the data at the address of the main storage RAM  22 _ 1 . At the same time, when the data and the address are both present in the buffer  247   c , a ROM controller  262  writes the data at the address of the firmware ROM  23 _ 1 . Further, when the data and the address are both present in the buffer  247   c , a register RW control circuit  263  writes the data in the buffer or the like identified by the address. 
         [0067]    Furthermore, when the address for reading is stored in the buffer  247   a  by the slave request processing circuit  247 _ 1 , data is read out from that address of the main storage RAM  22 _ 1  into the RAM controller  261 . The data read out is once stored in the buffer  247   a  and then, temporarily stored in a read data buffer  253  via the interconnect  40 . Subsequently, the data is transmitted to the CPUs  21 _A and  21 _B via the CPU bus interfaces  241   a  and  241   b . Similarly, when the read address is stored in the buffer  247   d , data is read out by the ROM controller  262  from this read address of the firmware ROM  23 _ 1 . The data read out is transmitted to the CPUs  21 _A and  21 _B via the buffer  247   d , the interconnect  40 , the read data buffer  253 , and the CPU bus interfaces  241   a  and  241   b . Similarly, when the address is stored in the buffer  247   f , data is read out by the register RW control circuit  263  from the register or the like identified by the address stored in the buffer  247   f . This data read out is transmitted to the CPUs  21 _A and  21 _B via the buffer  247   f , the interconnect  40 , the read data buffer  253 , and the CPU bus interfaces  241   a  and  241   b.    
         [0068]    A RAM base address register  264  is an element corresponding to the RAM address register  243 _ 1  of the first embodiment illustrated in  FIG. 3 . When starting the synchronization upon power-on, the firmware program stored in the firmware ROM  23 _ 1  is copied to the main storage RAM  22 _ 1 , but in the RAM base address register  264 , the address of a copy destination of the main storage RAM  22 _ 1  is stored. However, whether the address is the address of the firmware ROM  23 _ 1  or the address of the main storage RAM  22 _ 1  is distinguished by higher order bits, and in the RAM base address register  264 , the address on the higher-order-bit side of the main storage RAM  22 _ 1  is stored. 
         [0069]    Further, here, there is provided a ROM-address detecting circuit  266  that determines a match or a mismatch between a ROM base address stored in a ROM-base-address storage section  265  and the address output from the switch  241   f . This ROM-address detecting circuit  266  is an element corresponding to the ROM-address detecting circuit  242 _ 1  in the first embodiment illustrated in  FIG. 3 . However, in the ROM-base-address storage section  265  of the second embodiment in  FIG. 4 , only a part of higher-order-bit side of the address of the firmware ROM  23 _ 1  indicating a firmware program storage area is stored. Therefore, the ROM-address detecting circuit  266  determines a match or a mismatch for the address on the higher-order-bit side of the firmware ROM  23 _ 1 . 
         [0070]    In the address queue  251 , the write address or the read address is stored, but as for the lower-order-bit side of the address, the lower-order-bit side of the address output from the switch  241   f  is directly stored. As to the higher-order-bit side, the higher-order-bit side of the address output from the switch  241   f  or the higher-order-bit side of the address of the RAM  22 _ 1  stored in the RAM base address register  264  is output, depending on selection by a selector  268 . The operation after the address is stored in the address queue  251  has been described above. 
         [0071]    A copy flag register  269  is a register to be reset at the time of reset in this system board  20 _ 1 . In this copy flag register  269 , a copy flag is set at a stage where the firmware program in the firmware ROM  23 _ 1  is copied to the RAM  22 _ 1 , and the address of a copy destination is stored in the RAM base address register  264 . 
         [0072]    In an address-replacement permitting flag register  271 , an address-replacement permitting flag is set at the time of reset in this system board  20 _ 1 , in response to determination that a copy flag is stored in a copy flag register  267  by an AND gate  270 . In other words, in this address-replacement permitting flag register  271 , the address-replacement permitting flag is set at the time of reset for the resynchronization after occurrence of loss of synchronism between the two CPUs  21 _A and  21 _B. 
         [0073]    A resynchronization reset control section  272  is requested to carryout resynchronization reset. In response to the request of the resynchronization reset, the resynchronization reset control section  272  instructs the CPUs  21 _A and  21 B to carry out the reset. Then, the CPUs  21 _A and  21 _B perform reset processing for resynchronization, including reading and running of the firmware program. Then, in this resynchronization reset processing, when the address output from the switch  241   f  is the address of the firmware ROM  23 _ 1 , at which the firmware program is stored, the address is replaced with the address of the copy destination of the firmware program, of the main storage RAM  22 _ 1 . Therefore, the firmware program is read from the main storage RAM  22 _ 1  at a high speed, and the resynchronization is performed in a short time. 
         [0074]      FIGS. 5(A) and 5(B)  coupled with each other by connecting the same references ((a), (b), . . . , (e)) respectively are a diagram that illustrates an operating sequence of the firmware and the circuit in the second embodiment illustrated in  FIG. 4 . 
         [0075]    Here, “hardware”, “OS”, “CPU firmware” and “system firmware” are illustrated separately, and the operation of each part is depicted. Here, the “CPU firmware” and “the system firmware” are both components of the firmware program stored in the firmware ROM. 
         [0076]    Here, at first, a system firmware creates a single address map for all the main storage RAMs  22 _ 1 ,  22 _ 2 , and  22 _ 3  of the system boards across this entire information processing system so as to avoid overlaps among addresses, and sets the address in each of the main storage RAMs  22 _ 1 ,  22 _ 2  and  22 _ 3 . 
         [0077]    Next, in the system firmware, copying the firmware program to the main storage RAM is controlled, and the firmware program on the firmware ROM in the hardware is copied to the main storage RAM. Here, as described in the first embodiment, copying of the firmware program to the main storage RAM is sufficient if the firmware program is copied to the main storage RAM of either one of the main storages RAM of each system board. 
         [0078]    After this copying is finished, “register setting” is performed. In other words, here, the address of the copy destination in the main storage RAM to which the firmware program is copied is stored in the RAM base address register  264  (see  FIG. 4 ), and the copy flag is set in the copy flag register  269  (see  FIG. 4 ). 
         [0079]    When an error occurs in the CPU  21 _A (CPU A), a platform interrupt takes place, and processing of suspending the OS is performed by the CPU  21 _B (CPU B). Subsequently, the CPU firmware is notified of the occurrence of the platform interrupt, a request to carry out error handling is provided from the CPU firmware to the system firmware, and the error handling is performed in the system firmware. Here, the occurrence of the error due to the loss of synchronism is recognized, and it is determined that redundancy recovery is desired. In this redundancy recovery, blocking access from other CPU or IO to the dual CPUs (CPU A/CPU B) including the CPU A in which the loss of synchronism has occurred is instructed, and thereby access blocking is performed on the hardware. Further, the system firmware is instructed to save a context on the cache of the CPU A/CPU B, and context saving operation is controlled in the CPU firmware, and the context is saved to the main storage RAM. This context is data to continue, after the resynchronization, processing that had been handled by the CPU A/CPU B. 
         [0080]    Next, the reset of the CPU is instructed by the system firmware, and the resynchronization reset processing of the CPU A/CPU B is performed. In this resynchronization reset processing, the CPU firmware is read from the main storage RAM and thereby the CPU is set, and further, the system firmware is read from the main storage RAM and thereby the system setting is performed. At the time of this system setting, an error in synchronism is recognized, and reading of the context is instructed. Upon receipt of this instruction, the CPU firmware performs context reading processing, and the context saved into the main storage RAM on the hardware is read out. Subsequently, in the system control circuit firmware, release of blocking the access from others is instructed, and operation of releasing blocking of access from the other CPU and IO is performed on the hardware. Subsequently, an OS recovery is requested from the system firmware, and the OS recovers from a platform interrupt via the error handling by the CPU firmware. 
         [0081]    As a result, the CPUs A and CPU B are synchronized again, and the processing performed before the loss of synchronism occurs is continued. 
         [0082]    Next, a third embodiment of the present case will be described. 
         [0083]    In this third embodiment and a fourth embodiment to be described later, when loss of synchronism occurs in a CPU, there is performed processing of moving, to the other CPU, information to carry on the processing performed in the CPU before execution of reset for resynchronization. Processing of leaving continuation of the processing to the other CPU is performed by this processing. The resynchronization may be performed after the information is moved to the other CPU, and returning to a state with high reliability may be performed by merely stopping the OS for an extremely short time. 
         [0084]      FIG. 6  is a block diagram that illustrates a configuration of an information processing system according to the third embodiment of the present case. 
         [0085]    In this  FIG. 6 , for the following description, firmware or OS/application are taken out and illustrated clearly. These firmware and OS/application are programs each carrying out the following operation by being executed in a CPU. 
         [0086]    In the information processing system of the third embodiment illustrated in this  FIG. 6 , one system board includes two sets of dual CPUs  21 _A and  21 _B, and  21 _C and  21 _D. 
         [0087]    Here, suppose loss of synchronism has occurred in the CPU  21 _B (CPU B). In that case, the following processing is performed. 
         [0088]    1) The loss of synchronism in the CPU B is detected by the dual processing circuit  241 _ 1  controlling the dual CPUs including the CPU B in which the loss of synchronism has occurred, of the dual processing circuits  241 _ 1  and  241 _ 2  provided for each pair of the dual CPUs. When the loss of synchronism in the CPU B is detected by the dual processing circuit  241 _ 1 , an error notice is sent to an error handling section  274 . After detecting the loss of synchronism in the CPU B, the dual processing circuit  241 _ 1  performs switching to select the address of the CPU A, so that the CPU A alone continues the processing. 
         [0089]    2) The error handling section  274  provides the system management device  50  with an interrupt, by setting a bit representing the fact that one of the dual CPUs is retracted. The system management device  50  recognizes the one of the dual CPUs being retracted, by using the bit being set. 
         [0090]    3) The system management device  50  sets an interrupt register  272  of a system control circuit  24 . 
         [0091]    4) The system control circuit  24  interrupts the CPU by setting of the interrupt register  272 . 
         [0092]    5) In response to this interrupt, the CPU A calls the firmware. 
         [0093]    6) The firmware performs processing for separating the CPU A/CPU B from this information processing system. 
         [0094]    7) The firmware notifies the OS of separation of the CPU A/CPU B. 
         [0095]    8) The firmware sets a CPU reset register  271  of the system control circuit  24 . 
         [0096]    9) In response to this setting, the CPU reset register  271  resets the CPU A/CPU B. 
         [0097]    10) In response to this reset, initialization is performed by the CPU A/CPU B. 
         [0098]    11) Upon completion of the initialization, an interrupt register  273  of the system control circuit is set by the CPU A/CPU B. 
         [0099]    12) The system control circuit  24  provides the system management device  50  with an interrupt to indicate the completion of reset. 
         [0100]    13) The system management device sets an interrupt register  275 . 
         [0101]    14) In response to this setting, the interrupt register  275  provides the CPU C/CPU D with an interrupt, and in response to this interrupt, the CPU C/CPU D notifies the OS that the resource of the CPU A/CPU B has increased. 
         [0102]    By executing the above method, the OS is stopped only for a shot time to separate the CPU A/CPU B, and the OS stop time during the resynchronization is reduced. 
         [0103]    Incidentally, the processing of this third embodiment is effective in a case where the OS or application has a function of supporting dynamic deletion and dynamic addition of the CPU. When this function is not supported, it is effective to perform dynamic replacement of CPU as described below in a fourth embodiment. 
         [0104]      FIG. 7  is a block diagram that illustrates a configuration of an information processing system according to the fourth embodiment of the present case. 
         [0105]    The block diagram of the information processing system illustrated in this  FIG. 7  is similar to that of the information processing system illustrated in  FIG. 1 , and provided with the same reference characters as those in  FIG. 1 . A point different from  FIG. 1  is that a system board  20 _ 3  that is one of three system boards  20 _ 1 ,  20 _ 2  and  20 _ 3  is in an off-line state of being logically separated from this information processing system  10  in an initial stage illustrated in this  FIG. 7 . Further, in this  FIG. 7 , an OS is clearly illustrated for subsequent description. This OS performs operation along the following description by being executed in the CPU. 
         [0106]    Furthermore,  FIG. 8  to  FIG. 13  are diagrams sequentially illustrate operations when loss of synchronism occurs in the information processing system of the fourth embodiment illustrated in  FIG. 7 . 
         [0107]    As illustrated in  FIG. 8 , suppose an error (loss of synchronism) has occurred in a CPU B. At this moment, following each operation is executed. 
         [0108]    The error (loss of synchronism) of the CPU B is detected by a system control circuit  24 _ 1  responsible for the CPU B in which the loss of synchronism has occurred, and the occurrence of the error is reported to a system management device  50  ( FIG. 8 ). 
         [0109]    2) Upon receipt of the report on the occurrence of the error, the system management device  50  starts the system board  20 _ 3  ( FIG. 8 ). 
         [0110]    3) When the staring of the system board  20 _ 3  is completed, the system management device  50  provides an interrupt to the CPU A that is a CPU in normal operation paired with the CPU B in which the loss of synchronism has occurred. The CPU A sets each control circuit so that requests from other CPU and IO are stopped temporarily. At this moment, the OS halts ( FIG. 9 ). 
         [0111]    4) Information for restarting the OS of the CPU A is copied to CPU E/CPU F of the system board  20 _ 3  via a main storage RAM  22 _ 1  of the system board  20 _ 1 . When the copying is finished, the CPU A provides the CPU E/CPU F with a CPU ID for recognizing the CPU A. In exchange for this, the CPU_A receives a CPU ID used as the ID of the CPU E/CPU F till then, from the CPU E/CPU F. Further, in order to correctly send a packet from the IO to the CPU after the replacement, the setting of the new CPU ID is reflected on each of IO control circuits  31 _ 1 ,  30 _ 2  and  30 _ 3  ( FIG. 10 ). 
         [0112]    5) The setting of stopping the issuance of the requests from other CPU and IO performed in the above 3) is released, and the OS recovers ( FIG. 11 ). 
         [0113]    6) After the above 5) is completed, the system management device  50  is provided with an interrupt, and the system board  20 _ 1  is separated logically ( FIG. 12 ). Subsequently, in the system board  20 _ 1 , reset processing is performed, or the system board  20 _ 1  is replaced. 
         [0114]    In the case of this third embodiment, the OS is halted during the time from 4) to 5), i.e., for an extremely a short time. 
         [0115]      FIGS. 13(A) and 13(B)  coupled with each other by connecting the same references ((a), (b), . . . , (j)) respectively are a diagram that illustrates an operating sequence of each part of the information processing system in the fourth embodiment illustrated in  FIG. 8  through  FIG. 12 . Here, the system board  20 _ 1  and the system board  20 _ 3  illustrated in  FIG. 8  are expressed as a system board  1  and a system board  3 , respectively. 
         [0116]    When occurrence of a loss-of-synchronism error in the CPU B of the system board  1  is detected on hardware, a platform interrupt is given to the OS, and suspend processing of the OS is performed by the CPU A. Further, error handling of the platform is raised to a CPU firmware of the system board  1  and furthermore, the error handling is performed by a system firmware of the system board  1 . In this error handling, the error is reported to the system management device  50 , and board replacement control is performed by the system management device  50 . In other words, here, the system board  3  on standby before that moment is activated, initialization of the CPU E/CPU F is performed by the CPU firmware and further, system initialization on the system board  3  is performed by the system firmware. After this initialization, the system board  3  enters a loop state (a wait state) for a while. The system management device  50  further sets an interrupt flag in an interrupt register. Then, the platform interrupt by setting the flag is accepted by the CPU A, and the OS suspends. Interrupt handling by the platform interrupt is performed in the CPU firmware of the system board  1 , and the processing is transferred to the system firmware, and a halt of other CPU and IO is instructed by the system firmware. On the hardware, in response to this instruction, requests from other CPU and IO are stopped. Further, context saving processing is performed in the system firmware of the system board  1 , and the context is saved into the main storage RAM. Furthermore, in the system firmware of the system board  1 , exchange of CPU IDs between the CPU A and the CPU E/CPU F is performed, a new CPU ID is set in an interrupt destination setting register in each control circuit. In addition, the CPU ID received from the system board  3  is set by the CPU firmware of the system board  1  and then, the system board  1  is stopped, and replacement/standby or the like is performed. 
         [0117]    In the system board  3 , the CPU E/CPU F in the loop state (wait state) returns to an active state, and the CPU ID received from the system board  1  is set as the CPU ID of the CPU E/CPU F. Further, in the system firmware of the system board  3 , reading of the context is instructed, and context reading processing is performed by the CPU firmware of the system board  3 , and the reading of the context saved into the main storage RAM is performed. In the system firmware of the system board  3 , recovery of other CPU and IO is further instructed, and recovery processing of other CPU and IO is performed in order to accept requests from other CPU and IO again. Further, the OS recovers. 
         [0118]    According to the fourth embodiment described above, the OS may be stopped only for a short time until the operation of the system board  1  is transferred to the system board  3  and thus, the stop time after the occurrence of the loss of synchronism may be extremely short. 
         [0119]    As described above, according to each embodiment described above, the stop time after the loss of synchronism may be short. Further, the timeout may not be set as a long time and thus, general-purpose components may be used. 
         [0120]    All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although the embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.