A parallel processing system comprising: a network; a plurality of computers accessibly connected with each other through the network to distributively execute a plurality of processes, wherein each computer of the plurality of computers comprises: an operation processing unit configured to execute one process of the plurality of processes allocated to the computer; a local memory group comprising a first area and a second area; and an I/O control circuit, wherein each of the first area and the second area comprises a plurality of partial areas respectively corresponding to the plurality of computers, wherein the operation processing unit of each computer: is configured to execute the allocated process by using the first area as an access destination during a first period, and to execute the allocated process by using the second area as the access destination during a second period subsequent to the first period, and characterized in that the I/O control circuit comprises: an updating section configured to update data stored in the second area to the latest data of the plurality of computers by carrying out communication among the plurality of computers during the first period so that the same data is stored in the second area of each of the plurality of computers after the update of the data stored in the second area of the computer, and to update data stored in the first area to the latest data of the plurality of computers by carrying out communication among the plurality of computers during the second period so that the same data is stored in the first area of each of the plurality of computers after the update of the data stored in the first area of the computer. a network; a plurality of computers accessibly connected with each other through the network to distributively execute a plurality of processes, wherein each computer of the plurality of computers comprises: an operation processing unit configured to execute one process of the plurality of processes allocated to the computer; a local memory group comprising a first area and a second area; and an I/O control circuit, an operation processing unit configured to execute one process of the plurality of processes allocated to the computer; a local memory group comprising a first area and a second area; and an I/O control circuit, wherein each of the first area and the second area comprises a plurality of partial areas respectively corresponding to the plurality of computers, wherein the operation processing unit of each computer: is configured to execute the allocated process by using the first area as an access destination during a first period, and to execute the allocated process by using the second area as the access destination during a second period subsequent to the first period, and is configured to execute the allocated process by using the first area as an access destination during a first period, and to execute the allocated process by using the second area as the access destination during a second period subsequent to the first period, and characterized in that the I/O control circuit comprises: an updating section configured to update data stored in the second area to the latest data of the plurality of computers by carrying out communication among the plurality of computers during the first period so that the same data is stored in the second area of each of the plurality of computers after the update of the data stored in the second area of the computer, and to update data stored in the first area to the latest data of the plurality of computers by carrying out communication among the plurality of computers during the second period so that the same data is stored in the first area of each of the plurality of computers after the update of the data stored in the first area of the computer. an updating section configured to update data stored in the second area to the latest data of the plurality of computers by carrying out communication among the plurality of computers during the first period so that the same data is stored in the second area of each of the plurality of computers after the update of the data stored in the second area of the computer, and to update data stored in the first area to the latest data of the plurality of computers by carrying out communication among the plurality of computers during the second period so that the same data is stored in the first area of each of the plurality of computers after the update of the data stored in the first area of the computer.