A parallel computing system including a plurality of nodes each provided with a synchronization device operable to establish synchronization by awaiting incoming data and transmitting the data when all awaited data is received, each synchronization device comprising: a receiver configured to receive data from at least two synchronization devices establishing synchronization, and extract synchronization information and register selection information from the received data; a transmitter configured to transmit data in the form of packets to each of the at least two synchronization devices from among a plurality of synchronization devices; a first receiving state register configured to store the extracted synchronization information; a second receiving state register configured to store the extracted synchronization information; and a controller configured to store the extracted synchronization information into the first receiving state register and the second receiving state register alternately based on the register selection information, and control the transmitter to transmit data including the register selection information to each of the at least two synchronization devices when all awaited data is received and the extracted synchronization information is stored in one of the first receiving state register and the second receiving state register, wherein the transmitter is configured to transmit one packet to a synchronization device of another node and another packet to the receiver of its own node via a loop path, wherein each packet comprises a sequence number, wherein the sequence number is register selection information for selecting one of the two receiving state registers, and wherein the controller comprises: a check circuit configured to check consistency between the sequence number of the received packet and the number of packets received, wherein when the results of the check indicate that the sequence number is consistent with the number of packets, one of the two receiving state registers is selected in accordance with a least significant bit of the sequence number. a receiver configured to receive data from at least two synchronization devices establishing synchronization, and extract synchronization information and register selection information from the received data; a transmitter configured to transmit data in the form of packets to each of the at least two synchronization devices from among a plurality of synchronization devices; a first receiving state register configured to store the extracted synchronization information; a second receiving state register configured to store the extracted synchronization information; and a controller configured to store the extracted synchronization information into the first receiving state register and the second receiving state register alternately based on the register selection information, and control the transmitter to transmit data including the register selection information to each of the at least two synchronization devices when all awaited data is received and the extracted synchronization information is stored in one of the first receiving state register and the second receiving state register, wherein the transmitter is configured to transmit one packet to a synchronization device of another node and another packet to the receiver of its own node via a loop path, wherein each packet comprises a sequence number, wherein the sequence number is register selection information for selecting one of the two receiving state registers, and wherein the controller comprises: a check circuit configured to check consistency between the sequence number of the received packet and the number of packets received, wherein when the results of the check indicate that the sequence number is consistent with the number of packets, one of the two receiving state registers is selected in accordance with a least significant bit of the sequence number. a check circuit configured to check consistency between the sequence number of the received packet and the number of packets received, wherein when the results of the check indicate that the sequence number is consistent with the number of packets, one of the two receiving state registers is selected in accordance with a least significant bit of the sequence number.