A device for executing neural network operation, comprising: an on-chip interconnection module; and a plurality of neural network processing modules communicatively connected to the on-chip interconnection unit, wherein each of the plurality of neural network processing modules is capable of reading data from and writing data to other neural network processing modules of the plurality of neural network processing modules through the on-chip interconnection module; wherein each of the plurality of neural network processing modules comprises a neural network processing unit configured to read and write data, characterized in that each neural network processing unit further comprises a synchronization relation unit, wherein the synchronization relation unit is configured to execute a data synchronization instruction; the data synchronization instruction comprises a plurality of opcodes, and each of the opcodes indicates whether a respective neural network processing unit is to participate in a synchronization operation, so as to indicate that multiple neural network processing units are to participate in the synchronization operation; by executing the data synchronization instruction, the synchronization relation unit is configured to send a synchronization signal to the synchronization relation units of the other neural network processing units participating in the synchronization operation, so as to force the multiple neural network processing units to perform the synchronization operation, wherein the multiple neural network processing units performing the synchronization operation is that the multiple neural network processing units start to execute subsequent instructions only after all of the multiple neural network processing units have executed the data synchronization instruction. an on-chip interconnection module; and a plurality of neural network processing modules communicatively connected to the on-chip interconnection unit, wherein each of the plurality of neural network processing modules is capable of reading data from and writing data to other neural network processing modules of the plurality of neural network processing modules through the on-chip interconnection module; wherein each of the plurality of neural network processing modules comprises a neural network processing unit configured to read and write data, characterized in that each neural network processing unit further comprises a synchronization relation unit, wherein the synchronization relation unit is configured to execute a data synchronization instruction; the data synchronization instruction comprises a plurality of opcodes, and each of the opcodes indicates whether a respective neural network processing unit is to participate in a synchronization operation, so as to indicate that multiple neural network processing units are to participate in the synchronization operation; by executing the data synchronization instruction, the synchronization relation unit is configured to send a synchronization signal to the synchronization relation units of the other neural network processing units participating in the synchronization operation, so as to force the multiple neural network processing units to perform the synchronization operation, wherein the multiple neural network processing units performing the synchronization operation is that the multiple neural network processing units start to execute subsequent instructions only after all of the multiple neural network processing units have executed the data synchronization instruction.