A system for controlling communication between parallel computers

Abstract: A communication control system controls communication between parallel computers using a wormhole routing. The system comprises unit for connecting a plurality of computer nodes by channels within a network to continuously transfer a message divided into a plurality of minimum data units for transmission and storing unit provided in respective nodes for storing said minimum data units, the number of said storing means being corresponding to the number of relay channels from the originating node to the most remote node plus 1. Therefore, a deadlock in a communication is avoided, and a high speed communication can be realized.

Description: Background of the invention The present invention relates to a system for controlling communication between parallel computers in which a plurality of computer nodes is connected through a plurality of channels and more particularly to a system for controlling communication between parallel computers in which a plurality of computer nodes is connected in a 2-divisional lattice and a plurality of nodes exists in two division channels. One such communication control system is a packet system. This system is used, for example, to transmit a message from a node A to a node C through a node B. Node A transmits a packet to node B. Node B stores this packet and then transmits it to node C. Thus, a relay transmission is performed. In this system, called store and forward routing, the packet is stored in node B once, thus causing a large delay in transmission to node C. A system for communicating between parallel computers, using a "wormhole routing" method, is known. In wormhole routing, the message is divided into minimum transfer units called flits, for example, several bytes of data. The first flit, namely, header flit is transferred within the network through a relay route between a transmitting node and a receiving node. When a certain node receives the header flit of the message, a channel (communication path) forming a relay route is selected by a transfer destination node designated by the header. The header flit and the following data flits are then transferred to the receiving node through the channel. The message is transferred in a form such that it continuously occupies the relay route from the transmitting node to the receiving node. That is, the message is transmitted on a channel in a chaining manner. Before the last flit of the message is output from the transmitting node, the header flit sometimes arrives at the receiving cell. A flit other than the header flit does not contain routing information but a flit of the message is transferred on a continuous channel within the network, and is thus not interleaved by flits of other messages. When the header flit of the message is blocked, transfer of all the flits of the message is stopped and transfer of other messages requiring the same channel being used for the message transfer is also blocked. When wormhole routing is used in a network in which a plurality of nodes is connected in a torus manner, it is necessary to avoid a deadlock state in which transfer of the all messages is blocked. Figure 1A is an explanatory view of a deadlock in a network connected in a torus manner. In this figure, the network is formed by four nodes 1 ○, 2 ○, 3 ○ and 4 with four uni-directional channels (a) (b) (c) and (d) connecting them. In Figure 1A, when all 4 nodes forming the network start to perform clockwise data transfer by a wormhole routing method, the message from node 1 ○ is transferred using channel (a), the message from node 2 ○ is transferred using channel (b), the message from node 3 ○ is transferred using channel (c) and the message from node 4 ○is transferred using channel (d). However, when the following flit of the message is transferred at the next clock, the message from node 1 ○ cannot use channel (b) and is blocked because channel (b) is already being used for the transmission of the message from node 2 ○. The following flit of the message transmitted from node 2 ○ similarly cannot use channel (c) and is blocked. The message transmitted from nodes 3 ○ and 4 ○ are blocked in the same way and since all messages are blocked, the result is a deadlock state. This deadlock state is caused, for example, by the length of the data being greater than the length between two channels. In such a case, when data is transmitted from node 1 ○ to node 3 ○, the last part of data to be transmitted from node 2 ○ to node 4 ○ has the last portion remaining in channel (b). Thus, the data transmitted from node to 1 ○ node 3 ○ cannot enter channel (b). Therefore, transfer of data from node 1 ○ to node 3 ○ and from node 2 ○ to node 4 ○ is blocked. A virtual channel method is known to be an algorithm for avoiding such a deadlocked state. A communication system using this method is explained by referring to Figure 1B, in which a channel connecting two nodes is uni-directional and all nodes are virtually doubled. The channel connecting nodes 1 ○ and 2 ○ is uni-directional from node 1 ○to node 2 ○ and is formed of double channels comprising (a) and ( a ) . This double channel is originally virtual and comprises one channel in terms of hardware. However, two channels may be naturally provided in terms of hardware. Respective channels imaginarily doubled can store a flit of a message, one of the flits stored in channel (a) or ( a ) being transferred from node 1 ○ to node 2 ○. The channel through which respective flits are to be transferred is determined for every flit. A method of using channels which are doubled upon transmitting a message from respective nodes is as follows. 1 - (a) - 2 - (b) - 3 - (c) - 4 2 - (b) - 3 - (c) - 4 - (d) - 1 3 - (c) - 4 - (d) - 4 - (a) - 2 4 - (d) - 1 - (a) - 2 - (b) - 3 By doubling a channel, the deadlock state in which all messages are blocked can be avoided. Even in a system using a virtual channel, as shown in Figure 1B, there is a problem in that the transfer ability is lowered when a message is transferred clockwise through all the nodes. At the first block, node 1 ○ transmits the message through channel (a) ; next, node 2 ○ transmits the message through channel (b); then, node 3 ○ transmits the message through channel (c); and finally, node 4 ○ transmits the message through channel (d). Thus, all four channels are used. However, at the second clock, the message is transferred through only two channels. The message transmitted from node 1 ○ is also transmitted to node 3 ○ via channel (b) , and the message transmitted from node 4 ○ is kept waiting until the transfer of the message from node 1 ○ to node 3 ○ is completed and then is also transmitted to node 2 ○ via channel (a) . Similarly, the message transmitted from node 3 ○ is kept waiting until the transfer of the message from node 4 ○ to node 2 ○ is completed and then is transferred to node 1 via channel (d). The message transmitted from node 2 ○ is kept waiting until the transfer of the message from node 3 ○ to node 1 ○ is completed and then transferred to node 4 ○ via channel (c). This is because the length of the message is larger than the distance between two nodes. Thus, when the message is transmitted from node 1 ○ to node 3 ○ via node 2 ○, the last part of the message remains in channel (a). Further, the channels which can be used upon transmitting a message between the nodes have been predetermined. For example, in message M2, it is predetermined that channel ( c ) is not used but channel (c) is used upon transferring the message from node 3 ○ to node 4 ○. Thus, the start of transfer of message M2 from node 3 ○ to node 4 ○ is kept waiting until message M3 completes the use of channel (c). All messages are transferred as described above. The message is transferred via two channels for all clocks except the starting clock. As a result, the transfer capability is lowered even in a virtual channel. An example of the virtual channel method is disclosed in "DISTRIBUTED COMPUTING", vol.1, no.4, October 1986, Berlin, Germany, pages 187 to 196. This also discloses a communication control system according to the preamble of accompanying claim 1. An object of the present invention is to provide a wormhole routing system for communicating between parallel computers. This system avoids the deadlock state and does not decrease transfer capability. According to the present invention, there is provided a communication control system for controlling communication between parallel computers comprising: means for connecting a plurality of computer nodes by relay channels within a network to continuously transfer a message divided into a plurality of minimum data units for transmission; characterised by: storing means provided in respective nodes for storing said minimum data units, said storing means having a capacity sufficient to store at least a number of said minimum data units equal to the number of relay channels from an originating node to a most remote node, whereby a deadlock in communication is avoided. Thus, a system embodying the invention comprises means for connecting a plurality of computer nodes within a network by channels for continuously transferring a message divided into a plurality of minimum data units, and means provided in respective nodes for storing at least a number of minimum data units equal to the number of relay channels from the originating node to the most remote node. The present invention avoids a deadlock in communication, thereby realizing high speed communication. Reference is made, by way of example, to the accompanying drawings in which: Figure 1A is a block diagram of a network in which a plurality of computer nodes is connected in a torus state, Figure 1B shows a conventional example of a communication control system using virtual channels, Figure 2A is a block diagram showing the principle of the first embodiment of the present invention, Figure 2B is a block diagram showing the principle of the second embodiment of present invention, Figure 3A is a block diagram showing the entire structure of the communication network system between parallel computer nodes according to the present invention, Figure 3B is a block diagram showing the entire structure of a node, Figure 4 shows a format of the header flit of the message, Figure 5 is a timing chart for explaining the basic method of data transfer between adjacent nodes, Figure 6 is a timing chart for explaining the output system of the continuous data flit outputted from one node, Figure 7 is a timing chart for explaining the data relay system, Figure 8 is a timing chart for explaining the system for notifying the originating node buffer-full possibility to the node adjacent to the transmitting source node, Figure 9A shows a format of data subjected to a routing within the network, Figure 9B shows an embodiment of the format of the transmitted class information accompanied by respective data flits, Figure 10 is a block diagram of the entire structure of an embodiment of the routing unit, Figure 11 is a block diagram showing the structure of the sending and receiving control module MSRD, Figure 12A shows the relationship between a multiplexer 20 (MUX) in a module of sending and receiving control MSRD and a switching control signal, Figure 12B shows the relationship between an output of multiplexer 22 (MUX) in module of sending and receiving control MSRD and a switching control signal, Figure 13 is a block diagram showing the structure of module of bus direction switching MSRP, Figure 14 is a block diagram showing the entire structure of sending and receiving class control module SR_PSRU, Figure 15 is a block diagram showing the input and output states of signals with regard to sending and receiving class control module SR_PSRU, Figure 16 is a block diagram showing the structure of class control module PRIM_MCOMM, Figure 17 is a block diagram showing the structure of the 0 class control module PRIM_MCOMM_F, Figures 18A to 18D are state transition diagrams showing the relationships between the state transition of the counter in the class control module PRIM_MCOMM and the class signal for an output circuit of data output request signals, Figire 18E shows a logic circuit for an output circuit of data output request signals, Figure 19 is a block diagram showing the structure of a memory control module MCTRL, Figure 20 is a block diagram showing the structure of input control module MIN, Figure 21 is a block diagram showing the structure of output control module MOUT, Figure 22 is a block diagram showing the structure of input buffer module MIFIFO, Figure 23 is a block diagram showing the structure of output buffer module MOFIFO, Figure 24 is a block diagram showing the structure of input and output buffer module MIO, and Figure 25 is a block diagram showing the bus direction control module MIRB. Description of the Embodiment Figure 2A is a block diagram showing the principle of the first embodiment. In Figure 2A, a plurality of computer nodes 1a, 1b...1n are mutually connected by channels 2a, 2b...2n, thus forming a communication network between parallel computers. In this network a message is divided into a plurality of minimum data transfer units, for example, flits comprising several bytes, between computer nodes. Data storing means 3a, 3b...3n store flits whose number is equal to the number of relay channels from the originating node to the most remote node in the network, plus 1. Figure 2B is a block diagram showing the principle of the second embodiment. Each data storing means 3a, 3b...3n shown in Figure 2A is divided into a plurality of areas. Each of these areas can store each of the above recited minimum data transfer units, namely, each of flits. Class number 0 is given to the flit which the originating node starts to transfer and class numbers 1, 2...n are given to flits which message transfer source nodes input to the originating node through 1, 2, ...n relay channels. When data is transferred within a network, respective nodes output a flit within the region of the highest priority class number selected from among a plurality of flits provided in data storing means 3a, 3b...3n from respective originating nodes to the adjacent nodes. The network in Figure 2B comprises four nodes. For simplicity, buffers corresponding to data storing means 3a, 3b... 3n are divided into four regions with class numbers 0 to 3. These buffers use a system in which the class number increases in accordance with the number of relay channels. For example, a message is transmitted from node 1 ○ to node 3 ○ in the sequence class 0 of node 1 ○ → class of node 2 ○ → class 2 of node 3 ○. Similarly, a message is transmitted from node 1 ○ to node 4 ○ in the sequence class 0 of node 1 ○ → class 1 of node 2 ○ → class 2 of node 3 ○ → class 3 of node 4 ○. Furthermore, when a data transfer starts from other nodes, the buffers are used in the sequence class 0 → class 1 → class 2 → class 3. The data class is transmitted together with the data (i.e., the flits) to designate the class in which the flits are stored. Let's suppose the case where the data of all the nodes are simultaneously transferred clockwise, as shown in the conventional example of Figure 1A. That is, at the first clock, the first flit of the message from nodes 1 ○, 2 ○, 3 ○ and 4 ○ are respectively transmitted to class 1 of the buffers in nodes 2 ○, 3 ○, 4 ○ and 1 ○. At the next clock, the first flit from nodes 1 ○, 2 ○, 3 ○ and 4 ○ are transmitted to class 2 of node buffers 3 ○, 4 ○, 1 ○, and 2 ○ respectively. At the following clock, the first flits of respective messages are transmitted to class 3 of the buffers of respective nodes 4 ○, 1 ○, 2 ○ and 3 ○. In this way, the present invention can transfer data by using all the channels in all the clocks. Figure 2B shows, for simplicity, a one-dimensional torus-type network in a uni-directional channel. However, it is possible to apply the present invention to bi-directional or two-dimensional networks, as will be explained in detail for an embodiment of the invention. Figure 3A shows the entire structure of a system for controlling communication between parallel computer nodes in accordance with the present invention. Adjacent nodes are connected by bi-directional channels. The network comprises a maximum of 32 nodes in both the transverse (east - west) and vertical (north - south) directions. Figure 3B is a block diagram showing the structure of one node. The node 1 comprises routing unit 4 for controlling the transfer direction of a message within the communication network, and a CPU 5. It is connected to adjacent nodes in the east, west, south and north directions. In the network shown in Figure 3A, the data transferred from a node to a communication destination node is controlled by the header flit, i.e., the first flit of the input message. Figure 4 shows an embodiment of the header flit. In Figure 3A, it is supposed that the message is first routed in the right or left (east or west) direction and then in the lower or upper (south or north) direction. It is further supposed that respective flits of the message comprise 32 bits from 0 to 31. When the header flit comprises xs at the 31st bit to designate the transfer direction (X direction) of the message as either east or west, xrcid from the 24th to 30th bit to designate the transfer distance of the message in the east or west direction or the number of the relay channels, ys at the 23rd bit for designating the transfer direction (Y direction) of the message as south or north, and yrcid from the 16th to the 22nd bit to designate the transfer distance in either the south or north direction, namely, the number of relay channels. In Figure 4, when xs is 0, the message is transferred to the east direction; when xs is 1, the message is transferred to the west direction; when ys is 0, the message is transferred to the south direction; and when ys is 1, the message is transferred to the north direction. Then, as shown in Figure 3A, if the message is transferred three nodes to the east (right) and two nodes to the south (lower) of a particular node, xs = 0, xrcid = 3, ys = 0 and yrcid = 2 in the header flit of the message. The message is then first transferred three nodes to the east of the particular node and then 2 nodes to the south of the particular node, i.e., to the communication destination node. In this case, in a routing of the message in the network, the node number of the destination node counted from the transmission source node is expressed as (+3, +2). Every time the message proceeds one node to the east, namely, the x direction, X of the relative node number is decremented and when the header flit arrives at the node three nodes east of the transmission source node, the relative node number becomes (0, +2), thereby completing a routing in the x direction. The routing is then conducted in the north - south direction, i.e., the y direction, and when the relative node number becomes (0, 0), the routing is completed. Figure 5 shows the basic system for transferring data between adjacent nodes according to the present invention. As shown in Figure 2B, the data of the class number region in the data storing means, for example, in the buffer of the originating node, is transmitted to the adjacent node. The transmission buffer of the transmission node side is divided into 3 phases 1 to 3. At phase 1, the data in the highest class in the buffer of the originating node is determined as the transfer data. At the next clock, the process of phase 2 is conducted, i.e., the class number of the data is transmitted. At the following clock, the process of phase 3 is conducted, i.e., the data flit is transmitted. The class number transmitted by phase 2 on the transmission side is received on the receiving node side at the clock in which the process of the phase 3 of the transmission node side is conducted. At the next clock, the data flit is received from the transmission node. Namely, transmission and receipt of one data flit is conducted in a period of 4 clocks. The transmission class determined by phase 1 on the transmission node side is the class having a buffer which is not full in the receiivng node side and satisfies the condition that the class is the highest class selected from the classes transmitting the transmission request from the receiving node side. Figure 6 shows the system for continuous output of data flits from 1 node. The 3 data flits are output in a pipeline manner. The transmission phase 3 of the first data flit and the transmission phase 1 of the second data flit are processed at the same clock. Phase 2 of the first data flit and phase 1 of the second data flit may be simultaneously processed at the same clock. Also, phase 3 of the first data flit and phase 2 of the second data flit may be processed at the same clock. In the present invention, the data transmitted from the adjacent node may not be directed to the originating node, but further transmitted to the next adjacent node, thus using the receiving node as a relay node. In this case, the data is transmitted to the adjacent node immediately without being stored in the buffer, thus achieving high speed transfer. The embodiment of such data relay system is shown in Figure 7. At the data relay node, the existence or non-existence of the relay of data can be detected from the class of data received at receiving phase 1 at clock 3. When at clock 3, a class is selected for the data transfer in transmission phase 1, the data received at receiving phase 2 at clock 4 is transmitted to the next node in transmission phase 3 at clock 5 without being changed. In the transmission node, the class to be transmitted is determined at transmission phase 1, the determined class number is transmitted at transmission phase 2, and the data in the class is transmitted at transmission phase 3. In the relay node, the class is received at receiving phase 1 and the transmission data is received at receiving phase 2. In the relay system, at the same clock 3 as in receiving phase 1, the transmission class is determined at the transmission phase 1 and then, the transmission class and data are transmitted to the next node at transmission phases 2 and 3, respectively, to perform a relay operation, as shown in Figure 7. In the relay system shown in Figure 7, the buffer of the destination node of the data to be relayed may be full or although another class is selected as the transmission data the class of the relay data may not be transmitted. This data is then stored in the buffer of the originating node. The transmission source adjacent node which transmits the relay data to the originating node is notified that the buffer within the originating node may become full. Such a notifying system is shown in Figure 8. It is supposed that the data is transmitted from node A to node B and that node B stores the data transmitted from node A in a buffer in receiving phase 3. When at node B, the class received at receiving phase 1 at clock 3 is not transmitted, node A is notified at clock 4 of the possibility of the buffer becoming full. Thus, at clock 5, node A knows whether or not the buffer within node B becomes full. As shown in Figure 6, the data flit is transferred by the pipeline system. Thus, even if node A knows at clock 5 that node A has already transmitted the next data, the buffer within node B needs to store two data flits. Node A is notified when the data in the node B buffer has been transmitted and the buffer becomes empty. According to the present invention (described later), the class information at the clock of transmission phase 2 designates the class in which the buffer is empty as well as the class of the data to be transmitted. Figure 9A shows data subjected to a routing in a network and the format of the transmitted class information together with the data flit shown in Figure 5. Figure 9A shows a format of the data flit. End bit e is added to the data, i.e., data of 32 bits, which is the same as the header flit shown in Figure 4, thereby providing a format of 32 bits. When "1" is provided at an end bit, it is shown that the data flit is the last flit of the message. The class information shown in figure 9B is used to control a transfer of the data flit between adjacent nodes. This information comprises distance data dis, subtracting "1" from the number of relay channels from the originating node to the final position node in an east - west or a south - north directional routing, class information clss designating the class in a buffer of the transmission signal, transmission class enable signal ce designating an enable of transmitting class information, empty class information eclss designating an empty class in a buffer of the originating node for storing the received data from the destination of the transfer information and empty class enable signal ece for designating an enable of empty class information. As described previously, the network shown in Figure 3A comprises a maximum of 32 nodes in both the east - west and north - south directions. Furthermore, the channels connecting the nodes are bi-directional. Thus, the maximum distance of the transfer of the message in the east - west or south - north directions is 16 and the highest class number of the buffer within respective nodes is also 16. Thus, distance data dis, receiving class information clss, and empty class information eclss comprise data of 4 bits. The class information and the empty class information within the buffer in the originating node are transferred simultaneously. This is because the channels are bi-directional and can receive data from opposite nodes. Figure 10 shows a block diagram of the entire structure of routing unit 4 within node 1 shown in Figure 3B. The routing unit PSRU comprises the east - west direction module of sending and receiving device MSRD 6, the south - north direction module of sending and receiving device MSRD 7, the module of the processor interface with CPU side MAD 8, the module of the first in first out system for inputting the data of the processor of the originating node MIFIFO 9, the module of the output buffer of the first in first out system for outputting the data to the processor MOFIFO 10, the module of the input-output buffer MIO 11 between the east - west direction MSRD 6 and the south - north direction MSRD 7. Two modules of input control MIN 12 and 13 for controlling data input from input buffer MIFIFO 9 to MSRD 6 and from MIO 11 to MSRD 7 and two control outputs MOUT 14 and 15 for controlling the data output from MSRD 6 to MIO 11 and from MSRD 7 to MOFIFO 10. Figure 11 shows a block diagram of a module of a sending and receiving device MSRD 6 with an east - west structure the MSRD being provided in routing until PSRU and controlling data sending and receiving operation between the originating node and the adjacent node. Buffer memories 16 and 17 have two ports for storing the data in the east and west directions, respectively. As stated above, a data flit comprises 32 bits and the maximum number of classes is 16. Furthermore, it is necessary to store, in one class, two data flits, one from the east and the other from the west. In one class, the capacity of the memory becomes 2 x 16 x 33 bits, 2 corresponding to east and west, 16 corresponding to the number of classes, and 33 corresponding to the data. Sending and receiving class control modules SR_PSRU 18 and 19 manage the classes of the data output in the east and west directions, respectively. Multiplexers 20 and 21 switch the data to be outputted in the east and west directions, respectively. Multiplexer 22 switches the output data to the module of input and output buffer MIO 11. In the case of south and north direction MSRD 7, the output of multiplexer 22 is inputted to MOFIFO 10. Bus direction switch modules MSRP 23 and 24 are interfaces to the adjacent nodes in the east and west directions, respectively. When the originating node owns or occupies the channel to the adjacent node, i.e., the originating node owns or occupies the bus to the adjacent node, it alternately transmits the class information and data flit to the adjacent node. When the originating node does not own the bus, it receives the data and class information from adjacent nodes. The modules for controlling the bus direction MARB 25 and 26 control the direction of the channels between the originating and adjacent nodes in the east and west directions, respectively, i.e., the direction of the bus. As one channel is used bi-directionally, the transmission direction is switched once per four clocks by using units of four clocks for the transfer of one data, as shown in Figure 5. If, an adjacent node wants to occupy the bus, the node then owning and occupying the bus must release it at a certain time in response to the bus request from the adjacent node. When the node not occupying the bus has data to be output or an empty class in its buffer, it can output a transmission request to the other node and thus occupy the bus. Registers 27 and 28 temporarily store data in the east or west directions and output them to buffer memories 16 and 17. Therefore, the data stored in buffers 16 and 17 are delayed by one clock. Next, the data flow within the module of sending and receiving control device MSRD shown in Figure 11 is explained. The input data from the processor of the originating node, i.e., the input signal to MSRD 6 from MIFIFO 9 in Figure 10, is input to three multiplexers, 20, 21 and 22, through input data bus dataw_i 29 in Figure 11. When the data is transmitted to the adjacent node in the east direction, multiplexer 20 transmits it to the east side node through output data bus datas_d_e 30 and the bus direction switch module MSRP23 to east side node data bus buf_e 31. When the data is directed to the adjacent node in the south or north direction, multiplexer 22 transmits it to the input and output buffer module MIO11 in Figure 10. Next, data from the west side adjacent node is input to direction switch MSRP 24 through west side node data bus buf_w 32. Then, the data is input to multiplexer 22, register 27, multiplexer 20 and sending and receiving class control module SR_PSRU 18 through input data bus datarw_e 33 for the west side node. As recited above, respective data flits accompany the class information designating the class number of the data. First class information and then the data flits are transmitted in the network. Class information is input to the sending and receiving class control mosulw SR_PSRU 18 and the data flit is input to multiplexer 22, register 27 and multiplexer 20. The input data is transmitted to the east side node without being stored in the buffer memory. That is, when the data is relayed, it is transmitted to the east side adjacent node through the bus direction switch module MSRP 23 and multiplexer 20. If it is necessary to store the data in a buffer memory before outputting it, it is stored in buffer memory 16 through register 27, and when the buffer memory in the east side node becomes empty, it is output from multiplexer 20. When The input data from the west side node is directed to the processor of the originating node, it is then output from multiplexer 22 and transferred to the processor of the originating node through input and output buffer module MIO 11, sending and receiving device module MSRD 7, output FIFO module MOFIFO 10 and processor interface module MAD 8. Then, except when the data is transferred to the processor of the originating node, and in other words, when it is outputted to the adjacent node, the class information attached to the data flit is transmitted from sending and receiving class control module SR_PSRU 18 through the east side node output class information bus datas_c_e 34 to the bus direction switch module MSRP23 and the class information and data flit are alternatively outputted from bus direction switch module MSRP23 to east side node data bus buf_e 31. The sending and receiving class control module SR_PSRU 18 controls the class of the data to be transmitted to the east side node as recited above at the next time and it is necessary to know the state of the buffer in the transmission destination. Therefore, sending and receiving class control module SR_PSRU 18 of the originating node receives buffer full signal fulli_e 35 from the east side node. It also outputs buffer full output fullo_w 36 to the west side node to indicate that the buffer in the originating node is full. In order to control which node occupies east side node data bus buf_e 31, the module of controlling bus direction MARB 25 transmits and receives bus request output buso_e and bus request input busi_e 37 to and from the corresponding module in the adjacent node. The data input through data bus buf_e 31 from the east side node, processed in the same manner as the data from the west side node, is transmitted to the west side node through bus direction switch module MSRP 24 and is output from multiplexer 22 to input and output buffer module MIO 11. The control method is performed as recited above. The relation between the switching control signals of multiplexers 20 and 22 and the outputs of respective multiplexers are shown in Figure 12. The signal dir_mo is output from output control module MOUT 14 in Figure 10 and other signals are output from memory control module MCTRL in sending and receiving class control module SR_PSRU 18. The control signals to multiplexer 21 are the same as those to multiplexer 20. Figure 13 is a block diagram showing the structure of bus direction switch module MSRP 23 for switching east side node data bus buf_e 31 in Figure 11 and comprises a multiplexer 38, two registers 39 and 40, and a three-state buffer 41. The signals from output data bus datas_d_e 30 connected to multiplexer 20 and output class information bus datas_c_e 34 connected to sending and receiving class control module SR_PSRU 18 to the east side node are switched alternately at every clock signal clk2 to register 39 and three-state buffer 41, and then to east side node data bus buf_e 31. Control signal b_e of register 39 becomes "H" when the originating node has the right of bus and control signal pb_e of three-state buffer 40 is delayed by 1 clock from control signal b_e for register 39. Register 40 temporarily stores the data input from the east side node through east side node data bus buf_e 31. Figure 14 is a block diagram showing the entire structure of a module of sending and receiving class control SR_PSRU. This module receives class information from the adjacent node and a data output request from input control module MIN 12, and forms class information to be transmitted to the adjacent node or input and output buffer module MIO 11. Sending and receiving class control module SR_PSRU in Figure 14 corresponds to sending and receiving class control module SR_PSRU 18 in Figure 11 and comprises receiving control module MRECV 42 connected to the input data bus datarw_e 33 from the west node, a total of 17 class control modules PRIM_MCOMM 43, 44 1 , ...44 16 , a memory control module MCTRL 45 and sending control module MSEND 46 connected to the output class information bus datas_c_e 34 to east side node. In Figure 14, class control modules PRIM_MCOMM 44 1 , ...44 16 manage respective classes of the data in the buffer memory in the originating node. The 0 class control module PRIM_MCOMM_F 43 manages the class of the data from the processor of the originating node. A total of 16 class control module PRIM_MCOMM 44 1 to 44 16 manage the data corresponding to respective classes in the buffer memory in the originating node. As recited above, in the entire structure shown in Figure 3A, class control module PRIM_MCOMM 44 1 to 44 16 are provided to 16 respective classes corresponding to the maximum distance between the nodes. Receiving control module MRECV 42 decodes class information from the adjacent node and notifies receipt of the data to class control module PRIM_MCOMM for the corresponding class. Sending control module MSEND 46 selects the highest class from the transmission requests from class control module PRIM_MCOM_F0 to PRIM_MCOMM_F 16 43, and this class becomes the transmission class number clss in the format of class information recited in Figure 9B. In order to notify the empty class in the buffer memory for storing the data which the originating node receives from the east side node to the west side node, sending and receiving class control module SR_PSRU 19 shown in Figure 11 receives the class information from the east side node. The west direction data buffer memory in the originating node receives data from sending and receiving class control module SR_PSRU 19 and the signals designating the existence or non-existence of 17 empty classes. It then selects the highest of the empty classes and determines the empty class number eclss as the class information corresponding to the empty class. This forms the class information to the east side node. At the west side node, the class information is formed by adding "1" to the class number of the west side node, to provide class 1 of the originating node in the case of class 0 in the west side node. Therefore, it is necessary to notify the west side node that class 1 in the originating node is actually empty, thus determining the class information to be sent from the west side node to the originating node. In Figure 14 , 0 class control module PRIM_MCOMM_F 43 sends and receives control information for the data input through the input buffer module MIFIFO 9 from the processor of the originating node in Figure 10 with the input control module MIN12. Memory control module MCTRL 45 outputs a switching control signal to buffer memory 16 and multiplexers 20 and 22 in Figure 11. Furthermore, a control signal is sent and received between memory control module MCTRL 45 and output control module MOUT 14 in Figure 10. Figure 15 shows an input and output state of the signal for sending and receiving class control module SR_PSRU. In Figure 15, only one of 17 class control modules PRIM_MCOMM is shown. Receiving control module MRECV 42 comprises decrementer DECR 47, comparing module CMP48, and two decoders DECODE 49 and 50. As recited above, receiving control module MRECV42 receives data from the west side node through input data bus datarw_e 33. The data comprises 33 bits, as shown in Figure 9A, and only the end bit is directly input to class control module PRIM_MCOMM 43 and 44. The distance dis designating the number of the relay channels to the destination node in the east - west direction (or the north - south direction) in the input class information, -1 is input to decrementer DECR47 and comparing module CMP 48. When the data arrives at the originating node, the distance to the destination node is decreased by one and the output of decrementer DECR is input to respective class control modules PRIM_MCOMM 43 and 44 as the new distance data dis. Comparing module 48 detects whether or not the distance data is 0. If it is not, the east direction enable signal rva_e is transmitted to class control modules PRIM_MCOMM 43 and 44 for a relay operation. If it is, the originating processor direction enable signal rvb_e for designating the originating processor direction or designating a change in direction from the east side to the south or north directions is transmitted to respective class control modules PRIM_MCOMM 43 and 44. Decoder 49 uses transmission class number clss and transmission class enable signal ce shown in Figure 9B and obtains the signal designating the data whose class is received. That is, it decodes the transmission class number clss so that, when the data of class 0 is received, the received transmission class number clss is decoded and the signals to the class control PRIM_MCOMM 43 and 44 are formed so that the class number is incremented by 1, as shown in Figure 2A. This decoded signal is then output to respective class control modules PRIM_MCOMM_F 43 and 44 as receiving enable signal rve_e. This signal is used to increment counter 55 by 1 to manage the number of data in the buffer. Sending and receiving class control module SP_PSRU shown in Figure 15 receives the data from the west node. The module of sending and receiving class control SR_PSRU (not shown) for receiving the data from the east side node has the same structure as the above recited module for the west side. Decoder 50 (not shown) provided in the module of sending and receiving class control uses an empty class number eclss and an empty class enable signal ece transmitted from the east side node and forms the signal erve_e for designating the class in the buffer for which the east side node becomes empty. Signal erve_e enters the module of class control 43 and 44 of send ing a nd receiving class control module SR_PSRU(east)18 as the signal erve_e in Figure 15, and class control modules 43 and 44 detect whether or not the corresponding class in the buffer in the east side node is empty. That is, when the buffer of the east side node is not empty, counter 56 in the module of class control 43 and 44 is set to "1" in the full state by using the signal fulli_e designating the full state. When the receiving enable signal evre_e is transmitted from sending and receiving class control module SR_PSRU(west)19 state 1 is changed to state 0, i.e., the state in which the corresponding buffer class of the adjacent east node becomes empty. The empty class has the class number obtained by increasing the transmission class number to the east side of the originating node by 1. Therefore, the class of the buffer in the originating node transfers the data to the corresponding empty class of the adjacent node of the east side or the data from the adjacent node on the west side is relayed by the originating node and transmitted to the empty class of the buffer in the east side node. Further, when the data is transferred from the east side node to the west side node, decoder DECODE forms empty class receiving signal erve_w for designating which class in the buffer in the west side node is empty by using empty class number eclss and empty class enable signal ece transmitted from the west side node. It then outputs the empty class receiving signal erve_w to module of transmission and receiving class control, SR_PSRU19 shown in Figure 11, to control the data output class to the west side node in the originating node. Next, the module of sending control MSEND46 comprises two priority encoders PRIORITY ENCODE 51 and 52. Priority encoder 51 receives from the module of the respective class control PRIM_MCOMM 43 and 44 data sending request signal DA_e which is sent to the adjacent node and selects the highest class sending request signal. The highest class is determined at the transmission class number clss in Figure 9B and a signal designating the highest class is returned to respective module of class control PRIM_MCOMM 43 and 44 as acknowledge signals sde_e for confirming that data is sent to the east side node. Therefore, counter 55 provided in class control module 43 and 44 is decremented by 1. Sending and receiving control Module SR_PSRU(west) 19 provided for the data received from the east side node includes priority encoder (52) (not shown). Priority encoder (52) is provided corresponding to the priority encoder 52 shown in Figure 15 and in the module of sending and receiving control (west) 19. An empty class sending request signal em_w is formed by class control module 43 and 44 of the sending and receiving control module 18 shown in Figure 15 when the memory, namely, buffer memory 16 shown in Figure 11, for storing the data transmitted from the west side node the originating node becomes empty. This signal is used to input the empty class sending request signal em_e to priority encoder 52 in the module of sending and receiving class control SR_PSRU(west) 19. When encoder 52 selects the highest class signal by using the empty sending request signal em_e, the empty class number clss shown in Figure 9B is formed and empty class transmitting acknowledge signal esde_e (corresponding to the shown empty class sending acknowledge signal esde_w) for confirming that the empty signal was sent to the west node is produced from the priority encoder (52) (not shown). The empty class sending acknowledge signal esde_e is transferred to modules of class control 43 and 44 in Figure 15, thereby resetting counter 55 for managing the number of data in the buffer, as the empty class means that the number of data is 0. Priority encoder 52 provided in module of sending and receiving class control SR_PSRU 18 uses empty class transmission request em_w formed by transmission received class control module SR_PSRU 19 based on the empty state of buffer memory 17 in Figure 11, namely, the memory for storing data transmitted from the east side not in the originating node, and selects the highest empty class. It thereby forms the empty class number ecls shown in Figure 9B and simultaneously outputs the empty class sending acknowledge signal esde_w to the east side node of the module of transmission receiving class control SR_PSRU 19. Further, signals from respective class control modules PRIM_MCOMM 43 and 44 to the module of output control MOUT 14 for controlling the data output to input and output buffer MIO11, namely, to the originating processor side or to the south or north direction, in Figure 10, as signal DB_e. Then output control module 14 and sending acknowledge signal psde_e designating the highest input class are selected from among data sending request signals DB_e appearing 17 signal lines to respective class control modules PRIM_MCOMM 43 and 44. Therefore, counter 55 for managing the number of data of the buffer in the originating node is incremented by 1. Further, respective class control modules PRIM_MCOMM 43 and 44 receive transmission buffer full signals designating that the buffer at the east side node is full. Thereby, counter 56 for designating that the buffer is full is set to "1". The module of the sending control MSEND 46 transmits the distance data dis_e which is decremented by decrementer 47 in receiving control module MRECV 42, the sending class number and the empty class number as class information to the east side node through output class information bus datas_c_e34. Figure 16 is a block diagram showing the structure of module of class control PRIM_MCOMM 44a. It shows the module of class control other than module of 0 class control PRIM_MCOMM_F. The module of class control shown here comprises counters 53, 54, 55 and 56, register 57 and logic circuit 58. East direction enable signal rve_e from module of receiving control MRECV 42 and originating processor (MIO) direction enable signal rvb_e are input to counter 53, which designates the direction of the data in the corresponding class in the buffer. Receiving enable signal rve_e for designating the class to which the data is sent is received by counter 53. Counter 54 receives an end bit signal and a receiving enable signal rve_e from receiving control module 42 and determines an end bit designating, for example, the final data flit of the message, and detects the initial flit of the next message, namely, the head flit. Counter 55 receives receiving enable signal rve_e from receiving control module MRECV 42, a transmission acknowledge signal esde_e for designating an empty class number transmitted to the west side node and transmitted from sending control module MSEND within the module of the sending and receiving class control SR_PSRU19, a sending acknowledge signal esde_e for designating the class of the data to be output to the west side node, and sending acknowledge signal psde_e for designating the class of the data output to the originating processor (MIO11). Then, counter 55 manages the number of data in the corresponding class of the buffer. As recited above, respective buffers can store two of the data. Counter 56 receives empty class receiving signal erve_e for designating the buffer empty class in the east side node. This signal is then transmitted from the module of the receiving control MRECV in sending and receiving class control module SR_PSRU 19 and the buffer full signal fulli_e is transmitted from the east side node. Counter 56 then shows whether or not the corresponding buffer in the east side node is full. Register 57 stores distance data dis_e decremented by decrementer DECR47 in receiving control module MRECV 42 and outputs the distance data dis_e to transmission control module 46. As recited above, the distance data dis_e is decremented and the data is thus routed within the network. The node distance dis is significant only for the header flit. The node for receiving a header flit corresponding to class A from the west side node performs a further routing of the flit to the east side node if the node distance dis is not 0. If the node distance dis is 0, the flit is not subjected to a routing to the east side node, but to a node in the south or north direction or to the originating node. The direction of the message corresponding to class A of the buffer of the node is kept the same until the end of the message, i.e., until the end bit is detected. Accordingly, the transmission direction of the following data flit in class A is determined by the value of the node distance dis maintained in register 57 as the transfer direction information of class A and the content of the register is updated only when the header flit is received. The direction of a message corresponding to a class other than class A in the buffer of the node is completely independently controlled based on the value of the node distance dis. Logic circuit 58 of Figure 16 uses the input from counters 53, 54, 55 and 56 and outputs a west-side node transmission request signal DA_e to sending control module MSEND 46. It also outputs a data transmission request signal for input and output buffer module MIO 11 to output control module MOUT14. Logic circuit 58 outputs write request wr_e and read request rd_e to memory control module MCTRL 45 and comprises the lower 1 bit address of the buffer for performing a write and read of data oes_e, and oer_e. Logic circuit 58 also outputs a west-side-node empty class sending request signal em_e to sending control module MSEND within module of sending and receiving class control SR_PSRU(west) 19 of class control modules PRIM_MCOMM 44 1 to 44 16 . Class control modules PRIM_MCOMM 44 1 to 44 15 for managing classes 1 to 15 perform exactly the same operation, but class control module PRIM_MCOMM 44 16 for managing class 16 is different from the other modules in that the data sending request is not sent to sending control module MSEND 46 as the data corresponding to class 16 is addressed to the originating node. Figure 17 is a structural block diagram of 0 class control module PRIM_MCOMM_F 43. The operation is almost the same as that of class control module 44 for managing classes 1 to 16, differing only in that it does not manage the buffer. However, it manages the data transmitted to the processor of the originating node. Counter 59 designates whether or not the buffer in the east side node is full, in the same way as counter 56 shown in Figure 16. Logic circuit 60 receives signals DA_i_e, DB_i_e designating a data sending request for adjacent nodes and input and output buffer MIO11 from input control module MIN12. Logic circuit 60 further receives a transmission acknowledge signal sde_e transmitted from priority encoder 51 in sending control module MSEND46 to the west side node and receives the data to input and output buffer module MIO 11 input from output control module MOUT 14. Logic circuit 60 outputs an east-side-node sending request signal DB_e to sending control module MSEND 46 and transmits MIO11 side sending request signal DB_e to output control module MOUT 14. Figure 18 shows the relation between a state transition of a counter in class control module PRIM_MCOMM and a signal. Signals de, da, db, fst and full are the signals within the module. SM 1 in Figure 18A shows the state of counter 55 in which the number of data in the buffer is managed. 0 means that the number of data is 0. This state changes to state 1, i.e., the state in which the number of data in the buffer is "1", when a receiving enable signal rve_e equals 1, and both sending acknowledge signals, i.e., both sde_e and psed_e for the classes respectively outputted to the east side node and to input and output buffer module MIO11 are "0". In state 1, the condition recited above is established. State 1 is then changed to state 2, i.e., the state in which the number of data in the buffer is 2. State 2 is changed back to state 1 when sending acknowledge signal sde_e for the class outputted to the east side node or sending acknowledge signal psed_e for the class is outputted to input and output buffer MIO11 becomes 1. Under the same conditions, state 1 is changed to state 3, namely, the state in which the buffer become empty. State 3 changes to state 0 when empty class sending acknowledge signal esde_e to the west side node becomes 1. As to the signals corresponding to these four states become de=1 for states 1 and 2 and as to empty class sending request signal for west side node, em_e = 1 for state 3. SM 2 of Figure 18B designates a direction of data within the buffer, i.e., the state of counter 53, and there is a possibility that 2 data in the buffer have different directions. Thus, the state becomes 0 to 3. When the state is 0 or 1, da=1 which designates the adjacent node direction. When states are 2 and 3, db=1, which designates the direction of input and output buffer MIO11 as described later. SM 3 =0 indicates that the data is header data and when it is to be transmitted to input and output buffer MIO 11 side (rvb_e=1), state 0 changes to state 1. The change condition from states 1 to 2 is SM 1 ≠2. This means that the member of the data within the buffer is not 2, but either 1 or 0. When SM 1 =2, the directions of the 2 data in the buffer may be different. For example, suppose that data previously received in the buffer is in the "a" direction, i.e., in the direction of the east side node. (In this case, the state of SM 2 is 0, and a da=1 signal is output.) Thereafter, when the data which is in the "b" direction, i.e., in the direction of the originating processor (MIO11) is received, the data is stored in the buffer and SM 2 becomes 1. However, when the previous data is in the buffer, SM 1 is 2, the signal da is "1" and db is maintained as "0". When the previously received data is output from the buffer, SM 1 is in state 1 and SM 2 changes to state 2, thereby outputting the signal db=1. Thus, the data received first is always outputted first. When the data is header data, as SM 3 =0 again, and when the data is for the east side node (rva+_e=1), the state 2 is changed to 3. The transition condition from state 3 to state 0 is SM 1 ≠2, i.e., in the same manner as the change condition from state 1 to state 2. Figure 18C is a state transition diagram for explaining state SM 3 of counter 54, how it is determined, and whether or not the data is header data. State 0 indicates that the data is header data. State 0 is changed to state 1 (in which the data is not header data) when a received enable signal rve_e becomes "1". When it is detected that the end bit is "1", state 1 is changed to state 0. The signal corresponding to state 0 is fst=1, and the signal corresponding to the state 1 is fst=0. Figure 18D is a diagram of change of state SM 4 of counter 56 for designating whether or not the buffer in the adjacent node is full. State 0 indicates that the buffer is not full. State 0 is changed to state 1, in which the buffer in the adjacent node is full, when the buffer full input fulli_e is input. Upon receipt of empty class receiving signal erve_e, state 1 is changed to state 0. The signals corresponding to states 0 and 1 are respectively full=0 and 1. Figure 18E shows the circuit for outputting a data sending request signal for the east side node DA_e and data sending request signal DB_e for input and output buffer MIO 11. The signal DA_e is output by using the above various signals and the east side direction enable signal rva_e. The signal DB_e is output by using the above recited signals and the originating processor direction enable signal rvb_e. Figure 19 is a block diagram showing the structure of memory control module MCTRL 45 used in sending and receiving class control module SR_PSRU shown in Figure 14 and comprises a logic circuit 61 and a register 62. Logic circuit 61 receives east side node sending acknowledge signal sde_e transmitted from sending control module MSEND 46, sending acknowledge signal psde_e for designating a class of the data to be sent to input and output module MIO 11 and transmitted from output control module MOUT 14, write request signal wr_e, read request signal rd_e and addresses oes_e and oer_e designated by the lower 1 bit (total 5 bits) of the buffer. Logic circuit 61 outputs data to the west side node. This signal becomes "1" when the data write signal for the buffer of the originating node is produced. Further, logic circuit 61 outputs a control signal other than dir_mo to multiplexers 20 and 22 shown in Figure 11. Register 62 outputs two read-address/read-enable signal pairs and one write-address/write-enable signal pair to the buffer memory. Two read-address/read-enable signal pairs are output because the memory read is performed twice per clock. One of the clocks is for reading the data to the east side node and the other is for reading the data to the module of input and output buffer MIO 11. The memory read is conducted at sending phase 2 in Figure 5 and the memory write is conducted at receiving phase 3 of node B in Figure 8. Figure 20 is a block diagram showing the structure of input control module MIN provided in the routing unit PSRU shown in Figure 10. Input control module MIN 12 is for determining the sending direction of the data inputted from the originating node processor. The sending direction is the output direction from sending and receiving control device module 6 in Figure 10 and one of the directions for the east side node, west side node and input and output register module MIO 11. The transmission direction is shown in the header flit as shown in Figure 4 and the direction of the data is maintained the same from the head of the message, i.e., the header flit to the end bit. Input control module MIN 12 comprises a decoder 63, a counter 64 and a decrementer 65, as shown in Figure 20. Decoder 63 and decrementer 65 receive the data from input buffer module MIFIFO 9, shown in Figure 10, through input data bus dataw_i29, and counter 64 receives the signal rvew_i designating the existence of data from MIFIF 9. Decoder 63 determines the direction of the data from the content of the header flit and outputs one of the enable signals rva_e, rva_w or rvb_e, which correspond to the east side node, the west side node and input - output buffer module MIO 11 side, respectively, in respect of the sending direction of the data, to counter 64. Counter 64 outputs to sending and receiving control device module 6, shown in Figure 10. For one of the signals DA_i_e, DA_i_w and DB_i_e, which correspond to the data transmission request for the east side, the west side and input and output buffer module MIO 11 side, respectively, in Figure 20, the signal DB_i_w is always kept 0. Counter 64 receives an acknowledge signal sde_i for designating the sending of the data from sending and receiving control device module 6. Counter 64 detects the header flit by determining the end bit and maintains the sending direction of the data in accordance with the content of the header flit. Decrementer 65 decrements the information designating the destination node in the header flit, i.e., the distance to the destination node, and outputs it to sending and receiving device module 6 as the distance data dis_e (or dis_w). The distance is decremented before the data from the processor of the originating node is output to the adjacent node because the number of bits representing this distance is decreased. Thus, the distance data dis shown in Figure 9B may be 4 bits. Figure 21 is a block diagram showing the structure of output control module MOUT 14. This module is for controlling the data output to the input and output buffer module MIO 11 from sending and receiving control device module 6 and enables the data of the highest of the classes which sending and receiving device module MSRD 6 requests for the output from input and output buffer module MIO 11. The class of the output data is maintained the same from the header flit of the message to the end flit of the message, i.e., the detection of the end bit. Even if higher class data is input to the originating node while the message is being output, the data output of the higher class is not immediately provided and the higher class input data is output after the end bit of the message being output is detected. This control method is performed when the data is output from sending and receiving control data module MSRD 6 to input and output module 11 and a different class (higher class) data output is conducted for every flit when the data is output to sending and receiving control device module 6 to adjacent nodes. Output control module MOUT 14 comprises priority encoder 66 and counter 67. Priority encoder 66 receives sending request signals DB_e and DB_w from sending and receiving class control module SR_PSRU(east)18 and sending and receiving class control module SR_PSRU(west)19 of sending and receiving control device module 6, respectively. Counter 67 receives these inputs, signal or_full for designating a buffer full state from input and output buffer module MIO 11, the end bit designating the last data flit of the message from sending and receiving control device module 6 and the output of priority encoder 66. Priority encoder 66 selects the highest class of data sending request signals DB_e and DB_w and outputs it to counter 67. Counter 67 detects a header flit and maintains the state designating the class of the data to be outputted from sending and receiving control device module 6. At the same time, when counter 67 outputs the signal designating the data output datae_mo to input and output buffer module MIO 11, sending acknowledge signals psde_e or psde_w for designating the class of data to be sent input and output buffer module MIO 11 selected by output module MOUT 14, to sending and receiving control device module 6. The two sending acknowledge signals psde_e and psde_w correspond to output requests from sending and receiving class module SR_PSRU 18 and 19 for the east and the west sides, respectively. Counter 67 outputs control signal dir_mo to multiplexer 22 shown in Figure 11. Figure 22 shows a block diagram of FIFO buffer MIFIFO 9 shown in Figure 10 for storing data to be input from the processor of the originating node. It comprises counter 68 and buffer 69. Buffer 69 temporarily stores the data to be input to the data bus dbus through interface MAD8 from the processor side and outputs it to input data bus dataw_i 29 connected to sending and receiving control module 6. Counter 68 receives sending acknowledge signal sde_i from sending and receiving control module 6 and write signal write from the processor side and comprises a counter for designating the number of data stored in a buffer, a register for designating the output address of the data and a register for designating the input address to which the next data is input. When signal write becomes "H", and the data is input, the counter value and the input address are incremented, and when the data is output, the output address is incremented and the counter value is decremented. Counter 68 outputs to the processor side signal rvew_i for designating that data exists in the buffer to the input control module MIN 12. It then transmits to the processor an ir_full signal for designating that the buffer is full or an ir_nempty signal for designating that the buffer is empty. Figure 23 shows a block diagram of output FIFO first-in-first-out system buffer module MOFIFO 10 provided at the output side of the processor. Output module MOFIFO 10 comprises counter 70 and buffer 71, as does input module MOFIFO shown in Figure 22. Buffer 71 temporarily stores data from sending and receiving control module (south/north) 7 in the same manner as buffer 69 and outputs it to the data bus connected to the processor side. Counter 70 operates in the same way as counter 68 in input first-in-first-out module MIFIFO 9, receives the signal datae_mo outputted from output control module MOUT 15 for designating the data output from sending and receiving control module 7 and read signal read from the processor, and outputs the signal or_full for designating that the buffer is full to output control module MOUT 15 and signal or_nempty and the full signal or_full to the processor side. Figure 24 is a block diagram showing the structure of the input and output control module 11 shown in Figure 10. It comprises a data buffer inserted between two sending and receiving control modules MSRD 6 and 7. The input and output buffer module comprises generally available FIFO comprising counter 72 of 1 bit for representing a data enabel, and buffer 73. Buffer 73 temporarily stores the data input from the data bus from the input side and outputs it to sending and receiving control module MSRD (south/north)7. Counter 72 receives signal datae_mo designating the data output from output control module MOUT 14 and sending acknowledge signal sde_i from sending and receiving control module 6. Signal or_full designating the buffer full state is input to output control module MOUT 14 and signal rvew_i designating that the data exists in input and output control module MIO, is output to input control module MIN 13. Figure 25 is a block diagram of bus direction control module MARB shown in Figure 11. It comprises a counter 74 which receives channel obtaining request signal buso_e from sending control module MSEND 46 shown in Figure 15. The channel obtaining request is output to notify the east side node that the originating node has data to be sent or that the buffer storing the data to be sent from the east side node has become empty, that the OR signal of sending request signal DA_e has been input to sending control module MSEND 46 and sent to the east side node, and that empty class sending request em_w input from sending and receiving class control module SR_PSRU(west)19 has been sent to the east side node. Counter 74 receives channel obtaining request busi_e from the east side node. It then outputs channel obtaining request buso_e to the east side node and outputs signal b_e designating that the channel is obtained, to register 39 in bus direction switching module MSRP 23 shown in Figure 13, to receiving control module MRECV 42 shown in Figure 15, and to sending control module MSEND 46 for controlling sending and receiving class control SR_PSRU. As explained, according to the present invention, in a network in which a lot of computer nodes are interconnected two-dimensionally by bi-directional channels, the highest class data of the originating data is output from respective nodes to adjacent nodes, thereby effectively utilizing the channel. As recited above, when the output is produced from sending and receiving control module MSRD to the adjacent nodes, the class for producing the data can be changed at every data flit. Even if the message transfer is blocked when the change from the originating node to the east side node is used for a particular message transfer, it becomes possible to use a particular channel to transfer another message. As described above, according to the present invention, in a communication between parallel computers using wormhole routing, a deadlock in communication is avoided and decrease in transfer capability is prevented, thereby increasing communication speed.

Claims: A communication control system for controlling communication between parallel computers comprising: means for connecting a plurality of computer nodes (1a, 1b, ...1n) by relay channels (2a, 2b, ...2n) within a network to continuously transfer a message divided into a plurality of minimum data units for transmission; characterised by: storing means (3a, 3b, ...3n) provided in respective nodes for storing said minimum data units, said storing means having a capacity sufficient to store at least a number of said minimum data units equal to the number of relay channels from an originating node to a most remote node, whereby a deadlock in communication is avoided. A communication control system according to claim 1, wherein the capacity of said storing means (3a, 3b, ...3n) is equal to n+1 of said minimum data units where n is said number of relay channels. A communication control system according to claim 2, wherein each of said storing means (3a, 3b, ...3n) is divided into a plurality of class number regions including a first region with a class number of zero for storing a said minimum data unit which starts a transfer from the originating node and further regions with respective class numbers 1,2...n for storing minimum data units transferred from a message transfer source node to the originating node through respective 1,2...n relay channels, wherein when data to be transferred to adjacent nodes exist in said plurality of class number regions, data in one of said plurality of class number regions is transferred to the adjacent node. A communication control system according to claim 3, wherein empty class information is determined based on the classes in which data does not exist in the adjacent node. A communication control system according to claim 4, wherein data is sent from the class having the highest priority class number selected from among the classes in which data exist in the originating node to an empty class based on said empty class information. A communication control system according to claim 1, wherein each said storing means (3a, 3b, ...3n) has buffer regions corresponding to a plurality of classes, and the data transmitted from a sending node is stored in one of said buffer regions of the originating node when one of said buffer regions corresponding to a request sending class in said sending node is empty. A communication control system according to claim 1, wherein each node (1a, 1b, ...1n) is further provided with: means for determining a transmission class; means for transmitting class information; and means for transmitting transmission class data. A communication control system according to claim 7, wherein transmission class data corresponding to an empty class is transferred by using empty class information transmitted from the adjacent node. A communication control system according to claim 6, wherein each node (1a, 1b, ...1n) further comprises a sending and receiving class control module (6, 7, 18, 19) including receiving control means (42) which receives input information from a node, which decrements relative distance information and outputs the relative distance to a destination node, said receiving control means (42) comprising: means (48) for comparing said distance information with a reference distance (0) to detect whether or not the determination of the transmission data is made by the originating node; first decoding means (49) for decoding the transmission class; and second decoding means (50) for decoding the empty information. A communication control system according to claim 9, wherein a respective class control module (43, 44) is provided for each said class information, comprising; first counting means (55) for managing the number of data in said buffer regions within the originating node; second counting means (53) for designating the direction of the data in said buffer regions; third counting means (54) for designating whether the data is header data; fourth counting means (56) for designating whether or not the corresponding buffer regions within the adjacent node are full; and a logic circuit (58) connected to said first to fourth counting means for outputting the data output request signal of the data to be transmitted to an adjacent node, a transmission acknowledge signal for confirming that the data has been sent, and a signal from said sending and receiving control module designating whether the class corresponding to said buffer regions in the adjacent node is empty. A communication control system according to claim 10, further comprising a plurality of sending and receiving control modules, wherein said sending and receiving control modules comprises: a first priority encoder (51), operatively connected to said class control module (43, 44), for determining the transmission class by receiving a sending request signal from said class control module; a second priority encoder (52) operatively connected to said class control module, for obtaining information designating an empty state of said buffer regions of an adjacent node transmitted through said sending and receiving class control module within the originating node, for using the information designating an empty state of said buffer regions of the adjacent node, and for forming a signal designating said buffer regions of the class corresponding to said class control module of said sending and receiving control module another one of the sending and receiving signal in said sending and receiving control module comprises the first sending and receiving control module for transmitting the data from a first node to a second node; and a second sending and receiving control module for controlling the data transfer from said second node to said first node, wherein, by using said first sending and receiving control module and said second sending and receiving control module, empty class information of said buffer regions of the input data is input to said first sending and receiving module and is transmitted to said second sending and receiving control module, the empty class information received from the other node and input to said second sending and receiving control module enables the counter for designating whether the class of said buffer regions in the adjacent node is full and is provided in said class control module corresponding to respective classes, and the transmission sending class information stored in the input data from the adjacent node is utilized.

Inventors: HORIE TAKESHI, IKESAKA MORIO, ISHIHATA HIROAKI, HORIE, TAKESHI, IKESAKA, MORIO, ISHIHATA, HIROAKI
Applicants: FUJITSU LTD, FUJITSU LIMITED
Application year: 1990
Main classification: G06F15
IPC codes: G06F15