When a node receives a message, it must have a place in memory to store the message. This place may have been specified by a crecv or irecv call, but the message may arrive well before the node gets to either call. Even if the node has executed these calls, the specified location in memory may be paged out. In either case, the node needs an area to store the incoming message temporarily. The sending node should not send more data than the receiving node can handle.
Every node reserves an area of memory to store incoming messages.
The area, referred to as the XMSG buffer, is set to 1 Mbyte by default. You can set the size of this buffer at run time with the -mbuf option. Setting a larger XMSG buffer may allow deadlocked programs to run correctly. The size of the XMSG buffer can also impact a code s performance. This code may deadlock depending on the size of x,y and the XMSG buffer.