Mechanism for optimized intra-die inter-nodelet messaging communication

Point-to-point intra-nodelet messaging support for nodelets on a single chip that obey MPI semantics may be provided. In one aspect, a local buffering mechanism is employed that obeys standard communication protocols for the network communications between the nodelets integrated in a single chip. Sending messages from one nodelet to another nodelet on the same chip may be performed not via the network, but by exchanging messages in the point-to-point messaging buckets between the nodelets. The messaging buckets need not be part of the memory system of the nodelets. Specialized hardware controllers may be used for moving data between the nodelets and each messaging bucket, and ensuring correct operation of the network protocol.

FIELD

The present application relates generally to computers, and computer applications, and more particularly to computer architecture and more particularly to messaging in a semiconductor chip or die.

BACKGROUND

Electronic circuit chips (or integrated semiconductor circuit) are being built with increasing numbers components integrated on the chips. A single chip is fabricated to hold an integration of multiple nodelets. Even still, each nodelet on a single chip can have a number of processors. Processors in a nodelet can be homogeneous (i.e., of the same type) or heterogeneous (i.e., of different types). Each nodelet has its memory system, however, memory between nodelets are not shared. That is, each nodelet has a separate memory coherence domain.

In a multi-node system, nodes communicate between each other by using one or more network protocols. For many applications, the amount of communication between neighboring nodes is higher than between remote nodes. Similarly, communications between neighboring nodes is more frequent than between the more remote nodes. Mapping logically “close” nodes to physically neighboring nodes reduces latency and power consumption. By mapping logically close nodes to nodes on the same chip, significant part of the communication stays on the chip. Nodelets participate in a larger multi-node system by network connections using a network protocol, typically using Message Passing Interface (MPI) protocol.

Network communication, however, still involves overhead such as the work that needs to be implemented for network protocol tasks, transmitting packets, and receiving packets.

Message Passing Interface (MPI) is a programming paradigm used for high performance computing (HPC). The model has become popular mainly due to its portability and support across HPC platforms. Because MPI programs are written in a portable manner, programmers optimize application-related aspects, such as computation and communication, but typically do not optimize for the execution environment. In particular, MPI tasks are often mapped to the processors in a linear order.

Determining the communication patterns of applications have been studied by A. Aggarwal, A. K. Chandra, and M. Snir. On communication latency in PRAM computation. In Proceedings of the ACM Symposium on Parallel Algorithms and Architectures, pages 11-21, June 1989, and by A. Alexandrov, M. F. Ionescu, K. E. Schauser, and C. Scheiman. Log GP:Incorporating long messages into the Log P model for parallel computation. Journal of Parallel and Distributed Computing, 44(1):71-79, 1997.

Independently of such communication pattern studies, another category of existing technology provides a model to guide the MPI programmer. However, early models explicitly ignored hardware characteristics to simplify the model. More recent models (see, D. Culler, R. Karp, D. Patterson, A. Sahay, K. E. Schauser, E. Santos, R. Subramonian, and T. von Eicken.Log P: Towards a realistic model parallel computation. In Proceedings of the ACM SIGPLAN Symposium on Principles and Practices of Parallel Programming, May 1993; and M. I. Frank, A. Agarwal, and M. K. Vernon.LoPC: Modeling contention in parallel algorithms. In Proceedings of the ACM SIGPLAN Symposium on Principles and Practices of Parallel Programming, pages 276-287, June 1997) attempt to develop a theoretical model for generic networks. However, such modeling has not employed empirical data to improve the model accuracy. With the existing techniques, it is difficult to obtain performance benefits.

BRIEF SUMMARY

A method and system for intra-die inter-nodelet messaging communication may be provided. The method, in one aspect, may include allocating a bucket comprising a memory array and hardware control logic that supports message passing interface semantics, for communicating data between a first process on a first memory domain and a second process on a second memory domain, wherein the first memory domain and the second memory domain are not shared, and wherein the bucket is not part of the first memory domain or the second memory domain. The method may also include mapping the bucket to the first process. The method may further include writing, by the first process, message data to the bucket and invoking a send message passing interface function that raises a hardware signal to the second process. The method may yet further include mapping the buffer to the second process in response to the second process invoking a receive message passing interface function, wherein the second process is enabled to read the data in the mapped bucket.

A system for intra-die inter-nodelet messaging communication, in one aspect, may include a plurality of nodelets on a single chip, each of the nodelets having its own memory coherence domain that is not shared with the rest of the nodelets on the single chip, each nodelet comprising one or more process cores, wherein the plurality of nodelets comprise at least a first nodelet having a first process core and a first memory coherence domain, and a second nodelet having a second process core and a second memory coherence domain. The system may also include a bucket comprising a memory array and hardware control logic that supports message passing interface semantics, for communicating data across the plurality of nodelets, wherein the bucket is not part of the memory coherence domains of the nodelets. The first process core is enabled to map the bucket to the first process core, write message data to the bucket and invoke a send message passing interface function that raises a hardware signal to the second process core. In response to the second process core invoking a receive message passing interface function, the buffer is mapped to the second process core for enabling the second process core to read the data.

A method for intra-die inter-nodelet messaging communication, in another aspect, may include reserving a bucket comprising a memory array and hardware control logic that supports message passing interface semantics, for communicating data between a first process on a first memory domain and a second process on a second memory domain, wherein the first memory domain and the second memory domain are not shared, and wherein the bucket is not part of the first memory domain or the second memory domain. The method may also include setting a plurality of control bits to indicate exclusive read and write access for the first process only. The method may further include receiving a send call invoked by the first process. The method may yet further include setting the control bits to indicate shared read and write access for the first process and raising a hardware signal for the second process. The method may still yet include receiving a receive call invoked by the second process. The method may further include setting the control bits to indicate shared read and write access for the second process. The method may also include, in response to receiving an un-map call from the first process, setting the control bits to indicate exclusive read and write access for the second process. The method may also include, in response to receiving an un-map call from the second process, setting the control bits to indicate exclusive read and write access for the first process.

DETAILED DESCRIPTION

The most network traffic when running scientific and high performance applications within a complex multi-node system is between relatively local nodes, with only smaller part going to relatively remote nodes in a system. Thus, it would be beneficial to have fast and efficient way of communicating between the local nodes.

The present disclosure describes communication mechanisms across different memory domains. In one aspect, low-overhead, low-latency point-to-point intra-nodelet messaging support for nodelets on a single chip that obey MPI semantics is provided. In one aspect, a local buffering mechanism is employed that obeys standard communication protocols for the network communications between the nodelets integrated in a single chip. Sending messages from one nodelet to another nodelet on the same chip is performed not via the network, but by exchanging messages in the point-to-point messaging buckets between the nodelets in one embodiment of the methodology of the present disclosure. The messaging buckets are not part of the memory system of the nodelets. Specialized hardware controllers are used for moving data between the nodelets and each messaging bucket.

FIG. 1illustrates components of a multi-nodelet chip in one embodiment of the present disclosure. Multiple nodelets (e.g.,104,106) may be integrated in a chip102. Each nodelet (e.g.,104,106) may have one or more homogeneous or heterogeneous processor cores. Each nodelet (e.g.,104,106) also has its separate memory system. Memory between nodelets (e.g.,104,106) is not shared. Nodelets (e.g.,104,106) participate in a larger multi-node system using network connections114and the MPI protocol. Messages from a process within a nodelet to other processes located on different chips are sent vie network interface and network. Messages from a process within a nodelet to another process in a different nodelet on the same chip is not preformed via the network but by exchanging messages in the point-to-point messaging buckets (e.g.,108,110,112) between the nodelets. In one embodiment, point-to-point messaging buckets (e.g.,108,110,112) are used for transferring MPI data between the nodelets (e.g.,104,106) on the same chip102. A bucket (e.g.,108,110,112) comprises memory space and hardware control logic that obeys messaging protocol such as MPI, and can be used by all nodelets on the chip to transfer messages from one nodelet to another nodelet on the same chip. In one embodiment, messaging buckets (e.g.,108,110,112) are not a part of the memory system of any of the nodelets on the chip. In one embodiment, hardware controllers in each nodelet (e.g.,104,106) are used for accessing the messaging buckets (e.g.,108,110,112). Messaging buckets (e.g.,108,110,112) in one embodiment support the MPI network protocol. Messaging buckets contain memory and control logic for receiving messages from one nodelet, informing another nodelet about the message waiting, and ensuring exclusive access to a specific memory within the bucket as defined by the MPI protocol for message transfer between two nodes. Details on how this is implemented will be apparent from the description below.

FIG. 2is a diagram that illustrates a message communication mechanism between nodelets in one embodiment of the present disclosure. Process0216(e.g., a core) on nodelet0204of a chip202may send a short MPI point-to-point message to Process1218(e.g., a core) on nodelet1on the same chip202, for example, as follows. Message sending between nodelets on the same chip is performed by using messaging buckets. Prior to sending a message from Process0216to Process1218, an available bucket is identified for this transfer, and the identified bucket is reserved for the exclusive use for this message during the message transfer. This reservation is done transparently to an application by an operating system (OS). Process0216initiates this functionality, for example, via a function or utility call such my_addr=bucket_alloc. This function, bucket_alloc, identifies an available bucket. This is an analogous call to malloc, and causes memory from bucket to appear normally in the calling process's address space208. In one embodiment, the size of the bucket can be specified in the function as an argument. In another embodiment, the size of the buffer in the bucket can be predefined. Yet in another embodiment, different buckets have specified different sizes of buffers. Now, the new memory area appears as a part of the address space of the Process0, and this process can write to this memory by addressing it. For example, Process0may perform appropriate calculations or computations, and write data to the allocated address. This maps Bucket0208to Process0216, and Process0can perform an MPI_Send.

When Process0is ready to send a message from the Bucket0to Process1, it calls MPI_Send function. The specialized control hardware in the Bucket0informs Process1that there is a message for it to receive, and that its location is Bucket0. This triggers Process1to issue MPI_Recv call which will effectively map Bucket0to memory space of Process1. After Process0216and Process1218call MPI_Send and MPI_Recv respectively, Bucket0208is mapped to both processes216,218. With this, Bucket0belongs to the memory space of both processes, and both processes have full read and write access to it. However, to respect MPI syntax (or other network protocol syntax), a message, once sent, cannot be modified from the process which generated it any more, in one embodiment of the present disclosure. Similarly, in one embodiment of the present disclosure, if the message is still in the memory area of the Process0, it cannot be modified by any other Process (including Process1). Thus, when either of the process writes this Bucket0208, a “copy-on-write” protocol is triggered, where a new copy of the bucket is generated. The mapping is adjusted so that buckets now point to the correct owners.

FIG. 3is a flow diagram illustrating a nodelet-to-nodelet communication in one embodiment of the present disclosure. A process or core (e.g., referred to as Process0or first process) on a nodelet (e.g., referred to as nodelet0or first nodelet) may send a message, e.g., a short MPI point-to-point message, to another process or core (e.g., referred to as Process1or second process) on another nodelet (e.g., referred to as nodelet1or second nodelet). At302, a bucket is created and mapped to Process0. A function call may be provided to create a bucket and map a process to it. For example, send_addr=Bucket_alloc (size, −1) (Process0) may create a bucket and map to a process named Process0. After the bucket is mapped, Process0can write message data to the bucket in the same way it is accessing any other memory. Once the message is ready to be sent, at304, Process0sends a message by invoking a send function that is provided in the present disclosure. The send function specifies a number of parameters including information such as the address of the created bucket, the communicator information or group identifier of Process0, format of the data, and the identifier of the recipient node which should receive the message. Invoking the send function raises a hardware signal to the recipient, the specific core on nodelet1. The raised signal is received on nodelet1, and it can either contain information to identify the receiving node and may contain the specific bucket (e.g., bucket_id) information with an active message waiting to be received, or it is a signal for the receiving nodelet that a message is waiting, after which a function is invoked to identify which node is receiving a message and in which bucket the message is waiting. An example of such send function is MPI_Send (send_addr, communicator_info, data_type, Process1), where send_addr is a reference pointer to, or an address of the bucket, communicator_info is the group information that the recipient process is part of, data_type is the data format of the data being communicated, and Process1is the recipient of the message.

At306, the recipient process, Process1, gets the signal. At308, depending on whether Process1had called a receive function before or after the signal, two scenarios may occur. An example of such receive function is MPI_Recv. In one embodiment of the present disclosure, only after Process1calls MPI_Recv function, a message can be received. If Process1did not call the MPI_Recv, the hardware signal remains in the pending state. At310, after Process1calls MPI_Recv (recv_addr, communicator_info, data_type, Process0), recv_addr is mapped to bucket_id, and the bucket memory is mapped to the receive addresses in the receiver memory space. The receiving process, Process1, is enabled to read the data at the recv_addr. The message data already available in the bucket can be now accessed by Process1. Thus, Process1receives the message, and the message is already in its memory. At312, the hardware signal from the bucket is reset to reflect the status of the message being delivered.

FIG. 4illustrates how a bucket may be controlled for a write and read from a sender and receiver processes during a message transfer in one embodiment of the present disclosure. At402, a sender process may reserve a bucket. At404, control bits are set for exclusive read/write access on this bucket for the sender process only. At406, if the sender process invokes a send call or function, the control bits are set for shared read/write access for the sender process and a hardware signal is raised for a receiver process at408. At410, if the receiver process invokes a receive call or function, the control bits are set for shared read/write access for the receiver process. At414, if the sender process or receiver process invokes an un-map call or like function, the bucket is disconnected from that process, and the control bits are set for exclusive read/write access for the other process which still has the bucket mapped. At418, when both processes are disconnected from the bucket, the bucket is released.

FIG. 5illustrates an example of a message bucket in one embodiment of the present disclosure. The example shows two nodelet bucket implementation for point-to-point messaging. A bucket500may include an array of memory508and hardware control logic514for accessing the memory array that complies with a network protocol. A bucket may also include a status/control (SC) register502, and a set of access bits510for each element of the memory array508. SC register502stores an indication of types of access a nodelet has for the elements of the memory array508. The types of access that the SC register502may indicate may include, but not limited to, exclusive write, exclusive read, shared write, shared read, exclusive read and write, shared read and write, and others. In one embodiment, memory array508has two sets associated with each address, set0512and set1514. Consider that a process on nodelet0(N0)504allocated a buffer or memory array of bucket500. Bucket500assigns exclusive read/write (RW) status to Nodelet0504. Nodelet0504writes message data to the bucket502by using the set0512of the bucket for the data. Nodelet0504issues mpi_send specifying the address of the allocated bucket, e.g., addr1. Message bucket (MB) logic decodes addr1, and two scenarios may occur based on whether the addr1 is on the same chip, e.g., Nodelet1506, or whether addr1 is outside the chip. If addr1 is on the same chip, the logic of the message bucket502notifies Nodelet1506, for example, by issuing an interrupt. If addr1 is not on the same chip, the logic of the message bucket500notifies the network message unit (MU) connected to the chip that a message is ready to be sent via network interface on the chip (not shown). Nodelet1506maps allocated receive buffer to bucket500. Bucket500assigns read (R) status to Nodelet1506, and R status to Nodelet0504. The access status and sharing status per each process is indicated in the SC register502. If both processes issue only read accesses, data is read out from the Set0512, and data in set1is not specified and not accessed.

In this example, two words (w0 and w1) in memory are mapped to every address word, set0and set1. When a message is written, it is written in the first set, and all access bits510for all words written in the first word set are set to 11 for w0 and 00 for w1. If Nodelet0504or Nodelet1506issues a write to the allocated buffer after the access status was set to shared, the message bucket500detects a conflict. The message bucket500saves the modified word in the first set to w1 location of the memory bucket, and sets the access bits for the modified word only in w0 to 01, and for w1 to 10—thus, Nodelet1sees its buffer unmodified, and Nodelet0sees its modification. If Nodelet0504issues a write to the allocated buffer into a word with w0 11 and w1 00, it writes the modified word in the w1, and the message bucket sets bits for w0 to 10 and for w1 to 01. These two sets of access bits define for each memory element and for the two sets which process has access to each set. The two access bits define the visibility of the set in this memory entry for each of the two processes. Thus, for example value 01 for w0 and 10 for w1 indicate that Nodelet0504has access to the set1, but not to the set0, whereas Nodelet1has access to the set0but not to the set1. In this way, there are two different copies of the conflicting data which one of the processes modified after a mpi_send call was issued. Nodelet0504and Nodelet1506have their private copies of data which they can modify without changing the data for the other process. For memory entries which were not written by any of the processes after mpi_send was issued, data are placed in the set0, and access bits for this memory entry are 11 for w0 and 00 for w1.

FIG. 6is a flow diagram illustrating invoking of a buffer copy in one embodiment of the present disclosure. At602, a memory access request to a bucket from a process, e.g., referred to as process0is received. At604, control bit setting in the bucket for process0is checked. In one embodiment of the present disclosure, once a bucket is mapped, only sending process0and receiving process1have access to it. At606, it is checked whether process0has access to this bucket. If so, at608, it is checked whether the request is a request to write to the bucket. If so, at610, it is checked whether process0has exclusive access. If so, data is written to a first word set, set0, in bucket memory at612. Otherwise, if process0does not have exclusive access but shared, data is written to a second word set, set1in bucket memory at614. At608, if it is determined that the request is not a write request, at616, it is checked whether a modified word exists. If so, at618, data in the second word set, set1is returned. If not, at620, data in the first word set, set0is returned. While this flow chart describes access for the first process0, the access to the bucket from a receiver process, e.g., referred to as process1can be performed by always accessing the first word set, set0. For example, the first nodelet which starts writing to a shared word will start using set1words w1, and the other nodelet (a receiver nodelet) will continue using set0words w0. The access bits are modified accordingly and can be different from row to row in the memory array.

FIG. 7illustrates multiple nodelets on a die in one embodiment of the present disclosure. A die shown inFIG. 7may be a piece of semiconductor material or dielectric upon which one or more electrical components may be integrated, e.g., mounted, etched, or formed. In one embodiment, buckets (e.g.,712,714,716,718) are shared across all the nodelets (e.g.,704,706,708,710) on the multi-node chip702, and can be accessed by all nodelets on the chip. In one embodiment, only two nodelets—one sending and one receiving nodelet—are connected to a bucket. In another embodiment, each bucket (e.g.,712,714,716,718) may be mapped to more than one nodelet (e.g.,704,706,708,710) where one nodelet is sending a message, and other nodelets are receiving the message. For example, Process0on Nodelet0(704) can send the same message to all the other nodelets, Nodelets1(706), Nodelet2(708) and Nodelet3(710), in which case all of those nodelets point to the same bucket, and are mapping the memory area of that bucket to their memory spaces, and can access data from the same bucket.

FIG. 8illustrates an example scenario when a bucket buffer is used for sending a modified message by a process in one embodiment of the present disclosure. Initially, a process820on Nodelet0(804) sends a message to Nodelet1(806) and Nodelet3(810). Initially, Nodelet0(804) is a sender of the message in Bucket01812) and Nodelets1(806) and3(810) will point to Bucket01(812) as the recipients of that message. The scenario illustrated in this figure occurs if a process820in Nodelet0(804) modifies its buffer (812) to send it to another process, for example to Nodelet2(808). In this case, before the modified data is written to Bucket01, a new bucket (e.g.,814) with a copy of Bucket01data is generated to keep the message to Nodelet1(806) and Nodelet3(810). The new bucket (e.g.,814) holds the message that was in Bucket01(812) before the message in that bucket is modified. In one embodiment, hardware control logic makes a copy of Bucket01(812) into the new bucket (e.g.,814) and changes pointers, so that Nodelet1(806) and Nodelet3(810) point to the new bucket (e.g.,814). In another embodiment, new data (modified data) may be written to Bucket02instead, and Bucket01left intact with original data, in which case the pointers of Nodelet1806and Nodelet3810would be left to point to Bucket01812; and Nodelet2808would point to Bucket02814instead to receive the modified message.

FIG. 9illustrates an example of a message bucket, in which multiple nodelet bucket implementation is provided for communicating a message to multiple nodelets. A bucket900comprises memory array910, access bits912, status register902, and control logic914. The memory array910can be accessed from multiple nodelets904,906,908and contains a number of sets for each entry of the memory array in the buckets. Each row of the memory array within a bucket has L sets of words in parallel, with L words (w0, w1, . . . wl-1) mapped to each address. Thus, to any address in the memory space within the bucket, L words are mapped. In one embodiment, the number of words L is 2 or more, but is less or equal to the number of nodelets who can access it. Each memory array element has M access bits912assigned to it, where M is the number of nodelets who can access and map this bucket into their address simultaneously. The m-th bit for any word w-sub-k specifies the ownership of that word to the m-th nodelet. Initially, the ownership of the first set0containing words w0 is assigned to the sending and receiving nodelet. If the sending or receiving nodelet attempt to write to that set0and the bucket is in the shared mode, a conflict is detected and the new word is written into another word, e.g., word w1. The access bits for the words, w0 and w1, are modified to describe the new ownership. In this example, the access bits for word w1 are set to those of the access bits for word w0 before the attempt to write to that buffer. The access bits for w0 would be set to include the recipients of the rewritten buffer (w0). If the sending nodelet sends this buffer (w0) to the third or fourth nodelet, more ownership bits are set for each word owned by the sender. There can be up to L different versions of the buffer, with ownership specified by access bits, and participating nodelets and the bucket mode being specified in the SC register.

The buckets of the present disclosure can also be generalized to other programming models.

For example a universal program counter (UPC) creates a shared memory array by using calls such as upc_all_alloc, upc_global_alloc. These can be accessed by all the threads. In another aspect, MPI remote memory access (RMA) has a notion of memory windows, with each process exposing a specific memory size to all the processes, e.g., MPI_WIN_CREATE. When these processes or threads are within a node, the buckets can be used to expose this window or arrays to one-another. The buckets provide the necessary “coherency domain” for accesses to this arrays or Memory windows. These models have different memory consistency semantics, for example, UPC dictates either writes to be relaxed or strict. RMA also has the concept of local “vs” remote memory window, which need to be explicitly synched so that the memory is consistent. The bucket logic of the present disclosure in one embodiment can be extended to incorporate these programming model related semantics and consistency models.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including a hardware description language (HDL), an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages, a scripting language such as Perl, VBS or similar languages, and/or functional languages such as Lisp and ML and logic-oriented languages such as Prolog as applicable. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).