Patent Publication Number: US-10311013-B2

Title: High-speed inter-processor communications

Description:
TECHNICAL FIELD 
     The disclosed technology relates generally to high-speed inter-processor communications with a flexible interconnect topology. 
     BACKGROUND 
     Clusters of computing devices including interconnected computer nodes are sometimes employed to process high-volume data or computation tasks. A computing cluster is a set of computing devices, e.g., configured as a computing network comprising multiple computing devices. Various data communications technologies have been deployed to enable the computing devices to exchange data, e.g., Ethernet, Fiberchannel, etc. However, these technologies generally exchange data more slowly than processors are able to process data. Different techniques to reduce interconnection overhead and latency have been tried on both software and hardware levels, but such techniques are limited by conventional system architectures of data pathways. 
     To improve performance, some computing devices have been designed to accommodate multiple processors. More recently, specialized processors (e.g., math processors, graphic processing units (GPUs), field programmable gate arrays, etc.) have been adapted for use with various computational processes. These specialized processors are referenced herein as “accelerators,” but various terms are commonly used to refer to these types of processors. Typically, accelerators are used when intensive computation, typically parallel mathematical computation, is involved. However, current computational needs have outpaced even the capabilities of accelerators. Some computing devices can operate with multiple accelerators. However, accelerators can consume and generate data much more quickly than standard computing buses (e.g., Peripheral Component Interconnect Express, or “PCIe”) and so standard interconnections between accelerators become bottlenecks. Moreover, interconnection topologies are fixed and cannot easily be changed to satisfy application requirements. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram illustrating an example of a computing environment employing multiple computing devices, consistent with various embodiments. 
         FIG. 2  is a data path diagram illustrating a conventional GPU architecture. 
         FIG. 3  is a data path diagram illustrating a memory channel data transport architecture, consistent with various embodiments. 
         FIG. 4  is a block diagram illustrating a computing device implementing an inter-processor communications architecture, consistent with various embodiments. 
         FIG. 5  is a block diagram illustrating interconnected daughter cards, consistent with various embodiments. 
     
    
    
     The figures depict various embodiments of the disclosed technology for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments may be employed. 
     DETAILED DESCRIPTION 
     Overview of Technology 
     A high-speed inter-processor communications architecture with a flexible interconnect topology is described. In various embodiments, the architecture includes a motherboard, multiple daughter boards, and “interconnections” or “interconnect ports” (e.g. ports, cables, printed circuit boards (PCBs), etc.) between the multiple daughter boards. The motherboard may include one or more conventional central processing units (CPUs) and multiple connectors (e.g. PCIe connectors). Each daughter board may occupy one of the connectors, and may employ it to exchange data and receive signaling information from other devices connected to the motherboard, e.g., a CPU. Each daughter board may include one or more accelerators, memory, and one or more interconnection ports per accelerator. 
     In various embodiments, each daughter board may include two, four, eight, or more interconnection ports per accelerator. In some implementations, a motherboard can have three or more attached daughter boards. One or more interconnection ports of a daughter board may be communicably connected via an interconnection cable or PCB to an interconnection port of a different daughter board. In so doing, one or more of the accelerators associated with each of the two interconnected daughter boards may be able to signal, exchange data, or share memory access with each other. Thus, the interconnection can form a mesh, fabric, or indeed any type of topology. A mesh topology exists when every accelerator is directly interconnected with every other accelerator. A fabric topology exists when some accelerators are interconnected with some other accelerators, but typically each accelerator can communicate with any other accelerator via at least one intermediate accelerator. In some implementations, each accelerator can communicate with any other accelerator via no more than one intermediate accelerator. In various implementations, this allows inter-processor communication across daughter boards to be flexible, configurable, or separate from PCIe connections with the motherboard. 
     In various embodiments, the accelerators may be manufactured by different manufacturers and daughter boards having accelerators from different manufacturers may or may not be capable of being directly connected or mixed on a motherboard. Typically, however, a computing device would only have daughter boards having accelerators manufactured by a common manufacturer or at least having a common standard. 
     In some embodiments, the interconnection ports and interconnection cables may use industry standard high speed communications standards, e.g., Quad Small Form-factor Pluggable (QSFP) connectors and cables. Even if standard interconnection ports and cables are used, different manufacturers may select different pinout configurations. In other embodiments, the interconnection ports and interconnection cables may be proprietary to the manufacturer of the accelerators associated with the daughter boards or some other manufacturer or product designer. In various embodiments, the interconnection ports can be on any location and in any direction of the daughter card. 
     An operator can interconnect the daughter boards in different configurations, e.g., to create different interconnection topologies. As an example, a topology may be selected based on the computing intensive application that the accelerators will function with. Machine learning is an example of a type of intensive computational process that benefits from the use of accelerators. Different machine learning algorithms exist. One such example is “deep learning.” Deep learning involves several layers of nonlinear processing units (e.g., “neurons”) that extract and transform “features” in underlying data. Typically, these nonlinear processing units can operate independently and then share their results with other nonlinear processing units generally in a “higher” layer. Sometimes, nonlinear processing units may share results with other nonlinear processing units in the same (or even a “lower”) layer. Embodiments of the architecture described above can be adapted for almost any type of machine learning algorithm. As an example, one or more computing devices may be assigned to each layer so that computations of a layer can be performed very quickly and intermediate results can be shared between accelerators on an as-needed basis. If the machine learning algorithm is modified, the topology can be easily changed without having to purchase and deploy new hardware. Moreover, if new accelerator designs become available and desirable, other portions of the computing devices do not need to be replaced. Thus, embodiments described herein enable a highly flexible topology architecture for interconnecting processors using high-speed communications links. 
       FIG. 1  is a block diagram illustrating an example of a computing environment  100  employing multiple computing devices, consistent with various embodiments. The computing environment  100  may sustain high bandwidth data sharing and processing. For example, the computing environment  100  may be a computing cluster, a server rack, or a server tray. As illustrated for example, the computing environment  100  may include a disaggregated rack  102 . The disaggregated rack  102  can be a computer cluster in which functional elements of the computer cluster are separated into separate devices, e.g., a networking (e.g., input/output (IO) processing) device  104 A, a processing device  104 B, a cache device  104 C, a storage device  104 D, and a memory service device  104 E (collectively referred to as the “computing devices  104 ”). 
     The computing devices  104 A-E may be a computer, e.g., computer server, server sled, computer tray, desktop computer, or other types of computers. Each of the computing devices  104  may include a processor (e.g., CPU), a motherboard and a volatile memory. 
     In various embodiments, interconnects  106  coupled to ports  108  may be a medium for inter-processor data transportation. The ports  108  may enable computing devices  104  to exchange data via various high-speed interconnects  106 . The inter-processor interconnects  106  may be a bus or cable. The inter-processor interconnects  106  may be multi-lane or single lane and may transmit or receive data via optics, electric signals, electro-magnetic signals, or other means of data communication. In some embodiments, each of the computing devices  104  may also include a network card, e.g., an optical or Ethernet NIC (not illustrated). The network card may be connected via a peripheral component interconnect (PCI) bus on the computing device&#39;s motherboard. Conventional inter-device architectures may utilize the network card as a portal for inter-processor communication, where a cluster switch or router acts as a nexus for inter-processor communications. As an example, a computing device may employ the network card to communicate with various computing devices. 
     Disaggregation enables flexible allocation and/or re-allocation of system resources in the cluster computing environment  100  through customization of rack resources, and thus improving cluster scalability. For example, the networking device  104 A may include one or more network components  110 A (e.g., a switch or a network buffer); the processing device  104 B may include one or more processor components (e.g., accelerators)  1106 , the cache device  104 C may include one or more cache memories  110 C (e.g., solid state drives); the storage device  104 D may include one or more persistent storage devices  110 D (e.g., hard disks); and the memory service device  104 E may include one or more random access memory modules  110 E. The network components  110 A, the processor components  110 B, the cache memories  110 C, the persistent storage devices  110 D, and the random access memory modules  110 E may be collectively referred to as “the resource components  110 ”. Because the resource components  110  may serve the same client application, a same data set may be exchanged amongst multiple computing devices  104 . Each of the resource components  110  can be independently serviced and/or upgraded, e.g., to suit the needs of applications the computing devices  104  may be configured to execute. As examples, a database application may receive faster storage devices  110 D, a machine learning processing application may receive processor components  110 B designed to speed up machine learning, and a web application may receive larger cache memories  110 C. 
       FIG. 2  is a data path diagram illustrating a conventional GPU architecture  200 . In the conventional architecture  200 , a conventional computing device executes an application using its CPU  206 . The application  206  can then transfer data via a memory module  207 . For example, the application  206  can move the data set to a region of memory. The GPU  210  may then receive a signal via the PCIe bus to retrieve the data from the memory module  207  and software executing on it may process the data. After processing the data, the process is reversed. 
       FIG. 3  is a data path diagram illustrating an accelerator architecture  300 , consistent with various embodiments. A computing device (e.g., one of the computing devices  104 ) can execute an application at a first accelerator  306 . The application executed by the accelerator  306  can then directly access a memory module  308  that is on the same daughter board as the accelerator. A second accelerator  310 , typically associated with a different daughter card in the same computing device but can be associated with the same daughter card as the first accelerator  306 , can then access the data stored in the memory module  308 . This access can occur via a high speed interconnect as described in detail above. 
     Thus, the high speed inter-processor communications architecture completely bypasses conventional modules. A comparison of  FIG. 2  and  FIG. 3  illustrates that the inter-processor communications architecture is advantageous at least because of the ability to reduce data processing latency because communications over interconnection ports and cables can occur at much higher speeds than was typical. 
       FIG. 4  is a block diagram illustrating a daughter board environment  400  implementing an inter-processor communications architecture (e.g., the architecture  300 ), consistent with various embodiments. The daughter board  400  environment includes a daughter board  404 . The daughter board  404  may comprise any rigid material including plastic, metal, alloy, carbon fiber, or any combination thereof. For example, the daughter board  404  may be a printed circuit board mountable on a mother board via a PCIe connector (not illustrated). 
     The daughter board  404  may include a specialized processor  402  (e.g., accelerator). The processor  402  can be coupled to one or more memory modules  406 . In some implementations, processor  402  can include memory modules  406 . In various embodiments, the processor  402  may act as a memory controller. In other embodiments, a separate component may be implemented as the memory controller. 
     In some embodiments, computing devices or daughter boards of computing devices are interconnected. A port  416  of a daughter card may be interconnected with other ports of the same or a different daughter card (not illustrated) via an inter-processor interconnects  418 . The port  416  and interconnect  418  may conform with an industry standard, e.g., QSFP, or may be proprietary. 
     The daughter board  404  may also include one or more PCIe connectors  420 . The PCIe connector  420  provides a data signal path for components and modules of the computing device (not illustrated) to communicate with the processor  402 . 
       FIG. 5  is a block diagram illustrating interconnected daughter cards, consistent with various embodiments. As illustrated, a first daughter card  502  and a second daughter card  504  may each be interconnected via an inter-processor interconnect  510 , e.g., the inter-processor interconnect  418 . The daughter card  502  and the second daughter card  504  may communicate with other devices of their respective computing devices via PCIe connectors  506  and  508 , respectively. 
     The computing devices on which the described technology may be implemented may include one or more central processing units, memory, input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), storage devices (e.g., disk drives), and network devices (e.g., network interfaces). The memory and storage devices are computer-readable storage media that may store instructions that implement at least portions of the described technology. In addition, the data structures and message structures may be stored or transmitted via a data transmission medium, such as a signal on the communications link. Various communications links may be used, such as the Internet, a local area network, a wide area network, or a point-to-point dial-up connection. Thus, the computer-readable media can comprise computer-readable storage media (e.g., “non-transitory” media) and computer-readable transmission media. 
     Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. Accordingly, the disclosed technology is not limited except as by the appended claims.