Patent Publication Number: US-7716393-B2

Title: Network chip design for grid communication

Description:
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT 
   The present invention was made with the support of the government of the United States under contract NBCH020055 awarded by the Defense Advanced Research Projects Administration. The United States government may have certain rights in the present invention. 

   BACKGROUND 
   A typical computer system includes several components that are collectively used by a user to perform various functions such as, for example, preparing and generating a document with a word processor application. With the computer system, the user may input data to a computing portion using peripheral devices such as a keyboard or a mouse. Data may also be provided to the computing portion using data storage media, e.g., a floppy disk or a CD-ROM. The computing portion, using memory and other internal components, processes both internal data and data provided to the computing portion by the user to generate data requested by the user. The generated data may be provided to the user via, for example, a display device or a printer. 
   The computing portion of a computer system typically includes various components such as, for example, a power supply, disk drives, and the electrical circuitry required to perform the necessary and requested operations of the computer system. The computing portion may contain a plurality of circuit boards on which various circuit components are implemented. For example, a computing portion designed to have enhanced sound reproducing capabilities may have a circuit board dedicated to implementing circuitry that specifically operates to process data associated with the reproduction of sound. 
   On a circuit board, a clock provides a reference of time to various integrated circuit (IC) packages that are connected onto the circuit board. Those skilled in the art will recognize that the integrated circuit packages may be used to house and support various types of integrated circuits (e.g., application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microprocessors, and digital logic chips). The integrated circuit packages communicate with one another, i.e., pass data, using wires or traces of conductive material (e.g., copper or gold) embedded in the circuit board. 
   SUMMARY 
   According to one aspect of one or more embodiments of the present invention, a system comprises: at least one central processing unit; at least one memory unit; and at least two integrated circuits configured to communicate with one another by proximity I/O communication, the at least two integrated circuits further configured to propagate data packets between the at least one central processing unit and the at least one memory unit, where at least one of the two integrated circuits is configured such that a rotation of the at least one of the two integrated circuits by 90 degrees does not effect the proximity I/O communication between the at least two integrated circuits. 
   According to another aspect of one or more embodiments of the present invention, a system comprises: at least one central processing unit; and a plurality of integrated circuits configured for proximity I/O communication and configured to propagate data packets between the at least one central processing unit and a component of the system, where data propagation between the at least one central processing unit and the component is unaffected by a rotation of at least one of the plurality of integrated circuits by 90 degrees. 
   According to another aspect of one or more embodiments of the present invention, a method of performing system operations comprises: propagating data to a switching fabric formed of a plurality of integrated circuits configured for proximity I/O communication; propagating the data though the switching fabric, where propagating the data through the switching fabric is unaffected by a rotation of at least one of the plurality of integrated circuits by 90 degrees; and propagating the data from the switching fabric. 
   According to another aspect of one or more embodiments of the present invention, a method of manufacturing a system comprises: configuring a plurality of integrated circuits to communicate by proximity I/O communication; connecting at least one central processing unit to the plurality of integrated circuits, where the plurality of integrated circuits is configured such that data propagation to and from the at least one central processing unit through the plurality of integrated circuits is unaffected by a rotation of at least one of the plurality of integrated circuits by 90 degrees. 
   Other aspects of the invention will be apparent from the following description and the appended claims. 

   
     BRIEF DESCRIPTION OF DRAWINGS 
       FIG. 1  shows a portion of a system in accordance with an embodiment of the present invention. 
       FIG. 2  shows a switching fabric in accordance with an embodiment of the present invention. 
       FIG. 3A  shows integrated circuits configured for proximity communication. 
       FIG. 3B  shows an enlarged view of a portion shown in  FIG. 3A . 
       FIG. 4  shows an integrated circuit of a switching fabric in accordance with an embodiment of the present invention. 
       FIG. 5  shows a portion of an integrated circuit of a switching fabric in accordance with an embodiment of the present invention. 
       FIG. 6  shows a portion of an integrated circuit of a switching fabric in accordance with an embodiment of the present invention. 
       FIG. 7  shows a portion of an integrated circuit of a switching fabric in accordance with an embodiment of the present invention. 
       FIG. 8  shows an integrated circuit of a switching fabric in accordance with an embodiment of the present invention. 
   

   DETAILED DESCRIPTION 
   Exemplary embodiments of the invention will be described with reference to the accompanying drawings. Like items in the drawings are shown with the same reference numbers. Moreover, numerous specific details are set forth in the context of embodiments of the present invention in order to provide a more thorough understanding of the invention. In other instances, well-known features have not been described in detail to avoid obscuring the invention. 
   In an exemplary system in accordance with an embodiment of the present invention, the system is divided into a plurality of computing modules. As shown in  FIG. 1 , a module  100  includes one or more central processing units (CPUs)  102 , one or more memory units  104  (e.g., cache memory, memory controllers), and a switching “fabric”  106  for switching data packets between, among other things, (i) the CPUs  102  and the memory units  104  and (ii) between module  100  and another module (not shown). 
   A switching “fabric,” like that shown in  FIG. 1 , may be formed of a plurality of individual integrated circuits. In order to simplify the design of such a switching fabric, embodiments of the present invention relate to a technique for designing one or more integrated circuits for a switching fabric, where implementation and use of the integrated circuits within the switching fabric is invariant by rotation of one or more of the integrated circuits by 90 degrees. Further, embodiments of the present invention relate to a technique for designing one or more integrated circuits for a switching fabric, where implementation and use of the integrated circuits within the switching fabric is invariant to flipping one or more of the integrated circuits with respect to one of its diagonals. 
   As shown in  FIG. 2 , an exemplary switching fabric  110  in accordance with an embodiment of the present invention is formed of individual integrated circuits  112 ,  114 ,  116 ,  118 ,  120 ,  122 ,  124 ,  126 ,  128 ,  130 ,  132 ,  134 ,  136 ,  138 ,  140 ,  142 ,  144 ,  146 ,  148 ,  150 ,  152 ,  154 . Further, the overlapping corners of the individual integrated circuits  112 - 154  include regions configured for “proximity communication.” Proximity communication is based on the observation that fast, low-cost communication is possible over short distances. Integrated circuits that are positioned for proximity communication are arranged such that transmitting and receiving circuits in the integrated circuits are aligned with only microns of distance between them. 
     FIGS. 3A and 3B  show an exemplary embodiment of integrated circuits configured for proximity communication. In  FIGS. 3A and 3B , a first integrated circuit  90  is positioned for “proximity I/O communication” with a second integrated circuit  92 . In one embodiment, using a technique referred to as “capacitive coupling,” data is capable of being transferred (i) from transmitting circuits  94  (forming a “sending region”) of the first integrated circuit  90  to receiving circuits  96  (forming a “receiving region”) of the second integrated circuit  92  and (ii) from transmitting circuits  98  (forming a “sending region”) of the second integrated circuit  92  to receiving circuits  100  (forming a “receiving region”) of the first integrated circuit  90 . Capacitive coupling causes a voltage change on a transmitting circuit to induce a voltage change on a corresponding receiving circuit of the facing integrated circuit. This makes it possible to transmit signals directly between the integrated circuits without having to route the signal through intervening signal lines within a printed circuit board. One or more techniques for aligning integrated circuits for proximity I/O communication are described in U.S. Pat. Nos. 6,710,436 issued to Harris et al and 6,768,195 issued to Drost. 
   Those skilled in the art will note that in one or more embodiments of the present invention, proximity I/O communication may not be limited to capacitive coupling. For example, proximity I/O communication may be implemented using one or more of inductive coupling and electromagnetic coupling. 
   At least some, if not all, of the individual integrated circuits  112 - 154  (shown in  FIG. 2 ) may be configured for proximity communication. Referring now to  FIG. 4 , one or more of the individual integrated circuits  112 - 154  (of  FIG. 2 ) are described herein as having at least three components: proximity I/O regions  160 ,  161 ,  163 ,  165  at the corners; wires  162 ,  164 ,  166 ,  167 ,  168 ,  169 ,  171 ,  173 ,  175 ,  177 ,  179 ,  181 ,  183 ,  185 ,  187 ,  189 ; and switches (shown, but not labeled) selectively disposed (i) between wires  162 ,  164 ,  166 ,  167 ,  168 ,  169 ,  171 ,  173 ,  175 ,  177 ,  179 ,  181 ,  183 ,  185 ,  187 ,  189  and (ii) between wires  162 ,  164 ,  166 ,  167 ,  168 ,  169 ,  171 ,  173 ,  175 ,  177 ,  179 ,  181 ,  183 ,  185 ,  187 ,  189  and proximity I/O regions  160 ,  161 ,  163 ,  165 . The proximity I/O regions  160 ,  161 ,  163 ,  165  of an exemplary integrated circuit (e.g.,  136  in  FIG. 2 ) are configured for proximity I/O communication with other integrated circuits (e.g.,  128  in  FIG. 2 ). As described above with reference to  FIGS. 3A and 3B , the proximity I/O regions  160 ,  161 ,  163 ,  165  are divided into sending regions and receiving regions. 
   Wires  162 ,  164 ,  166 ,  167 ,  168 ,  169 ,  171 ,  173 ,  175 ,  177 ,  179 ,  181 ,  183 ,  185 ,  187 ,  189 , positioned to link proximity I/O regions  160 ,  161 ,  163 ,  165 , are used for communication within the integrated circuit  170 . The wires  162 ,  164 ,  166 ,  167 ,  168 ,  169 ,  171 ,  173 ,  175 ,  177 ,  179 ,  181 ,  183 ,  185 ,  187 ,  189  shown in  FIG. 4  represent four independent sub-networks, where each sub-network is formed of one or more wires interconnected by one or more switches. In the exemplary embodiment of  FIG. 4 , wires  162 ,  167 ,  169 ,  171  propagate data North and East (the “North-East” sub-network), wires  164 ,  173 ,  175 ,  177  propagate data North and West (the “North-West” sub-network), wires  166 ,  179 ,  181 ,  183  propagate data South and West (the “South-West” sub-network), and wires  168 ,  185 ,  187 ,  189  propagate data South and East (the “South-East” sub-network). 
   Those skilled in the art will note that the specific quantities of components described with reference to  FIG. 4  are exemplary only. Accordingly, in one or more embodiments of the present invention, an integrated circuit of a switching fabric may have any number of sub-networks with any number of wires interconnected by any number of switches. 
   Still referring to  FIG. 4 , a 90-degree rotation of the integrated circuit  170  does not vary the layout of the wires. In other words, the arrangement of wires within the integrated circuit  170  for propagating data to/from North, East, South, and West is invariant to a 90-degree rotation of the integrated circuit  170  with respect to inputs and outputs of the integrated circuit  170 . 
   As described above, the proximity I/O regions  160 ,  161 ,  163 ,  165  are each divided into sending regions and receiving regions. After flipping an integrated circuit across one of its diagonals, the receiving regions should be in front of the sending regions. In other words, each type of proximity interconnect region (either sending or receiving) is different from the one of its image by the flipping across the diagonal. As there are four sub-networks entering and exiting each of the proximity I/O regions  160 ,  161 ,  163 ,  165  in integrated circuit  170 , there are a total of eight sending and receiving regions. Exemplary sending and receiving regions of the North-West proximity I/O communication region are shown in  FIG. 5 . Those skilled in the art will note that the difference in size between the sending and receiving regions in  FIG. 5  is shown for clarification purposes. In actual implementation, the sending and receiving regions may be of the same size or varying sizes and shapes. 
     FIG. 6  shows a portion of an exemplary integrated circuit in accordance with an embodiment of the present invention. Particularly,  FIG. 6  shows exemplary sending and receiving regions  190 ,  191  of a North-West proximity I/O communication region as exemplarily connected to wires and switches shown in  FIG. 4 . 
   Referring again to  FIG. 4 , switches (shown, but not labeled), located along each of wires  162 ,  164 ,  166 ,  168  are used to switch data packets within independent sub-network described above. For example, in the North-East sub-network, data received by a wire  162  is directed North and East by appropriate switches located along the wire  162 . Those skilled in the art will note that the shown proximity I/O regions  160 ,  161 ,  163 ,  165  represent the sum of what belongs to two proximity I/O communication integrated circuits. Accordingly, for a particular proximity I/O communication region  160 ,  161 ,  163 ,  165  shown in  FIG. 4 , four switches for switching packets from any of the entry points to any of the two exit points are implemented between integrated circuit  170  and an integrated circuit (not shown) proximity I/O interconnected to integrated circuit  170  at the particular proximity I/O communication region  160 ,  161 ,  163 ,  165 . 
   Switches (shown, but not labeled) are placed in a manner such that they are “balanced,” i.e., a switch that is placed before a sending/receiving region corresponds to a switch that is placed after a receiving/sending so that there are no unnecessary switches. As shown in  FIG. 4 , four switches are present in each proximity I/O communication region  160 ,  161 ,  163 ,  165 .  FIG. 7  shows that each switch may be divided into two “half-switches” (1 to 2 way switches), where part of the necessary switching is configured to take place before a sending/receiving region (of a first integrated circuit) and part is configured to take place after a receiving/sending region (of a second integrated circuit proximity interconnected to the first integrated circuit). Those skilled in the art will note that in the arrangement shown in  FIG. 7 , paths traverse the same regions (or complementary regions) and the same number of logic gates, thereby having equal “costs” (in terms of, for example, latency, power, and area). 
   Linking the switches, proximity I/O communication regions, and wires is dependent on the layout of a particular integrated circuit in a switching fabric.  FIG. 8  shows a portion of an integrated circuit  180  of a switching fabric in accordance with an embodiment of the present invention. Particularly,  FIG. 8  shows the positioning of switches, proximity I/O communication regions, and wires for the North-West sub-network of the integrated circuit  180 . The North-West sub-network has two exits (i.e., North and West are both exiting the integrated circuit  180  to cross the proximity I/O communication regions). As the image of the North-West sub-network by flipping across the diagonal extending from the bottom-left corner to the top-right corner of the integrated circuit  180  is the South-East sub-network, these two exits enter the South-East sub-network. The switch is a “full” switch that is either before or after the proximity I/O communication regions. The South-East sub-network image by the same flipping is the previous North-West sub-network and shares the “full” switch with it. 
   The North-East sub-network image by the same flipping is still the North-East sub-network. However, the North sub-network becomes the East sub-network. Thus, a North exit enters an East entry. A “half” switch is thus positioned to switch between North and West. The “half” switch is used before or after the proximity I/O communication regions depending on which sub-network the data comes from and which sub-network the data will propagate to. 
   The South-West sub-network remains unchanged by the flipping. The West sub-network becomes the South sub-network. Thus, a West exit is linked to a South entry. 
   Those skilled in the art will note that the specific quantities of components described with reference to  FIGS. 4-8  are exemplary only. Accordingly, in one or more embodiments of the present invention, an integrated circuit of a switching fabric may have any number of sub-networks with any number of wires interconnected by any number of switches. 
   Further, in one or more embodiments of the present invention, because one or more integrated circuits for a switching fabric are invariant to 90-degree rotations, it may be possible to reduce the number of different chips to produce for the switching fabric. 
   Further, in one or more embodiments of the present invention, because one or more integrated circuits for a switching fabric are invariant to flipping the integrated circuits, it may be possible to reduce the number of different chips to produce for the switching fabric. 
   Further, in one or more embodiments of the present invention, because one or more integrated circuits for a switching fabric are invariant to 90-degree rotations, costs associated with the production of numerous integrated circuit masks may be reduced or avoided. 
   While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this disclosure, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as disclosed herein. Accordingly, the scope of the invention should be limited only by the attached claims.