Abstract:
A system and method providing address broadcast synchronization using multiple switches. The system for concurrently providing addresses to a plurality of devices includes a first switch and a second switch. The first switch is coupled to receive address requests from a first plurality of sources. The first switch is configured to output the address request from the first plurality of sources. The second switch is coupled to receive address requests from a second plurality of sources. The second switch is configured to receive the address request from the first plurality of sources from the first switch. The second switch is further configured to delay the address request from the second plurality of sources prior to arbitrating between ones of the address request from the second plurality of sources and ones of the address request from the first party of sources received from the first switch. The second switch selects a selected address request, and the first and the second switch are further configured to broadcast concurrently a corresponding address to the selected address request. A method is also contemplated for concurrently providing addresses to a plurality of devices. A method of arbitrating in a first switch and a second switch between requests to the first switch and the second switch is disclosed where the arbitrated outcomes in both the first switch and the second switch are identical.

Description:
BACKGROUND OF THE INVENTION  
         [0001]    1. Field of the Invention  
           [0002]    This invention relates to cache synchronization, and more particularly to address broadcast synchronization to a plurality of potentially responding devices.  
           [0003]    2. Description of the Relevant Art  
           [0004]    Maintaining cache coherency in an N-way system, where N is the number of processors in the system, is essential. In a system where N is small (N&lt;4), the address buses of all cacheable devices may be physically connected together. Therefore, all cacheable devices may see a cache miss address simultaneously. On the other hand, when a system of N is large (N&gt;4), it becomes electrically unfeasible to connect the address buses of all cacheable devices together.  
           [0005]    One approach for achieving cache coherency in a system with large N, is by broadcasting the cache miss addresses to all cacheable devices simultaneously, through an address broadcast network. The address broadcast network has an address-in and an address-out connection to each of the cacheable devices. When a device sends a cache miss address to the address broadcast network, the address gets buffered, and then broadcast to all devices concurrently, so that all devices may check or update their tags appropriately.  
           [0006]    One problem with building an address network in hardware for large systems (N&gt;4) is that one needs a very large pin count ASIC (Application Specific Integrated Circuit) to accommodate all address-ins and address-outs for all cacheable devices to maintain address synchronization. The expense of building a large pin count ASIC to accommodate all address-ins and all address-outs for all cacheable devices limits this solution to only a very small number of computer systems.  
           [0007]    Another possible solution is to slice the address network into X (X&gt;1) slices for a small ASIC solution. The problem with address slicing is that using typical request and grant flow control techniques between address slices to maintain address synchronization requires a computer system performance degradation that is unacceptable.  
           [0008]    What is needed is a mechanism for achieving synchronization between address network slices without substantial performance degradation. The request and grant flow control technique used should require a minimum number of control signals passing between each switch.  
         SUMMARY OF THE INVENTION  
         [0009]    The problems outlined above are in large part solved by a system and method providing address broadcast synchronization using multiple switches. Each switch may be an application specific integration circuit (ASIC) or a separate switching device. By dividing address requests between more than one switch, addresses may be broadcast concurrently to a plurality of devices, which may advantageously provide for a higher system performance at a lower cost.  
           [0010]    In one embodiment, the system for concurrently providing addresses to a plurality of devices includes a first switch and a second switch. The first switch is coupled to receive address requests from a first plurality of sources. The first switch is configured to output the address request from the first plurality of sources. The second switch is coupled to receive address requests from a second plurality of sources. The second switch is configured to receive the address request from the first plurality of sources from the first switch. The second switch is further configured to delay the address request from the second plurality of sources prior to arbitrating between ones of the address request from the second plurality of sources and ones of the address request from the first party of sources received from the first switch. The second switch selects a selected address request, and the first and the second switch are further configured to broadcast concurrently a corresponding address to the selected address request.  
           [0011]    A method is also contemplated, in one embodiment, for concurrently providing addresses to a plurality of devices. In one embodiment, the method comprises receiving at a first switch a first address and a corresponding first request from a first device. The method receives at a second switch a second address and a corresponding second request from a second device, with the first switch being different from the second switch. The method transfers the second address and the corresponding second request to the first switch. The method delays the corresponding first request in the first switch. The method arbitrates in the first switch between the corresponding first request and the corresponding second request but rather the first address or the second address will comprise a first transmission. The method concurrently broadcasts to a plurality of devices the first transmission from the first switch and the first transmission from the second switch where the first transmission from the first switch and the first transmission from the second switch are identical.  
           [0012]    In another embodiment, a system for concurrently providing addresses to a plurality of devices includes a first switch and a second switch. The first switch is coupled to receive address requests from a first plurality of sources. The first switch is configured to output the address request from the first plurality of sources. The second switch is coupled to receive address requests from a second plurality of sources. The second switch comprises a broadcast buffer, an incoming buffer, a delay circuit, and a broadcast arbiter. The broadcast buffer is coupled to receive addresses of the address requests from the second plurality of sources. The incoming buffer is coupled to receive addresses of the output of the address requests from the first plurality of sources from the first switch. The delay circuit is coupled to receive the address requests from the second plurality of sources. The delay circuit is configured to delay the address requests from the second plurality of sources for a predetermined length of time. The broadcast arbiter is coupled to arbitrate between ones of the address request from the second plurality of sources and ones of the output of the address request from the first plurality of sources from the first switch for a selected address request. The first switch and the second switch are further configured to broadcast concurrently a corresponding address to the selected address request selected in the broadcast arbiter.  
           [0013]    In still another embodiment, a method of arbitrating in a first switch and a second switch between requests to the first switch and the second switch is disclosed. The comprises tracking which switch was most recently selected and tracking which switch is next to be selected. In response to a reset, the method selects the first switch and indicates that the second switch is next to be selected. In response to only a local request to the first switch or only a remote request to the second switch, the method selects the first switch and indicates that the first switch is next to be selected. In response to only a local request to the second switch or only a remote request to the first switch, the method selects the second switch and indicates that the second switch is next to be selected. In response to both a local request and a remote request concurrently, the method selects the switch which was not most recently selected, and the method indicates that the switch not most recently selected will be the next to be selected. Otherwise, the method selects the first switch and indicates the switch most recently selected as the next to be selected.  
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0014]    Other objects and advantages of the invention will become apparent upon reading the following detailed description and upon reference to the accompanying drawings in which:  
         [0015]    [0015]FIG. 1 is a block diagram of an embodiment of a computer system including two switches that concurrently provide addresses to a plurality of devices;  
         [0016]    [0016]FIG. 2 is a block diagram of an embodiment of the two switches shown in FIG. 1; and  
         [0017]    [0017]FIGS. 3A and 3B are a flowchart of an embodiment of a method for arbitrating in a first switch and a second switch between request to the first switch and the second switch. 
     
    
       [0018]    While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present invention as defined by the appended claims.  
       DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0019]    Similar features are designed herein using identical reference numerals. It is noted that the use of a reference numeral with an additional letter may designate a particular one of a group that may referenced as a while with the reference numeral by itself.  
         [0020]    [0020]FIG. 1—Computer System Including Two Switches  
         [0021]    [0021]FIG. 1 is a block diagram of a computer system including two switches, switch  110 A and switch  110 B. As shown, the computer system includes CPUs  115 A- 115 H, input and output devices (I/O)  120 A- 120 D, and memories  125 A- 125 D. Data signals beginning with a P have a processor  115  as a destination, and data signals beginning with an I/O have an I/O device  120  as a destination. Switches  110 A and  110 B are shown receiving input from various groupings of the processors  115  and the I/O devices  120 . The switches  110 A and  110 B are also shown outputting signals to various ones of the processors  115 , the I/O devices  120 , and to the memories  125 .  
         [0022]    A plurality of processors (CPUs)  115 A- 115 H (eight as shown), each receives an input, preferably addresses, appropriately referenced as P0-P7. Each of the processors  115 A- 115 H outputs an output, preferably an address and an address request, such as an address request packet, to one of the two switches  110 A and  110 B. As shown, switch  110 A also accepts address request packets from I/O device  120 A and I/O device  120 B. Also as shown, switch  110 B accepts address request packets from I/O device  120 C and I/O device  120 D. Switch  110 A outputs an output signal, preferably address signals, to the CPUs  115 A- 115 D, the I/O devices I/O0-I/O1, and memories  125 A- 125 B. Switch  110 B outputs an output signal, preferably address signals, to processors  115 E- 115 H, I/O devices I/O2-I/O3, and memories  125 C- 125 D. Switch  110 A and switch  110 B also exchange data, preferably including addresses and address requests.  
         [0023]    It is noted that while a particular number of processors  115 , I/O devices  120 , and memories  125  are illustrated, any number of processors, I/O devices, and/or memories, or other devices are contemplated. It is also noted that while unidirectional data paths are illustrated, bi-directional data paths may also be used as desired.  
         [0024]    [0024]FIG. 2—Address Broadcast Synchronization Switches  
         [0025]    [0025]FIG. 2 is a block diagram of one embodiment of the switches  110 A and  110 B. As shown, each switch  110  includes a plurality of input FIFOs (First-In, First Out buffers)  205 , a request arbiter  215 , an input multiplexer (MUX)  210 , a broadcast FIFO  225 , an incoming FIFO  230 , a delay circuit  235 , a broadcast arbiter  240 , and an output MUX  245 . The switches  110  exchange output requests from their respective request arbiters  215  and output addresses from their respective input MUXes  210 .  
         [0026]    As illustrated, switch  110 A accepts addresses P0P3 and I/O0-I/O1, as well as address requests P0_req-P3_req and I/O0_req and I/O1_req. Switch  110 A outputs address signals P0-P3, I/O0-I/O1, and M0-M1. Each incoming address P0-P3 and I/O0-I/O1 is received into an input FIFO  205 A- 205 F. The address requests that correspond to the addresses received in the input FIFOs  205 A- 205 F are received at a request arbiter  215 A. In the preferred embodiment, the request arbiter  215 A is a round-robin arbiter, although any other means of arbitration may be used as desired for choosing requests received by request arbiter  215 A. When the request arbiter  215 A chooses (or arbitrates) for a particular address request, the request arbiter  215 A controls the selection at input MUX  210 A with regard to the output of the input FIFOs  205 A- 205 F. The selected address request is output as SW0_req to delay circuit  235 A. The output of input MUX  210 A, shown as signal  220 A, is provided to a broadcast FIFO  225 A. It is noted that output signal  220 A is also provided to switch  110 B, and that the address request SW0_req is also provided to switch  110 B.  
         [0027]    Switch  110 A is also coupled to receive the address request SW1_req from switch  110 B, as well as address output signal  220 B. Signal  220 B is received at incoming FIFO  230 A. As shown, broadcast FIFO  225 A and incoming FIFO  230 A each output data to output MUX  245 A, broadcast FIFO  225 A as ‘0’ (zero) and incoming FIFO  230 A as ‘1’ (one). Address request SW0_req is delayed for a period of time in delay circuit  235 A before being provided to broadcast arbiter  240 A. The period of time of the delay may be a predetermined period of time. It is noted that in a preferred embodiment, the predetermined period of time is equal to the time required for switch  110 A to receive the address request SW1_req and the address output signal  220 B. Broadcast arbiter  240 A chooses (or arbitrates) between request SW1_req and request SW1_req. The broadcast arbiter  240 A controls the output of output MUX  245 A choosing between ‘0’ and ‘1’. The output of output MUX  245 A, the selected address for the first transmission, is provided concurrently to various groups of the processors  115 , I/O devices  120 , and/or memories  125  through signals P0-P3, I/O0-I/O1, and M0-M1.  
         [0028]    As illustrated, switch  110 B accepts addresses P4-P7 and I/O2-I/O3, as well as address requests P4_req-P7_req and I/O2_req and I/O3_req. Switch  110 B outputs address signals P4-P7, I/O2-I/O3, and M2-M3. Each incoming address P4-P7 and I/O2-I/O3 is received into an input FIFO  205 G- 205 L. The address requests that correspond to the addresses received in the input FIFOs  205 G- 205 L are received at a request arbiter  215 B. In the preferred embodiment, the request arbiter  215 B is a round-robin arbiter, although any other means of arbitration may be used as desired for choosing requests received by request arbiter  215 B. When the request arbiter  215 B chooses (or arbitrates) for a particular address request, the request arbiter  215 B controls the selection at input MUX  210 B with regard to the output of the input FIFOs  205 G- 205 L. The selected address request is output as SW1_req to delay circuit  235 B. The output of input MUX  210 B, shown as signal  220 B, is provided to a broadcast FIFO  225 B. It is noted that output signal  220 B is also provided to switch  110 A, and that the address request SW1_req is also provided to switch  110 A.  
         [0029]    Switch  110 B is also coupled to receive the address request SW0_req from switch  110 A, as well as address output signal  220 A. Signal  220 A is received at incoming FIFO  230 B. As shown, broadcast FIFO  225 B and incoming FIFO  230 B each output data to output MUX  245 B, broadcast FIFO  225 B as ‘1’ (one) and incoming FIFO  230 B as ‘0’ (zero). Address request SW1_req is delayed for a period of time in delay circuit  235 B before being provided to broadcast arbiter  240 B. The period of time of the delay may be a predetermined period of time. It is noted that in a preferred embodiment, the predetermined period of time is equal to the time required for switch  110 B to receive the address request SW0_req and the address output signal  220 A. Broadcast arbiter  240 B chooses (or arbitrates) between request SW1_req and request SW1_req. The broadcast arbiter  240 B controls the output of output MUX  245 B choosing between ‘0’ and ‘1’. The output of output MUX  245 B, the selected address for the first transmission, is provided concurrently to various groups of the processors  115 , I/O devices  120 , and/or memories  125  through signals P4-P7, I/O2-I/O3, and M2-M3.  
         [0030]    It is noted that the delay circuits  235 A and  235 B may include any circuit that is configured to delay the output of a received signal. In one embodiment, a delay circuit  235  delays the received signal longer than the minimum time required to propagate the received signal through delay circuit  235 . In another embodiment, delay circuit  235  includes one or more flip-flops. It is also noted that in various embodiments various incoming and outgoing signals to and from switches  110 A and  110 B may be buffered at input to the switch  110  and/or on output from the switch  110 .  
         [0031]    Generally speaking, the system of FIG. 1 operates as described herein. The first switch  110 A is coupled to receive address requests from a first plurality of sources. For example, one plurality of sources may be processors  115 A- 115 D and/or I/O devices  120 A- 120 B. The first switch  110 A is configured to output a received address request from the first plurality of sources.  
         [0032]    The second switch  110 B is coupled to receive address requests from a second plurality of sources. For example, the second plurality of sources may include processors  115 E- 115 H and/or I/O devices  120 C- 120 D. Switch  110 B is also configured to receive the address request from the first plurality of sources from the first switch  110 A. The second switch is further configured to delay internally address requests from the second plurality of sources. It is noted that the length of the delay may be predetermined, and is preferably equal in length of time to the time delay in receiving the address request from the first plurality of sources from the first switch. The second switch  110 B is further configured to arbitrate between ones of the address requests from the second plurality of sources and ones of the address request from the first plurality of sources output from the first switch. The arbitration between the address requests is to determine a selected address request. Once a selected address request has been selected, the first switch and the second switch are further configured to broadcast concurrently the corresponding address to the selected address request. It is noted that the corresponding address will broadcast to any or all devices, including the CPUs  115 A- 115 H, I/O devices  120 A- 120 B, and memories  125 A- 125 D.  
         [0033]    In one embodiment, the second switch  110 B is further configured to output the address request from the second plurality of sources, and the first switch  110 A is further configured to receive this request from the second plurality of sources. First switch  110 A is further configured to delay internally the address request from the first plurality of sources. The time of the delay of the address request from the first plurality of sources may be a predetermined length of time and is preferably a length of time approximately equal to the time required for the second switch  110 B to provide the address request in the second plurality of sources to first switch  110 A. The first switch is further configured to arbitrate between ones of the address request from the first plurality of sources and ones of the address requests from the second plurality of sources from the second switch. The arbitration is to determine the selected address request, as noted above for the second switch  110 B. It is noted that the selected address provided by the first switch  110 A and the selected address provided by the second switch  110 B are the same and are concurrently provided to the devices as described above.  
         [0034]    FIGS.  3 A- 3 B—Arbitration by a Broadcast Arbiter  
         [0035]    [0035]FIGS. 3A and 3B illustrate a flowchart of an embodiment of a method for operating an arbiter, such as broadcast arbiters  240 A and  240 B. The method tracks which switch was most recently selected, and the method also tracks which switch is next to be selected. At decision block  305 , the method checks to see if reset has been asserted. If reset has been asserted in decision box  305 , then an output MUX selects output ‘0’ (i.e. switch  110 A) and the next granted switch will be the other switch (i.e. switch  110 B) (step  310 ).  
         [0036]    If reset has not been asserted in decision block  305 , then the method determines if only a local request has been made to the first switch  110 A or only a remote request has been made to the second switch  110 B in decision block  315 . If only a local request has been made to the first switch  110 A or only a remote request is made to the second switch  110 B, then the method selects output MUX output ‘0’ and the next granted switch will be the same switch (step  320 ).  
         [0037]    If there has not been only a local request to the first switch  110 A or only a remote request to the second switch  110 B, then the method moves to decision block  325 . If only a local request has been made to the second switch  110 B or only a remote request has been made to the first switch  110 A in decision box  325 , then the method selects output MUX output ‘1’ and the next granted switch will be the same switch (step  330 ).  
         [0038]    If only a local request to the second switch  110 B or only a local request to the first switch  110 A has not been made in decision block  325 , then the method moves to decision block  335 . In decision block  335 , if both a local request and a remote request have concurrently been made, and the current granted switch is switch  110 A, then the output MUX selects ‘1’ and the next granted switch is switch  110 A (step  340 ). If in decision block  335  both the local request and remote request have been made concurrently but the current granted switch is not switch 0, then the method moves to decision block  345 .  
         [0039]    In decision block  345 , if both the local request and a remote request have been made concurrently and the current granted switch is switch  110 B, then the output MUX selects ‘0’ and the next granted switch is switch  110 A (step  350 ). It is noted that in decision blocks  335  and  345 , an affirmative decision is made in either case when a local request and a remote request have both been made concurrently. In either case the selected output MUX output is to the switch not most recently selected and the indicated switch as the next granted switch is also the switch not most recently selected.  
         [0040]    The default action when all decision blocks are negative, is for the outgoing MUX to select ‘0’, and the next granted switch is the current granted switch (step  355 ).  
         [0041]    In various embodiments, the switches  110 A and  110 B may be application specific integrated circuits ASCIC0 and ASCIC1. In one embodiment, ASCIC0 and ASCIC1 are location strapped via jumpers. It is noted that ASCIC0 preferably will have a pull-up resistor, while ASIC1 preferably has a pull-down resistor, both of which get latched on reset to identify which is ASCIC0 and which is ASCIC1. Note that the priority toggles between the broadcast arbiters based on the switch that had the last request granted and the current outstanding request. The method disclosed may advantageously ensure that both arbiters are synchronized to each other without a need for request/grant flow control mechanisms beyond the address and the corresponding address request that was initially received.  
         [0042]    As an example of an embodiment of the operations of switches  110 A and  110 B, right after a reset, both processors  115 A and  115 E have an outstanding address packet in the address network. The P0 address packet is received in switch  110 A&#39;s input FIFO  205 A from processor  115 A, whereas the P4 address packet is received and stored in switch  110 B&#39;s input FIFO  205 G from processor  115 E. The request arbiter  215 A in switch  110 A will receive the P0 request associated with the address stored in input FIFO  205 A. Similarly, request arbiter  215 B receives the P4_req address request associated with the P4 address stored in input FIFO  205 G.  
         [0043]    Request arbiter  215 A in switch  110 A controls input MUX  210 A to output the address associated with input signal P0 as output signal  220 A, which is provided to broadcast FIFO  225 A and to incoming FIFO  230 B. Likewise, request arbiter  215 B controls input MUX  210 B to output the address from P4 as output signal  220 B. Output signal  220 B is provided to broadcast FIFO  225 B and also to incoming FIFO  230 A. Concurrently with the addresses being routed from the input FIFO  205  to the broadcast FIFOs  225  and incoming FIFOs  230 , switch  110 A has asserted SW0_req line indicating the presence of an address from switch  110 A in broadcast FIFO  225 A and incoming FIFO  230 B.  
         [0044]    As a finite amount of time is required for the address and the request line to be provided from one switch  110  to the other switch  110 , in this case from switch  110 A to switch  110 B, signal SW0_req is first provided to a delay circuit  235 A, before being provided to broadcast arbiter  240 A. In the preferred embodiment, the delay circuit  235 A delays the address request SW1_req by approximately an equal amount of time as required for switch  110 A to receive the address and corresponding address request from switch  110 B. In this embodiment, broadcast arbiter  240 A receives notice that an address is present in the broadcast FIFO  225 A concurrently with an address being available in the incoming FIFO  230 A. The broadcast arbiter  240 A chooses (or arbitrates) for priority between the SW0_req and SW1_req. The preferred arbitration method is described above with respect to FIGS. 3A and 3B. Broadcast arbiter  240 A selects either ‘0’ or ‘1’ denoting the address from switch  110 A or switch  110 B, respectively, in controlling the output of the output multiplexer  245 A.  
         [0045]    It is noted that since SW1_req and SW1_req are both required to cross from one switch to the other, the signals endure a delay, such as two clock cycles in one embodiment. Therefore, each switch  110 A and  110 B delays the address request that it sends, SW0_req and SW1_req, respectively, to the broadcast arbiter  240  of the other switch by an equivalent time period of 2 clock cycles. This delay ensures that the broadcast arbiters  240 A and  240 B in each switch  110 A and  110 B receive the address request concurrently.  
         [0046]    Switch  110 A has the P0 address placed in its broadcast FIFO  225 A and the P4 address placed in incoming FIFO  230 A. Switch  110 B has the P0 address placed in its incoming FIFO  230 B and P4 packet placed in broadcast FIFO  225 B. At this time broadcast arbiter  240 A has received address request SW0_req and address request SW1_req, whereas broadcast arbiter  240 B has likewise received address request SW1_req and address request SW1_req.  
         [0047]    The arbitration method described above with respect to FIGS. 3A and 3B illustrates a preferred embodiment of how the broadcast arbiter  245  works for each address request that it receives. After a reset, the last granted switch defaults to switch  110 A, so that switch  110 A broadcast arbiter now has the highest priority. When the broadcast arbiter  240 A has highest priority, then both broadcast arbiter  240 A and broadcast arbiter  240 B will select the ‘0’ of the multiplexer  245 B. It is noted that both broadcast arbiter  240 A and broadcast arbiter  240 B are at decision block  345  of FIG. 3B. Both a local request and a remote request have been received and the current granted switch is switch  110 B (the default upon a reset), therefore the output MUXes  245 A and  245 B both select ‘0’ and the next granted which will be switch  110 A (step  350 ). Thus, the address from P0 is provided as output  250 A and output  250 B, concurrently on address lines P0-P7, I/O0-I/O3, and M0-M3.  
         [0048]    Continuing, at decision block  325 , as the request is now only the request from switch  1110 B, the output MUXes  245  will select ‘1’ and the next granted will be switch  110 B (step  330 ). It is noted that broadcast arbiter  240 A and broadcast arbiter  240 B, following an arbitration method similar to that disclosed in FIGS. 3A and 3B, make selections between local and remote requests which are identical in all cases. It is also noted the broadcaster arbiter  240 A knows that upon a reset that it will have priority just as broadcast arbiter  240 B knows that after a reset it will not have priority.  
         [0049]    Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.