Abstract:
The invention provides a chassis, a scalable router that includes a plurality of chassis and a method of upgrading a router. Each chassis includes a plurality of processing modules and a programmable interconnection module. Data connections are provided between each processing module on each chassis and the interconnection module on that chassis, and a data connection is provided between the interconnection module on each chassis and the interconnection module on at least one other chassis. In this way, the inter-chassis and intra-chassis connections pass through the programmable interconnection module, which may be used for concentrating the location of resources, such as opto-electronic and electro-optical converters, in a single card or unit. Moreover, the programmable nature of the switch fabric in each interconnection module is amenable to reconfiguration so as to support a change in that chassis&#39; interconnection pattern that may be required for expanding the router&#39;s capacity.

Description:
FIELD OF THE INVENTION 
   The present invention relates to internetworking routers and, more particularly, to high-capacity routers having multiple interconnected chassis. 
   BACKGROUND OF THE INVENTION 
   Ever-increasing usage of the Internet is expected to lead to even higher demands on the capacity of Internet routers than those which already persist today. However, the rate of growth of traffic through a given Web site or traffic point in the Internet may vary considerably amongst different Internet Service Providers (ISPs). In some cases, the growth may be sudden and staggering, requiring huge increases in capacity on an almost instantaneous basis. In other cases, an anticipated increase in capacity might still be some time away, although a router with a basic switching capacity may need to be purchased immediately in order to satisfy an existing demand. 
   A conventional approach to upgrading the routing capacity through a given traffic point is to simply replace the existing router with a new, higher-capacity device. The old router is either decommissioned or relegated to a less traffic-intensive area of the ISP&#39;s internal network. Unfortunately, this approach requires a significant capital expenditure on the part of the ISP, since higher-capacity routers tend to be disproportionately more expensive than lower-capacity routers. Moreover, a capital expenditure of this nature is necessitated each time a capacity increase is required or desired. Additional disadvantages include the “down time” associated with installation of a new router, testing of new connections, changing suppliers and so on. 
   Clearly, it would be advantageous to provide a scalable solution to the problem of accommodating traffic growth through a router. However, as now described, conventional router design makes this a near impossible feat. Specifically, a scalable router typically has two or more chassis, each of which contains multiple switch cards and line cards. The line cards have ports for interfacing with an external network. Internally to the router, the switch cards are connected to the line cards and to one another by a backplane on each chassis, and by direct interconnections across multiple chassis. 
   In order to enhance the available switching capacity of the router, it may appear plausible to add one or more extra chassis but it should also be apparent that these additional chassis must somehow be connected to the existing chassis. As a result, existing hardware connections, both within and between he existing chassis, must be disconnected and then re-connected according to a different inter-chassis topology and a different intra-chassis interconnect pattern. Thus, while avoiding part of the capital expenditure associated with an outright replacement of the existing router, the conventional solution has the disadvantage of requiring added installation and testing efforts, both of which are labour-intensive and prone to error. 
   Hence, there remains a strong need in the industry to provide a scalable router that would be designed to accommodate changes in capacity without requiring replacement or disconnection of the existing inter-chassis or intra-chassis connection hardware. 
   SUMMARY OF THE INVENTION 
   The present invention endeavours to obviate or mitigate one or more disadvantages of the prior art and may be summarized according to a first broad aspect as a router that includes a plurality of chassis, each chassis including a plurality of processing modules and a programmable interconnection module. Data connections are provided between each processing module on each chassis and the interconnection module on that chassis, and a data connection is provided between the interconnection module on each chassis and the interconnection module on at least one other chassis. 
   In one specific embodiment, the data connections between the processing modules on each chassis and the interconnection module on that chassis are electrical and the data connections between the interconnection modules on different chassis are optical. 
   In another specific embodiment, the interconnection module on each chassis includes a plurality of electrical input ports, a plurality of electrical output ports and a programmable switch fabric disposed therebetween, for providing selective connections between individual ones of the electrical input ports and corresponding ones of the electrical output ports. Additionally, each processing module on each chassis includes a plurality of electrical input ports, a plurality of electrical output ports and a processing fabric disposed therebetween. Moreover, the plurality of electrical input ports of each processing module on each chassis is connected to a respective subset of the electrical output ports of the interconnection module on that chassis, while the plurality of electrical output ports of each processing module on each chassis is connected to a respective subset of the electrical input ports of the interconnection module on that chassis. 
   In this way, the inter-chassis and intra-chassis connections pass through the programmable interconnection module, which may be used for concentrating the location of resources, such as opto-electronic and electro-optical converters, in a single card or unit. Moreover, the programmable nature of the switch fabric in each interconnection module is amenable to reconfiguration so as to support a change in that chassis&#39; interconnection pattern that may be required for expanding the router&#39;s capacity. 
   According to a second broad aspect, the present invention may be summarized as a chassis for use in building a scalable router. The chassis includes a plurality of processing modules, each processing module including a plurality of electrical input ports, a plurality of electrical output ports and a processing fabric disposed therebetween. The chassis also includes a programmable interconnection module, including a plurality of electrical input ports, a plurality of electrical output ports and a programmable switch fabric disposed therebetween, for selectively establishing connections between individual ones of the electrical input ports and corresponding ones of the electrical output ports in accordance with a connection map. 
   A data connection is established between each processing module and the interconnection module, whereby a subset of the plurality of electrical input ports of each processing module on each chassis is connected to a respective subset of the electrical output ports of the interconnection module on that chassis and whereby a subset of the plurality of electrical output ports of each processing module on each chassis is connected to a respective subset of the electrical input ports of the interconnection module on that chassis. 
   The chassis additionally includes a plurality of optical input ports and a plurality of optical output ports, for external connection to one or more other chassis of the router. The chassis also has a plurality of optical-to-electrical conversion units, each optical-to-electrical conversion unit being connected between a respective one of the optical input ports and a respective subset of the electrical input ports of the interconnection module. Finally, the chassis includes a plurality of electrical-to-optical conversion units, each electrical-to-optical conversion unit being connected between a respective subset of the electrical output ports of the interconnection module and a respective one of the optical output ports. 
   According to a third broad aspect, the present invention may be summarized as a method of upgrading a router including a plurality of original chassis, each original chassis having a plurality of processing modules and a programmable interconnection module, wherein a data connection exists between each processing module on each original chassis and the interconnection module on the same original chassis and wherein a data connection exists between the interconnection module on each original chassis and the interconnection module on at least one other original chassis. The method includes providing at least one additional chassis, each additional chassis comprising a plurality of processing modules and a programmable interconnection module, wherein a data connection exists between each processing module on each additional chassis and the interconnection module on the same additional chassis. The method then includes establishing a data connection between the interconnection module on each additional chassis and the interconnection module on at least one original chassis, and establishing a data connection between the interconnection module on each additional chassis and the interconnection module on at least one other additional chassis. Finally, the interconnection modules of the various original chassis are reprogrammed. The interconnection modules of the various additional chassis can be programmed prior to, or after, their connection to the original chassis. 
   These and other aspects and features of the present invention will now become apparent to those of ordinary skill in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying drawings. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
     In the accompanying drawings: 
       FIG. 1  depicts an interconnect pattern existing within a chassis in accordance with an embodiment of the present invention; 
       FIG. 2  illustrates an example of the internal structure of an interconnection module in the chassis of  FIG. 1 ; 
       FIG. 3A  shows a possible inter-chassis topology for building a router from two chassis; 
       FIG. 3B  shows a possible intra-chassis interconnection pattern for each of the chassis of  FIG. 3A ; 
       FIG. 4A  shows a possible inter-chassis topology for building a router from three chassis; 
       FIG. 4B  shows a possible intra-chassis interconnection pattern for each of the chassis of  FIG. 4A ; 
       FIG. 5A  shows a possible inter-chassis topology for building a router from six chassis; 
       FIG. 5B  shows a possible intra-chassis interconnection pattern for each of the chassis of  FIG. 5A ; and 
       FIG. 6  illustrates, in schematic form, a router made up of a set of “clusters” of chassis. 
   

   DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
   With reference to  FIG. 1 , there is shown a chassis (shelf)  100 , and more specifically a data interconnect pattern within the chassis  100 , according to an embodiment of the present invention. Two or more chassis  100  of the type shown in  FIG. 1  may be used to build a router, in a manner to be described later on with reference to  FIGS. 3A through 5B . The chassis  100  includes a plurality of line cards  110  for interfacing with an external network  120 , a plurality of processing cards  130  for providing processing and switching power, and an interconnection module  140 . 
   The line cards  110  provide an optical or electrical interface to the network  120 . Data in the form of packets (or ATM cells or SONET frames or the like) are exchanged with the network  120  via the line cards  110 . In a specific non-limiting example embodiment, the line cards  110  may be implemented as disclosed in U.S. patent application Ser. No. 09/870,766 to Norman et al., filed on Jun. 1, 2001 and hereby incorporated by reference herein. In the embodiment illustrated in  FIG. 1 , the line cards  110  are equipped with bi-directional functionality. In other embodiments, some of the line cards  110  may be input line cards (for receiving packets from the network  120 ), while other ones of the line cards  110  may be output line cards (for transmitting packets to the network  120 ). 
   The total number of line cards  110  in the illustrated embodiment is sixteen, each of which may be inserted into a corresponding slot (not shown) of the chassis  100 . The sixteen line cards  110  are grouped into two sets denoted more precisely as  110   0,0 - 110   0,7  (set 0) and  110   1,0 - 110   1,7  (set 1). Each of the line cards  110  has a set of ports used for establishing one or more full-duplex electrical paths with a corresponding one of the processing cards  130 . By way of example, each of the line cards  110  may have four ports for establishing two full-duplex 2.5 Gbps (in each direction) electrical paths with a corresponding one of the processing cards  130 . It should be understood, however, that the present invention is not limited to a specific number of line cards  110  or to a specific number of ports or paths per line card or to a specific path bandwidth. 
   The total number of processing cards  130  in the chassis  100  of the illustrated embodiment is four, each of which may be inserted into a corresponding slot (not shown) of the chassis  100 . The four processing cards  130  include a set of two first-stage processing cards (denoted  130   0,0  and  130   0,1 ) and a set of two second-stage processing cards (denoted  130   1,0  and  130   1,1 ). Each of the first-stage processing cards includes a set of line-side ports for establishing sixteen full-duplex electrical paths L with a corresponding set of the line cards  110 . Each of the first- and second-stage processing cards  130  includes a set of switch-side ports for establishing either sixteen (first-stage) or thirty-two (second-stage) full-duplex electrical paths P with the interconnection module  140 . 
   As shown in  FIG. 1 , the line-side ports of each of the first- and second-stage processing cards  130  include a set of line-side input ports  132  and a set of line-side output ports  134 . Similarly, the switch-side ports of each of the first- and second-stage processing cards  130  include a set of switch-side input ports  136  and a set of switch-side output ports  138 . Additionally, each of the first- and second-stage processing cards  130  has a a processing fabric  135 , such as a switch fabric, connected between the input ports  132 ,  136  and the output ports  134 ,  138 . In a specific non-limiting example embodiment, the processing cards  130  may be implemented as disclosed in U.S. patent application Ser. No. 09/870,766 to Norman et al., filed on Jun. 1, 2001 and hereby incorporated by reference herein. 
   The paths L joining the line cards  110  and the processing cards  130  may be established through the use of a backplane or midplane configuration. In the illustrated embodiment, these paths are denoted L 0 -L 31 , where, for the line cards  110   0,0  to  110   0,7  in set 0, paths L 0 -L 1  are established between line card  110   0,0  and processing card  130   0,0 , paths L 2 -L 3  are established between line card  110   0,1  and processing card  130   0,0 , paths L 4 -L 5  are established between line card  110   0,2  and processing card  130   0,0 , paths L 6 -L 7  are established between line card  110   0,3  and processing card  130   0,0 , paths L 8 -L 9  are established between line card  110   0,4  and processing card  130   0,0 , paths L 10 -L 11  are established between line card  110   0,5  and processing card  130   0,0 , paths L 12 -L 13  are established between line card  110   0,6  and processing card  130   0,0  and paths L 14 -L 15  are established between line card  110   0,7  and processing card  130   0,0 . A similar interconnect pattern joins the line cards  110   1,0  to  110   1,7  in set 1 to processing card  130   0,1 . 
   The paths P joining the processing cards  130  to the interconnection module  140  may also be established through the use of a backplane or midplane configuration. In the illustrated embodiment, there are ninety-six such paths and these are denoted P 0 -P 95 , where paths P 0 -P 31  are established between processing card  130   1,0  and the interconnection module  140 , paths P 32 -P 63  are established between processing card  130   1,1  and the interconnection module  140 , paths P 64 -P 79  are established between processing card  130   0,0  and the interconnection module  140  and paths P 80 -P 95  are established between processing card  130   0,1  and the interconnection module  140 . 
   Due to the fact that all first- and second-stage processing cards  130  are connected to the interconnection module  140 , data arriving from the network  120  can be processed by one or several processing cards  130  in sequence. For example, data can be routed back into the network  120  after reaching one of the first-stage processing cards  130   0,0 ,  130   0,1  or it can be forwarded to one of the second-stage processing cards  130   1,0 ,  130   1,1  via the interconnection module  140 . In another embodiment of the invention, the entire set of line cards  110  could be connected to the entire set of processing cards  130 , obviating a need for the distinction between “first-stage” and “second-stage” processing cards. Although the processing cards  130  are shown in  FIG. 1  to be mutually non-interconnected, those skilled in the art should appreciate that it is within the scope of the present invention to provide additional data paths between pairs of processing cards  130 . This may be advantageous when implementing a hypercube-based interconnect pattern, for example. 
   It should be understood that the present invention is not limited to any particular number of processing cards  130 , nor to any particular number of stages of processing cards  130 , nor to any particular number of line-side ports or switch-side ports per processing card  130 , nor to any particular interconnection pattern between the processing cards  130  and the interconnection module  140 , nor to any particular implementation for achieving a connection between the processing cards  130  and either the line cards  110  or the interconnection module  140 . 
   The interconnection module  140  may be a separate card in the chassis  100  and includes a plurality of switch-side ports that establish paths P 0 -P 95  with the switch-side ports of the processing cards  130 . The interconnection module  140  further includes a plurality of optical ports A, B, C, D for establishing a plurality of full-duplex optical paths with the external world (i.e., with optical ports of other chassis of the router, to be described later on with reference to  FIGS. 3A to 5B ). In the illustrated embodiment, optical port A establishes 32 optical paths denoted A 0 -A 31 , optical port B establishes 24 optical paths denoted B 0 -B 23 , optical port C establishes 24 optical paths denoted C 0 -C 23 , and optical port D establishes 32 optical paths denoted D 0 -D 31 . However, it is to be understood that the present invention limits neither the number of optical ports nor the number of paths per optical port. 
   The internal structure and functionality of the interconnection module  140  are now described with reference to  FIG. 2 . Paths P 0 -P 95 , which join the interconnection module  140  to the processing cards  130 , are connected to electrical interfaces of a programmable switch fabric  200 . More specifically, since each of the ninety-six paths P 0 -P 95  is full-duplex, they are connected to ninety-six input electrical interfaces (denoted IN 0 -IN 95 ) and ninety-six output electrical interfaces (denoted OUT 0 -OUT 95 ) of the switch fabric  200 . 
   In addition, a set of 112 additional input electrical interfaces (denoted IN 96 -IN 207 ) lead from the optical ports A, B, C, D via a bank of opto-electronic receivers  210 , while a set of  112  additional output electrical interfaces (denoted OUT 96 -OUT 207 ) lead to the optical ports A, B, C, D via a bank of electro-optical transmitters  220 . More specifically, electrical interfaces IN 96 -IN 127  and OUT 96 -OUT 127  are associated with optical paths A 0 -A 31 , electrical interfaces IN 128 -IN 151  and OUT 128 -OUT 151  are associated with optical paths B 0 -B 23 , electrical interfaces IN 152 -IN 175  and OUT 152 -OUT 175  are associated with optical paths C 0 -C 23 , and electrical interfaces IN 176 -IN 207  and OUT 176 -OUT 207  are associated with optical paths D 0 -D 31 . 
   Thus, by virtue of its input electrical interfaces IN 0 -IN 207 , the switch fabric  200  can be on the receiving end of up to a total of 96 electrical data signals from the processing cards  130  and a further  112  electrical data signals from other chassis (via the optical ports A, B, C, D), for a total of  208  received electrical signals. Similarly, by virtue of its output electrical interfaces OUT 0 -OUT 207 , the switch fabric  200  can transmit up to a total of 96 electrical data signals to the processing cards  130  and a further  112  electrical data paths to other chassis (via the optical ports A, B, C, D), for a total of 208 transmitted electrical signals. 
   In order to accommodate the switching requirement of the switch fabric  200 , the latter may be implemented as a single, non-blocking cross-point switch matrix of the requisite size (208×208). This would allow any of the electrical paths P 0 -P 95  to be connected to any of the optical paths A 0 -A 31 , B 0 -B 23 , C 0 -C 23 , and D 0 -D 31 . However, it is noted that not all of the electrical signals received from the opto-electronic receivers  210  will need to be relayed to the processing cards  130 . Rather, some of these may need to be immediately re-routed back to the optical ports via the electro-optical transmitters  220 . Similarly, some of the electrical signals received from the processing cards  130  will need to be immediately re-routed back to the processing cards  130 , although possibly to a different processing card than the one it originated from. Both of these types of immediate re-routing functionality can be referred to as “loopback” functionality. 
   As a result of this requirement for some degree of loopback functionality, the switch fabric  200  can be constructed from two or more interconnected cross-point switches of smaller dimensionality, as illustrated in  FIG. 2 . Specifically, switch  230  takes care of switching a subset of 96 electrical signals (from the processing cards  130  to the external world), while switch  240  takes care of switching a subset of  112  electrical signals (from the external world to the processing cards  130 ). Loopback functionality is made possible through an interconnection of a relatively small number of connections, in this case twenty-eight (28), between the two cross-point switches  230 ,  240 . 
   Hence, the requisite functionality of a massive 208×208 switch fabric  200  can be attained using one 124×124 switch matrix (where 124=96+28) and one 140×140 switch matrix (where 140=112+28). In some circumstances, it may be advantageous to over-provision slightly and use two identical switch matrices, which in this case translates into a requirement for two 140×140 cross-point switch matrices. Cross-point switches of this magnitude are available from Velio Semiconductor Corp. as part number VC 3003. Of course, other variations are possible, especially with respect to the number of input and output ports on each of the cross-point switches  230 ,  240  and the number of intra-fabric connections provided for loopback purposes. It should be appreciated that signal conditioning functionality (e.g., regeneration and re-timing) may additionally be provided within the switch fabric  200  or in connection therewith. 
   When signal conditioning is performed at the periphery of the switch fabric  200 , this may be achieved through the use of a dedicated signal conditioning module. 
   It should also be mentioned at this point that if the signals being handled by the chassis  100  need to remain in an optical form throughout their journey through the chassis  100 , MEMS (micro-electro-mechanical switch) devices or the like may be used in the switch fabric  200  instead of the cross-point switches  230 ,  240 . This may also require signal conditioning, albeit of a different type (e.g., re-timing and possibly multi-mode to single-mode conversion or vice-versa). 
   The connection map applied by the switch fabric  200  is controlled by a controller  250 , which may be embodied as a microprocessor, FPGA, EEPROM, etc. The format of the connection map output by the controller  250  will, of course, depend on the internal structure of the switch fabric  200 . In the case of the illustrated embodiment, the controller  250  would be responsible for providing two 140×140 connection maps, one to each of the cross-point switches  230 ,  240 . By changing the content of the connection maps, the controller  250  can change the mutual interconnection of the processor cards  130  within the chassis  100  and also the interconnection defined between the processor cards  130  in the chassis  100  and the external world relative to the chassis, which includes other chassis in a multi-chassis configuration. 
   The controller  250  may be located in the chassis  100  itself, either on a separate controller card or on the interconnection module  140 . The controller  250  may be accessed through the backplane via a dedicated external communication channel or it may be accessed through one of the line cards  110  via one of the paths P 64 -P 95 . The controller  250  may be responsive to instructions transmitted via a modem or other interface device (e.g., a communications adapter) connected over a transmission medium such as a tangible medium (e.g., optical or analog communications lines) or a medium implemented using wireless techniques (e.g., microwave, infrared or other transmission schemes). 
   As previously mentioned, a router may be built from two or more identical chassis  100  of the type illustrated in  FIG. 1 . The chassis are interconnected to one another via their interconnect modules&#39; optical ports in accordance with an inter-chassis topology. Each full-duplex optical path travelling between optical ports may be carried on two separate optical fibers or on separate wavelengths of the same fiber or in any other suitable way known to those of ordinary skill in the art. Many inter-chassis topologies are within the scope of the present invention. In some topologies, it is desired to mesh all the chassis using the available optical ports. In other topologies, multiple chassis may be connected in a ring- or star-like configuration. In still other topologies, the chosen interconnect strategy will seek to keep the maximum number of “hops” between chassis to below a given upper bound. 
   In either case, the router so created can be upgraded by simply adding one or more supplementary chassis to the existing group of chassis, adding new interconnections between previously idle optical ports and re-programming the interconnection modules  140  in all chassis. It is noted that the interconnection modules in the supplementary chassis may be pre-programmed prior to their interconnection to the other chassis, or they may be programmed once the connections have been established. Also of note is the fact that the hardware connections within each chassis and between the chassis remain fixed, and are merely appended to as the router is scaled; rather, it is the software that adapts to the growing size of the router. Hence, the router can be scaled without ever having to replace any hardware (thus minimizing the cost) and without having to disconnect any physical connections between chassis or within any of the chassis (thus minimizing the down time). Thus, the use of a programmable interconnect module  140  within each chassis greatly simplifies scaling. 
   The connection map that is provided to the switch fabric  200  of an interconnection module  140  of a given chassis in the router will be a function of the chosen inter-chassis topology and the position of the given chassis within that topology. Care must therefore be taken to properly program each switch fabric  200  so as to allow the desired connectivity to take place.  FIGS. 3A ,  4 A and  5 A provide examples of an inter-chassis topology, for a router offering a progressively larger capacity. The accompanying  FIGS. 3B ,  4 B and  5 B provide examples of suitable intra-chassis interconnection possibilities that enable the router to achieve the requisite functionality at the corresponding stage of growth. 
   Accordingly,  FIG. 3A  shows a router  300  comprising only two interconnected chassis  310 ,  320 . For ease of reference, each of the chassis  310 ,  320  is identical to the chassis described previously with reference to  FIGS. 1 and 2 . Therefore, each of the chassis  310 ,  320  includes four optical ports; specifically, chassis  310  includes optical ports A 310 , B 310 , C 310 , D 310  and chassis  320  includes optical ports A 320 , B 320 , C 320 , D 320 . The inter-chassis topology is defined by optical port A 310  of chassis  310  being connected to optical port A 320  of chassis  320  by an optical fiber bundle  399   A . In this case, optical fiber bundle  399   A  establishes 32 full-duplex optical paths between optical ports A 310  and A 320 , which can be carried as 64 unidirectional paths on 64 optical fibers or multiplexed in any suitable way. 
     FIG. 3B  provides a detailed view of the interconnection pattern established within each of the chassis  310 ,  320 . It has been assumed that all paths are full-duplex paths, although each such full-duplex path may be implemented by a multiplicity of physical links. For ease of illustration, the line cards have been omitted. Also for ease of illustration, the input and output interfaces IN, OUT of the switch fabric in the interconnection module  140  are not specifically shown but they can be unambiguously determined from the correspondence established above with electrical paths P 0 -P 95  (in the case of IN 0 -IN 95  and OUT 0 -OUT 95 ), optical paths A 0 -A 31  (in the case of IN 96 -IN 127  and OUT 96 -OUT 127 ), optical paths B 0 -B 23  (in the case of IN 128 -IN 151  and OUT 128 -OUT 151 ), optical paths C 0 -C 23  (in the case of IN 152 -IN 175  and OUT 152 -OUT 175 ) and optical paths D 0 -D 31  (in the case of IN 176 -IN 207  and OUT 176 -OUT 207 ). 
   Turning first to chassis  310  in  FIG. 3B , the interconnection module  140  is programmed to establish the following connection groups:
     (i) paths P 64 -P 71  to paths P 0 -P 7  (loopback)   (ii) paths P 72 -P 79  to optical paths A 0 -A 7  of optical port A 310      (iii) paths P 80 -P 87  to paths P 8 -P 15  (loopback)   (iv) paths P 88 -P 95  to optical paths A 8 -A 15  of optical port A 310      (v) paths P 16 -P 31  to optical paths A 16 -A 31  of optical port A 310      

   With continued reference to  FIG. 3B , the interconnection module  140  of chassis  320  is programmed to establish the following connection groups:
     (vi) paths P 64 -P 71  to paths P 0 -P 7  (loopback)   (vii) paths P 72 -P 79  to optical paths A 16 -A 23  of optical port A 320      (viii) paths P 80 -P 87  to paths P 8 -P 15  (loopback)   (ix) paths P 88 -P 95  to optical paths A 24 -A 31  of optical port A 320      (x) paths P 16 -P 31  to optical paths A 0 -A 15  of optical port A 320      

   With additional reference to  FIG. 4A , the capacity of the router  300  of  FIG. 3A and 3B  can be augmented by adding a third chassis  330  to the existing set of chassis  310 ,  320 , resulting in the creation of a router  400 . Chassis  330  has a plurality of optical ports denoted A 330 , B 330 , C 330 , D 330 . A fiber bundle  399   B  is connected between optical ports B 310  and B 330  on chassis  310  and  330 , respectively, while a fiber bundle  399   C  is connected between optical ports C 320  and C 330  on chassis  320  and  330 , respectively. It is noted that fiber bundle  399   A , which joins optical ports A 310  and A 320  on chassis  310  and  320 , respectively, remains intact and that none of the hardware within any chassis needs to be replaced, disconnected to re-connected. 
     FIG. 4B  provides a more detailed view of one possible intra-chassis interconnection pattern which can be established by the interconnection module  140  of each chassis  310 ,  320 ,  330  and which allows the router  400  to function as a scaled version of the router  300 . It is recalled that the actual connections (i.e., from electrical interface to electrical interface) of the switch fabric within each interconnection module  140  can be derived from the illustrated interconnection pattern and from the correspondence between the interfaces and the paths, which was described earlier with reference to  FIGS. 2 and 3B . Specifically, the following connection groups have been established within chassis  310 :
     (i) paths P 64 -P 71  to optical paths A 0 -A 7  of optical port A 310      (ii) paths P 72 -P 75  to paths P 0 -P 3  (loopback)   (iii) paths P 76 -P 79  to paths P 36 -P 39  (loopback)   (iv) paths P 80 -P 87  to optical paths A 16 -A 23  of optical port A 310      (v) paths P 88 -P 91  to paths P 32 -P 35  (loopback)   (vi) paths P 92 -P 95  to paths P 4 -P 7  (loopback)   (vii) paths P 8 -P 15  to optical paths A 8 -A 15  of optical port A 310      (viii) paths P 16 -P 23  to optical paths B 0 -B 7  of optical port B 310      (ix) paths P 40 -P 47  to optical paths A 24 -A 31  of optical port A 310      (x) paths P 48 -P 55  to optical paths B 8 -B 15  of optical port B 310      
   With continued reference to  FIG. 4B , the interconnection module  140  of chassis  320  is programmed to establish the following connection groups:
     (xi) paths P 64 -P 67  to optical paths A 8 -A 11  of optical port A 320      (xii) paths P 68 -P 71  to optical paths A 24 -A 27  of optical port A 320      (xiii) paths P 72 -P 75  to paths P 0 -P 3  (loopback)   (xiv) paths P 76 -P 79  to paths P 36 -P 39  (loopback)   (xv) paths P 80 -P 83  to optical paths A 12 -A 15  of optical port A 320      (xvi) paths P 84 -P 87  to optical paths A 28 -A 31  of optical port A 320      (xvii) paths P 88 -P 91  to paths P 32 -P 35  (loopback)   (xviii) paths P 92 -P 95  to paths P 4 -P 7  (loopback)   (xix) paths P 8 -P 15  to optical paths A 0 -A 7  of optical port A 320      (xx) paths P 24 -P 31  to optical paths C 0 -C 7  of optical port C 320      (xxi) paths P 40 -P 47  to optical paths A 16 -A 23  of optical port A 320      (xxii) paths P 48 -P 55  to optical paths C 8 -C 15  of optical port C 320      

   With continued reference to  FIG. 4B , the interconnection module  140  of chassis  330  is programmed to establish the following connection groups:
     (xxiii) paths P 64 -P 67  to optical paths B 0 -B 3  of optical port B 330      (xxiv) paths P 68 -P 71  to optical paths B 8 -B 11  of optical port B 330      (xxv) paths P 72 -P 75  to optical paths C 0 -C 3  of optical port C 330      (xxvi) paths P 76 -P 79  to optical paths C 8 -C 11  of optical port C 330      (xxvii) paths P 80 -P 83  to optical paths B 4 -B 7  of optical port B 330      (xxviii) paths P 84 -P 87  to optical paths B 12 -B 15  of optical port B 330      (xxix) paths P 88 -P 91  to optical paths C 4 -C 7  of optical port C 330      (xxx) paths P 92 -P 95  to optical paths C 12 -C 15  of optical port C 330      

   Thus, it has been shown that the capacity of a router designed according to an embodiment of the present invention can be increased by (1) adding a new chassis; (2) connecting one or more additional fiber bundles between the optical ports of the chassis; and (3) re-programming the interconnection module  140  in each chassis via the controller  250 . No existing connection within any chassis or between any pair of chassis needs to be physically dismantled or re-established, resulting in a truly scalable solution to the problem of increasing router capacity. It is also noted that the interconnection module in new chassis may be pre-programmed prior to its interconnection to the existing chassis, or it may be programmed once the connections to the existing chassis have been established. 
   With additional reference now to  FIG. 5A , it is shown how the capacity of the router  400  of  FIGS. 4A and 4B  can be further augmented without requiring the disconnection or re-connection of equipment, and without requiring existing equipment to be replaced. Specifically, the addition of three more chassis  340 ,  350 ,  360  results in the creation of a router  500 . Chassis  340  has a plurality of ports denoted A 340 , B 340 , C 340 , D 340 , chassis  350  has a plurality of ports denoted A 350 , B 350 , C 350 , D 350  and chassis  360  has a plurality of ports denoted A 360 , B 360 , C 360 , D 360 . 
   A fiber bundle  399 * A  is connected between optical ports A 330  and A 340  on chassis  330  and  340 , respectively. A fiber bundle  399 * B  is connected between optical ports B 320  and B 340  on chassis  320  and  340 , respectively. A fiber bundle  399 * C  is connected between optical ports C 310  and C 340  on chassis  310  and  340 , respectively. A fiber bundle  399   D  is connected between optical ports D 310  and D 350  on chassis  310  and  350 , respectively. Finally, a fiber bundle  399 * D  is connected between optical ports D 320  and D 360  on chassis  320  and  360 , respectively. It is noted that fiber bundles  399   A ,  399   B  and  399   C  remain intact and that none of the hardware within any given chassis needs to be replaced, disconnected to re-connected. 
     FIG. 5B  provides a more detailed view of suitable connections that can be established by the interconnection module within each of the chassis  310 ,  320 ,  330 ,  340 ,  350  and  360 , thus demonstrating scalability of the three-chassis router  400  of  FIG. 4A  to the six-chassis router  500  of  FIG. 5A . Specifically, the following connection groups are established for chassis  310 :
     (i) paths P 64 -P 67  to optical paths A 0 -A 3  of optical port A 310      (ii) paths P 68 -P 71  to optical paths B 0 -B 3  of optical port B 310      (iii) paths P 72 -P 75  to optical paths C 0 -C 3  of optical port C 310      (iv) paths P 76 -P 77  to paths P 0 -P 1  (loopback)   (v) paths P 78 -P 79  to paths P 34 -P 35  (loopback)   (vi) paths P 80 -P 83  to optical paths A 8 -A 11  of optical port A 310      (vii) paths P 84 -P 87  to optical paths B 8 -B 11  of optical port B 310      (viii) paths P 88 -P 91  to optical paths C 8 -C 11  of optical port C 310      (ix) paths P 92 -P 93  to paths P 32 -P 33  (loopback)   (x) paths P 94 -P 95  to paths P 2 -P 3  (loopback)   (xi) paths P 4 -P 7  to optical paths A 4 -A 7  of optical port A 310      (xii) paths P 8 -P 11  to optical paths B 4 -B 7  of optical port B 310      (xiii) paths P 12 -P 15  to optical paths A 12 -A 15  of optical port A 310      (xiv) paths P 16 -P 19  to optical paths D 0 -D 4  of optical port D 310      (xv) paths P 20 -P 23  to optical paths C 4 -C 7  of optical port C 310      (xvi) paths P 36 -P 39  to optical paths A 12 -A 15  of optical port A 310      (xvii) paths P 40 -P 43  to optical paths B 12 -B 15  of optical port B 310      (xviii) paths P 44 -P 47  to optical paths A 28 -A 31  of optical port A 310      (xix) paths P 48 -P 51  to optical paths D 4 -D 7  of optical port D 310      (xx) paths P 52 -P 55  to optical paths C 12 -C 15  of optical port C 310      (xxi) optical paths A 16 -A 23  to optical paths D 8 -D 15  (loopback)   (xxii) optical paths B 16 -B 23  to optical paths D 16 -D 23  (loopback)   (xxiii) optical paths C 16 -C 23  to optical paths D 24 -D 31  (loopback)   
   With continued reference to  FIG. 5B , the interconnection module  140  of chassis  320  is programmed to establish the following connection groups:
     (xxiv) paths P 64 -P 65  to optical paths A 4 -A 5  of optical port A 320      (xxv) paths P 66 -P 67  to optical paths A 12 -A 13  of optical port A 320      (xxvi) paths P 68 -P 71  to optical paths B 0 -B 3  of optical port B 320      (xxvii) paths P 72 -P 75  to optical paths C 0 -C 3  of optical port C 320      (xxviii) paths P 76 -P 77  to paths P 0 -P 1  (loopback)   (xxix) paths P 78 -P 79  to paths P 34 -P 35  (loopback)   (xxx) paths P 80 -P 81  to optical paths A 6 -A 7  of optical port A 320      (xxxi) paths P 82 -P 83  to optical paths A 14 -A 15  of optical port A 320      (xxxii) paths P 84 -P 87  to optical paths B 8 -B 11  of optical port B 320      (xxxiii) paths P 88 -P 91  to optical paths C 8 -C 11 of optical port C 320      (xxxiv) paths P 92 -P 93  to paths P 32 -P 33  (loopback)   (xxxv) paths P 94 -P 95  to paths P 2 -P 3  (loopback)   (xxxvi) paths P 4 -P 5  to optical paths A 0 -A 1  of optical port A 320      (xxxvii) paths P 6 -P 7  to optical paths A 8 -A 9  of optical port A 320      (xxxviii) paths P 8 -P 11  to optical paths B 4 -B 7  of optical port B 320      (xxxix) paths P 12 -P 15  to optical paths A 16 -A 19  of optical port A 320      (xl) paths P 16 -P 19  to optical paths D 0 -D 3  of optical port D 320      (xli) paths P 20 -P 23  to optical paths C 4 -C 7  of optical port C 320      (xlii) paths P 36 -P 37  to optical paths A 2 -A 3  of optical port A 320      (xliii) paths P 38 -P 39  to optical paths A 10 -A 11  of optical port A 320      (xliv) paths P 40 -P 43  to optical paths B 12 -B 15  of optical port B 320      (xlv) paths P 44 -P 47  to optical paths A 20 -A 23  of optical port A 320      (xlvi) paths P 48 -P 51  to optical paths D 4 -D 7  of optical port D 320      (xlvii) paths P 52 -P 55  to optical paths C 12 -C 15  of optical port C 320      (xlviii) optical paths A 24 -A 31  to optical paths D 8 -D 15  (loopback)   (xlix) optical paths B 16 -B 23  to optical paths D 16 -D 23  (loopback)   (l) optical paths C 16 -C 23  to optical paths D 24 -D 31  (loopback)   

   The connections for the other chassis (namely, chassis  330 ,  340 ,  350  and  360 ) can similarly be obtained from inspection of  FIG. 5B  by a person of ordinary skill in the art. 
   From the above, it is again seen how the capacity of a router designed according to an embodiment of the present invention can be increased by simply (1) adding a set of additional chassis; (2) connecting additional fiber bundles between the optical ports of the existing and additional chassis; and (3) re-programming the interconnection module  140  in each chassis via the controller  250 . No existing connection within any chassis or between any pair of chassis needs to be physically dismantled or re-established, resulting in a truly scalable solution to the problem of increasing router capacity. 
   It also should be noted that for the router  500  of  FIGS. 5A and 5B , there are six chassis but only four optical ports per chassis and hence there are more chassis in the router  500  than there are optical ports per chassis. When this occurs, it may be useful to divide the router into “clusters” of chassis where, in this case, there are up to four chassis per cluster. For each chassis, three optical ports per chassis are reserved for connections with other chassis within the same cluster, while the remaining (fourth) optical port is used to connect to a chassis in another cluster. In general, for an N-port chassis, where M−1 of N ports are reserved for intra-cluster connections, the total number of M-chassis clusters that can be accommodated is (N−(M−1))+1=N−M+2. In the case where N=M (as in  FIG. 5A ), the maximum number of clusters is two, with M (or N, since N is equal to M) chassis in each cluster. 
   It should be appreciated that the number of chassis required to achieve a certain switching capacity may exceed the maximum number of chassis that can be accommodated by clustering. Nonetheless, it is still possible to build a scalable router in accordance with an embodiment of the present invention. In such a case, an additional component needs to be introduced, namely a “chassis interconnection module” (CIM). 
   By way of example,  FIG. 6  shows a router  600  composed of three clusters  610 A,  610 B and  610 C. Within cluster  610 A are contained five chassis  640  and an optional switching element  660 , all of which connected to a common CIM  630  via fiber optic bundles (or DWDM fibers)  650 . The CIM  630  for a given cluster may have a larger number of ports than any one chassis and provides programmable optical interconnections (e.g., using MEMS devices). The illustrated embodiment shows that none of the chassis within a given cluster is connected directly to another chassis within that cluster Such chassis-to-chassis connections are provided by “loopback” functionality of the CIM  630 , which allows connecting two chassis together within the same cluster and connecting two other CIMs via the CIM  630 . Nevertheless, it is within the scope of the present invention to provide chassis-to-chassis connections within the same cluster, which may be advantageous when implementing a hypercube-based topology. 
   The clusters themselves are interconnected via the CIM in each cluster. An ultra-dense optical link may interconnect the CIMs in the various clusters. For instance, in the illustrated embodiment, link  620  connects the CIMs in clusters  610 A and  610 B, while link  622  connects the CIMs in clusters  610 A and  610 C. A link  624  emanates from CIM  630  and is unused for the time being but is available for future use, in case another cluster is added to the router  600 . The ultra-dense links  620 ,  622 ,  624  may be DWDM optical fibers or they may comprise bundles of single-carrier or coarsely multiplexed optical fibers. 
   In this way, different numbers of clusters can be interconnected to form a router. Scalability of the router is achieved by adding one or more clusters (with respective CIMS), adding selected connections between CIMs and re-programming the CIMs. No intra-cluster connections need to be disabled or re-connected. Careful observation of this architecture reveals similarity with the architecture of  FIG. 1 , where the CIM of  FIG. 6  plays the role of the interconnection module  140  of  FIG. 1 . Thus, one skilled in the art who has understood how the interconnection module can be reprogrammed to provide the required connectivity will also understand how the same can be achieved by re-programming the CIM  630 . 
   It should be understood, of course, that the above described functionality of the chassis and routers can apply to multiple individual streams of data flow, sometimes referred to as “data planes”. Thus, although the above description has been formulated in terms of a single data plane, it should be understood that the present invention is equally applicable to the transmission of information across multiple independent planes which draw upon the resources of a common set of chassis. The line cards and processing cards in each chassis may either be dedicated to a particular data plane or may be shared amongst two or more date planes. 
   Also, the term “card” is meant to be interpreted broadly so as to cover not only a printed circuit board that has connectors disposed primarily along an edge thereof, but also other modules that contain circuitry, software and/or control logic capable of providing the requisite functionality. A large-area wafer comprising all the requisite internal circuitry of one or more “cards” is also envisaged. 
   While specific embodiments of the invention have been described and illustrated, those skilled in the art will appreciate that further modifications and variations may be made without departing from the scope of the invention as defined in the claims appended hereto.