Patent Publication Number: US-8995456-B2

Title: Space-space-memory (SSM) Clos-network packet switch

Description:
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT 
     This work was supported in part by National Science Foundation grant number 0435250. 
    
    
     BACKGROUND 
     As the Internet continues to grow, high-capacity switches and routers are needed for backbone networks. Several approaches have been presented for high-speed packet switching systems. Most high-speed packet switching systems use a fixed-sized cell in the switch fabric. Variable-length packets are segmented into several fixed-sized cells when they arrive, switched through the switch fabric, and reassembled into packets before they depart. 
     For implementation in a high-speed switching system, there are mainly two approaches. One approach is a single-stage switch architecture. An example of the single-stage architecture is a crossbar switch, where identical switching elements are arranged on a matrix plane. However, the number of I/O pins in a crossbar chip may limit the switch size. This makes a large-scale switch difficult to implement cost-effectively, as the number of chips becomes large. Another approach is to use a multiple-stage switch architecture, such as a Clos-network switch. The Clos-network switch architecture, which is a three-stage switch, is scalable. Three-stage Clos-network switches use small switches as modules in each stage to build a switch with a large number of ports and less hardware than that of a single-stage switch of the same size. 
    
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 
       The foregoing and other features of the present disclosure will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. Understanding that these drawings depict only several embodiments in accordance with the disclosure and are, therefore, not to be considered limiting of its scope, the disclosure will be described with additional specificity and detail through use of the accompanying drawings. 
       In the drawings: 
         FIG. 1  is a schematic diagram of an example SSM Clos-network switch; 
         FIG. 2  is another schematic diagram of the example SSM Clos-network switch shown in  FIG. 1 ; 
         FIG. 3  is another schematic diagram of the example SSM Clos-network switch shown in  FIG. 1 ; 
         FIG. 4  is a flowchart showing the operation of another example SSM Clos-network switch; 
         FIG. 5  is a flowchart showing the operation of yet another example SSM Clos-network switch; and 
         FIG. 6  is a block diagram illustrating an example computing device that is arranged for Clos-network switch implementations, all arranged in accordance with at least some embodiments of the present disclosure. 
     
    
    
     DETAILED DESCRIPTION 
     In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented here. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the Figures, may be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which are explicitly contemplated and make part of this disclosure. 
     This disclosure is drawn, inter alia, to methods and systems related to Clos-network switch systems and methods for efficiently implementing high-density network switches. An example embodiment generally relates to a switch architecture and a method for configuring the switch that are based on a Clos-network architecture, which implements multiple small switches to build a large-scale switch. 
     Currently, there are three broad types of Clos-network switches: Space-Space-Space (SSS or buffer-less) architecture, Memory-Space-Memory (MSM) architecture, and Memory-Memory-Memory (MMM) architecture. A SSS Clos-network switch has no memory in any of the three stages. Although the design of the switch modules is rather simple, the SSS Clos-network switch may require a complex matching process and a long resolution time. A variety of matching schemes for SSS Clos-network switches have been proposed. A MSM Clos-network switch uses buffers in the first-stage and third-stage modules to simplify the configuration complexity of Clos-network switches. In this way, the scheduling of packets becomes a dispatching issue. However, the buffers in the first-stage and third-stage modules need to work with speedup (or the implementation of one or more parallel algorithms). This makes the implementation infeasible. A MMM Clos-network switch has buffers in all three stages. This may help to resolve contention from different first-stage modules. However, switches with buffers in the second-stage modules may suffer from serving packets out-of-sequence, which is undesirable, as re-sequencing packets increases the switch&#39;s complexity and cost. These switches and schemes, although they are very efficient with benign admissible traffic, require long communication delays among arbiters, or require speedup. The port rates and switch implementation are limited to those delays and speedup. Switch and router builders have not provided an efficient way to implement high-density switches. Further, current switches have a limited port capacity. 
       FIG. 1  is a schematic diagram of an example Space-Space-Memory [SSM] Clos-network switch  10  that is arranged in accordance with at least some embodiments of the present disclosure. The example SSM Clos-network switch  10  includes a three-stage switch architecture with cross-point buffers in the third-stage modules and virtual output queues (VOQs)  12  at the input ports. VOQs are input queuing mechanisms in which each input port maintains a separate queue for each output port. The first-stage  14  may include input modules (IMs) such as IM( 0 )  20 , the second-stage  16  may include central modules (CMs) such as CM( 0 )  22 , and the third-stage  18  may include output modules (OMs) such as OM( 0 )  24 . 
     The example three-stage SSM Clos-network switch  10  utilizes buffers in the crossbars at the third-stage modules. The memory implemented in the buffered crossbar in the third-stage module does not need speedup and may simplify the switch configuration process. The configuration method may be used to provide connectivity between input and output ports of SSM Clos-network switches by performing matching at the module level in a simple and efficient way and by avoiding the matching of ports. The decision of which output ports are connected to the input port may be achieved by allowing the output ports to select a packet from the cross-point buffers at the third-stage modules so that matching may not be needed. The implementation of a scheduler capable of matching thousands of ports in large-size switches may have prohibitively large complexity. To decrease the scheduler complexity, the module-to-module method hierarchizes the matching process and may make the implementation of large switches feasible by requiring arbiters of relatively small size. 
       FIG. 2  is another schematic diagram of the example SSM Clos-network switch shown in  FIG. 1 , arranged in accordance with at least some embodiments of the present disclosure. As shown in  FIG. 2 , the number of IMs/OMs  20 / 24  is signified by k, while the number of CMs  22  is signified by m. The number of input ports (IPs)  28  and output ports (OPs)  30  in each IM/OM  20 / 24  is signified by n. i signifies the IM  20  number, where 0≦i≦k−1. j signifies the OM  24  number, where 0≦j≦k−1. r signifies CM  22  number, where 0≦r≦m−1. h signifies IP/OP  28 / 30  number in each IM/OM  20 / 24 , respectively, where 0≦h≦n−1. The IM(i) module  32  signifies the (i+1)th input module, where 0≦i≦k−1. The CM(r) module  34  signifies the (r+1)th central module, where 0≦r≦m−1. The OM(j) module  36  signifies the (j+1)th output module, where 0≦j≦k−1. The IP(i,g) input port  28  signifies the (g+1)th input port at IM(i)  32 , where 0≦g≦n−1. The OP(j,h) output port  30  signifies the (h+1)th output port at OM(j)  36 , where 0≦h≦n−1. VOQ(i,g,j,h)  26  signifies the virtual output queue at IP(i,g)  28  that is destined for OP(j,h). The cross-point buffer at OM(j)  36  that stores cells from CM(r)  34  to OP(j,h) is signified by CXB(r,j,h)  38 . 
     In the illustrated example, each IP(i,g)  28  has N=n×k VOQs  26  to avoid head-of-line (HOL) blocking, a phenomenon that may severely degrade switch performance by limiting a switch&#39;s  10  throughput. N signifies the total number of ports of the Clos-network switch. Each OM(j)  34  has N=n×k cross-point buffers  38  to store cells going from VOQ(i,g,j,h)  26  to OP(j,h). Further, L I (i; r) signifies an output link at IM(i)  32  that may be coupled to CM(r)  34 . L C (r; j) signifies an output link at CM(r)  34  that is coupled to OM(j)  36 . 
       FIG. 3  is another schematic diagram of the example SSM Clos-network switch shown in  FIG. 1 , arranged in accordance with at least some embodiments of the present disclosure. As shown in  FIG. 3 , the SSM Clos-network switch  10  may include a scheduler  82 , S M , configured to perform module matching, which matches the IM(i)  32 -OM(j)  36  pairs. There may also be input arbiters  80  and output arbiters  72  at the inputs and OM(j)  36 , respectively, configured to arbitrate the packets at the VOQs  26  and cross-point buffers  38  for dispatching. 
     In some embodiments, cells traverse the SSM Clos-network switch  10  as follows with reference to  FIGS. 1-3 . First, cells arrive in the VOQs  26  at the inputs and the module scheduler  82  is notified. This scheduler  82  may match IMs  20  to OMs  24 . Input arbiters  80  of the matched IMs may select a packet from the VOQ  26  with the longest occupancy (typically, the packet that has been in the queue the longest amount of time) to forward a cell to the destined OM  24 . The selected cells may be sent to and stored at their corresponding cross-point buffers  38  at OMs  24 . Output arbiters  72  may then select a packet from the cross-point buffer  38  with the longest occupancy (again, typically, the packet that has been in the queue the longest amount of time) to be forwarded to the OP  30 . Packets selected by the output arbiters  72  may then be forwarded to the OPs  30 . 
     The use of buffers  38  in the SSM Clos-network switch  10  may make port matching needless, and thereby may reduce the configuration time. Although some embodiments of the SSM Clos-network switch  10  may use cross-point buffers  38  in the third-stage modules  18 , the third-stage modules  18  are not required to work with a memory speedup. 
     Matching schemes used to configure SSM Clos-network switches  10  may adopt two phases: port matching first and routing assignment thereafter. However, executing these two phases may be complex, as output contention and path routing may need to be resolved for every time slot before the cell transmission occurs. The configuration process in some described embodiments of the SSM Clos-network switch  10  may consist of route assignment only, as port matching may not be needed. Port matching may not be needed because several input ports may send cells to a single output in a given time slot, and the buffers at the output port may store all those cells while dispatching one cell out of the port. 
     Additional embodiments may include a configuration matching scheme for the SSM Clos-network switch  10 . This matching scheme may include a weighted module-first and none-port matching scheme (WMF-NP) for the SSM Clos-network switch  10 . In some embodiments, the WMF-NP matching scheme may perform module-to-module matching first, and then may match an input to the output-links  74  of input modules  20 , which may be executed at the same time the output port arbitration is executed. Input and output arbitrations may be performed separately instead of matching input  28  and output  30  ports. Output arbiters  72  at the output ports  30  in the third-stage modules  18  may select a packet from the cross-point buffers  38  in an independent manner. This may be an improvement over previously proposed weight-based module-first matching schemes for SSS Clos-network switches. The WFM-NP scheme may reduce the scheduler  82  and arbiter sizes and response time of SSM Clos-network switches  10 . Furthermore, the memory implemented in the buffered crossbar  38  in the third-stage module  18  may not require speedup to achieve high switching performance. 
     The present disclosure considers that the WMF-NP scheme may use queue occupancy as the selection policy or weight, for example. In some embodiments, the WMF-NP matching scheme, VOQs  26  and IM output-link arbiters  74 , and the input  80  and output arbiters  72  may all use the longest queue-occupancy first as the selection policy. For example, the WMF-NP matching scheme may consider the occupancy of all input ports  28  in an IM  20  for module matching. Further, the VOQs  26  and output-link arbiters  74  may determine what CM  22  cells will use based on the occupancy of CMs  22 . Even further, the input arbiter  80  may first select a cell from the queue with the longest occupancy among non-empty VOQs  26  in order to forward the cell to the cross-point buffers  38 , and the output arbiters  72  at the third-stage module  18  may select the cross-point buffer  38  with the longest occupancy to forward a cell to the OP  30 . As in  FIG. 2 , a flow control mechanism may be used to indicate to VOQ(i,g,j,h)  26  which CXB(r,j,h)  38  may be available, so that VOQ(i,g,j,h)  26  may forward a cell. 
     In some embodiments, to determine the weight for the IM(i)  32 -OM(j)  36  matching, a switch  10  may implement a VOQ module counter, or VMC(i,j), to count the number of cells in IM(i)  32  that are destined to OM(j)  36 . The switch  10  may perform i iterations of matching between VOQs  26  and IM output-link arbiters  74 , and I M  iterations for module matching. Each L I (i,r) may have an available/matched flag FL I (i,r) and each L C (r,j) may have an available/matched flag FL C (r,j). These flags may indicate whether or not a link (and, therefore, the configuration of CM(r))  34  is selected. These flags may be used to define eligibility of an OM  24  in the module-matching phase. OM(j)  36  may be considered eligible to match IM(i)  32  if there is at least one path (and L I (i,r 1 ) and L C  (r 2 ,j), where r 1 =r 2 ) available connecting these two modules. 
     In some example embodiments, the WMF-NP matching scheme may be implemented as follows. In the first iteration of the WFM-NP matching scheme, module matching may occur. The module matching process follows a request-select-accept approach and includes the following operations: 
     Operation 1 (Request):
         Each VMC with a count larger than zero may transmit a request to the destined and eligible OM arbiter  72  at the S M . The transmitted requests may include the number of cells for an OM  24 .       

     Operation 2 (Select):
         If an unmatched OM arbiter at the S M  receives requests, the unmatched OM arbiter may select the request from the VOQ  26  with the largest occupancy.       

     Operation 3 (Accept):
         If an unmatched IM arbiter at the S M  receives one or more selections, the IM arbiter may accept the request from the VOQ  26  with the largest occupancy. The FL I  and FL C  flags of the matched links may be set as matched.       

     The present disclosure considers that, in the first iteration of the WFM-NP matching scheme (but after the module matching process), the VOQ  26  selection process and the matching process within the IM  20  occurs. In some embodiments, these processes may occur simultaneously. In some other embodiments, the IM  20  matching process must occur after the VOQ  26  selection process. 
     In the VOQ  26  selection process, each input arbiter (I A )  80  may select one non-empty VOQ  26  for the matched OM(j)  36  by using a “longest queue first,” or LQF, selection policy. Other selection policies may also be implemented. For example, a “largest queue first” selection policy may be implemented. 
     In the IM  20  matching process, each L i  may be matched to an input and includes the following operations: 
     Operation 1 (Request):
         Each input with cells to OM(j)  36  sends a request to all k L i (i,r) arbiters.       

     Operation 2 (Select):
         Each L i  arbiter then selects the request of an input whose weight is the largest and sends a grant to the input.       

     Operation 3 (Accept):
         Each input accepts one grant.       

     The IM  20  matching may perform m iterations among those unmatched L i  and inputs. In the I M th iteration of the WFM-NP matching scheme, module matching may again be performed. Note that I M  may have a value of up to k=N/n iterations. In each of these iterations, modules that meet the following criteria are matched:
         Criteria 1: Modules with one or more unselected, non-empty input ports  28 ; and   Criteria 2: r 1  of IM output link L i (i, r 1 ) is equal to r 2  of CM output link L C  (r 2 , j), where r 1  and r 2  are the index of CM(r)  34 .       

     These same modules may also be considered for VOQ  26  selection after each module matching iteration. 
     After the module matching and selection processes are executed for all iterations, output arbiters  72  at each output port in the OMs  24  may use the LQF policy to select a buffered cell among non-empty cross-point buffers  38  in order to forward a cell to the output port  30 . 
     In some embodiments, the scheduler  82  may include module-input arbiters  76  and module-output arbiters  78 . The module-input arbiters  76  and module-output arbiters  78  may assist in matching the input arbiters  80  and output arbiters  72  to facilitate efficient transmission. The scheduler  82  may communicate with input arbiters  80  and output arbiters  72  to inform them of existing packets for switching and to inform queues of which packets have been granted for switching, among other things. In the example of  FIG. 3 , the scheduler  82  (and corresponding module-input arbiters  76  and module-output arbiters  78 ) may be distributed throughout the CMs  22 . In another example, the scheduler (and corresponding module-input arbiters  76  and module-output arbiters  78 ) may be a component separate from, but in communication with the modules (IMs  20 , CMs  22 , and OMs  24 ). 
       FIG. 4  is a flowchart showing the operation of another example SSM Clos-network switch that is arranged in accordance with at least some embodiments of the present disclosure. The example embodiments may include one or more of processing operations  38 ,  40 ,  42 ,  44 ,  46 ,  48 ,  50  and  52 . 
     Processing begins at operation  38 , which may include receiving packets partitioned into fixed-size cells at a selected one of the plurality of input ports of the network switch architecture. 
     Processing flows from operation  38  to operation  40 . Operation  40  may include storing the fixed-size cells in a selected one of the virtual output queues associated with the selected one of the plurality of input ports 
     Continuing from operation  40  to operation  42 , operation  42  may include detecting the arrival of the fixed-size cells in the selected one of the virtual output queues with the input arbiter and notifying the switch scheduler. 
     Proceeding to operation  44 , operation  44  may include matching one of the plurality of input modules with one of the plurality of output modules to provide a matched input module and a matched output module with the switch scheduler, wherein the matched input module is associated with the selected one of the plurality of input ports. 
     Continuing to operation  46  from operation  44 , operation  46  may include selecting the fixed-size cells from the selected one of the plurality of virtual output queues with an input arbiter associated with the matched input module based, at least in part, on a first selection criteria. 
     Continuing to operation  48 , operation  48  may forwarding the selected fixed-size cells from the selected one of the plurality of virtual output queues through the matched input module to a selected one of the plurality of cross-point buffers of the matched output module. 
     Continuing to operation  50  from operation  48 , operation  50  may include selecting the fixed-size cells from the selected one of the plurality of cross-point buffers of the matched output module with an output arbiter associated with the matched output module based, at least in part, on a second selection criteria. 
     Proceeding to operation  52 , operation  52  may include forwarding the selected fixed-size cells from selected one of the plurality of cross-point buffers of the matched output module to an output port of the network switch architecture. 
       FIG. 5  is a flowchart showing the operation of yet another example SSM Clos-network switch arranged in accordance with at least some embodiments of the present disclosure. Example embodiments may include one or more of processing operations  54 ,  56 ,  68  and  60 . 
     Processing begins at operation  54 . Operation  54  may include matching one of the plurality of input modules with one of the plurality of output modules to provide a matched input module and a matched output module. 
     Continuing from operation  54  to operation  56 , operation  56  may include selecting a packet to transmit based, at least in part, on a first selection criteria. 
     Processing may continue at operation  58 , which may include transmitting the selected packet from the matched input module to a selected one of the plurality of cross-point buffers of the matched output module 
     Proceeding to operation  60 , operation  60  may include outputting the packet based, at least in part, on a second selection criteria. 
     In an example embodiment, a Clos-network architecture  10  may be configured to implement the method of  FIG. 5 . 
     With reference to  FIG. 6 , depicted is a block diagram illustrating an example computing device  600  that is arranged for Clos-network switch implementations arranged in accordance with at least some embodiments of the present disclosure. In a very basic configuration  601 , computing device  600  typically includes one or more processors  610  and system memory  620 . A memory bus  630  may be used for communicating between the processor  610  and the system memory  620 . 
     Depending on the desired configuration, processor  610  may be of any type including but not limited to a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. Processor  610  may include one more levels of caching, such as a level one cache  611  and a level two cache  612 , a processor core  613 , and registers  614 . The processor core  613  may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP Core), or any combination thereof. A memory controller  615  may also be used with the processor  610 , or in some implementations the memory controller  615  may be an internal part of the processor  610 . 
     Depending on the desired configuration, the system memory  620  may be of any type including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof. System memory  620  may include an operating system  621 , one or more applications  622 , and program data  624 . Application  622  may include a Clos-network switch system algorithm  623  that is implemented to efficiently manage network resources. Program Data  624  may include Clos-network switch system data  625 . In some embodiments, application  622  may be arranged to operate with program data  624  on an operating system  621  to effectuate the efficient management of network resources. This described basic configuration is illustrated in  FIG. 6  by those components within dashed line  601 . 
     Computing device  600  may have additional features or functionality, and additional interfaces to facilitate communications between the basic configuration  601  and any required devices and interfaces. For example, a bus/interface controller  640  may be used to facilitate communications between the basic configuration  601  and one or more data storage devices  650  via a storage interface bus  641 . The data storage devices  650  may be removable storage devices  651 , non-removable storage devices  652 , or a combination thereof. Examples of removable storage and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard-disk drives (HDD), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSD), and tape drives to name a few. Example computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. 
     System memory  620 , removable storage  651  and non-removable storage  652  are all examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which may be used to store the desired information and which may be accessed by computing device  600 . Any such computer storage media may be part of device  600 . 
     Computing device  600  may also include an interface bus  642  for facilitating communication from various interface devices (e.g., output interfaces, peripheral interfaces, and communication interfaces) to the basic configuration  601  via the bus/interface controller  640 . Example output devices  660  include a graphics processing unit  661  and an audio processing unit  6862 , which may be configured to communicate to various external devices such as a display or speakers via one or more A/V ports  663 . Example peripheral interfaces  670  include a serial interface controller  671  or a parallel interface controller  672 , which may be configured to communicate with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral devices (e.g., printer, scanner, etc.) via one or more I/O ports  673 . An example communication device  680  includes a network controller  681 , which may be arranged to facilitate communications with one or more other computing devices  690  over a network communication via one or more communication ports  682 . The communication connection is one example of a communication media. Communication media may typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. A “modulated data signal” may be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared (IR) and other wireless media. The term computer readable media as used herein may include both storage media and communication media. 
     Computing device  600  may be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a cell phone, a personal data assistant (PDA), a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions. Computing device  600  may also be implemented as a personal computer including both laptop computer and non-laptop computer configurations. 
     According to one embodiment, computing device  600  is connected in a networking environment such that the processor  610 , application  622  and/or program data  624  may perform with or as a Clos-network switch system in accordance with embodiments herein. 
     The herein described subject matter sometimes illustrates different components contained within, or connected with, different other components. It is to be understood that such depicted architectures are merely exemplary, and that in fact many other architectures may be implemented which achieve the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively “associated” such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality may be seen as “associated with” each other such that the desired functionality is achieved, irrespective of architectures or intermedial components. Likewise, any two components so associated may also be viewed as being “operably connected”, or “operably coupled”, to each other to achieve the desired functionality, and any two components capable of being so associated may also be viewed as being “operably couplable”, to each other to achieve the desired functionality. Specific examples of operably couplable include but are not limited to physically mateable and/or physically interacting components and/or wirelessly interactable and/or wirelessly interacting components and/or logically interacting and/or logically interactable components. 
     With respect to the use of substantially any plural and/or singular terms herein, those having skill in the art may translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth herein for sake of clarity. 
     It will be understood by those within the art that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes but is not limited to,” etc.). It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases “at least one” and “one or more” to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to inventions containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and/or “an” should typically be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of “two recitations,” without other modifiers, typically means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to “at least one of A, B, and C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, and C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). In those instances where a convention analogous to “at least one of A, B, or C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, or C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” will be understood to include the possibilities of “A” or “B” or “A and B.” 
     While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.