Patent Publication Number: US-11044539-B1

Title: Mouse over elephant

Description:
BACKGROUND 
     Data centers are a collection of devices and network switches. They may also be cloud scale datacenters, hyperscale data centers or High Performance Computers (HPC). 
     Devices are organized into a device plane of compute related nodes made available to a composer or orchestrator, such as: a server, a multi CPU server, multiple servers clustered in a chassis, hot storage, warm storage, cold storage, CPU, memory, SSD, JBOD, GPU, FPGA, ASIC; and exclude non-compute nodes used to operate, manage, and maintain the datacenter, such as: environmental control devices, management devices, laptops. 
     Network switches transport data between a port on a device to a port on another device. Packet switches buffer, route, and switch data packets flowing between devices. A data plane physically transports packets between ports. A control plane routes packets based on address or other parameters, determining which switch port the packet is moved between. The packets may be OPA, PCIe, Ethernet, or other protocol packets. An Optical to Electrical to Optical (O-E-O) conversion is required. Many bit times of delay are added. OSI layers 1-3 are required. The terms packet switch and mouse switch are used interchangeably. 
     Packet switches are limited in the bandwidth of a single switch. A switch may contain many pieces of switch fabric within a single chassis. Multiple layers of switches are required to yield a switch plane with a sufficient number of ports. Adding one new layer adds two new hops of latency through the switch plane. A switch plane is a collection of switch layers, for instance: Top Of Rack (TOR), aggregation, leaf, and spine. 
     As used within this paper, east/west data flows are within a layer. One hop latency is added. For example, from one server to another or one leaf switch to another leaf switch. 
     North/south data flows are between layers. Data that goes north returns south again, adding 2 hops latency. For example, from a leaf layer to a spine layer back to a leaf layer. 
     Faster, higher bandwidth networks have historically helped the latency problem. However, this is expensive, and the approach is not extensible to ever growing device planes and the need for higher bandwidth faster networks. 
     Mouse flows are characterized by groups of a small number of packets to a single destination, followed by another small number of packets to a different destination. Packet switches (also called mouse switches) are particularly suited to mouse flows. 
     Elephant flows are characterized by groups of a large number of packets to a single destination. Circuit switches (also called elephant switches) are particularly suited to elephant flows. 
     Circuit switches move data from port to port, without inspecting the packet. 
     Transparent optical circuit switches are Optical to Optical (O-O), with no conversion to electrical. The packet cannot be read or modified by the switch. The data plane and control planes are independent. The control plane provides software definition of the switch plane. Switch plane latency is only speed of light wire propagation delay. 
     A bare metal cloud is an environment where selected hardware is dedicated to a customer. Layer 4-7 may be elastic within the cloud, but layer 1 is frozen. 
     Stranded resources are available online devices which are not used due to unacceptably high network latency to other devices they may be clustered with. Connections between different racks or different pods have increased latency as the data must be passed through more layers of packet switches to reach the intended destination. 
     One existing low-radix transparent optical circuit switch has 320 ports and 3.5 dB insertion loss. Another has 384 ports and 2.7 dB insertion loss. Low cost QSFP/OSFP optics modules may have a 5 dB link budget, thus only one pass through a switch is possible. Use is limited to between TOR and leaf layers, a virtual pod. 
     U.S. Pat. No. 9,210,487 “Implementation Of A Large-Scale Non-Blocking Optical Circuit Switch” stitches together multiple low-radix fabrics into a single box. Insertion loss is reduced by only using select lower loss ports and by using end to end power monitoring for switch alignment. This complicates switch alignment, compounding 6 orthogonal control variables into a single measurement. A 6 dB insertion loss requires expensive high power optics modules. 
     SUMMARY 
     High port-count low insertion-loss non-blocking transparent optical circuit switches enable an all optical circuit switch plane. Multiple interconnected optical circuit switches form an optical circuit switch layer. One or more switch layers form a switch plane. A low insertion loss switch is required to interconnect multiple switches. The circuit switch plane may have 100k, 1M, or more ports. 
     A circuit switch plane is added between the device plane and packet switch plane, adding a new dimension of flow, up/down. Up/down data flows are between circuit switch and packet switch planes. Whereas, north/south flows are between layers and east/west flows are within a layer. Circuits may be configured by the orchestrator, OS, or application. This new architecture enables previously unattainable performance. 
     Speed of light latency circuits may be created between any device and any other device in any rack or pod, or to any port in any packet switch layer. The data center is temporally shrunk, as devices may be accessed without encountering a packet switch. Application performance may be improved. 
     Elephant flows need not traverse any mouse switches. A circuit may be created for elephant flows, and destroyed when the flow is complete. Only mice need flow through mouse switches. This reduces the amount of traffic in the packet plane, allows the packet network to be oversubscribed, reduces the number of packet switch ports, and significantly reduces packet switch spend and energy consumption. Omni Path Architecture (OPA) may be a preferred protocol. 
     Remote devices may be direct attached, localizing devices across racks and pods. Servers may be clustered across pods. Network drives may have nearly the same performance as local drives. GPU may be clustered or attached to a server. FPGA, ASIC, or other specialized hardware can be purchased only as needed, and shared with many different customers in many different pods. 
     Al HPC may share neural network FPGA or ASIC resources across the datacenter. Neural network ASICs may be reconnected for wider or for deeper configurations. GPU may be connected for image processing functions. Memory may be connected as storage. Devices may be connected with non-standard protocols. 
     The topology of the network may conform to the data/application. Point to point, ring, mesh, tree, fat tree, star, CLOS, leaf/spine, or hybrid topologies are a matter of optimization and software definition. 
     HPCs may be configured for optimal network performance. Unused HPC resources may be put in a physically partitioned cloud, and time sold to merchant customers. 
     Bare metal may be bent to meet merchant customer requirements. 
     Network Function Virtualization (NFV) reduces specialized switch hardware. Ethernet may be eliminated from inside the compute plane. Ethernet may remain outside of the compute plane, as a WAN interface and to connect to operations and maintenance equipment. An Ethernet NIC may be the only hardware needed. 
     Hadoop requests may move the network, not the data. Storage devices may have multiple network ports and may have advanced features, such as ability to respond to Hadoop queries. A port on the storage device may be connected to each server, the storage device appears local, latencies are nearly local, and relatively little data may be actually moved. If data must actually be moved, data may be replicated from one storage device to another storage device over a dedicated circuit, without burdening the packet switches or servers. Also, terabyte scale ram memory may have Hadoop features, and data replicated from storage to memory. 
     Stranded resources are recovered. Orchestration may be more efficient. Clustered devices need not be located in the same physical rack or same physical pod to meet latency requirements. A Top Of Cluster (TOC) packet switch may serve the entire cluster. 
     Packet switch congestion may be avoided. Additional ports may be dynamically connected where they are needed. 
     The Top of Rack (TOR) packet switch layer may be eliminated and replaced with a Top Of Pod (TOP) switch. 
     Backup may not burden the packet network. Backup applications may request connections and request file transfer between intelligent devices. 
     Devices may be optically connected with a secure tunnel, eliminating the need for crypto. The connection may be compatible with quantum cryptography. Software may define physically isolated networks. 
     The circuit switch plane is transparent and agnostic. Speeds, lambdas, and protocols may be mixed anywhere within the switch, given the devices at each end of the circuit have compatible optics modules and protocols. FPGA, ASIC, and GPU need not use a standard protocol. 
     The network may be self-healing. Fiber and port failures may be diagnosed and traffic rerouted. Connectivity may be verified, eliminating the impact of human wiring errors. 
     Technology refresh may be asynchronous. The orchestrator may select the preferred resource. 
     Technology adoption may be sped. Limited availability, evaluation, or costly hardware may be accessed from anywhere in the datacenter. 
     Additional applications exist in telecommunications, test, hardware validation, and software validation. The circuit switch fabric is resilient to cosmic rays, whereas packet switch fabric may experience induced errors. 
    
    
     
       DESCRIPTION OF DRAWINGS 
         FIG. 1  is a diagram of a packet switched network over a circuit switched network. 
         FIG. 2  is a diagram of a packet switched network over a circuit switched network with A/B redundancy and fixed packet routes. 
         FIG. 3  is a diagram of a packet switched network over a circuit switched network with 3 layers. 
         FIG. 4 a    illustrates a GPU cluster with FPGA and ASIC assist and backup. 
         FIG. 4 b    illustrates a server cluster and Hadoop replication. 
         FIG. 4 c    illustrates packet congestion and latency reduction. 
         FIG. 4 d    illustrates NFV. 
         FIG. 4 e    illustrates a composable server. 
         FIG. 4 f    illustrates an Al HPC cloud. 
         FIG. 5  is a method to orchestrate a circuit switched network. 
     
    
    
     DETAILED DESCRIPTION 
       FIG. 1  is a diagram of a packet switched fiber optic network over a circuit switched fiber optic network  100 . A circuit switch plane  120  is added to the datacenter network between the device plane  110  and packet switch plane  130 . Circuit switch plane  120  has a circuit leaf layer  122  with one or more leaf circuit switches  123 , and packet switch plane  130  has a packet leaf layer  132  with one or more leaf packet switches  133 . Circuit switch plane  120  may have a circuit spine layer  124  with one or more spine circuit switches  125 , and packet switch plane  130  may have a packet spine layer  134  with one or more spine packet switches  135 . A WAN  140  connection may exist. 
     Devices  112  may be housed in racks  114 , and racks  114  organized into pods  116 . Pods  116  may be containers. 
     Devices  112  have a network port. Ports may be Omni Path Architecture (OPA) with PSM4 optics. PSM ports may be switched independently, in 25 GBPS increments from a 100 GBPS module. Other protocols may be used, such as Ethernet, Infiniband, FibreChannel, or other protocol over fiber, or any mix of the above. Other optics may by used, such as CWDM4, coarse WDM, dense WDM, BX, LX, SX, or other optics module, or any mix of the above. Each end of a link must have compatible protocol and optics. 
     Switches may have a number of ports for east/west connectivity to other switches in the same layer and a number of ports for north/south connectivity to higher or lower layers. Higher layers may span multiple physical locations. In the context used within this paper, east/west does not mean within a datacenter, and north/south outside of the datacenter. A novel up/down routing dimension may be available between circuit plane  120  and packet plane  130 . This is a mouse network over an elephant network. East/west is illustrated on the page as left/right, north/south as up/down, and up/down with squiggly lines. East/west and north/south flows may exist on both the circuit and packet planes. A single circuit connection may have multiple east/west and/or north/south components. 
     A packet layer may have the same or different topology as a circuit layer. Elephant traffic need not traverse a mouse switch, saving many mouse switch ports. The up/down direction may be oversubscribed by 7:1, 3:1, 2:1, or other circuit:packet ratio. The oversubscription ratio may have local variations. The number of packet switches over a circuit switch may vary. Some circuit switches may not have a directly corresponding packet switch. Connections to packet switches elsewhere in the network may be made. 
     High port-count low insertion-loss non-blocking transparent optical circuit switches are required for large switch planes. An example of such a switch is described in copending Ser. No. 16/041,815 “Optical Switch”, and is incorporated by reference. 2 or 3 switch layers may have an insertion loss of &lt;=3 dB, or &lt;=5 dB. Switches may have 500, 1000, 2000, 4000, or more ports. Planes may have 20K, 50K, 200K, 1M, or more ports, with an insertion loss of &lt;=3, 4, or 5 dB. Insertion losses exclude interconnect fiber. Existing low port-count switches have insufficient connectivity to form a large plane. Too many ports are used for up/down, east/west, north/south traffic compared to the number of device ports. Existing high insertion-loss switches limit hops through multiple optical switches. 
       FIG. 2  is a diagram of a packet switched network over a circuit switched network with A/B redundancy and fixed packet routes  200 . A/B redundant network  200  may be similar to network  100 , with the addition of one or more layers have A and B sub-layers. A/B/C or A/B/C/D sub-layers are also possible. Not all layers may require the same level of redundancy. For instance, the leaf circuit layer may only have an A sub-layer and the spine circuit layer may have A and B sub-layers. Or, a sub-layer may be present in only a portion of a layer. For devices with a single PSM port, a cable may split fibers between sub-layers. 
     Connections between A/B sub-layer add another routing dimension. This allows traffic to load balance between A/B sub-layers, and not be distinct redundant networks until necessary. This may be advantageous in relieving congestion. If more than one circuit switch is required in a pod, they may be partitioned horizontally as A/B layers, instead of vertically with each switch serving different racks. 
     Dashed lines in circuit switch  223 A indicate fixed (not switched) connections between a device and a packet switch. This may be desired if all nodes have some amount of packet switched traffic. Conversely, if most nodes have point to point traffic, such as GPU clusters, HPC, direct connection, or storage replication, fixed connections to a packet switch may waste ports. Fixed connections may also be used with non-redundant network  100 . This split of fixed ports from a PSM4 fiber bundle may be done inside of the switch, but without connecting to the switch fabric, to simplify datacenter cabling. 
       FIG. 3  is a diagram of a packet switched network over a circuit switched network with 3 layers  300 . Three layer network  300  may be similar to network  100 , but with an additional super spine layer  326  and  336 . This allows a high port-count low insertion-loss switch of 1024 ports and 0.4 dB insertion loss, with 5 dB link budget optics, to realize a 268M port network. Or, a 512 port 0.4 db insertion loss switch to form a  33 M port network. Compared to the 2 layer network  100 , a 524K port and 131K port network may be realized, respectively. Preferably, the insertion loss of the circuit plane is &lt;=3 dB, but may be &lt;=3.5, 4, or 5 dB. 
       FIG. 4 a - e    illustrates some possible configurations of network  100 . Highlighted routes illustrate connections used in the example application. Preferably devices  112  may forward packets, for low bandwidth requirements this eliminates the need for a connection to a packet switch. 
       FIG. 4 a    illustrates a GPU cluster with FPGA and ASIC assist. Also, backup is illustrated. As an example, GPU  112   k  may be fully meshed, memory  112   e  may be in a star topology with GPU  112   k  and server  112   a , FPGA  112   l  may assist GPU  112   k . GPU  112   k  and FPGA  112   l  need not use a standard packet format. GPU  112   k  may be added in rack scale increments. Specially configured racks with a mix of GPU  112   k  and server  112   a  are no longer necessary. This configuration may be useful for rendering and compressing video and allowing asymmetric hardware refresh. 
     Backup network load need not burden a packet switch, or other network traffic. Circuit switched plane  120  may connect SSD  112   f  and cold store  112   j . SSD  112   f  and cold store  112   j  are preferably able to stream files, with server  112   a  (not shown) running the backup application only needing to generate transfer requests and confirm completion. Optionally, some devices, such as cold store  112   j , may have a low percentage of use within its pod and may be hard connected directly to the spine layer. Files may also be moved between store  112   c , SSD  112   f , JBOD, NVM, hard disk, hot storage  112   h , warm storage  112   i , cold storage  112   j , backup, or other device. 
       FIG. 4 b    illustrates a server cluster and Hadoop replication. Servers  112   a  may be clustered in a topology selected to best meet data or process load. Servers  112   a  may be located in different pods. The route cost of the north/south and east/west connections to the server in the remote pod are nearly identical. Server  112   a  may connect directly to a spine packet switch  135 , avoiding the latency of leaf packet switch  133 . Server  112   b  may be used interchangeably with server  112   a  when not used as a WAN interface. 
     Storage  112   c  may connect to multiple servers  112   a  in various pods  116 . Storage  112   c  may be intelligent storage, capable of processing Hadoop requests. Hadoop replication may be a logical function, moving the network and not moving the data. Additionally, storage  112   c  may connect to other storage  112   c , allowing replication between devices without burdening servers  112   a  or the packet network  130 . This may be necessary due to limited ports on storage  112   c . Alternatively, a packet switch  133  may be used for aggregation. 
       FIG. 4 c    illustrates packet congestion and latency reduction. Multiple ports of server  112   a  may connect to one or more switches  133  for added bandwidth. Leaf switches  132  may connect to other leaf switches  132 , instead of a spine switch  134 , adding additional hybridized layers of switching. Or, additional switch ports may be connected for added bandwidth. 
     Packet switch  133  may be a Top Of Cluster (TOC) switch. All nodes of the cluster connect to the same switch, regardless of which rack of pod they may be in. Any node within the cluster may reach any other node within the cluster with a one hop latency. 
       FIG. 4 d    illustrates Network Function Virtualization (NFV). Ethernet may be nearly eliminated from the network. An ethernet NIC and appropriate WAN optics may be added to server  112   b . No other modifications are required, as the circuit switch plane  120  is wavelength and protocol agnostic. Ethernet WAN  140  may connect to the circuit switch plane  120  for server redundancy. Server  112   b  may take on routing functions and connect to spine switch  136  or leaf switch  133 . Additional packet ports may be used as necessary from switches in other pods. Some network functions are more efficiently implemented in FPGA  1121  or ASIC  112   m  than in server  112   b  software. FPGA and servers may be used for switching functions during peak hours, and released to other customers off peak. 
       FIG. 4 e    illustrates a composable server. A server may be disaggregated into building blocks, CPU  112   d , memory  112   e , storage  112   f , and GPU  112   g . Memory  112   e  may have a mix of SRAM, DDR, flash, or other memory types and a memory manager with a network port. These components may be composed to create servers better sized for a particular task. 
       FIG. 4 f    illustrates an Al HPC cloud. An Al application may require a neural network ASIC  112   m . ASIC  112   m  may be connected to memory  112   e  to record neural evolution. FPGA  1121  may assist ASIC  112   m . Server cluster  112   a  may reside in a different pod. Given network latency constraints and using only an existing packet network, ASIC  112   m  may be too costly to deploy throughout the datacenter and reserving server  112   a  time within a pod may also be too costly. 
     Other functionality (not illustrated) may include diagnostic  112   o  with TDR, power meter, or other diagnostic hardware which may be scripted to validate switch ports, fiber, and fiber connectivity. Human errors in fiber connectivity might be corrected by updating port mapping, instead of moving fiber to correct the wiring error. 
       FIG. 5  is a method to compose and orchestrate a circuit switched network  500 . 
     Resource requests may be accepted in step  510 . The request may contain: customer contract requirements; direct connect requests; hardware allocation request; drive mount/unmount requests; open/close stream request; Hadoop requests; packet switch traffic statistics; packet switch latency; composition script request; application request; and/or other sources. Certain OS functions may be modified to simplify request generation. 
     Direct connect, drive mount/unmount, and open/close stream requests may configure a circuit switched route between source and destination devices. 
     Hadoop replication requests may configure circuit switched routes between multiple servers and a single storage device, moving the network instead of moving the data. 
     Latency within a packet switch is dependent on source and destination ports. Use of QOS or packet traffic statistics may be used to configure the circuit switched network to reorder packet switch ports. 
     Backup applications may configure various routes between source and destination. 
     FEA, HPC, or Al HPC applications may cluster servers into a network topology that best matches the data topology. 
     Clustered devices may have a packet switch port. All packet switch ports for the cluster may connect to a single TOC packet switch. 
     A composer may provide a requested resources and connection topology. 
     Lowest cost of available resource allocations may be determined in step  520 . Cost of resources required to meet the resource request may be calculated using: number and type of ports and devices consumed; packet switch port to port latency; wire latency; minimum quantity of unallocated resources; billable cost; and/or other factors. If a sufficiently low cost route is not available, existing routes may be moved to a higher cost route that still within acceptable cost. 
     Verify proposed resource allocation in step  530 . If the allocation fails to meet: latency requirements; physical isolation requirements; and/or other factors, the request may be rejected. 
     Network and devices are configured in step  540 . 
     The orchestration request returns in step  550 . 
     The previous examples are intended to be illustrative. Countless additional variations and applications are readily envisioned. Planes and layers may be partially implemented. Resource orchestration will vary.