# Evolution of System Embedded Optical Interconnect in Sub-Top-of-Rack Data Center Systems
###### Abstract
In order to assess the viability of embedding optical links within modern sub-TOR data center architectures, three generations of data center systems were developed over the past 5 years based on current storage switch enclosure form factors. In this paper, we review the evolution of system embedded optical interconnect technologies in three key data center demonstration prototype platforms produced by Seagate Systems in the UK: LightningValley, ThunderValley2 and Pegasus, which were adapted from different data storage switch enclosures to allow selected internal high-speed electronic transmission lines to be converted to optical links. LightningValley is a partially optically interconnected data storage system. It was developed based on a modified 4U24 OneStor enclosure from Seagate Technology, in which 12 Gb/s Serial Attached SCSI (SAS) traffic was conveyed optically between the SAS protocol switches on two internal controller cards along 24 PCB embedded polymer optical waveguide channels, thereby showing, for the first time, how in-system optical channels could be successfully deployed within a 12G SAS architecture [4]. ThunderValley2 is a fully optically enabled data storage array that was developed based on a 2U24 OneStor enclosure from Seagate Technology, in which all internal high-speed links were implemented optically. This required the deployment of commercial midboard optical transceivers, an electro-optical midplane and proprietary pluggable optical connectors for hard disk drives [5]. Pegasus is a fully optically enabled 24 drive Ethernet data storage, switch and compute platform, for which interchangeable optical transceiver mezzanine cards and a proprietary electro-optical drive connector were developed. Aurora is a test and measurement platform developed to allow comparative characterization of different types of more advanced optical interconnect technologies including embedded and discrete polymer and glass waveguide circuits, optical PCB connectors and transceiver and switch technologies including advanced silicon photonics devices [6]. A converged data center test rack, Nexus, was developed to allow different optically enabled platforms to be interconnected and validated with respect to each other. In particular, this allows for more advanced, less mature technologies such as glass waveguide circuits and silicon photonic transceivers to be quickly validated in fully operational, optically enabled data center systems. Table 1 lists the key parameters of the three optically enabled data center system demonstration platforms showing both the communication protocol used in the reported demonstrators and the next generation of communication protocols, for which the need for system-level optical interconnect would be greater. Given the low bandwidth requirements of these sub-TOR systems compared to switch enclosures, there has been very limited effort beyond the research and development described in this paper to address optically enabling these systems, though the work described herein sets out a technology ecosystem, which laid the foundation for CPO. Figure 1 provides a schematic overview of all platforms reported in this paper. The outlook for the next generation of polymer-based optical interconnects and its combination with silicon photonics and nanophotonics is then reviewed. The LightningValley, ThunderValley2, and Pegasus demonstrations indicate how data center infrastructure architects may be able to configure rack systems in order to maximize the data bandwidth through top-of-rack (TOR) switches. The enablers for this are storage systems that utilize a combination of optical interface cards and burst mode transmission of data. If adopted, this strategy would provide another building block in the shift to disaggregated infrastructure: racks consisting of storage alone could be accessed by a remote server while fully utilizing the bandwidth of the TOR switch.