# Evaluating the Impact of Optical Interconnects on a Multi-Chip Machine-Learning Architecture
## 1 Introduction  
Advances in algorithms, increases in computing power, and collections of huge datasets enable deep learning to outperform alternatives in numerous application areas. In particular, Convolutional Neural Networks (CNNs) provide superior performance in image recognition compared to conventional solutions. Hence, it has rapidly gained popularity, leading to the invention of various CNN models [1; 2; 3; 4; 5; 6]. Because CNNs demand huge computing power and can be massively parallelized, many studies on computing systems have utilized _accelerators_ designed especially for CNNs as they can improve performance by maximizing parallelism in computing and energy-efficiency only with essential components [7]. Meanwhile, as CNNs become deeper (more layers being populated), more storage capacity and computing capability are required for accelerators. Therefore, scalability (flexibility in size) is a critical factor for machine-learning computing systems. Many of the proposed accelerator architectures are used to build a _cluster_ where multiple accelerators are interconnected through a network [7; 8; 9]. However, they do not focus enough on improving the performance of communication through a network, but rather focus on increasing parallelism and/or reducing memory access in an accelerator. In this paper, we first examined how much inter-accelerator communication affects the performance of three CNN models for image recognition: AlexNet [1], VGG [2], and ResNet [3]. We estimated _training_ time, which is as critical as _inference_ time for intelligent decision makers and information providers to enhance their prediction models. We modeled DaDianNao [7], a custommulti-chip machine-learning architecture designed for CNN, and estimated its execution time (training) with our arithmetic performance model described in Section 4.2. When allocating a CNN model and input/intermediate data to each accelerator to reduce inter-chip communication, we applied _model parallelism_[10] to AlexNet as well as _data parallelism_ to VGG and ResNet. Consequently, for various _batch sizes_ (the numbers of input images concurrently propagating through CNN), the portion of communication time in total training time is dominant for VGG and ResNet. In addition, it is also quite considerable on AlexNet. A direct way to enhance the communication performance is by replacing existing electrical interconnects with optical ones that provide higher bandwidth. This is reasonable because photonics technology has been dramatically improved in terms of bandwidth, optical module size, and on-package integration. For example, 100 G (gigabit) Ethernet is one of the mainstream for datacenter-scale networks [11] while 400 G Ethernet is expected to enter the market in a couple of years [11]. Moreover, silicon optical interposers have been available [12; 13; 14], which can integrate multiple chips, optical components, and waveguides implemented on the interposer in a single package by 2.5D integration. In this paper, we show the advantages of optical interconnects for multi-chip architecture targeting machine learning by demonstrating performance improvement through replacing electrical interconnects (25.6 Gbps) with optical ones (100 Gbps or 400 Gbps) in an existing multi-chip system. To show the effects of optical interconnects, we assume highly practical implementation. Considering the feasibility with current technologies, DaDianNao [7] is selected as the baseline architecture of the machine-learning accelerator cluster where 100 Gbps optical interconnects are applied, which can be implemented both on-chip and off-chip by current photonics technology [11; 12; 13; 14; 15]. Because directly interconnecting small DaDianNao accelerators (about 8 \(\times\) 8 mm\({}^{2}\)) with optical cables is too costly, we propose and assume to integrate several accelerator chips in a package by 2.5D integration with a silicon optical interposer and to interconnect multiple packages with off-chip (off-package) fibers [14] at a reasonable cost.