# Optical Convolutional Neural Networks: Methodology and Advances (Invited)
## 2 Development of Optical Convolution Neural Network
### Optical CNN Based on Optical Interference  
Benefitting from the development of photonic integration technology, matrix multiplication has been extensively focused on light interference [82; 83; 84]. Since the convolution operation can be mathematically converted to matrix multiplication, it has become a very popular scheme to realize optical CNNs by exploiting the interference properties of light [85; 86; 87; 88; 89; 90; 91]. As the basic unit to build a coherent unitary matrix for discrete convolution operation, the structure diagram of the MZI is shown in Figure 4a, where the splitting ratio of two optical couplers is fixed at 50:50, the internal phase shifter provides an additional phase \(\theta(0\leq\theta\leq\pi)\) to control the splitting ratio of the MZI, and the external phase shifter adds a relative phase \(\varphi(0\leq\varphi\leq 2\pi)\) to two output arms. Ignoring the optical propagation loss, the transfer matrix \(T(\theta,\varphi)\) for a single MZI unit is written as  
\[\begin{array}{ll}T(\theta,\varphi)&=P_{\varphi}CP_{\theta}C\\ &=\frac{1}{2}\begin{bmatrix}e^{i\varphi}&0\\ 0&1\end{  
where \(P_{\varphi}\) is the transfer matrix of the external phase shifter, \(C\) is the splitting ratio of the optical coupler, \(P_{\theta}\) represents the internal phase shifter, and \(t\) is a simplified representation of the elements in the matrix. An arbitrary unitary matrix was formed by constructing an MZI-based special mesh network. Two mainstream unitary matrix mesh networks (one was rectangular architecture [82] and the other was triangular architecture [83]) are shown in Figure 4b,c. In both architectures, \(N(N-1)/2\) MZIs were needed to make up an \(N\)-dimensional unitary matrix mesh. The transfer matrix of a single MZI unit in the mesh network connecting channels \(m\) and \(n(m=n-1)\) can be written as  
\[T_{m,n}^{(k)}(\theta^{(k)},\varphi^{(k)})= \begin{bmatrix}1&&&&\\ &1&&&&\\ &&\ddots&&&\\ &&&t_{11}&t_{12}&&\\ &&&t_{21}&t_{22}&&\\ &&&&\ddots&&\\ &&&&&&1\\ &&&&&&&1\\ \end{bmatrix} , \tag{6}\]  
where \(k\) is the serial number of the MZI. The transfer matrix of the entire mesh \(T\) can be decomposed into  
\[T=\prod_{(m,n)\in S}T_{m,n}^{(k)}(\theta^{(k)},\varphi^{(k)}), \tag{7}\]  
where \(S\) defines the specific ordered sequence. For the triangular unitary MZI mesh shown in Figure 4c, according to Equation (6), the transfer matrix of four rows of MZIs can be written as  
\[T_{2,1}^{k=1,3,6,10}= \begin{bmatrix}t_{11}&t_{12}&&\\ t_{21}&t_{22}&&\\ &&1&&\\ &&&1&\\ &&&&1&\\ \end{bmatrix} ,T_{3,2}^{k=2,5,9}= \begin{bmatrix}1&&\\ &t_{11}&t_{12}&&\\ &t_{21}&t_{22}&&\\ &&1&\\ &&&1&\\ \end{bmatrix} , \tag{8}\] \[T_{4,3}^{k=4,8}= \begin{bmatrix}1&&&&\\ &1&&\\ &&t_{11}&t_{12}&\\ &&t_{21}&t_{22}&\\ \end{bmatrix} ,T_{5A}^{k=7}= \begin{bmatrix}1&&&&\\ &1&&\\ &1&&\\ &&t_{11}&t_{12}&\\ &&t_{21}&t_{22}&\\ \end{bmatrix} .\]  
Figure 4: Structure of a single MZI unit and two classic unitary transform MZI mesh architectures. (**a**) The structure diagram of the MZI with two phase shifters. (**b**) Rectangular structure \(5\times 5\) unitary transformation MZI mesh. (**c**) Triangular structure \(5\times 5\) unitary transformation MZI mesh. Sequentially, according to Equation (7), the entire transfer matrix of the unitary MZI mesh can be inferred as  
\[T=T_{2,1}^{10}T_{3,2}^{9}T_{4,3}^{8}T_{5,4}^{7}T_{2,1}^{6}T_{3,2}^{5}T_{4,3}^{4} T_{2,1}^{3}T_{3,2}^{2}T_{2,1}^{1}. \tag{9}\]  
Since the MZI mesh can represent any unitary matrix, it is worth using the MZI mesh to represent any real-valued matrix through singular value decomposition (SVD) [92]. Any real-valued matrix \(M\) with dimensions of \(p\times q\) can be decomposed by SVD as  
\[M=U\Sigma V^{\dagger}, \tag{10}\]  
where \(U\) is a \(p\times p\) unitary matrix, \(\Sigma\) is a \(p\times q\) rectangular diagonal matrix with non-negative real numbers on the diagonal, and \(V^{\dagger}\) is the complex conjugate of the \(q\times q\) unitary matrix \(V\). From Equation (7), unitary matrices \(U\) and \(V^{\dagger}\) can be implemented with the MZI mesh.