## 3 Deep Neural Networks Assisted Nanophotonics Design for Silicon Platform
### Training Generative Deep Neural Networks as Inverse Models
#### 3.2.3 Unsupervised Generative Neural Network  
In addition to learning from the data samples, DNN can also work as a trainable function and merge with EM-simulator based optimization algorithms. In this way, training data samples are not required.  
We take reference [100] as an example to illustrate the process of building and training an unsupervised neural network as an inverse model to generate irregular metagratings. The irregular geometries of metagrating are also produced by a well-trained generator for the given optical response as well as random noise. As shown in Figure 10, the device geometry \(\mathbf{G}\) is the function of diffraction wavelength \(\lambda\), angle \(\theta\) and random noise \(\mathbf{n}\) as shown in Equation (21):  
\[\mathbf{G}=f_{1}(\lambda,\theta,\mathbf{n};\mathbf{W}), \tag{21}\]  
where \(f_{1}(\cdot)\) is the function described by the generator with trainable parameters matrix \(\mathbf{W}\). With known metagrating geometry \(\mathbf{G}\), the optical response \(\mathbf{E_{G}}\) can be calculated by forward EM simulation as in Equation (22):  
\[\mathbf{E_{G}}=f_{2}(\mathbf{G}), \tag{22}\]  
where \(f_{2}(\cdot)\) is the non-analytical function computed by EM simulation. The loss function is an analytical function defined by designer as in Equation (23):  
\[L=f_{3}(\mathbf{E_{T}},\ \mathbf{E_{G}}). \tag{23}\]  
The aim of \(f_{3}(\cdot)\) is to diminish the difference between target optical response \(\mathbf{E_{T}}\) and generated optical response \(\mathbf{E_{G}}\). To train this deep generator, back-propagation is applied as in Equation (24):  
\[\frac{\partial L}{\partial\mathbf{W}}=\frac{\partial L}{\partial\mathbf{E_{G} }}.\frac{\partial\mathbf{E_{G}}}{\partial\mathbf{G}}.\frac{\partial\mathbf{G }}{\partial\mathbf{W}}=\frac{\partial f_{3}}{\partial\mathbf{E_{G}}}.\frac{ \partial f_{2}}{\partial\mathbf{G}}.\frac{\partial f_{1}}{\partial\mathbf{W}}. \tag{24}\]  
The first term \(\frac{\partial f_{3}}{\partial\mathbf{E_{G}}}\) is an explicit function, which is easy to calculate analytically. The second term \(\frac{\partial f_{2}}{\partial\mathbf{G}}\), which is an implicit function, can be solved through adjoint method with an inverse simulation as derived in Section 2. The last term \(\frac{\partial f_{1}}{\partial\mathbf{W}}\), is the loss function of deep generator which can be trained via back-propagation similar to other DNNs.  
In this way, the generator is no longer trained by prepared data samples but with the assistance of EM simulation in a chain rule. After the training process, the generator is  
Figure 10: Merging DNN to optimization algorithms.  
able to produce multiple metagratings with different geometric parameter sets for a target optical response.  
Reinforcement learning which merges DNN with EM simulator is another approach without the need of training data samples. DNNs based on reinforcement learning generate new data features which are evaluated by EM simulator and fed back for the training of DNNs. [PERSON] et al. utilized reinforcement learning for silicon-based color printing design [101]. Different from examples discussed above which make use of gradient of EM simulator, reinforcement learning makes decisions on the rewards of short-term and long-term gains.