# Inverse Design of Distributed Bragg Reflectors Using Deep Learning
## 2 Materials and Methods
### Simulation Neural Network  
A fully connected neural network (FCNN) with four hidden layers was implemented, where network inputs were processed through a batch normalisation layer to improve convergence, and eliminate the need for further standardisation [21]. The input and output layers hold 6 and 81 neurons, which represent the 6 layer thicknesses and the discretised reflectance spectra, respectively. The mean squared error (MSE) loss function was chosen to suit the regression task, and the Adam optimiser was chosen for combined convergence speed and quality [22]. Finally, a learning rate of 0.001 was used. All layers initially used the ReLU activation function, except for the final layer, which used a sigmoid function to bound the predictions to between 0 and 1.  
Further refinements looked at adding model complexity through altering layer sizes and adding dropout. Table 2 summarises some of the key metrics produced by different layer architectures. It was found that optimal performance was achieved by increasing only the final layer to 1000 neurons, but that additional neurons saw no significant improvements.  
Following these trials, a structure of 6-500-500-500-1000-81 neurons was chosen, with 2% and 1% dropout in hidden layers 1 and 2, respectively, as this presented the best trade-off between predictive accuracy and overfitting.  
Dropout was not added to the later layers, as, this late in the network, there is less opportunity for the model to correct the potential errors that it could introduce.  
To help optimise the convergence, the model was updated to take an adaptive learning rate. This makes use of the accuracy of a small learning rate where needed, whilst reducing the chance of getting trapped in local minima at an early stage [21]. It works by reducing the learning rate if the validation loss plateaus for more than a specified number of epochs (known as the patience) [22]. After some experimentation, this was chosen as 6, and the learning rate decreased logarithmically each time it updated until reaching \(1\times 10^{-6}\), below which the network training became unfavourably slow.  
Lastly, to try and reduce the final training errors, the ReLU activation functions were swapped for leaky ReLU functions. The difference between these is that the leaky variety has a user-defined non-zero gradient for negative values. This helps to solve the 'dying neuron' problem, as, now, if certain weights enter the negative region, their training will not completely stop [23]. The hyperparameter \(\alpha\) specifies how much the function 'leaks'--in other words, the gradient of the function for negative values of x. Here, \(\alpha\) has been chosen as 0.1.  
A summary of the current network architecture is included in Table 3.  
The model was trained on the first half of the dataset mentioned in Section 2.1, where the data were first shuffled and then split into training, validation, and test data at roughly a 6:1:0.8 split.  
Following this, the model was saved so that it could be imported into a new file later, with its parameters already determined.