# Development of a CFD-Suitable Deep Neural Network Model for Laminar Burning Velocity
## 2 Developed Deep Neural Network Model for Laminar Burning Velocity
\tag{2}\]  
The original model used only ReLU functions [18]; however, the switch to _tanh_ function in the first layer gave accuracy improvement to the new model. Further inclusions of _tanh_ functions did not produce significant further improvement, even though in the literature there is an example of LBV-DNN with all _tanh_ activation functions [11]. Since _tanh_ function takes longer to calculate and did not give a significant advantage when used in deeper layers, we left the _ReLU_ activation function in those layers. Result improvement might have been achieved since the _tanh_ activation function managed to give primary predictions which later were adjusted by layers with ReLU bending curve so that it would fit data better. However, since the number of neurons is not large, it is likely that with small laminar burning velocities (0 m/s or close to it), the model might predict values below zero. To avoid negative predictions, the Equation (2) was also applied to the final output function. For model optimization RMSprop optimizer was used [48]. As studies show, it is one of the best methods for optimization [49]. In addition, the regularization parameters were changed compared to the original model. The first layer had Ridge (L2) regularization parameter equal to 0.000002 and other hidden layers had Lasso (L1) regularization equal to 0.0004. These values were selected after multiple tests with varied values. The effects of the decrease in errors due to L1 and L2 are well documented in [50; 51]. In general, they prevent DNN from overfitting, which makes the network more stable and reliable. To improve the reliability of the new model, MAE and MSE errors were combined and minimized as weighted errors (equal weights were selected after several test cases) for the training of the model as a loss function. Most of the influence was displayed by MAE since velocity was in meters per second and lower than 1 m/s. On the other hand, MSE helped in cases when MAE became almost constant. We believe this approach could give predictions closer to real values, as performed analysis shows that averaging MAE and MSE indeed gives values in between for the data gathered for this research. This can be helpful in cases with areas lacking observations. Also, it is worth noticing that in cases of unique (single) observations, the MAE and MSE will point to the same location. A simple explanation of this could be the idea that while MAE minimizes the model towards median and MSE--towards mean, the combination of them would predict the value in between as visualized in Figure 3.  
To obtain predictions closer to original experimental values, the study did not modify the testing set; however, the training set for improved DNN was adjusted by making some outliers equal to the mean. Outliers were considered values that are different by over 10% from the median value when results from three or more different experimental sources under the same or similar conditions were available. These values were considered outliers since the experimental spread of more than 10% shows higher experimental uncertainties, and given several sources, aberrant points are the most suspect. Around 3-4% of all points were adjusted. Yet, considering that this can detect many data points near zero, those values were ignored. Due to the way in how the model was trained, outliers still could have influenced the final model. However, due to the mean value, this influence is reduced. This allowed the creation of a model which is stable and can perform well on data sets that have outliers. To use this model in the CFD solver, it was exported by saving the weights \(w_{i_{j-1},i_{j},j}\), biases \(b_{ij}\) and performing matrix multiplication with them. Here, \(j\) denotes a layer number, and \(i_{j}\) denotes a weight number in the \(j\)-th layer. \(j\) can obtain values from 1 to \(k\) and \(i_{j}\) can obtain values from 1 to \(n_{j}\).