# A Neural Network-Inspired Matrix Formulation of Chemical Kinetics for Acceleration on GPUs
## Appendix A Substitution with Approximate Artificial Neural Networks
The only restrictions are that the input features are functions of temperature and the output dimensionality is \(N_{R}\) (assuming without loss of generality that all reactions in the mechanism are reversible). To highlight key points, two examples of approximate ANN architectures are shown in Figure 11. Architecture 1 uses the same input features as the exact ANN but allows for variation \(N_{L}\) and \(N_{H}\) (referred to as the modified Gibbs neurons). Architecture 2 is similar but only utilizes one input feature, namely \(\log(T)\). Nonlinearity is imposed in both architectures through the activation functions \(\sigma_{l}\). In Architecture 1, since several functions of temperature are already included in the input, a simple rectified linear unit (relu) activation function can be used:  
\[\forall x\in\mathbb{R},\quad\sigma_{l}(x)=\text{relu}(x)=\max(0,x). \tag{2}\]On the other hand, since Architecture 2 utilizes only \(\log(T)\) in the input layer, the more expensive exponential linear unit (elu) activation can be used [37] to allow the model to extract dependence on powers of \(T\) as needed during the training process:  
\[\forall x\in\mathbb{R},\quad\sigma_{l}(x)=\text{elu}(x)= \begin{cases}x&\text{if $x \geq 0$,}\\ e^{x}-1&\text{if $x<0$.}\end{cases}  \tag{10}\]  
In the above scenario, it is reasonable to expect that the computational advantage offered by the smaller input size of Architecture 2 is offset by the more expensive activation function. In light of this, Architecture 2 can be modified to use the relu activation, although this lessens the expressive power of the ANN. For example, with only \(\log(T)\) as the input, an ANN using Equation (11) throughout cannot easily represent a function composed of polynomials of \(T\); its architecture must be modified to at least include \(T\) itself as an additional input (akin to Architecture 1). Moreover, Architecture 2 can be modified to include \(1/T\) as an additional input, and Architecture 1 can be modified to remove the higher order temperature terms (\(T^{2}\), \(T^{3}\), and \(T^{4}\)). In the tests conducted by the authors, such modifications were found not to significantly affect the end results--all variations were sufficiently accurate when trained until convergence using a mean-squared error loss function on \(\log(\mathbf{K}_{c})\). The key takeaways are that the formulation of the approximate ANN for the equilibrium constants involves reductions and re-castings of the exact form in a few major aspects: (1) the re-interpretation of the exact Gibbs neurons (Figure 10, left) into the modified Gibbs neurons (Figure 10, middle and right); (2) the fact that the approximate ANN evaluation is not treated in distinct temperature partitions (in the exact setup of Equation (9), the polynomial coefficients for a particular species, which are interpreted as the weights, are temperature-dependent whereas the approximate ANN weights are not); and (3) the customized form of the input layer and nonlinearity in the approximate ANN. Such reductions and changes are necessary to motivate the approximate ANN from a computational efficiency and simplicity aspect with respect to the exact form. The above concepts were presented in the context of replacing the equilibrium rate constant with an approximate ANN, but the same techniques can be applied for other components such as the forward rate constant. For example, using an approximate ANN to consolidate the standard, falloff, and pressure-log forward rate constant computations into a single matrix multiplication operation could be a promising application of the technique.