## 2 Artificial Neural Network Flow Law
### Artificial Neural Network Equations
#### 2.1.4 Derivatives of the Neural Network
As expected, the ReLU activation function gives the worst results with a final value of \(\text{E}_{\text{MS}}=32\times 10^{-6}\), mainly due to the low number of neurons and the fact that this function is a piecewise linear function and not able to efficiently approximate the nonlinear behavior of the material. The other five activation functions enhance more or less the same behavior. The final value of the E\({}_{\text{MS}}\) is pretty much the same for all of them, and around \(\text{E}_{\text{MS}}=12\times 10^{-6}\). Table 2 reports the results of the training phase, the errors reported in this Table are computed using the whole dataset (both train and test parts). Figure 4: Global convergence of the six ANN models. From the data, it is evident that all models require approximately the same training time to complete the specified number of iterations, with the complexity of activation functions influencing this duration; notably, the training time is a bit greater for Swish and Softplus functions compared to ReLU. We also reported in this table the real values of the \(\mathrm{E_{RMS}}\) and \(\mathrm{E_{MAR}}\) concerning the flow stress \(\sigma\) using the whole experimental database. From the latter, we can see that the \(\mathrm{E_{RMS}}\) is about 0.6 MPa for all activation functions except the ReLU one, where the value is above 0.8 MPa. Concerning the \(\mathrm{E_{MAR}}\), the value of all models is around 1%, while it is more than 2% for the ReLU function. Of the six activation functions, the Exponential function gives the best results in terms of solution quality, while the ReLU function gives the worst, as reported by computing the global error using the following expression \(\Delta E=\sqrt{\mathrm{E_{RMS}^{2}+E_{MAR}^{2}}}\). But we must note that the results of all models, except the ReLU one are very close at the end of the training stage, as illustrated in Figure 4 and Table 2. Depending on when the train is stopped, a particular model may yield the best performance due to the varying slopes of convergence among the models. Figure 5 reports a comparison of the experimental stress acquired during the Gleeble compression tests (reported as dots in Figure 5) and the predicted stress \(\sigma\) using the ANN for the strain rate \(\dot{\epsilon}=1\) s\({}^{-1}\). From this observation, we can infer that all ANNs effectively replicate the experimental results, except for the ReLU activation function. In the case of ReLU, as depicted in Figure 5, the predicted flow stress exhibits a piecewise linear behavior. Of the six activation functions introduced, as detailed in Section 2.1.2, the exponential function stands out due to its unique features. The computation of the function and its derivative in a single step necessitates only one evaluation of the exponential function, as indicated by \(f^{\prime}(x)=f(x)\) in Equation (10). If we analyze the results reported in Table 2 and Figure 5 concerning the exponential activation function, we can see that this one has a \(\mathrm{E_{RMS}}=0.688\) MPa, \(\mathrm{E_{MAR}}=1.176\%\) and the global behavior of the flow stress for \(\dot{\epsilon}=1\) s\({}^{-1}\) is similar to the Sigmoid, Tanh, Swish or Softplus functions. In terms of the global performance of the 3-15-7-1-exp ANN, Figure 6 reports the comparison of the experimental data (dots) and the ANN flow stress for all strain rates and temperatures defined in Section 1.2. We can see that over the whole temperatures \(T\) and strain rates \(\dot{\epsilon}\), the performance of the model based on an exponential function is very good overall. This model will therefore be retained for the remainder of the comparative study. It is well known that artificial neural networks are able to interpolate data correctly within their learning domain, but behave unsatisfactorily when we wish to evaluate results outside the boundaries of this learning domain. The ANNs developed in this study follow this general rule, but with different degrees of progress depending on the nature of the activation function used.