## 2 Artificial Neural Network Flow Law
### Artificial Neural Network Equations
#### 2.1.4 Derivatives of the Neural Network  
As presented in Section 1, in order to implement the ANN as a Fortran routine in Abaqus, we need to compute the three derivatives of \(\sigma\) with respect to \(\varepsilon\), \(\dot{\varepsilon}\) and \(T\). We can compute those derivatives using differentiation of the output with respect to the inputs. As illustrated in Figure 2, we are using here a two hidden layers neural network. Therefore, as Equations (2)-(4) are used to compute \(\overrightarrow{y}_{[k]}\) and \(\overrightarrow{y}_{[k]}\) for each hidden layer and the output \(s\) from the input vector \(\overrightarrow{x}\) of the ANN, we can write the derivative \(\overrightarrow{s}^{\prime}\) of a two hidden layers network as follows:  
\[\overrightarrow{s}^{\prime}=\mathbf{w}_{[1]}^{T}\cdot\Big{[}\Big{(}\mathbf{ w}_{[2]}^{T}\cdot\Big{(}\overrightarrow{w}^{T}\circ f^{\prime}(\overrightarrow{y}_{[2]}) \Big{)}\Big{)}\circ f^{\prime}(\overrightarrow{y}_{[1]})\Big{]}, \tag{13}\]  
where \(f^{\prime}(\square)\) is the activation function's derivative introduced by Equations (5)-(10) and \(\circ\) is the Hadamard product (the so-called element-wise product). Because of the pre and post processing of the values introduced in Section 2.1.3, the derivative of the flow stress \(\sigma\) with respect to the inputs \(\varepsilon\), \(\dot{\varepsilon}\) and \(T\) is then given by:  
\[ \begin{cases}\partial\sigma/\partial\varepsilon=s_{1}^{\prime}\cdot\Delta \sigma/\Delta\varepsilon\\ \partial\sigma/\partial\dot{\varepsilon}=s_{2}^{\prime}\cdot\Delta\sigma/( \dot{\varepsilon}\cdot\Delta\varepsilon)\\ \partial\sigma/\partial T=s_{3}^{\prime}\cdot\Delta\sigma/\Delta T\end{cases} , \tag{14}\]  
where \(s_{i}^{\prime}\) is one of the three components of the vector \(\overrightarrow{s}^{\prime}\) defined by Equation (13). Finally, Equations (2)-(4), (11)-(14) and the requested activation function iterations (6000 iterations, lasting around 1 h), on a Dell PowerEdge R730 server running Ubuntu 22.04 LTS 64-bit, equipped with 96 GB of RAM and 2 Intel Xeon CPU E5-2650 2.20 GHz processors. Concerning the dataset used for the training of the ANN, as already introduced in the previous sections, this dataset is composed of 21,030 quadruplets (\(\epsilon\), \(\dot{\epsilon}\), \(T\) and \(\sigma\)) acquired during the Gleeble experiments described in Section 1.2. A chunk of 75% of the dataset is used for training while the rest is used for the test during the training of the ANN. All details about this procedure can be found in Pantale [36] where the interested reader can download the source, data and results of this training program. Regarding the training procedure, specifically the starting point, all models are initialized with precisely identical weights and biases. However, due to different activation functions, the initial solution varies from one model to another. Error evaluation of the models uses the Mean Square Error (E\({}_{\text{MS}}\)), the Root Mean Square Error (E\({}_{\text{RMS}}\)) and the Mean Absolute Relative Error (E\({}_{\text{MAR}}\)) given by the following equations:  
\[ \begin{cases}\text{E}_{\text{MS}}=\frac{1}{N}\sum_{i=1}^{N}\left(\square_{i}^ {e}-\square_{i}\right)^{2}\\ \text{E}_{\text{RMS}}(\text{MPa})=\sqrt{\text{E}_{\text{MS}}}\\ \text{E}_{\text{MAR}}(\%)=\frac{1}{N}\sum_{i=1}^{N}\left|\frac{\square_{i}- \square_{i}^{e}}{\square_{i}^{e}}\right|\times 100\end{cases} , \tag{15}\]  
where \(N\) is the total number of points for the computation of those errors, \(\square_{i}\) is the \(i\)th output value of the ANN, and \(\square_{i}^{e}\) is the corresponding experimental value. Figure 4 shows the evolution of the common logarithm of the Mean Square Error, i.e., \(\log_{10}(\text{E}_{\text{MS}})\), of the output \(s\) of the ANN during the training, evaluated using only the test data (25% of the dataset). By examining this figure, we can assess and compare the convergence rates of various ANNs, concluding that a stable state was more-or-less achieved for all analyzed ANNs after 6000 iterations.