# Efficient Deep Learning for Gradient-Enhanced Stress Dependent Damage Model
## 3 Training Data
### Back-Propagation Algorithm  
Though the back-propagation algorithm can be traced back to the 1970s, its popularity grew with the seminal paper of [PERSON] et al. (2018). It has become the 'backbone' of many efficient machine learning tools such as Pytorch or Tensorflow. The back-propagation algorithm basically minimizes the error function in weight space by the method of gradient descent. Mathematically, the input and output are in matrix form \((\mathbf{X},\mathbf{Y})\) where \(X\in\mathbb{R}^{d\times n}\) and \(Y\in\mathbb{R}^{k\times n}\), \(d\) designate the number of input parameters, \(k\) is number of output parameters and \(n\) is number of training data. After calculating the output from the input of a mini-batch \(\mathbf{X}\), the activation \(\mathbf{y}^{l}\) at each hidden layer are saved where \(l=1,\cdots,m\) is the order of hidden layers. At the output layer, the derivative of the loss function with respect to \(z\) are calculated \(\mathbf{e}^{m}=\frac{\partial J}{\partial\mathbf{z}^{m}}\) where \(J=\frac{1}{n}\sum_{i=1}^{n}\left\|\mathbf{\overline{y}}_{i}-\mathbf{y}_{i}\right\|_{2 }^{2}\). Hence, the gradient can be computed \(\frac{\partial J}{\partial\mathbf{w}^{m}}=\mathbf{y}^{m-1}\mathbf{e}^{m}\). For \(l=m-1,\cdots,1\), the derivatives \(\mathbf{e}^{l}=\left(\mathbf{w}^{l+1}\mathbf{e}^{l+1}\right)\otimes f^{\prime}(\mathbf{z}^{l})\) are determined where the operator \(\otimes\) means element wise product. Finally, the gradient is updated by \(\frac{\partial J}{\partial\mathbf{w}^{l}}=\mathbf{y}^{l}\mathbf{e}^{l}\) which apparently requires the continuity and differentiability of the error function. Applicatively in the three-point bending specimen, the training data will be collected by the gradient-enhanced damage models and the training process based on the \((1-100-100-100-62)\) architecture in Figure 3. Because of the symmetric of the specimen, the loading deflection curve are quite similar in the left from \(\frac{L}{2}-\Delta_{p}\) to \(\frac{L}{2}\) and in the right from \(\frac{L}{2}\) to \(\frac{L}{2}+\Delta_{p}\) with regards to the middle point. Hence, it is a good idea to divide the data into two parts, the left part and the right part with respect to the middle point. Thus, the training is separated into two processes with 61 data for each case.  
\begin{table}
\begin{tabular}{l c c} \hline
**Name** & **Equation** & **Derivative** \\ \hline Identity \((-\infty,+\infty)\) & \(f(z)=z\) & \(f^{\prime}(z)=1\) \\ \hline Logistic \((0,1)\) & \(f(z)=\frac{1}{1+e^{-z}}\) & \(f^{\prime}(z)=f(z)\left[1-f(z)\right]\) \\ \hline SoftPlus \((0,+\infty)\) & \(f(z)=\ln(1+e^{z})\) & \(f^{\prime}(z)=\frac{1}{1+e^{-z}}\) \\ \hline TanH \((-1,1)\) & \(f(z)=Tanh(z)\) & \(f^{\prime}(z)=1-f(z)^{2}\) \\ \hline Rectified Linear Unit \((\text{ReLU})\)\((0,+\infty)\) & \(f(z)=\left\{\begin{array}{cc}0,&z<0\\ z,&z\geq 0\end{array}\right.\) & \(f^{\prime}(z)=\left\{\begin{array}{cc}0,&z<0\\ 1,&z\geq 0\end{array}\right.\) \\ \hline \end{tabular}
\end{table}
Table 2: Activation functions.  
Figure 2: The various forces with respect to the location in three-point bending specimen.