## 3 Case Study
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Step Number** & **Step Name** & **Description** \\ \hline \multirow{2}{*}{1} & \multirow{2}{*}{Network initialization} & The network architecture consists of an input layer (\(x\)), three hidden layers (\(h_{1},h_{2},h_{3}\)), and an output layer (\(y\)). The input layer takes two parameters: system error \(c(t)\) and reference action \(r(t)\). \\ \hline \multirow{4}{*}{2} & \multirow{4}{*}{Weight initialization} & The neural network parameters \(k_{p}\), \(T_{i}\), \(k_{1}\), \(k_{2}\), \(k_{3}\), and \(\alpha\) and other adaptive parameters are initialized randomly with a normal distribution (or using the Xavier/He method). In this case, the weight matrices for the hidden layers and the bias for each layer are initialized as \\  & & \(W_{ij}^{(l)}\sim N\left(0,\frac{1}{\sqrt{n}}\right)\), \(b^{(l)}=0\). \\ \hline \multirow{4}{*}{3} & \multirow{4}{*}{Forward pass} & For each \(l\)-th hidden layer, an input linear combination is computed. For the first hidden layer (error handling):\(z_{1}=W_{1}\cdot[e(t),r(t)]+b_{1}\), \(\alpha_{1}=\text{SmoothReLU}(z_{1})\) \\  & & For the second hidden layer (the regulatory effect computation): \(z_{2}=W_{2}\cdot a_{1}+b_{2}\), \(a_{2}=\text{tanh}(z_{2})\). \\  & & For the third hidden layer (switching parameters integration): \(z_{3}=W_{3}\cdot a_{2}+b_{3}\), \(a_{3}=\text{SmoothReLU}(z_{3})\). \\  & & For the output layer (the output coordinate compute): \(z_{4}=W_{4}\cdot a_{3}+b_{4}\). \\ \hline \multirow{4}{*}{4} & \multirow{4}{*}{Error computation} & The output error is computed as the difference between the prediction and the actual value, and a loss function, \(L\), is applied, which could be, for example, \(MSE\): \\  & & \(L=\frac{1}{h}\cdot\frac{n}{i}(y_{i}-x(t_{i}))^{2}\). \\  & & To compute the gradients, the sensitivity function \(\Xi(t)\) is used, which takes into account the parameters’ influence on the system. \\ \hline \multirow{4}{*}{5} & \multirow{4}{*}{Backpropagation} & Backpropagation involves computing gradients at each layer. For the output layer: \(\delta_{4}=\frac{\partial L}{\partial y}\cdot\sigma^{\prime}(z_{4})\), \\  & & \(\sigma^{\prime}(z_{4})\) is the activation function’s derivative at the output. \\  & & For the remaining layers, starting from the third, the error is calculated based on the activation function derivative: \\  & & \(\delta_{3}=\left(W_{4}^{T}\delta_{4}\right)\cdot\sigma^{\prime}(z_{3})\), \(\delta_{2}=\left(W_{3}^{T}\delta_{3}\right)\cdot\sigma^{\prime}(z_{2})\), \(\delta_{1}=\left(W_{2}^{T}\delta_{2}\right)\cdot\sigma^{\prime}(z_{1})\). \\ \hline \multirow{4}{*}{6} & \multirow{4}{*}{Updating weights with regularization} & For each layer, the weights and biases are updated, taking into account the regularization \(\lambda\). The gradients for the weights and biases are calculated as \\  & & \(\frac{\partial L}{\partial W^{(l)}}=\alpha^{(l-1)}\cdot\left(\delta^{(l)}\right) ^{T}+\lambda\cdot W^{(l)}\), \(\frac{\partial L}{\partial W^{(l)}}=\delta^{(l)}\). \\  & & The weights and biases are updated using the gradient descent equation: \\  & & \(W^{(l)}=W^{(l)}-\eta_{l}\cdot\frac{\partial L}{\partial W^{(l)}}\), \(b^{(l)}=b^{(l)}-\eta_{l}\cdot\frac{\partial L}{\partial W^{(l)}}\), \\  & & where \(\eta_{l}\) is the dynamic training rate, depending on the change in the optimization functional: \\  & & \(\eta_{l}=\frac{\eta}{1+\alpha\cdot[\Xi(l)]}\), \\  & & & where \(\alpha\) is the training rate adaptation coefficient, and \(|\Xi(t)|\) is the change magnitude in the functional at the current step. \\ \hline \multirow{4}{*}{7} & \multirow{4}{*}{Parameter’s adaptation} & Network parameters such as \(k_{p}\), \(T_{i}\), \(k_{1}\), \(k_{2}\), \(k_{3}\), and \(\alpha\) are trained via gradient descent, \\  & & taking into account regularization and the sensitivity function \(\Xi(t)\). These parameters include both weights and adaptive coefficients that change based on changes in the loss function. \\ \hline \multirow{4}{*}{8} & \multirow{2}{*}{Training repetition} & The training process is repeated for each batch of data or all training data, with weights and parameters updated at each step. It continues until stopping after the maximum number of epochs or the error improves. \\  & & \(\text{Performance}\) \\  & & \(\text{Once training is complete, the network is tested on test data to assess its ability to handle}\) \\  & & \(\text{new data by predicting output values given nonlinearities and delays in the system}\). \\ \hline \hline \end{tabular}
\end{table}