# Analysis of Instantaneous Energy Consumption and Recuperation Based on Measurements from SORT Runs
## 3 Results
### Neural Network  
For the model describing ICRE, a four-layer neural network (9) was used. For this network, an identity function (linear activation) was chosen as the activation function (8). Faster contributions of the objective function (11) were obtained for this activation function (see Table 6). The total number of parameters of the neural network is 84. The number of training epochs is 300. A validation split was used during training on a set containing a 15% batch. MSE was selected as the function of the quality (loss) of training the neural network for the regression model and the validation method. The Figure 8 shows that in the first 70 epochs of training, the most significant decrease in the objective function value occurred. The coefficient of determination \(R^{2}\) value for NN equals 0.9206, which is higher than for the linear model but slightly lower than for KNN. For the NN (6) model, the residuals were analysed. The blue curve represents training loss, which is consistently less than the green curve representing validation loss. As is typical with neural network models, this small discrepancy suggests some overfitting. Both curves are rather close to each other, though, so it would seem that the model does a decent job of fitting the training data while also being able to generalise to new data. After the model reaches convergence, the MSE settles around a value of about 20, which represents the residual error. There appears to be diminishing returns in terms of optimising the model after 100 epochs, as the curves become increasingly flat. This indicates that adding more training epochs does not result in noticeable gains. Convergence happens early in the training phase, which is indicative of \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Layer (Type)** & **Output Shape** & **Parameters** & **Activation** \\ \hline Input Layer & (1, 10) & 0 & \\ Dense 1 & (1, 5) & 55 & linear \\ Dense 2 & (1, 3) & 18 & linear \\ Dense 3 & (1, 2) & 8 & linear \\ Output Dense & (1, 1) & 3 & linear \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Neural network architecture. Figure 8: Values of objective function (loss) during training and validation of the neural network. the model's effective learning process. The trends that have been observed support this conclusion. The lack of significant overfitting or underfitting is supported by the model's consistent performance across the training and validation datasets. This provides strong evidence that the model's hyperparameters and architecture are optimal for the provided data and task. Important insights into the performance of the neural network can be found in the residual analysis, which is shown in Figure 9. The model seems to have captured the majority of the data variability, as indicated by the histogram and density function of the residuals (top-left figure), which demonstrate that the residuals are nearly symmetrically distributed around zero. There are a number of lags in the ACF plot (top-right) that display correlations. It appears that the residuals might show some link over time and are not totally independent. The normal Q-Q plot (bottom-left) reveals deviations from the theoretical quantiles, particularly in the tails. The residuals show a clear departure from the red reference line, indicating that the residuals do not follow a perfectly normal distribution. The presence of these deviations suggests that the model struggles to predict extreme values accurately, which can impact overall model performance, particularly in high-variance regions of the data. In the bottom-right corner of the CDF plot, we can see a comparison between the normal distribution (blue) and the empirical distribution of residuals (green). A discrepancy between the theoretical and empirical curves is evident in the tails, providing more evidence that the residuals do not follow a perfectly normal distribution.