## 4 Results
### Automation of Pit Stop Decisions Using a Neural Network
#### 4.2.1 Choice of Neural Network Hyperparameters
\begin{table}
\begin{tabular}{l l} \hline \hline
**Hyperparameter** & **Value** \\ \hline Number of hidden layers & 3 \\ Number of neurons per layer & 64 \\ Activation functions & ReLU (hidden layers), sigmoid (output layer) \\ L2 regularization & 0.0005 \\ Optimizer & Nadam \\ Loss function & Binary cross entropy \\ Training class weights & no pit stop: 1, pit stop: 5 \\ Training batch size & 256 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Hyperparameters of the neural network for the pit stop decision. progress, while still allowing it to increase quickly in the case of sudden events, e.g., FCY phases. This would require input data from more than a single lap. We therefore switched to recurrent neural networks (RNNs), which operate with time-series input data. Their neurons store an internal state, which allows them to learn relationships between consecutive steps. In our case, we insert series of consecutive laps. Pre-processing of these series is as described in Section 3.2.2. Instead of a single lap, units of several consecutive laps are processed with the same pipeline. The only difference is that time series that include laps before the first lap must be artificially filled with 'neutral values', cp. Table 10.  
As with the FFNN, many different architectures were tested. We found that pure RNNs can reach high scores on the test data (\(F_{1}\approx 0.9\)), but tend to output many incomprehensible predictions. An example of this is given in Figure 7. In particular, [PERSON]' predictions in the last part of the race are clearly wrong, since the remaining pit stops feature is zero there. This demonstrates that the \(F_{1}\) score alone has little significance. The predictions of the RNN jump strongly from lap to lap, which is why it predicts significantly less false positives and false negatives than the FFNN, where the predicted probability increases slowly but steadily. However, as the example shows, the pit stop predictions of the RNN are often placed completely incorrectly. This is much worse for making automated pit stop decisions than predicting the pit stop just a few laps too early or too late, as was the case with the FFNN. We therefore combined the advantages of both types by creating a hybrid NN. The following procedure gave the best results in our tests. The FFNN is first created and trained as described. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Feature** & **Lap -1** & **Lap 0** & **Lap 1** & **Lap 2** \\ \hline Race progress & 0.0 & 0.0 & 0.02 & 0.04 \\ Tire age progress & 0.04 & 0.04 & 0.06 & 0.08 \\ Position & 3 & 3 & 2 & 2 \\ Relative compound & Soft & Soft & Soft & Soft \\ Race track category & 2 & 2 & 2 & 2 \\ FCY status & 0 & 0 & 0 & 0 \\ Remaining pit stops & 2 & 2 & 2 & 2 \\ Tire change of pursuer & False & False & False & False \\ Close ahead & False & False & True & True \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Example time series input data for a length of 4 laps showing how values before lap 1 are artificially filled with ‘neutral values’. Figure 7: Progressions of pit stop probabilities for three drivers predicted by a recurrent NN (RNN) using real input data from the 2019 Brazilian Grand Prix. The vertical lines indicate that a real pit stop was made by the respective driver. The horizontal line at 50% probability separates the no pit stop from the pit stop region. The race was excluded from the training data. Driver abbreviations: [PERSON]—[PERSON], GAS—[PERSON], [PERSON]—[PERSON]. Then a new model with an equivalent structure to that of the FFNN is created, and a single LSTM (long short-term memory) neuron is added after the original output. Subsequently, the weights for all layers that existed in the FFNN are copied to the hybrid NN and marked as untrainable. Finally, the hybrid NN is trained with data sequences. Only the weights of the LSTM neuron are then adjusted. It thus learns to modify the output of the FFNN such that its predictions become increasingly accurate. Table 11 gives an overview of the additional hyperparameter choices.