# Contact Force Surrogate Model and Its Application in Pantograph-Catenary Parameter Optimization
## 2 Surrogate Model Training by Feedforward Neural Network
### Training of Surrogate Model  
Map \(X_{r}\) of all samples of the three sets to the interval \([-1,1]\) and the processed input parameter value can be obtained as  
\[x_{r}=\frac{2(X_{r}-X_{\rm min})}{X_{\rm rmax}-X_{\rm rmin}}-1, \tag{1}\]  
where \(X_{\rm rmin}\) is the lower limit of \(X_{r}\) in the value space, and \(X_{\rm rmax}\) is the upper limit. Map \(\sigma\) to the interval \([0,1]\) and the processed output parameter value, i.e., \((\sigma-\sigma_{\rm min})/(\sigma_{\rm max}-\sigma_{\rm min})\) can be obtained, where \(\sigma_{\rm min}\) and \(\sigma_{\rm max}\) are the minimum and maximum of \(\sigma\) for all examples. A three-layer feedforward neural network (Figure 1a) was employed to construct a surrogate model that predicts the SDCF based on pantograph-catenary parameters. The \(r\)-th neuron \(x_{r}\) in the input layer is connected to the first hidden layer's \(h\)-th neuron with a  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Input Parameters** & **Unit** & **Value Range** & **Value Interval** & **Reference Value** \\ \hline \multirow{2}{*}{\(v\)} & \multirow{2}{*}{km/h} & \([250,380]\) & 2 & \multirow{2}{*}{/} \\ \cline{3-3}  & & (380,450] & 1 & \\ \hline \(m_{3}\) & kg & \([5,11]\) & 0.02 & 7.94 \\ \hline \(m_{2}\) & kg & \([5,20]\) & 0.05 &  
connection weight \(\omega_{th}\). \(\sum\limits_{r=1}^{15}\omega_{th}x_{r}-\theta_{h}\) is applied to a specific activation function to compute the output value of that neuron. Similarly, the output values from the previous hidden layer are transmitted to the next hidden layer through connection weights until the output value of the \(h\)-th neuron in the third hidden layer, \(b_{h}\), is obtained. \(b_{h}\) is then transmitted to the output layer with a connection weight \(v_{h}\), and the output value of the output layer's neuron is calculated by applying \(\sum\limits_{h=1}^{H_{3}}v_{h}b_{h}-\gamma\) to an activation function. After denormalization, this yields the SDCF. Among them, \(H_{1}\), \(H_{2}\), and \(H_{3}\) are the neuron numbers of the three hidden layers, \(\theta_{h}\) is the threshold of the \(h\)-th neuron of the first hidden layer, and \(\gamma\) is the threshold of the output layer's neuron. The neuron numbers and activation functions of hidden layers and output layers are shown in Table 2.  
The training of the SDCF surrogate model was accomplished by taking the minimizing of the mean squared error on the training set as the objective, using the connection weights and thresholds for the hidden and output layers as the decision vectors, and employing Adam Optimizer to find the optimal solution. Firstly, the connection weights and thresholds were respectively initialized to a set of normally distributed random numbers with a mean of 0 and a standard deviation of 0.05. Following the Adam update rules, these weights and thresholds were updated, and the corresponding mean squared errors for the training and validation datasets were computed. This process iterated until the maximum of 500,000 iterations. The optimizer's learning rate was set for 0.0005, and each training \begin{table}
\begin{tabular}{c c c} \hline \hline
**Neural Network Layer** & **Number of Neurons** & **Activation Function** \\ \hline
1 st hidden layer & 20 & tanh \\ \hline
2 nd hidden layer & 10 & tanh \\ \hline
3 rd hidden layer & 5 & sigmoid \\ \hline output layer & 1 & relu \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Hyper-parameter settings of neural network layer in surrogate model. Figure 1: Computational procedure. (**a**) Surrogate model based on feedforward neural network, (**b**) Sensitivity analysis based on variance-based method, (**c**) Parameter optimization based on SCSA and surrogate model. batch consisted of 1024 samples. It could be observed that as the number of iterations increases, the mean squared errors for both the training and validation datasets tend to converge (Figure 2).