## 4 Experiments and Results
### Experiments in Cost Prediction of WMS
After normalizing the input and output features of the datasets, the MTR-MLS algorithm is applied, and the performance of the MTR-MLS-based cost prediction model on the test set is tested by a five-fold cross-validation. The optimal parameter settings corresponding to the MTR-MLS algorithm for different datasets are given in Table 5, and the performance results of the algorithm on different subentry costs are shown in Table 6.  
In Table 6, the test errors for different maintenance grades do not differ much. Among them, the prediction accuracy of manufacturing cost (\(D_{3}\)) and period cost (\(D_{5}\)) is around 12%, which is significantly better than the prediction results of the other three subentry costs. This is because the variability in maintenance time between WSM tasks of the same category and grade is not significant, and the fluctuations in manufacturing and period costs are smaller. Therefore, it is reasonable to estimate the manufacturing and period costs of new events based on historical data of the same category and grade of WSM. In order to better compare the prediction accuracy of itemized cost with the total cost, the final total cost prediction is obtained by inverse-normalization based on the prediction results of the algorithm on subentry cost in Dataset 4. Similarly, based on the same strategy, we adopt the representative multi-output support vector regression (mSVR) and BPNN to predict the total cost of secondary maintenance of surface combatant ships. In BPNN, the topology was set as 23-15-5. Sigmoid and purelin were exploited as the activation function and in the hidden layer and the transfer function in the output layer, respectively. We set the maximum number of convergences as 150, while the learning rate and the error equaled 0.1 and 0.001, respectively. In mSVR, the penalty parameter \(c\) and kernel parameter \(\sigma\) were \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**MAPE (\%)** & **Dataset 1** & **Dataset 2** & **Dataset 3** & **Dataset 4** & **Dataset 5** & **Dataset 6** \\ \hline \(D_{1}\) & 13.7 & 11.6 & 21.0 & 20.3 & 15.8 & 14.9 \\ \(D_{2}\) & 13.0 & 26.1 & 16.3 & 15.1 & 11.1 & 14.9 \\ \(D_{3}\) & 7.9 & 14.5 & 18.6 & 12.0 & 10.7 & 12.3 \\ \(D_{4}\) & 15.3 & 12.5 & 14.8 & 15.3 & 16.5 & 17.7 \\ \(D_{5}\) & 12.3 & 6.3 & 15.6 & 13.3 & 17.5 & 9.3 \\ Average & 12.6 & 13.4 & 17.5 & 15.2 & 14.5 & 14.0 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: MAPE(%) values of MTR-MLS algorithm on WSM datasets. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Ship Category** & **Maintenance Grade** & **Dataset Size (N)** & **Denotes** \\ \hline combat support ships & second-grade & 44 & Dataset 1 \\  & third-grade & 25 & Dataset 2 \\ \hline surface combat ships & second-grade & 35 & Dataset 3 \\  & third-grade & 57 & Dataset 4 \\ \hline Submarines & second-grade & 27 & Dataset 5 \\  & third-grade & 26 & Dataset 6 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Description of the WSM datasets. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Datasets** & \(\Lambda_{\mathbf{1}}\) & \(\Lambda_{\mathbf{2}}\) & \(\Lambda_{\mathbf{3}}\) & \(\sigma\) \\ \hline Dataset 1 & \(10^{-3}\) & \(10^{-2}\) & \(10^{-3}\) & \(10^{-3}\) \\ Dataset 2 & \(10^{-3}\) & \(10^{-2}\) & \(10^{-2}\) & \(10^{-3}\) \\ Dataset 3 & \(10^{-4}\) & \(10^{-4}\) & \(10^{-5}\) & \(10^{-4}\) \\ Dataset 4 grid search in interval [\(10^{-3}\), \(10^{3}\)]. The prediction results of some samples in Dataset 4 are shown in Table 7.  
The predicted values of the different methods for samples in dataset 4 in terms of total costs and the corresponding errors are shown in Figure 5. In particular, the predicted values of the WSM samples' total costs are obtained by summing the predicted subentry cost values. The total cost prediction results obtained by summing multiple subentry costs of WSM using the proposed MTR method are significantly better than the traditional multi-output algorithms such as mSVR and BPNN.