# Simulation Tool for Winter Navigation Decision Support in the Baltic Sea
## 8 Verification and Validation of the Simulation Model
### Validation
#### 8.2.4 Parameter Variability--Sensitivity Analysis  
According to this method, the input parameters of the model are varied and the effects on the output behavior are noted. Firstly, a qualitative analysis is performed, where the directions of output changes are compared with those expected in the real system. The expected behavior of the real system is obtained through interviews with field experts. Next, the magnitudes of outputs are also compared. This approach is similar to the first validation approach (comparison with other models), the difference being that instead of comparing other models, the comparison is made using field experts' descriptions of expected output behavior. The difference between face validity and parameter sensitivity is that in parameter sensitivity the various system parameters are varied one at a time and their effect on the system is observed. This helps to validate that the parameters are captured correctly and that their interaction with the system is modelled correctly as well. In face validity, overall system behavior is observed and validated. It is possible that even though face validity is successful, parameter variability sensitivity analysis may bring out inaccuracies (if any) in the modelling of specific parameters. The current model contains more parameters than the previous efforts and hence this approach allows for a complete validation study. Table 5 shows the parameters of the simulation model along with their ranges and default values. These are vessel speed threshold (the speed below which icebreaker assistance is required), the maximum waiting time allowable for any vessel before it receives icebreaker assistance, whether convoys are being built or not (convoy mode on/off), and maximum allowable convoy size (or minimum convoy speed).  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Output** & **Current Model** & **[PERSON] et al. (2018)**[14] \\ \hline
**Number of assistances** & Increase & Increase \\ \hline
**Total system waiting time** & Increase & Increase \\ \hline
**Average waiting time** & Increase & Increase \\ \hline \hline \end{tabular}
\end{table}
Table 4: Validation by comparison with other models for increase in vessel speed threshold.  
Additionally, vessel fleet size (the number of vessels in the system) is parameterized, such that it is possible to change the number of vessels for the period in order to gauge how model performance scales up with increase in the number of vessels. Multiple parameter variation experiments have been performed. The results of the qualitative study are presented in Table 6. The quantitative studies are described in detail in the following section.  
Figure 8: (**a**) Historical and simulated arrival times comparison: Set 1. (**b**) Historical and simulated arrival times comparison: Set 2.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **Range** & **Default** \\ \hline
**Vessels in system (N)** & 1, 25, 50, 100, 116 & 116 \\ \hline
**Vessel speed threshold (\(s_{n}\))** & (3, 6) kn & 3 kn \\ \hline
**Maximum allowable waiting time (\(t_{max}\))** & (12, 24) h & 12 h \\ \hline
**Convoys allowed (\(C_{on}\))** & 0, 1 & 1 \\ \hline
**Maximum convoy size (\(C_{max}\))** & (2, 6) & 2 \\ \hline
**Minimum convoy speed (\(s_{c}\))** & (6, 8) kn & 6 kn \\ \hline \hline \end{tabular}
\end{table}
Table 5: Range of parameters.  
From the validation studies, it is confirmed that the behavior of the model is aligned with the real system behavior as well as that of the previous simulation efforts. The next section presents some computational experiments with parameter variation to provide further insight into the functioning of the model.