## 4 Computational Experiments and Statistical Analysis
Figures 5 and 6 present the interaction plots between factors CVST and CVPT for the E[CI] and E[PI] objectives, respectively. The axes of the plots are the levels of the coefficients of variation. These plots confirm that for both qualitative objectives, there exists an interaction effect between the coefficients of variation of the processing times (CVPT) and the coefficients of variation of the setup times (CVST), which means that the best solution selected with the AHP method, in terms of the qualitative criteria, varies depending on the variability of the processing and setup times. This leads again to the relevance of including uncertainties in the optimization process and the execution of an accurate distribution fitting. Figure 4: Main effects plots for S(E/T). Figure 3: Main effects plots for E[E/T]. To conduct the experimental design and validate the validity and objectivity, the assumptions of homoscedasticity, normality, and independence were tested. Because the homoscedasticity and normality tests were not fulfilled, we performed a Friedman test to corroborate the ANOVA results. Tables 9 and 10 present the detailed Friedman tests for E[E/T] and SD[E/T] in terms of the factors PDPT, CVPT, PDST, and CVST. To the best of our knowledge, this is the only work that has studied these four objective functions in an SPFSP. We present three indicators for the multi-objective problems proposed by [PERSON] et al. (2018) and [PERSON] et al. (2018), which can be used for future comparisons. For this work, these indicators were adjusted for the four objective functions analyzed:  
* Number of Pareto solutions (NPS), which means the number of nondominated points for each instance;
* Mean ideal distance (MID), which is a measure of the closeness between Pareto solutions and an ideal point (0,0,0,0). The quality of a Pareto frontier is higher as the value of MID decreases. Equation (1) shows the function for this indicator;
* Spread of the nondominance solution (SNS), which is an indicator of the diversity of Pareto points. The Pareto frontier presents more diversity as the value of SNS increases. Equation (3) shows the function of this indicator. Figure 5: Interaction plots of the CVST-CVPT for E[E/T]. Figure 6: Interaction plots of the CVST-CVPT for S(E/T). \[MID=\frac{\sum_{i=1}^{NPS}c_{i}}{NPS} \tag{1}\]  
where:  
\[c_{i}=\sqrt{E[E/T]_{i}^{2}+SD[E/T]_{i}^{2}+E[CI]_{i}^{2}+E[PI]_{i}^{2}} \tag{2}\]  
\[SNS=\sqrt{\frac{\sum_{i=1}^{NPS}(MID-c_{i})^{2}}{NPS}} \tag{3}\]  
Tables 11-13 show the averages of the NPS, MID, and SNS for each instance size, each combination of the PDPT with CVPT, and each combination of the PDST with CVST, respectively. In Table 11, it can be seen that the MID and SNS increase as the number of jobs or machines increases, and due to the increment of the jobs or machines, the expected and standard deviation of tardiness present a crescent behavior. Moreover, the NPS also tends to augment, giving more possible solutions to chose in scenarios with higher variability. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Factor**} & **Levels of Factor** & **N** & **Expected Median** & **Sum of Ranks** & **Overall Median** & **DF** & \(p\)**-Value** \\ \hline PDPT & Lgn & 25,920 & 5243.6 & 44,156 & 5090.4 & 1 & 0 \\  & Unf & 25,920 & 4937.2 & 33,604 & & & \\ \hline CVPT & 0.3 & 17,280 & 4450.2 & 22,359 & 5093 & 2 & 0 \\  & 0.4 & 17,280 & 5059.4 & 34,166 & & & \\  & 0.5 & 17,280 & 5769.4 & 47,155 & & & \\ \hline PDST & Lgn & 25,920 & 5247 & 43,715 & 2112.9 & 1 & 0 \\  & Unf & 25,920 & 4978.8 & 34,045 & & & \\ \hline CVST & 0.3 & 17,280 & 4954.2 & 25,591 & 5296.1 & 2 & 0 \\  & 0.4 & 17,280 & 5265.5 & 34,108 & & & \\  & 0.5 & 17,280 & 5668.6 & 43,981 & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Friedman test for SD(E/T) versus the factors PDPT, PDST, CVPT, and CVST.