# Linking Scheduling Criteria to Shop Floor Performance in Permutation Flowships
## 4 Computational Results  
### Dominance Relationships among Scheduling Criteria  
A first goal of the experiments is to establish which scheduling criterion is more related to the different shop floor performance measures. To check the statistical significance of the results, we test a number of hypotheses using a one-sided test for the differences of means of paired samples (see, e.g., [60]) for every combination of \(m\) and \(n\). More specifically, for each pair of scheduling criteria \((A,B)\) and a shop floor performance measure \(\zeta\), we would like to know whether the sequence resulting from the minimisation of scheduling criteria \(A\) yields a better value for \(\zeta\), denoted as \(\zeta(A)\), than the sequence resulting from the minimisation of scheduling criteria \(B\). More specifically, we want to establish the significance of the null hypothesis \(H_{0}:\zeta(A)\)_better than_\(\zeta(B)\) to determine whether criterion \(A\) is more aligned with SF indicator \(\zeta\) than criterion \(B\), or vice versa. Note that _better than_ may express different ordinal relations depending on the performance measure, i.e., it is better to have a higher \(TH\), but it is better to have lower \(ACT\) and \(WIP\), therefore we specifically test the following three hypotheses for every combination of scheduling criteria \(A\) and \(B\):  
\[H_{0}:TH(A)>TH(B)\]  
\[H_{1}:TH(A)\leq TH(B)\]  
with respect to throughput, and  
\[H_{0}:WIP(A)\leq WIP(B)\]\[H_{1}:WIP(A)>WIP(B)\]  
and  
\[H_{0}:ACT(A)\leq ACT(B)\]  
\[H_{1}:ACT(A)>ACT(B)\]  
with respect to average completion time and work in process, respectively. The results for each pair of scheduling criteria \((A,B)\) are shown in Tables 1-10 for the different testbeds, where \(p\)-values are given as the maximum level of significance to reject \(H_{0}\) (\(p\) represents the limit value to reject hypothesis \(H_{0}\) resulting from a \(t\)-test, i.e., for every level of significance \(\alpha\leq p\), \(H_{0}\) would have to be rejected, whereas for every \(\alpha>p\), \(H_{0}\) would not be rejected. A high \(p\) indicates that \(H_{0}\) can be rejected with high level of significance, and therefore \(H_{1}\) can be accepted.) To express it in an informal way: a value close to zero in the column corresponding to the performance measure \(\zeta\) in the table comparing the pair of scheduling criteria \((A,B)\) indicates that minimizing criterion \(A\) leads to better values of \(\zeta\) than minimizing criterion \(B\), whereas a high value indicates the opposite. To make an example of the interpretation of the procedure adopted, let us take the column \(TH\) for any of the testbeds in Table 1 (all zeros). This column shows the \(p\)-values obtained by testing the null hypothesis that makespan minimisation produces solutions with higher throughput than those produced by using flowtime minimisation as a scheduling criterion. Since these \(p\)-values are zero for all problem sizes, then the null hypothesis cannot be rejected. As a consequence, we can be quite confident (statistically speaking) that makespan minimisation is more aligned with throughput increase than completion time minimisation. In view of the results of the tables, the following comments can be done. * Regarding Table 1, it is clear that makespan outperforms the total completion time regarding throughput, and that the total completion time outperforms the makespan regarding average cycle time. These results are known from theory and, although they could have been omitted, we include them for symmetry. The table also shows that completion time outperforms makespan with respect to work in process, a result that cannot be theoretically predicted. This results is obtained for all instance sizes and different methods to generate the processing times. As a consequence, if shop floor performance is measured using primarily one indicator, \(C_{max}\) would be the most aligned objective with respect to throughput, whereas \(\sum C_{j}\) would be the most aligned with respect to cycle time and work in process.