# A Modified Analytic Hierarchy Process Suitable for Online Survey Preference Elicitation
## 3 Results
### Respondents' Preferences for Each Method (a User Perspective)  
At the end of the survey, respondents were also asked to indicate which approach they thought was easiest. This was asked using both approaches. There was no apparent strong preference for one approach or another (Figure 8), with preferences distributed across the range for both. The first pair of distributions in Figure 8 (i.e., the blue distributions) represent the preference for the MAHP approach (\"Modified\") derived from the pairwise (AHP) and grouped (MAHP) comparisons, respectively, while the second pair (i.e., the green distributions) represent the preference for the traditional AHP (\"Pairwise\"), again derived from the pairwise (AHP) and grouped (MAHP) comparisons, respectively. As there were only two alternatives, there are no inconsistencies, and the distributions are reciprocal. Figure 8: Perceptions of ease of use of each approach (derived using each approach). The black dot represents the mean of the distribution. The first part of the label represents the alternative considered, while the second part of the label represents the approach used to elicit the preference. For example, “Modified.AHP” represents the user preference for the use of the MAHP approach but derived using the traditional AHP. Conversely, “Pairwise.MAHP” represents the user preference for the use of the traditional AHP approach but derived using the MAHP. The light blue distributions represent the preferences for the use of MAHP estimated using both methods (AHP and MAHP respectively), and the green distributions represent the preferences for the pairwise AHP estimated using both methods. Figure 7: Consistency indicator of responses for the four-alternative comparison: (**a**) MAHP and (**b**) AHP. Blue bars indicate consistent responses; red bars indicate inconsistent responses. The black vertical line indicates the GCI where the CR = 0.1. The paired \(t\)-test suggested that there was no statistically significant difference between the preference scores of the individuals based on the method used (Table 6). However, comparing the samples as a whole, the mean of the preference for the MAHP approach was statistically significantly different (and slightly higher) than the mean of the preference for the pairwise (AHP) approach when using the MAHP (\(t=2.146\), DF = 48), but not statistically significantly different when using the pairwise AHP approach (\(t=-0.1345\), DF = 36). When comparing preference ranks, the preference ranking changed for 11 of the 18 individuals across the two methods (i.e., the derived weight was equal to or higher than 0.5 for one of the options using one approach, but less than 0.5 using the other). We define \"rank consistency\" in Figure 9 as observations that were consistently ranked first or second using both approaches (compared with the GCI measure of inconsistency). While potentially spurious, the respondents who were most consistent in terms of rank of the two alternatives were those who also had the greatest preference for the MAHP approach. In all the cases above, the paired \(t\)-test and ranking comparisons were only applied to respondents who provided a response to both the AHP and the MAHP version of the questions in the survey. Several respondents systematically did not complete the pairwise (AHP) comparisons but completed the grouped (MAHP) comparisons (Figure 10), possibly suggesting that these individuals did not feel confident in completing the pairwise comparisons. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Alternative**} & \multicolumn{2}{c}{**Mean Score**} & \multirow{2}{*}{**Mean Difference**} & \multirow{2}{*}{**\(t\)-Statistic**} & \multirow{2}{*}{**Probability**} \\ \cline{2-2} \cline{5-6}  & **MAHP** & & & & \\ \hline MAHP & 0.557 & 0.492 & 0.065 & 0.774 & 0.449 \\ AHP & 0.443 & 0.508 & \(-0.065\) & \(-0.774\) & 0.449 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Paired \(t\)-tests between the derived weights from each approach.