# A Modified Analytic Hierarchy Process Suitable for Online Survey Preference Elicitation
## 4 Discussion  
### Advantages of MAHP over Traditional AHP for Online Surveys  
The increased availability of online survey providers and the decreasing costs of data collection through such providers offers opportunities for analysts to increase the range of stakeholders and their geographic distribution surveyed to support decision-making. The loss of direct interaction with survey respondents, however, also comes at a potential cost of increased inconsistency in the results (and with that, potentially biased weightings), as the ability to feed back to individuals is limited. This has also sparked an increase in the number of methods to try and retrospectively adjust AHP scores to reduce the influence of inconsistencies (e.g., [8; 57; 58]). As noted in the introduction, inconsistencies can arise for a number of reasons, including as an artefact of the discrete nature of the responses when a fractional preference score may be required to totally eliminate inconsistencies [27; 59]. The MAHP is still susceptible to this, although its effect on the results was relatively minor. For the four-comparison example (i.e., dessert), seven of the 25 MAHP responses had a GCI = 0, while a further seven had a GCI \(<\) 0.05. The largest inconsistency observed was GCI = 0.24, well below the threshold value of 0.353 for four variables. In contrast, only one of the 18 respondents who completed the bivariate AHP comparisons had a GCI = 0, with all others having a GCI greater than the critical value (i.e., GCI \(>\) 0.353), with a maximum value of GCI = 1.7. The substantial difference between the proportion of inconsistency between the two approaches suggests that the discrete nature of the responses has only a relatively small impact on the level of inconsistency, with the other causes of inconsistency identified in the literature having a substantially greater impact. In our study, differences between the MAHP- and the AHP-derived scores increased with increasing complexity. With three or fewer options to compare, both approaches produced statistically similar results (i.e., not statistically different). With four comparisons, the proportion of inconsistent responses increased substantially for the AHP but not for the MAHP. The derived weights were also statistically significantly different for three of the four dessert options, reflecting the effects of inconsistencies on the derived preference weights. The preference weight rankings were also found to change in some cases with the approach used. With the MAHP, the relative rankings of alternatives form an implicit key role of the elicitation process, as all alternatives are compared at the same time. As a result, the relative rankings of the alternatives are obvious to the respondent and will influence how they score each alternative. In contrast, the traditional pairwise AHP compares only  
Figure 10: The number of responses to each question. The red indicates a non-response, which occurred only for bivariate (AHP)-related questions. All MAHP questions were answered by all respondents. two alternatives at a time, and the implicit set of ranking across all alternatives (if not preference weights) is obfuscated. With only one direct comparison in the \"water\" scenario, both approaches produced similar ranks (with only two exceptions). In contrast, with the four-alternative scenario, the ranking of each option was more varied between the two elicitation approaches. This does not explain, however, the contradictory outcomes in the final scenario where the two approaches themselves were compared for ease of use. In this case, both may have been influenced by some of the many other drivers of inconsistency in responses, particularly as the cake was waiting and respondents may have rushed the last question to ensure they got their fair share (a consequence of the tragedy of the commons). The ease with which the respondents can complete the survey task is an indicator of the acceptability of the weighting approach (e.g., [47; 60; 61]).