# On the Use of Data Envelopment Analysis for Multi-Criteria Decision Analysis
## 4 Discussion  
The underlying use of hierarchical structures in MCDA is relatively common. Different management objectives often have a different number of sub-objectives, or a different number of indicators relating to each objective. Introducing a hierarchical structure in the assessment of management alternatives using DEA provides a number of benefits. Foremost, it allows an assessment of how each management alternative relates to each objective as an overall benefit of the doubt measure over all objectives and indicators. While not considered in this study, the efficiency score associated with each group of indicators (i.e., potentially representing a separate management objective) could be considered as well as the overall performance measure of the management option. The use of an intermediate step also has benefits in terms of saving degrees of freedom when data are limited. The number of indicators that can be compared in an analysis is limited by the number of available observations. The discrimination power and accuracy of DEA with respect to the ability to estimate the performance of DMUs decreases as the number of DMUs decreases or the numbers of inputs and outputs increases [46]. While there is no definitive estimate of the number of observations required to ensure robust estimation of the efficiency scores, a commonly applied rule of thumb is that the number of observations (DMUs) should be greater than \(max[3(m+n),m\times n]\), where \(m\) and \(n\) are the number of outputs and inputs, respectively [45]. In our hypothetical example, with nine outputs and one input, 30 observations would be required to meet this. With 100 generated DMUs, this was not a concern. For the waste management example, however, degrees of freedom issues may have contributed to the large number of efficient alternatives when all the data were used, but substantially fewer when breaking the problem up into essentially four separate estimation stages (one each for each objective and one for the combined objective level analysis). For example, when all data were used, the problem required 24 observations (i.e., \(3\times(7+1)\) alternatives) although only 22 alternatives were available. In contrast, estimating the efficiency score associated with just the environmental objectives required only 15 observations (i.e., \(3\times(4+1)\)). If decision makers do have differing preferences for the different objectives, then weights can be applied also at each stage to provide a weighted overall efficiency score. While not explicitly covered in this study, an example of how importance weights can be applied in DEA is given by [PERSON] et al. [18]. If weights were available for the waste management example, it is expected that the correlation between the MCDA and DEA model rankings would have been greater. In the hypothetical analysis, each observation was treated as a separate management option for consideration. In many MCDA studies, multiple sets of observations are associated with each management option reflecting uncertainty in management outcomes. These may be derived through stochastic simulation using models or through (differing) \begin{table}
\begin{tabular}{l l l l l l} \hline \hline  & \multirow{2}{*}{**Electre III**} & \multicolumn{3}{c}{**Radial DEA**} & \multicolumn{3}{c}{**Slacks-Based Model**} \\ \cline{3-6}  & & **All** & **Hierarchy** & **All** & **Hierarchy** \\ \hline ELECTRE III & 1.000 & & & & \\ Radial all & 0.369 & 1.000 & & & \\ Radial & 0.228 & 0.405 & 1.000 & & \\ Hierarchy & 0.510 & 0.907 & 0.356 & 1.000 & \\ SBM & 0.621 & 0.509 & 0.762 & 0.627 & 1.000 \\ Hierarchy & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Rank correlations between different models, waste management example. expert opinion from a range of experts. The DEA framework offers particular advantages in this case, as each potential outcome set is compared with all other sets.