## 3 Methodology: Development of the Evaluation Framework
### Building the Cost-Benefit Function
#### 3.3.3 Building a Panel Method Scheme: Analytic Hierarchy Process  
Panel methods to elicit the criteria highly depend on the selected method (questionnaires, interviews, or group discussions), panel composition (experts, laypeople, or stakeholders), procedure (single-round, Delphi) and outcome (consensus, statistical analysis of results) [32]. In the case of this evaluation method, and according to the IPD principles, the panel will be composed by the stakeholders being articulated through the IPD contract, and must result into a consensus to elicit the weighting parameters. There are several panel methods which can be implemented, being those more relevant: direct rating, Simos' cards method, ranking the criteria, pair-wise comparison (for example in the analytic hierarchy process, AHP), and the classical multi-attribute value theory (MAVT). which may be adapted for other procedures [33]. Below, the main scheme of an analytic hierarchy process (AHP) is presented in Figure 3, which establishes that [34]: \"a decision making process can be decomposed into 4 steps, namely: defining the goal, structure the decision hierarchy, construct a set of pairwise comparison matrices and use the priorities obtained from the comparison to weight the priorities in the level immediately below\". The AHP process is a well-known method that serves to establish the relevance for each criterion through their comparison per pair. As the state-of-the-art demonstrates, it is a consistent method that can be used in order to solve different decision making problems. In this case, the problem to be solved is the selection of the best scenario from the alternatives through considering the priorities of the stakeholders. Thus, this tool proposes and integrates a pair-wise comparison strategy that allows generating a weighting scheme through comparing per pairs the main criteria with respect to the cost/benefit index and the sub-criteria (level 2 of indicators) with respect to the criteria ENE09. The values related to the intensity of importance, according to [35], \"have to be set by consensus of the decision making panel (or experts' panel for fixed weighting schemes), in which each participant is asked individually to compare the criterion in a pairwise method setting whether a criterion is more important than the pair, and by how much\". Table 2 shows the scale of numbers used to reflect the intensity of importance of the indicators. Figure 3: Hierarchy for the decision-making process. Following this approach, the process considers the construction of the pairwise comparison matrices that can establish the relevance of each criterion for a defined goal. In this evaluation method, there are three matrices that allow comparing:  
* Pairwise comparison matrix of the six environmental and economic criteria with respect to cost. * Pairwise comparison matrix of the six energy and comfort criteria with respect to the benefit. * Pairwise comparison matrix of the six criteria for the renewables with respect to ENE09. In the following Tables 3-5, these three comparison matrices are shown. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**MATRIX 2** & **ENE01** & **ENE02** & **ENE06** & **ENE09** & **ENE13** & **COM01** \\ \hline ENE01 & 1 & ENE01/ENE02 & ENE01/ENE06 & ENE01/ENE09 & ENE01/ENE13 & ENE01/COM01 \\ \hline ENE02 & ENE02/ENE01 & 1 & ENE02/ENE06 & ENE02/ENE09 & ENE02/ENE13 & ENE02/COM01 \\ \hline ENE06 & ENE06/ENE01 & ENE06/ENE02 & 1 & ENE06/ENE09 & ENE06/ENE13 & ENE06/COM01 \\ \hline ENE09 & ENE09/ENE01 & ENE09/ENE02 & ENE09/ENE06 & 1 & ENE09/ENE13 & ENE09/COM01 \\ \hline ENE13 & ENE13/ENE01 & ENE13/ENE02 & ENE13/ENE06 & ENE13/ENE09 & 1 & ENE13/COM01 \\ \hline COM01 & COM01/ENE01 & COM01/ENE02 & COM01/ENE06 & COM01/ENE09 & COM01/ENE13 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Pairwise comparison matrices for Matrix 2 (Benefits Level 1).