# Prioritization of Research Proposals Using the Analytic Hierarchy Process – AHP

Deise Dos Santos Oliveira, Irenilza De Alencar Nääs, Flávio Martins Barros

► To cite this version:

Deise Dos Santos Oliveira, Irenilza De Alencar Nääs, Flávio Martins Barros. Prioritization of Research Proposals Using the Analytic Hierarchy Process – AHP. 20th Advances in Production Management Systems (APMS), Sep 2013, State College, PA, United States. pp.347-352, 10.1007/978-3-642-41263-9_43. hal-01451777

HAL Id: hal-01451777

https://inria.hal.science/hal-01451777

Submitted on 1 Feb 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Prioritization of research proposals using the analytic hierarchy process - AHP

Deise R. M. S. Oliveira<fnref n="1" />,<fnref n="2" />, Irenilza de A. Näas<fnref n="1" />, Flávio Margarito Martins Barros<fnref n="3" />

**Abstract.** This research is a prospective analysis of prioritizing research proposals using pre-established criteria and applying the Analytic Hierarchy Process-AHP. The prioritized projects should ensure the competitive advantage of the organization. For this analysis, criteria which take into account other factors, which are not purely technical, were applied. The guiding hypothesis of this study was that it is possible to maintain a competitive portfolio of research proposals, from the analysis of pre-established criteria, using the methodology of the Analytic Hierarchy Process, giving priority to innovation and technology transfer. A key finding was the importance of retrieving research projects' history in the company, and stating gaps to be controlled in future analysis. Six ongoing projects were analyzed by 17 interviewed researchers and scores were given. Results obtained using the multicriteria analysis were similar to that previously applied by the selection committee, validating the hypothesis. We concluded that AHP may assist in the identification and prioritization criteria for project portfolio management.

**Keywords:** multicriteria analysis, research proposals selection, project portfolio.

## 1. Introduction

In the globalized world and today's dynamic, it is essential for organizations to maintain their competitive edge. The essence of this differential is aligned to updated strategies, contributing to the achievement of organizational goals. Without this strategic planning any further action may lead to failure. The connection between strategy and the selection and implementation of initiatives is through the funding of proposals and the project portfolio management is responsible for this alignment [1]. According to the Project Management Institute, portfolio management is the collection and management of projects or programs that are grouped together to facilitate the achievement of the strategic objectives of the organization. Therefore,

<footnote n="1">¹ Paulista University-UNIP, Graduate Program in Production Engineering, Dr. Bacelar St.
1212, São Paulo, Brazil</footnote>

<footnote n="2">²Embrapa Agricultural Informatics, Dr. André Tosello Avenue, 209,
Barão Geraldo, 13083-886, Campinas, SP, Brazil
{Deise R. M. S. Oliveira, deise.oliveira@gmail.com}</footnote>

<footnote n="3">³ State University of Campinas - Unicamp, Av. Cândido Rondon, 501
Barão Geraldo, 13083-875, Campinas, SP, Brazil
{deise.oliveira@gmail.com; irenilza@gmail.com; flaviomargarito@gmail.com}</footnote>
when managing a portfolio, the first premise is that it should be aligned with the
organization's strategies. In order to approve the proposals a main approach is that
they should be, firstly, be in accordance to the organizations' mission, goals and
strategic [6].

Current literature describes various types of problems related to portfolio
management, such as proposals without proper strategic alignment, high demand for
funding, presenting decision-making problems, and without reliable information. All
these issues are usually responsible for poor performance of the portfolio, as the
projects either have low impact or present failures, above the acceptable, in their
development [1; 4; 6]. One way to prevent this to happen within the organization is
working effectively in the management of the portfolio in all its stages.

The preliminary analysis of the proposals that will be part of the portfolio is
essential to the efficiency of the portfolio management. Therefore, classification and
selection of these proposals must be a careful process, adopting clear pre-established
criteria, which have the supervision of a senior manager. The authors [1] warn that
determining the strategic focus of the portfolio should be conducted in a senior
management levels because it involves the strategic goals of the institution.

Many organizations organize their portfolio subjectively, not adopting specific
criteria or methodologies in selecting their proposals. The decision in which proposal
should be funded and developed is still tied to an outsider/insider reviewer analysis,
and arbitrary choice, according to the reviewer viewpoint. This exposes the project to
risk, and maximizes the value assignment sometimes presenting no significant
differences between the proposals, leading to difficulties in selecting the best one [6].
This subjective assessment, based on the expertise of the reviewers/evaluators is
somehow essential in the evaluation process. When proposals are selected by the
same criteria, using mathematical and methodological rigor it promotes more
confidence to the decision makers [3].

Nowadays, there are several tools that work with the classification, selection and
prioritization of proposals. One of them, developed in the 1970s, is the Analytic
Hierarchy Process (AHP), a mathematical model to support decision making [9]. AHP
is a method that is characterized by the ability to analyze a problem and propose a
decision-making through the construction of hierarchical levels. The problem is
analyzed by pre-established criteria. The criteria are decomposed into sub-criteria up
to a certain level. These criteria are organized into a hierarchy descending where the
ultimate goals should be at the top, followed by their sub-goals, immediately below,
and, finally, the various possible outcomes or alternatives are selected. The scenarios
determine the likelihood of achieving the goals. The AHP usually run from the
general to the more particular and concrete goal [2; 5].

The objective of this study was to propose a methodology for prioritization and
selection of proposals in a company's Research, Development and Innovation (RD &
I) using the analytic hierarchy process - AHP.
## 2. Methodology

In this study, the criteria were selected with the application of questionnaires which were organized following the guidelines of the AHP. Five questionnaires were distributed to members of the Embrapa Information Technology Headquarters, and Internal secretary of the Technical Committee. The criteria were listed for the purpose of assisting in the evaluation of proposals to be prioritized.

The questionnaire consisted of 8 questions (yes or no options), and the evaluators could choose what they consider relevant in making a decision to prioritize research proposals. They could also add any other criteria that they found beneficial. One evaluator added four additional criteria. With that, the number of criteria was adjusted to the theory of AHP. The author [9] says that must be established between 3-7 criteria, not exceeding the number 9. The total number of criteria listed was in a total of 6 and 15 sub-criteria. The alternatives were the 6 proposals. Figure 1 shows the division of each criterion into sub-criteria in level 1 and 2, respectively.

<table><thead><tr><th>Goal</th><th>Level 1</th><th>Level 2</th></tr></thead><tbody><tr><td rowspan="6">Prioritizing criteria for selecting research projects</td><td>Adaptation to PDE/PDU<sup>1</sup></td><td>Aligned to the organization<br>mission<br>Aligned to the organization<br>strategic</td></tr><tr><td>Creativity</td><td>New idea<br>Process improvement</td></tr><tr><td>Technical aspects</td><td>Team<br>Risk<br>Technical quality<br>Results expected</td></tr><tr><td>Budget</td><td>Organization funding<br>Other sources funding</td></tr><tr><td>Possibility of development</td><td>Product<br>Service</td></tr><tr><td>Possibility of technology<br>transfer</td><td>Contract/patent<br>Spin off<br>Public knowledge</td></tr></tbody></table>

<caption>Figure 1. Scheme of the goal and the two levels of criteria adopted in the prioritization process of research projects.</caption>

Pairwise comparisons were made between each pair of factors at a given level of the hierarchy in regards to their contribution to the factor at the immediately preceding level. These pairwise comparisons yield a reciprocal (n, n) matrix A, where

<footnote n="1">¹ Organization Strategic Programs</footnote>
$a_{ii} = 1$ (diagonal elements) and $a_{ji} = 1/a_{ij}$. Suppose that only the first column of matrix A is provided to state the relative importance of factors 2, 3, ..., n with respect to factor 1. If the judgments were completely consistent, then the remaining columns in the matrix would be completely determined by the transitivity of the relative importance of the factors. However, there would be no consistency except for that created by setting $a_{ji} = 1/a_{ij}$. Therefore, the comparison needs to be repeated for each column of the matrix; specifically, independent judgments must be made for each pair. A is consistent if and only if $\lambda_{max} = n$. However, the inequality of $\lambda_{max} > n$ always exists; therefore, the average of the remaining eigenvalues can be used as a "consistency index" (CI; Eq. 1), which is the difference between $\lambda_{max}$ and n divided by the normalizing factor $(n - 1)$.

$$CI = (\lambda_{max} - n)/(n - 1) \qquad \text{Eq. 1}$$

The CI of the studied problem is compared with the average random index (RI) obtained from associated random matrices of order n to measure the error due to inconsistency [9]. A consistency ratio (CR = CI/RI) with a value $\le 0.1$ should be maintained for the matrix to be consistent; otherwise, the pairwise comparisons should be revised. The homogeneity of factors within each group, a small number of factors in the group, and a better understanding of the decision problem would improve the consistency index.

After selecting the criteria, evaluators were interviewed, and then gave the scores to the 6 research projects and the calculation of the collected data was done in real-time using the online web based software [7].

### 3. Results and Discussion

Data from the weight given by each evaluator were organized in tables containing the criteria for the 6 projects. Mean values and standard deviation of the values were calculated for each criterion, as seen in Table 1.

<table><caption>Table 1. Proportional weight of the criteria adopted by the projects' evaluators (%)</caption><thead><tr><th rowspan="2">Criteria</th><th colspan="6">Project</th><th rowspan="2">SD</th></tr><tr><th>A</th><th>B</th><th>C</th><th>D</th><th>E</th><th>F</th></tr></thead><tbody><tr><td>Adaptation to PDE/PDU<fnref n="1" /></td><td>29</td><td>28</td><td>30</td><td>26</td><td>29</td><td>31</td><td>2</td></tr><tr><td>Technical aspects</td><td>18</td><td>18</td><td>19</td><td>17</td><td>18</td><td>18</td><td>1</td></tr><tr><td>Creativity</td><td>11</td><td>10</td><td>12</td><td>13</td><td>11</td><td>11</td><td>1</td></tr><tr><td>Budget</td><td>8</td><td>7</td><td>8</td><td>8</td><td>8</td><td>9</td><td>1</td></tr><tr><td>Possibility of technology transfer</td><td>14</td><td>18</td><td>13</td><td>16</td><td>15</td><td>14</td><td>2</td></tr><tr><td>Possibility of development</td><td>19</td><td>20</td><td>18</td><td>19</td><td>19</td><td>17</td><td>1</td></tr></tbody></table>

Total mean score of the weights were calculated and final values are presented in Table 2. The results were ranked and finally compared to the real score the research

<footnote n="1">¹ Organization Strategic Programs; SD = Standard deviation</footnote>
projects obtained during their previous evaluation by the research committee. Results were similar to those previously obtained, indicating that the methodology using multicriteria analysis is feasible to be used.

<table><caption>Table 2. Comparison of the total scores of each research project considering all criteria and the general classification</caption><thead><tr><td>Project</td><td>Average of score</td><td>Classification using AHP</td><td>Classification by Embrapa selecting committee</td></tr></thead><tbody><tr><td>A</td><td>16</td><td>4°</td><td>Rejected</td></tr><tr><td>B</td><td>20</td><td>1°</td><td>Approved</td></tr><tr><td>C</td><td>17</td><td>3°</td><td>Approved</td></tr><tr><td>D</td><td>14</td><td>5°</td><td>Rejected</td></tr><tr><td>E</td><td>14</td><td>5°</td><td>Rejected</td></tr><tr><td>F</td><td>19</td><td>2°</td><td>Approved</td></tr></tbody></table>

The success of project portfolio management in an RD & I organization is directly related to the adaptation to organizational strategies and innovation management. In the present study, two methods have been proposed to support the management of a portfolio of Institution RD&I proposals. This analysis was based on the retrospective aspect, as the proposals results were known. This knowledge based on specific criteria was necessary to understand the gaps in research and planning for the future of research in the organization.

The use AHP tool to assist in the prioritization criteria for proposals' portfolio management was successful. The AHP is a decision support methodology, which establishes the construction of a hierarchical structure and value judgment for decision making [3; 8]. This methodology does not eliminate the figure of the selecting committee, but it facilitates its performance, using the knowledge for the individual pairwise comparison of the criteria involved in the analysis.

The application of AHP in evaluating 6 projects enabled the prioritization of a project over another, based on technical criteria, considering both the objective and subjective views of these criteria, allowing a greater likelihood of success in project selection. To review and project prioritization, the tool proved to be adequate due to its ability to adapt to different situations that may occur in the proposals evaluation. The application of the tool in a larger set of projects with more evaluators should be considered as a continuation of this work.

## 4. Final Remarks

The success of project portfolio management in an RD & I organization is directly related to significant aspects, such as an adaptation to organizational strategies. We used the AHP technique to assist in the prioritization of research proposals within a project portfolio. The AHP is a decision support methodology, which establishes the construction of a hierarchical structure and value judgment for decision making. This methodology does not eliminate the figure of the reviewer, but it facilitates their
performance, using his knowledge for the individual pairwise comparison of the
criteria involved in the analysis.

The application of AHP in evaluating the proposals enabled the prioritization of a
proposal over another, based on technical criteria. The tool proved to be adequate due
to its ability to adapt to different situations that may occur in the evaluation process.

References

1. Archer, N. P.; Ghasemzadeh, F. An integrated framework for project portfolio selection. International Journal of Project Management 17(4):207-216 (1999).

2. Barros, M. A.; Moreira, M. A.; Rudorff, B. F. T. Processo analítico hierárquico na identificação de áreas favoráveis ao agroecossistema cafeeiro em escala municipal. Pesquisa Agropecuária Brasileira 42(12):1769-1777 (2007).

3. Belton, V.; Stewart, T. J. Multiple criteria decision analysis: an integrated approach. Boston: Kluwer Academic, 372 p. (2002).

4. Cooper, L. P. A research agenda to reduce risk in new product development through knowledge management: a practitioner perspective. Journal of Engineering and Technology Management 1(2): 117-140 (2003).

5. Granemann, S. R.; Gartner, I. R. Modelo Multicriterial para Escolha Modal/Submodal de Transporte. Anais do XIV Congresso de Pesquisa e Ensino em Transportes, ANPET, Gramado, (2000).

6. Killen, C. P.; Hunt, R. A.; Kleinschmidt, E. J. Project portfolio management for product innovation. International Journal of Quality & Reliability Management 25(1): 24-28 (2008).

7. MakeitRational®. Available at http://www.makeitrational.com. Accessed on: March 2013.

8. Montevechi, J. A. B.; Pamplona, E. O. Análise hierárquica em análise de investimentos. Itajubá. Escola Federal de Engenharia de Itajubá, 1999. Available at: <http://www.iepg.unifei.edu.br/arnaldo/pesquisa/artigos/ahpinvest.html>. Accessed on: Nov. 2012. (2012).

9. Saaty, T. L. Método de análise hierárquica. São Paulo: Makron, 367 p. (1991).