# Selecting Normalization Techniques for the Analytical Hierarchy Process

Nazanin Vafaei, Rita A. Ribeiro, Luis M. Camarinha-Matos

► To cite this version:

Nazanin Vafaei, Rita A. Ribeiro, Luis M. Camarinha-Matos. Selecting Normalization Techniques for the Analytical Hierarchy Process. 11th Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), Jul 2020, Costa de Caparica, Portugal. pp.43-52, 10.1007/978-3-030-45124-0_4 . hal-03741563

HAL Id: hal-03741563

https://inria.hal.science/hal-03741563v1

Submitted on 1 Aug 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Selecting Normalization Techniques for the Analytical Hierarchy Process

Nazanin Vafaei, Rita A. Ribeiro, Luis M. Camarinha-Matos

CTS-UNINOVA and Faculty of Sciences and Technology, Nova University of Lisbon
2829-516 Caparica, Portugal
{Nazanin.vafaei, rar, cam}@uninova.pt

**Abstract.** One of the matters which has influence on Multi-Criteria Decision Making (MCDM) methods is the normalizing procedure. Most MCDM methods implement normalization techniques to produce dimensionless data in order to aggregate/rank alternatives. Using different normalization techniques may lead to different rankings. So, selecting a more suitable normalization technique is a requirement in the decision process. Specially, by the advent of big data and its role in developing life's quality, finding the best normalization technique in MCDM models are more challenging. Collecting data from sensors causes more complex decision problems, thus, providing accurate normalized values (in the same unit) is more critical in these types of contexts. In this research, we analyze and evaluate the effect of different normalization techniques on the ranking of alternatives in one of the Multi-Criteria Decision Making (MCDM) methods called Analytical Hierarchy Process (AHP) using our developed assessment framework. An illustrative example (smart car parking) is used to discuss the suitability of the framework and recommend more proper normalization technique for AHP. Furthermore, the developing of technological innovation is expected by using the evaluation framework which can raise the accuracy of the normalized values in decision problems.

**Keywords:** Normalization, MCDM, AHP, Decision making, data fusion, aggregation, Big data.

## 1 Introduction

During the last decades, Multi-Criteria Decision Making (MCDM) has received much attention from researchers due to its abilities to deal with complex decision problems that depend on several criteria. Each MCDM problem is defined by a decision matrix that includes a set of alternatives $A_i$ ($i=1, ..., m$), criteria $C_j$ ($j=1, ..., n$), the relative importance of the criteria (or weights) $W_j$, and $ri_j$, corresponding the rating of alternative $i$ with respect to criteria $j$ [1]. In most MCDM problems, criteria are measured in different units (e.g. velocity, fuel consumption, design, etc., in selecting a car problem) while they should be defined in the "same scale" to make an effective comparison. Therefore, the pre-processing for making dimensionless data from heterogeneous input data is called normalization. The normalization procedure is the first step in most MCDM methods and using different normalization techniques may lead to different ranking/ordering of alternatives and may cause deviation from optimal
ranking/ordering. Thus, choosing the suitable normalization techniques plays an
important role in the final results of decision problems.

From another point of view, the role of normalization techniques is extended by
developing technological innovations that contribute to the growth of life's quality by
exploring new ideas in different discipline like big data which is a target discipline for
new and evolved normalization techniques. Big data are collected from heterogeneous
sensors and multiple other sources which need a suitable normalization technique to
make them applicable for data fusion/aggregation. These reasons and important roles
of the normalization process motivated us to propose an assessment framework to
evaluate different normalization techniques in the MCDM methods.

In this work, the main research question that we address is: *Which normalization technique is more suitable for usage with the AHP method?*

This paper is an extended version of a preliminary study [2] which assessed the suitability of four normalization techniques in AHP method using Pearson and Spearman correlation (a part of the on-going evaluation framework). In this work, we assess the chosen normalization techniques with the additional developed evaluation assessment that was introduced in the recent submitted work by the authors [29] and recommend the most proper normalization technique for AHP method. In order to ensure the robustness of results, the same illustrative example (smart car parking place) from the above paper [2] was borrowed.

## 2 Contribution to Life Improvement

Life quality is defined as the level of wellbeing in terms of health, comfort, happiness, etc. for humans. Some elements like technological innovations have the power to change and improve life quality in different ways [3]. Nowadays, new developments in Artificial Intelligence, Machine Learning, Big Data, and Internet of Things (IoT) / Cyber-Physical Systems, are likely to change our daily lives with new ideas [4]. For instances, in smart car parking, input data is collected from sensors and then delivered to data centers and after analyzing and finding the best parking place, related data will be transferred to the driver with the help of IoT. In the data center, all received data should be normalized and then options ranked by implementing one of the MCDM methods. Then data related to the best parking place should be transferred to the driver, saving time in finding where to park, often a stressful situation. This decision problem shows the improvement of life quality by increasing comfort, especially in big cities. In this paper, we aim to select the best normalization technique for using in AHP when ranking alternatives in a smart car parking decision problem.

## 3 Normalization

Numerous normalization techniques have been proposed in the literature and most MCDM methods use one of these techniques. Jahan and Edwards [1], [5] pointed to some important features that have influencing effects on capability of normalization
techniques and should be considered when developing and evaluating of techniques
(Fig. 1).

<figure><img src="image_3.png" /><figcaption>Fig. 1: Expected aspects for normalization techniques ([1], [5])</figcaption></figure>

As Fig. 1 shows, one of these aspects is the capability of removing scales, which is the basic role of normalization technique in converting the different measurement units of criteria (in MCDM models) into dimensionless units and making comparable decision matrices [1], [5]. Symmetry is another feature that belongs to some normalization techniques which can convert cost criteria into benefit one [1], [5]. This aspect would reduce the calculation process but is always not necessary for MCDM methods [1], [5]. The next property is rank reversal that causes ranking changes by adding or removing alternatives [1], [5]. Rank reversal could happen by selecting a unsuitable normalization technique [1], [5]. Handling negative values is an important capability for a normalization technique when dealing with negative values in MCDM methods [1], [5]. Fig. 1 also depicts data types and topology as the last aspect which is added in the work of Jahan and Edwards [1].

Previous research done by the authors proved that the type of input data caused influencing effects on the normalized values and ranking alternatives as well in MCDM method [6]. For example, including zero or decimal numbers in the input data using Sum or Logarithmic normalization techniques are not recommended because of producing undefined and infinite normalized values with the mentioned techniques [6].

Several normalization techniques are proposed in literature. For instance, Jahan and Edwards [1] listed 31 normalization techniques and categorized them based on their applicability for material selection problems. They also, elaborated pros and cons of some techniques such as Max-Min which is affected by the number of alternatives because of changing maximum and minimum values by adding or removing alternatives [1]. Some other studies discussed the Max-Min normalization techniques which are very commonly used with MCDM methods [7], [8]. Another common normalization technique is Vector normalization which is implemented for TOPSIS method [1], [7].

Although there are many normalization techniques in the literature, since this paper is an extension of a previous study [2], we selected exactly the same normalization technique that were used in [2]. Therefore,, we compare the suitability of the five more well-known normalization technique (Table 1) for using in AHP method.
<table><caption>**Table 1:** Well-known Normalization techniques (adapted from [2]).</caption><thead><tr><th>Normalization technique</th><th>Condition of use</th><th>Formula</th></tr></thead><tbody><tr><td rowspan="2">Linear: Max (N1)</td><td>Benefit criteria</td><td>n<sub>ij</sub> = <sup>r<sub>ij</sub></sup>&frasl;<sub>r<sub>max</sub></sub></td></tr><tr><td>Cost criteria</td><td>n<sub>ij</sub> = 1 - <sup>r<sub>ij</sub></sup>&frasl;<sub>r<sub>max</sub></sub></td></tr><tr><td rowspan="2">Linear: Max-Min (N2)</td><td>Benefit criteria</td><td>n<sub>ij</sub> = <sup>r<sub>ij</sub> - r<sub>min</sub></sup>&frasl;<sub>r<sub>max</sub> - r<sub>min</sub></sub></td></tr><tr><td>Cost criteria</td><td>n<sub>ij</sub> = <sup>r<sub>max</sub> - r<sub>ij</sub></sup>&frasl;<sub>r<sub>max</sub> - r<sub>min</sub></sub></td></tr><tr><td rowspan="2">Linear: sum (N3)</td><td>Benefit criteria</td><td>n<sub>ij</sub> = <sup>r<sub>ij</sub></sup>&frasl;<sub>&sum;<sub>i=1</sub><sup>m</sup> r<sub>ij</sub></sub></td></tr><tr><td>Cost criteria</td><td>n<sub>ij</sub> = <sup>1/r<sub>ij</sub></sup>&frasl;<sub>&sum;<sub>i=1</sub><sup>m</sup> 1/r<sub>ij</sub></sub></td></tr><tr><td rowspan="2">Vector normalization (N4)</td><td>Benefit criteria</td><td>n<sub>ij</sub> = <sup>r<sub>ij</sub></sup>&frasl;<sub>&radic;(&sum;<sub>i=1</sub><sup>m</sup> r<sub>ij</sub><sup>2</sup>)</sub></td></tr><tr><td>Cost criteria</td><td>n<sub>ij</sub> = 1 - <sup>r<sub>ij</sub></sup>&frasl;<sub>&radic;(&sum;<sub>i=1</sub><sup>m</sup> r<sub>ij</sub><sup>2</sup>)</sub></td></tr><tr><td rowspan="2">Logarithmic normalization (N5)</td><td>Benefit criteria</td><td>n<sub>ij</sub> = <sup>ln(r<sub>ij</sub>)</sup>&frasl;<sub>ln(&prod;<sub>i=1</sub><sup>m</sup> r<sub>ij</sub>)</sub></td></tr><tr><td>Cost criteria</td><td>n<sub>ij</sub> = <sup>1 - ln(r<sub>ij</sub>)</sup>&frasl;<sub>m - 1</sub></td></tr></tbody></table>

## 4 Assessment Framework for Evaluation of Normalization Techniques

As mentioned above, normalization is the unavoidable step in most MCDM problems to make dimensionless criteria from heterogeneous data. Several studies focused on the effect of the normalization techniques and introduced some metrics that can help decision makers to select the more appropriate technique for using in aggregation/ranking process [1], [5], [8]–[25]. Among these articles some of them are more interesting due to the characteristic of the metric to be used for MCDM methods. For example, Celen [19] (Max-Min, Max and Sum) used consistency conditions to analyze the effects of three normalization techniques and recommended the more suitable one for the TOPSIS method. Furthermore, Charaborty and Yeh [18], [20] discussed the suitability of Vector, Max-Min, Max, and Sum normalization techniques and assessed the best technique using Ranking Consistency Index (RCI). In another study, Mathew et al. [24] presented Max-Min as the best normalization for the weighted aggregated sum product assessment (WASPAS) method using Spearman correlation. Moreover, Jahan [5] proposed the range target-based normalization technique and compared the efficiency of three types of normalization techniques (Non-monotonic, Comprehensive, and Target-based (point and range)) using ANOVA.

As mentioned, several research studies are done related to assessing normalization techniques while there is a need to define a general framework for the most well-known
MCDM methods. The observed gaps motivated us to design and develop an assessment framework to recommend the most proper normalization techniques for more well-known MCDM decision models.

The preliminary framework was proposed in [2], [6], [26], [27], [29] and consists of a number of steps such as calculating RCI, correlation, Standard Deviation, Minkowski distances, and so on. The authors introduced the developed version of the framework that consists of three levels as shown in Fig. 2 to select the best normalization technique for MCDM decision models [29].

<figure><img src="image_4.png" /><figcaption>Fig. 2. Three level of the evaluation framework (adapted from [29])</figcaption></figure>

As Fig. 2 shows, the first level of the developed framework identifies the topology of the input data set. The second level refers to the selection of the normalization techniques from different categories that are classified as linear (Max, Max-Min, and Sum), semi linear (Vector and Target-based) and non-linear (Logarithmic and Fuzzification) techniques. Also, considering the topology of the input data set (from first level) should be taken in this level in order to exclude the normalization techniques which are not fitted with the input data set. Vafaei et al. [6] showed that when the input data contains zero or decimal numbers, the elimination of the Logarithmic and Sum normalization technique is necessary because of producing infinite and undefined normalized values with these techniques (see [6]). Finally, in the third level of the assessment framework, we implement several metrics to analyze the effect of the selected normalization techniques on the MCDM problems and recommend the most appropriate technique. Further information about these levels are explained in the next section.
# 5 Comparison of Normalization Techniques with an Illustrative Example for Analytical Hierarchy Process (AHP) Method

We are going to compare the suitability of the selected normalization techniques when applied to a small illustrative example that is borrowed from [23] for ranking 7 parking sites (alternatives) with respect to a set of criteria to find the best location for parking place using the AHP method. This example consists of 3 criteria (C1, C2, and C3) which are C1= time to park, C2= distance, and C3= size of parking space; also, 7 alternatives (A1, A2, A3,..., and A7) are defined as sites for parking locations. The goal of this illustrative example is to finding the best parking place for the car, C1 and C2 are cost criteria (i.e. the lower values the better) and C3 is the benefic criteria (i.e. the higher values the better).

In the original illustrative example [2], four pairwise comparison matrices were defined (three pairwise comparison matrices for each criterion and one pairwise comparison matrix between criteria) for AHP method and then selected normalization techniques were used to rank the alternatives. Vafaei et al. [2] showed that based on the characteristic of the Logarithmic normalization technique, using this technique for the related illustrative example produces zero or infinite normalized values which is not desirable for AHP method. So, the authors excluded the Logarithmic method and implemented Max, Max-Min, Sum, and Vector normalization techniques to normalized data. Furthermore, the authors also mentioned that "since AHP requires the columns of the pairwise matrices to sum up 1, the techniques: linear max, linear max-min and vector normalization techniques had to be re-normalized with linear sum before being compared."

In this study, we borrowed the normalized values of the decision matrix using four normalization techniques (Max, Max-Min, Sum, Vector) from [2] and implement the developed evaluation framework to assess the suitability of the chosen normalization techniques and recommend the most proper one for the AHP method. For more information about the AHP method and the normalizing process for this illustrative example please see [2]. Table 2 shows the global weights (re-normalized values) and ranking of alternatives for the smart car parking example that are borrowed from [2].

<table><caption>**Table 2:** Global weight (G) and Ranking (R) of alternatives for the smart parking example.</caption><thead><tr><th rowspan="2"></th><th colspan="2">Max</th><th colspan="2">Max-Min</th><th colspan="2">Sum</th><th colspan="2">Vector</th></tr><tr><th>G</th><th>R</th><th>G</th><th>R</th><th>G</th><th>R</th><th>G</th><th>R</th></tr></thead><tbody><tr><td>A1</td><td>0.1972</td><td>2</td><td>0.1925</td><td>2</td><td>0.1505</td><td>4</td><td>0.1693</td><td>2</td></tr><tr><td>A2</td><td>0.0681</td><td>6</td><td>0.0634</td><td>6</td><td>0.0762</td><td>6</td><td>0.1165</td><td>6</td></tr><tr><td>A3</td><td>0.1143</td><td>5</td><td>0.1161</td><td>5</td><td>0.0993</td><td>5</td><td>0.1297</td><td>5</td></tr><tr><td>A4</td><td>0.2469</td><td>1</td><td>0.2658</td><td>1</td><td>0.2876</td><td>1</td><td>0.1755</td><td>1</td></tr><tr><td>A5</td><td>0.0460</td><td>7</td><td>0.0291</td><td>7</td><td>0.0749</td><td>7</td><td>0.1101</td><td>7</td></tr><tr><td>A6</td><td>0.1765</td><td>3</td><td>0.1869</td><td>3</td><td>0.1598</td><td>2</td><td>0.1450</td><td>4</td></tr><tr><td>A7</td><td>0.1509</td><td>4</td><td>0.1462</td><td>4</td><td>0.1517</td><td>3</td><td>0.1538</td><td>3</td></tr></tbody></table>

Table 2 also shows that using different normalization techniques caused different global weights and different ranks for alternatives. It is not possible to select the best rank for the decision problem just by observing the results. So, we applied the
evaluation framework (Fig. 2) to assess the normalization techniques and recommend the most proper one for AHP method.

The first level of the assessment framework (Fig. 2) indicates the topology of the input data sets. As mentioned in [2], input data in AHP are defined as pairwise comparison matrices using a [1-9] scale (corresponding to semantic interpretations; e.g. A1 is more important than A2 due to a criterion). So, considering the characteristic of input data sets in AHP, we skip this level and proceed to the next levels. In the second level, we should select normalization techniques from three categories (Linear, semi-linear, and non-linear). We initially selected Max, Max-Min, and Sum normalization techniques from the linear group the Vector technique from the semi-linear group, and the Logarithmic one from non-linear group. However, taking into account the reasons mentioned above, we excluded the Logarithmic technique from selected normalization techniques and continued the evaluation with four normalization techniques (Max, Max-Min, Sum, and Vector).

In the third level of the evaluation framework (Fig. 2), we deal with four types of metrics including outlier detection; measures of location; measures of scale; and comparison methods. Outlier detection and measure of location are applied to the input data set [29]. As mentioned above, the input data for the AHP method are defined as pairwise comparison matrices in the scale of [1-9] and detecting outliers and measuring location are meaningless for this method. So, we omitted all metrics which are applicable just by input data (such as Outlier detection, Measure of location, ANOVA, MSE, ...) and proceed the evaluation with some other metrics that work with normalized values and rank of alternatives such as calculation Minkowski distances, Standard Deviation, Mean Ks values, and Ranking Consistency Index (RCI) from comparison metrics. We calculated Minkowski distances (Manhattan, Euclidean, and Chebyshev), as well as Standard deviation (STD) and Mean Ks values (the average of Pearson correlation) from Table 2 data. For more information about calculating Minkowski distances and Standard deviation please see [6] and for calculation of Pearson correlation and Mean Ks value and Ranking Consistency Index (RCI) please see [27]. Table 3 shows the results of the above metrics using the smart park illustrative example with the AHP method.

<table><caption>**Table 3:** Results of applied metrics on illustrative example for AHP method</caption><thead><tr><th></th><th>Manhattan</th><th>Euclidean</th><th>Chebishev</th><th>STD</th><th>Mean Ks</th><th>RCI</th></tr></thead><tbody><tr><th>Max</th><td>1.8468</td><td>0.4642</td><td>0.2010</td><td>0.0716</td><td>0.9606</td><td>18</td></tr><tr><th>Max-Min</th><td>2.0783</td><td>0.5258</td><td>0.2367</td><td>0.0811</td><td>0.9564</td><td>18</td></tr><tr><th>Sum</th><td>1.7155</td><td>0.4758</td><td>0.2127</td><td>0.0734</td><td>0.9029</td><td>16.3333</td></tr><tr><th>Vector</th><td>0.6520</td><td>0.1638</td><td>0.0655</td><td>0.0253</td><td>0.9263</td><td>17.6667</td></tr></tbody></table>

In order to sort the obtained results from different metrics, their interpretation regarding each metric is needed. So, based on the previous studies done by the authors [6], [27], and [29], their interpretations are as follow: for Minkowski distances (Manhattan, Euclidean, and Chebyshev), STD, Mean Ks, and RCI the higher values are better. For more information and the logics about this interpretation please see ([6], [27], and [29]). As shown in Table 3 and Table 4 each metric ranked normalization
techniques differently. So, still it is impossible to say that which normalization
technique would be the best choice for the AHP method. Vafaei et al. [29] suggested to
use plurality voting from social choice method [28] to find the best normalization
technique. This method selects the alternative with the largest number of times that has
the first rank/order in the decision problems. Therefore, we implemented the plurality
voting method for each normalization technique with respect to the different metrics.
The results are shown in Table 4.

<table><caption>**Table 4:** Ordering of normalization techniques with respect to the metrics and using plurality voting</caption><thead><tr><th></th><th>Manhattan</th><th>Euclidean</th><th>Chebishev</th><th>STD</th><th>Mean Ks</th><th>RCI</th><th>Plurality Voting</th></tr></thead><tbody><tr><th>Max</th><td>2</td><td>3</td><td>3</td><td>3</td><td>1</td><td>1</td><td>2</td></tr><tr><th>Max-Min</th><td>1</td><td>1</td><td>1</td><td>1</td><td>2</td><td>1</td><td>5</td></tr><tr><th>Sum</th><td>3</td><td>2</td><td>2</td><td>2</td><td>4</td><td>4</td><td>0</td></tr><tr><th>Vector</th><td>4</td><td>4</td><td>4</td><td>4</td><td>3</td><td>3</td><td>0</td></tr></tbody></table>

Summing up, using the plurality voting method showed that the best normalization
technique for the AHP method is the Max-Min and the second best is the Max
normalization technique. Comparing these results with the preliminary study conducted
in [2] proves the robustness of the evaluation assessment and implemented metrics in
order to help decision makers and recommend the best normalization technique for
different MCDM methods.

## 6 Conclusions

The objective of this article was to recommend the most suitable normalization
technique for the AHP method. The behaviour of four normalization techniques (Max,
Max-Min, Sum, Vector) were analysed and an assessment framework was applied for
selecting the normalization technique. To clarify the approaches, a smart car parking
example was used. The evaluation results showed that the Max-Min is the best
technique for the AHP method and that the Max normalization technique is the second
best for the mentioned illustrative example. We propose the use of the suggested
assessment framework and mentioned metrics to select adequate normalization
techniques for the case studies (decision problems) involving ranking of alternatives as
results may change for other case studies.

In order to generalize our results for selecting the most suitable normalization
technique, a simulation with multiple representative scenarios will be performed.
Moreover, we plan to continue developing the assessment framework by adding more
metrics to ensure the robustness of the results. Testing and validating this framework
with several real-world case studies and adding more normalization techniques are also
ongoing and future work.

**Acknowledgements.** This work was funded in part by the Center of Technology and Systems (CTS) and the Portuguese Foundation for Science and Technology (FCT)
through the Strategic Program UID/EEA/00066/2019 and UIDB/00066/2020.

References

1. A. Jahan and K. L. Edwards, "A state-of-the-art survey on the influence of normalization techniques in ranking: Improving the materials selection process in engineering design," *Mater. Des.*, vol. 65, pp. 335-342, (Jan. 2015), DOI: 10.1016/j.matdes.2014.09.022.

2. N. Vafaei, R. A. Ribeiro, and L. M. Camarinha-Matos, "Normalization Techniques for Multi-Criteria Decision Making: Analytical Hierarchy Process Case Study," in *DoCEIS 2016. IFIP Advances in Information and Communication Technology Technological Innovation for Cyber-Physical Systems*, L. M. Camarinha-Matos, A. Falcao, N. Vafaei, and S. Najdi, Eds. Lisbon, Portugal: Springer, Cham, (2016), pp. 261-269, DOI:10.1007/978-3-319-31165-4_26.

3. L. M. Branscomb, "Technological Innovation," *Int. Encycl. Soc. Behav. Sci.*, pp. 15498-15502, (Jan. 2001), DOI: 10.1016/B0-08-043076-7/03208-3.

4. L. M. Camarinha-Matos, S. Tomic, and P. Graça, Eds., *Technological Innovation for the Internet of Things*, vol. 394. Springer, Berlin, Heidelberg: Springer Berlin Heidelberg, (2013), DOI: 10.1007/978-3-642-37291-9.

5. A. Jahan, "DEVELOPING WASPAS-RTB METHOD FOR RANGE TARGET-BASED CRITERIA: TOWARD SELECTION FOR ROBUST DESIGN," *Technol. Econ. Dev. Econ.*, vol. 24, no. 4, pp. 1362-1387, (Jul. 2018), DOI: 10.3846/20294913.2017.1295288.

6. N. Vafaei, R. A. Ribeiro, L. M. Camarinha-Matos, and L. R. Valera, "Normalization techniques for collaborative networks," *Kybernetes*, p. K-09-2018-0476, (Jun. 2019), DOI: 10.1108/K-09-2018-0476.

7. S. Opricovic and G.-H. Tzeng, "Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS," *Eur. J. Oper. Res.*, vol. 156, no. 2, pp. 445-455, (Jul. 2004), DOI: 10.1016/S0377-2217(03)00020-1.

8. A. S. Milani, A. Shanian, R. Madoliat, and J. A. Nemes, "The effect of normalization norms in multiple attribute decision making models: a case study in gear material selection," *Struct. Multidiscip. Optim.*, vol. 29, no. 4, pp. 312-318, (Apr. 2005), DOI: 10.1007/s00158-004-0473-1.

9. A. Chawade, E. Alexandersson, and F. Levander, "Normalizer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets," *J. Proteome Res.*, vol. 13, no. 6, pp. 3114-3120, (Jun. 2014), DOI: 10.1021/pr401264n.

10. M. Eftekhary, P. Gholami, S. Safari, and M. Shojaee, "Ranking Normalization Methods for Improving the Accuracy of SVM Algorithm by DEA Method," *Modern Applied Science*, vol. 6, no. 10. p. 26, (26-Sep-2012).

11. A. Jahan, K. L. Edwards, and M. Bahraminasab, *Multi-criteria decision analysis for supporting the selection of engineering materials in product design*, 2nd edn. Butterworth-Heinemann, Oxford, (2016).

12. A. Krylovas, N. Kosareva, and E. Zavadskas, "Scheme for statistical analysis of some parametric normalization classes," *Int. J. Comput. Commun. Control*, vol. 13, no. 6, pp. 972-987, (2018).

13. D. Migilinskas and L. Ustinovichius, "Normalisation in the selection of construction alternatives," *Int. J. Manag. Decis. Mak.*, vol. 8, no. 5/6, p. 623, (2007), DOI: 10.1504/IJMDM.2007.013422.

14. S. C. Nayak, B. B. Misra, and H. S. Behera, "Impact of Data Normalization on Stock Index Forecasting," *Int. J. Comput. Inf. Syst. Ind. Manag. Appl.*, vol. 6, pp. 257-269, (2014).
[15] D. M. Pavlicic, "Normalization affects the results of MADM methods," *Yugosl. J. Oper. Res.*, vol. 11, no. 2, pp. 251-265, (2001).

[16] M. Yazdani, A. Jahan, and E. K. Zavadskas, "ANALYSIS IN MATERIAL SELECTION: INFLUENCE OF NORMALIZATION TOOLS ON COPRAS-G," *Econ. Comput. Econ. Cybern. Stud. Res.*, vol. 51, no. 1, (2017).

[17] E. K. Zavadskas, A. Zakarevicius, J. Antucheviciene, E. Kazimieras ZAVADSKAS, A. Zakarevicius, and J. Antucheviciene, "Evaluation of Ranking Accuracy in Multi-Criteria Decisions," *Informatica*, vol. 17, no. 4, pp. 601-618, (2006).

[18] S. Chakraborty and C.-H. Yeh, "A simulation comparison of normalization procedures for TOPSIS," in *2009 International Conference on Computers & Industrial Engineering*, (2009), pp. 1815-1820, Troyes: IEEE, DOI: 10.1109/ICCIE.2009.5223811.

[19] A. Celen, "Comparative Analysis of Normalization Procedures in TOPSIS Method: With an Application to Turkish Deposit Banking Market," *INFORMATICA*, vol. 25, no. 2, pp. 185-208, (2014).

[20] S. Chakraborty and C.-H. Yeh, "A simulation based comparative study of normalization procedures in multiattribute decision making," in *AIKED'07 Proceedings of the 6th Conference on 6th WSEAS Int. Conf. on Artificial Intelligence, Knowledge Engineering and Data Bases - Volume 6*, (2007), pp. 102-109, World Scientific and Engineering Academy and Society (WSEAS) Stevens Point, Wisconsin, USA.

[21] P. Chatterjee and S. Chakraborty, "Investigating the Effect of Normalization Norms in Flexible Manufacturing System Selection Using Multi-Criteria Decision-Making Methods," *J. Eng. Sci. Technol.*, vol. 7, no. 3, pp. 141-150, (2014).

[22] A. Jahan and E. K. Zavadskas, "ELECTRE-IDAT for design decision-making problems with interval data and target-based criteria," *Soft Comput.*, vol. 23, no. 1, pp. 129-143, (Jan. 2019), DOI: 10.1007/s00500-018-3501-6.

[23] J. Papathanasiou, N. Ploskas, T. Bournaris, and B. Manos, "A Decision Support System for Multiple Criteria Alternative Ranking Using TOPSIS and VIKOR: A Case Study on Social Sustainability in Agriculture," Springer, Cham, (2016), pp. 3-15, DOI:10.1007/978-3-319-32877-5_1.

[24] M. Mathew, S. Sahu, and A. K. Upadhyay, "Effect Of Normalization Techniques In Robot Selection Using Weighted Aggregated Sum Product Assessment," *Int. J. Innov. Res. Adv. Stud.*, vol. 4, no. 2, (2017).

[25] S. Baghla and S. Bansal, "Effect of normalization techniques in VIKOR method for network selection in heterogeneous networks," in *2014 IEEE International Conference on Computational Intelligence and Computing Research*, (2014), pp. 1-6, IEEE, DOI: 10.1109/ICCIC.2014.7238357.

[26] N. Vafaei, R. A. Ribeiro, and L. M. Camarinha-Matos, "Selection of Normalization Technique for Weighted Average Multi-criteria Decision Making," in *Technological Innovation for Resilient Systems. DoCEIS 2018. IFIP AICT, vol 521.*, Springer, Cham, (2018), pp. 43-52, DOI:10.1007/978-3-319-78574-5_4.

[27] N. Vafaei, R. A. Ribeiro, and L. M. Camarinha-Matos, "Data normalisation techniques in decision making: case study with TOPSIS method," *Int. J. Inf. Decis. Sci.*, vol. 10, no. 1, p. 19, (2018), DOI: 10.1504/IJIDS.2018.090667.

[28] A. d'Angelo, A. Eskandari, and F. Szidarovszky, "Social choice procedures in water-resource management," *J. Environ. Manage.*, vol. 52, no. 3, pp. 203-210, (Mar. 1998), DOI: 10.1006/jema.1997.0156.

[29] N. Vafaei, R. A. Ribeiro, and L. M. Camarinha-Matos, "Comparison of Normalization Techniques on Data Sets with Outliers," *Computer Science and Information Systems* (2020), [To appear]