# An Examination of Ranking Quality for Simulated Pairwise Judgments in relation to Performance of the Selected Consistency Measure

Authors: Paul Thaddeus Kazibudzki
Journal: Advances in Operations Research
 (2019)

---

## Abstract

An overview of current debates and contemporary research devoted to modeling decision making processes and their facilitation directs attention to techniques based on pairwise judgments. At the core of these techniques are various judgment consistency measures which, in a sense, control the prioritization process which leads to the establishment of decision makers’ unknown preferences. If judgments expressed by decision makers were perfectly consistent (cardinally transitive), all available prioritization techniques would deliver the same solution. However, human judgments are consistently inconsistent, as it were; thus the preference estimation quality significantly varies. The scale of these variations depends, among others, on the chosen consistency measure of pairwise judgments. That is why it seems important to examine relations among various consistency measures and the preferences estimation quality. This research reveals that there are consistency measures whose performance may confuse decision makers with the quality of their ranking outcome. Thus, it introduces a measure which is directly related to the quality of the preferences estimation process. The main problem of the research is studied via Monte Carlo simulations executed inWolfram Mathematica Software. The research results argue that although the performance of examined consistency measures deviates from the exemplary ones in relation to the estimation quality of decision makers preferences, solutions proposed in this paper can significantly improve that quality.


---

## Body

## 1. Introduction

Overwhelming scientific evidence indicates that the unaided human brain is simply not capable of simultaneous analysis of many different, competing factors and then synthesizing the results for the purpose of making a rational decision. Indeed, numerous psychological experiments, e.g., [1], including the well-known Miller [2] study put forth the notion that humans are not capable of dealing accurately with more than about seven (±2) things at a time (the human brain is limited in its short term memory capacity, its discrimination ability, and its range of perception).Humans learn about anything by two means: the first involves examining and studying some phenomenon from the perspective of its various properties and then synthesizing findings and drawing conclusions; the second entails studying some phenomenon in relation to other similar phenomena and relating them by making comparisons. The latter method leads directly to the essence of the matter, i.e., judgments regarding a phenomenon. Judgments can be relative or absolute. An absolute judgment is the relation between a single stimulus and some information held in short or long term memory. A relative judgment, on the other hand [3], is defined as identification of some relation between two stimuli both present to the observer. Certainly humans can make much better relative than absolute judgments. Thus, a pairwise comparisons method was proposed in order to facilitate the process of relative judgments.Some authors proclaim [4, 5] that this method dates back to the beginning of the 20th century and was firstly applied by Thurstone [6]; however, its first scientific applications can be found in Fechner [7]. In reality, the method itself is much older and its idea goes back to Ramon Lull who lived in the end of 13th century. It is a fact that its popularity comes from an influential paper of Marquis de Condorcet [8]; see, e.g., [9, 10], who used this method in the election process where voters rank candidates based on their preference. It has been perfected in many papers, e.g., [4, 11–20].Also, the method is the core of a decision making methodology—the Analytic Hierarchy Process (AHP), developed by Saaty [21]. Although it is criticized (see [22–25, 25–28]) it is also continuously perfected and as such very often validated [29] and developed from the perspective of its utility (see [5, 30–37]). Thus, considering the stable increase of the AHP application (see, e.g., [38–44]) as well as its steady position among other theories of choice, e.g., [31–37, 45–47], the problematic issues of AHP should be continuously examined and questions which arise thereof should be simultaneously addressed.The fundamental objective of this research is to determine the answer to the question:Does the reduction of PCM inconsistency lead to improvement of the priority ratios estimation process quality?


## 2. Background

The AHP uses the hierarchical structure of the decision problem, pairwise relative comparisons of the elements in the hierarchy, and a series of redundant judgments which enable measurement of judgment consistency. To make a proposed solution possible, i.e., derive ratio scale priorities on the basis of verbal judgments, a scale is utilized to evaluate the preferences for each pair of items. Probably, the most known scales are Saaty’s numerical scale which comprises integers, and their reciprocals, from one (equivalent to the verbal judgment, “equally preferred”) to nine (equivalent to the verbal judgment, “extremely preferred”), and a geometric scale which usually consists of the numbers computed in accordance with the formulaf(n)=cn/2forn∈-8,…,8∧n∈I wherec denotes its parameter which commonly equals 2. Other arbitrarily defined numerical scales are also available, e.g., composed of arbitrary integers from one ton and their reciprocals.The key issue in AHP is priority ranking on the basis of true or approximate weights, i.e., judgments. If the relative weights of a set of activities are known, they can be expressed as a Pairwise Comparison Matrix (PCM):A(w)=(wi/wj),  i, j=1,…,n. PCM in the AHP reflects decision makers’ preferences (their relative judgments) about considered activities (criteria, scenarios, players, alternatives, etc.). On the basis of A(w), it is possible to derive true weights; i.e., decision makers priority ratios wi, where:i=1,…, n, are selected to be positive and normalized to unity: ∑inwi=1. For uniformity,  w is referred hereafter to its normalized form. If the elements of a matrix A(w) satisfy the condition wij=1/wji for alli, j=1,…,nthen matrix A(w) is calledreciprocal. If the elements of a matrix A(w) satisfy the condition wikwkj=wij for alli, j, k=1,…,n, and the matrix isreciprocal, then it is calledconsistent orcardinally transitive.Certainly, in real life situations when AHP is utilized, there is noA(w) which would reflect weights given by the vector of priority ratios. As was stated earlier, the human mind is not a reliable measurement device. Assignments, such as “Compare – applying a given ratio scale – your feelings concerning alternative 1 versus alternative 2”, do not produce accurate outcomes. Thus, A(w) is not established but only its estimate A(x) containing intuitive judgments, more or less close to A(w) in accordance with experience, skills, specific knowledge, personal taste, and even temporary mood or overall disposition. In such case, consistency property does not hold and the relation between elements of A(x) and A(w) can be expressed as xij=eijwij where eij is a perturbation factor fluctuating near unity. In the statistical approach,eij reflects a realization of a random variable with a given probability distribution.Besides the prioritization procedure (PP) proposed by the creator of AHP, right principal eigenvector method (REV), there are alternative PPs devised to cope with the priority ratios estimation problem; their demonstrative review can be found, e.g., in [47]. Many of them are optimization based and seek a vector  w, as a solution of the minimization problem given by the formula min⁡DAx,Aw subject to some assigned constraints, such as positive coefficients and normalization condition. Because the distance function D  measures an interval between matrices A(x)   and A(w), different definitions of the distance function lead to various prioritization concepts and prioritization results. As an example, eighteen PPs in [48] are described and compared for ranking purposes although some authors suggest there are only fifteen that are different. Furthermore, since the publication of the above-mentioned article, a few additional procedures have been introduced to the literature; see, e.g., [43, 49–51]. Probably the most popular alternative to the REV is the Logarithmic Least Squares Method (LLSM) developed by Crawford and Williams [26, 52]. It is given by the following formula:(1)wLLSM=min⁡∑i=1n∑j=1nln2⁡aijwjwiThe LLSM solution also has the following closed form and is given by the normalized products of the elements in each row:(2)wiLLSM=∏j=1naij1/n∑i=1n∏j=1naij1/nThus, it is also known as the geometric mean method and it is utilized in this research which strives to improve the reliability of the pairwise comparisons process which is also the core element of AHP.


## 3. Problem and Research Methodology

There are several PCM consistency measures (PCM-CMs) provided in literature called consistency or inconsistency indices (CIs). The most popular one is the PCM-CM proposed by Saaty [21]. He proposed his CI as determined by the formula(3)CIREV=λmax-nn-1wheren indicates the number of alternatives within the particular PCM and λmax denotes its maximal eigenvalue. The significant disadvantage of the PCM-CM is the fact that it can operate exclusively with reciprocal PCMs. In the case of nonreciprocal PCMs, this measure is useless (its values are meaningless) which in consequence seriously diminishes the value of the whole approach; see, e.g., [33]. It was also recently found to be incorrect; see, e.g., [4, 10, 53, 54].However, as mentioned earlier, there are a number of additional PCM-CMs. Some of them, as in the case ofCIREV, originate from the PPs devised for the purpose of the priority ratios estimation process. Their distinct feature is the fact that all of them can operate equally efficiently in conditions where reciprocal and nonreciprocal PCMs are accepted. Probably the most known example from that set of propositions is PCM-CM proposed by Aguaron and Moreno-Jimenez [55] given by the following formula:(4)CILLSM=2n-1n-2∑i<jlog2⁡aijwjwiNoticeably, there are a few definitions of PCM-CMs which are not connected with any PP and are devised on the basis of the PCM consistency definition. Koczkodaj’s idea [56] attracts attention and is the first to be scrutinized. Koczkodaj’s PCM-CM is grounded in his concept of triad consistency.In order to clarify this, for any three distinguished decision alternatives A1, A2, and A3, there are three meaningful priority ratios, i.e.,aij,ajk, andaik, which have their different locations in a particular  A=aijn×n. For some differenti≤n, j≤n, andk≤n, the tuple (aij,ajk,aik) is called atriad. If the matrix  A=aijn×n is consistent, thenaijajk =aik for all triads.In consequence, either of the equations1-aik/aijajk=0 and 1-aijajk/aik=0 have to be true. Taking the above into consideration, Koczkodaj proposed his measure for triad inconsistency by the following formula:(5)TIaij,ajk,aik=min⁡1-aikaijajk,1-aijajkaikFollowing his idea, he then proposed the following PCM-CM of any reciprocal PCM=A:(6)KA=maxi<j<k⁡TIpaij,ajk,aikp=1,…,n3where the maximum value forK(A) is taken from the set of all possible triads in the upper triangle of a given PCM.On the basis of Koczkodaj’s idea of triad inconsistency, Grzybowski [5] presented his PCM-CM determined by the following formula:(7)ATIA=1n3∑i=1n-2∑j=i+1n-1∑k=j+1nmin⁡1-aikaijajk,1-aijajkaikFinally, following the idea thatlnaik/aijajk=-lnaijajk/aik, Kazibudzki [57] redefined triad inconsistency and proposed– Two formulae for its measurement:(8)LTI1aij,ajk,aik=ln⁡aijajkaik(9)LTI2aij,ajk,aik=ln2⁡aijajkaik– One meaningful formula for PCM-CM: (10)ALTIxA=1n3∑i=1n-2∑j=i+1n-1∑k=j+1nLTIxaij,ajk,aikwherexdenotes the formula for triad inconsistency measurement, i.e.,LTI1 orLTI2.It behooves us to mention thatALTI(A) can be calculated on the basis of triads from the upper triangle of the given PCM when it is reciprocal or all triads within the given PCM when it is nonreciprocal.As was already stated earlier, the fundamental question which should be asked by researchers who deal with the problem of priority ratios estimation quality in relation to a PCM consistency measure is as follows:Does the reduction of PCM inconsistency lead to improvement of the priority ratios estimation process quality?The common reason why one strives to improve the consistency of the PCM, when it seems unsatisfactory, is to increase the quality of the priority ratios estimation process. However, the above question remains open and the answer to it is not evident. Even the creator of AHP stated once that improving consistency does not mean getting an answer closer to the “real” life solution [21]. It can be illustrated in the following example.Considered is the true PV (denoting true weights of examined alternatives), i.e.,w=7/20, 1/4, 1/4, 3/20 and A(w) derived from that PV, which can be presented as follows:(11)
A
w
=
1
7
5
7
5
7
3
5
7
1
1
5
3
5
7
1
1
5
3
3
7
3
5
3
5
1Then two PCMs are considered, i.e.,R(x) and A(x) produced by a hypothetical decision maker (DM). It is assumed that DM is very trustworthy and is able to express judgments very precisely at the same time being still somehow limited by the necessity of expressing judgments on a scale (the example utilizes Saaty’s scale). In the first scenario, entries of A(w) are rounded to Saaty’s scale and the entries are made reciprocal (a principal condition for a PCM in the AHP) producing(12)Rx=1112111211121212121In the second scenario, only entries of A(w) are rounded to Saaty’s scale (nonreciprocal case) producing(13)
A
x
=
1
1
1
2
1
2
1
1
2
1
2
1
1
2
1
2
1
2
1
2
1It should be noted thatR(x) is perfectly consistent and A(x) is not. Tables 1 and 2 present selected values of the PPs related PCM-CMs (that is, CIREV and CILLSM) forR(x) and A(x) together with PVs derived from R(x)   and A(x); Mean Absolute Errors (MAEs), formula (14), among  w∗(PP) and  w for the case; Spearman Rank Correlation Coefficients (SRCs) among   w∗(PP) and w for the case.(14)MAEw∗PP,w=1n∑i=1nwi-wi∗PPTable 1
Values of the PCM-CMs denoted as CI(PP) for R(x) and proposed characteristics of PVs estimates (∗) quality in relation to the true PV for the case.
PP Estimates Performance measures CI(PP) MAE SRC REV 0.285714,0.285714,0.285714,0.142857 T 0.0 0.0357143 0.8164966 LLSM 0.285714,0.285714,0.285714,0.142857 T 0.0 0.0357143 0.8164966 ( ∗ ): derived from R(x) with application of a particular PP.Table 2
Values of the PCM-CMs denoted as CI(PP) for A(x) and proposed characteristics of PVs estimates (∗) quality in relation to the true PV for the case.
PP Estimates Performance measures CI(PP) MAE SRC REV 0.309401,0.267949,0.267949,0.154701 T –0.0893164 0.0202995 1 LLSM 0.314288,0.264284,0.264284,0.157144 T 0.0400378 0.0178559 1 ( ∗ ): derived from A(x) with application of a particular PP.Surprisingly, a very interesting phenomenon can be noted on the basis of information provided in Tables1 and 2. The nonreciprocal version of the analyzed PCM contains nonzero values for the selected PCM-CMs. In cases similar to this example, the value of Saaty’s PCM-CM always becomes negative which makes it inexplicable and in consequence useless under such circumstances (as already mentioned earlier). The other exemplary measure taken into consideration is positive and higher values than zero which indicates that particular PCM is inconsistent. On the basis of the same indicators in the case of the reciprocal version of the analyzed PCM, its perfect consistency is apparent because all selected PCM-CMs in this case are equal to zero. However, the estimation precision measures (MAE and SRC), i.e., characteristics of the particular PV estimation quality, indicate something quite opposite. Surprisingly, smaller values of MAEs are apparent, as are perfect correlations of ranks between estimated and genuine PV for nonreciprocal version of the analyzed PCM. Certainly, this conclusion concerns all analyzed PPs and it is very true in the situation when the particular PCM is less consistent (on the basis of selected exemplary PCM-CMs). It remains to be mentioned that the inconsistency problem (hence errors) does not exist for simplified pairwise comparisons [53].Taking into consideration only such a trivial example like the one above, it becomes apparent that the relation between performance of a consistency measure and the quality of the priority ratios estimation process is of great importance. That is why to examine the phenomenon further to improve the quality of the pairwise comparisons based prioritization process was decided. Thus, the simulation framework for this purpose was adopted from [5, 57] as the only way to examine the said phenomena through computer simulations. The simulation algorithm SAK is comprised of the following phases.Phase 1.
Randomly generate a priority vectorw=w1,…,wnT of assigned size [n x 1] and related perfect PCM(w)=K(w).Phase 2.
Randomly select an elementwxy forx<y ofK(w) and replace it with wxyeB whereeB is a relatively significant error, randomly drawn (uniformdistribution) from the interval eB∈[2;4]. Errors of that magnitude are basically considered “significant”; see, e.g., [50, 58].Phase 3.
For each other elementwij,i<j≤n, select randomly a valueeij for the relatively small error in accordance with the given probability distribution π (applied in equal proportions asgamma,log-normal,truncated normal, anduniform distribution) and replace the element wij with the element wijeij whereeij is randomly drawn (uniformdistribution) from the interval eij∈[0,5;1,5].Phase 4.
Round all values ofwijeij fori<jofK(w) to the nearest value of a considered scale.Phase 5.
Replace all elementswij fori>j ofK(w) with 1/wij.Phase 6.
After all replacements are done, return the value of the examined index as well as the estimate of the vectorw denoted as  w∗(PP) with application of assigned prioritization procedure. Then return the mean absolute error (MAE) and mean relative error (MRE) (formula (15)) between  w and  w∗(PP). Remember values computed in this phase as one record.(15)MREw∗PP,w=1n∑i=1nwi-wi∗PPwiPhase 7.
RepeatPhases from Phase 2 to Phase 6Nn times.Phase 8.
RepeatPhases from Phase 1 to Phase 7Nm times.Phase 9.
Saveall records as one database file.The above algorithm allows examining the performance of the selected CM in relation to the quality of the priority ratios estimation process. Its framework resembles steps scrutinized in the example provided earlier in this paper and was thoroughly described in [5]. Thus, for brevity, it will not be analyzed in detail herein.For formality, all parameters of the applied PDs,gamma,log-normal,truncated normal, anduniform, in the simulation algorithm SAK are set in such a way that the expected value EV(eij)=1. The simulation begins fromn=4. Simulations forn=3 are not interesting due to direct interrelation of considered PCM consistency measures; see, e.g., [58, 59]. For the sake of objectivity, the simulation data is gathered in the following way: all values of selected consistency measures are split into 15 separate sets designated by the quantilesQ of orderp from 1/15 to 14/15. The 15 intervals are defined as the first is from 0 to the quantile of order 1/15, i.e., VRCM1=[0, Q1/15), where VRCM representsa Value Range of the Selected PCM Consistency Measure; the second denotes VRCM2=[Q1/15,Q2/15), and so on… to the last one which starts from the quantile of order 14/15 and goes on to infinity, i.e., VRCM15=Q14/15,∞. The following variables are examined: mean VRCMn, average MAE within VRCMn between  w and  w∗(PP), MAE quantiles of the following orders, 0.05, 0.1, 0.5, 0.9, and 0.95, and relations between all of them. The application of the rounding procedure was also assumed which in this research operates according to Saaty’s scale. Lastly, the scenario takes into account the compulsory assumption in conventional AHP applications, i.e., the PCM reciprocity condition. The outcome of the simulation program is presented for the most popular PP=LLSM, and the most attractive PCM-CM=K(A). It must be emphasized that other PPs and PCM-CMs were also examined but results with their application will not be presented in this research paper because this is beyond its scope. The results are based onNn=20, andNm=500, i.e., 10,000 cases.


## 4. Results and Discussion

One could assume that MAE and MRE quantiles of any order should monotonically increase concurrently with the growth of the selected PCM-CM, e.g., VRCM index. The same relation should occur for meanVRCMn and average MAE and MRE for VRCMn. The results of the proposed simulation framework or any other similar simulation scenario which would contradict such a pertinent relationship would unequivocally lead to the conclusion that the examined PCM-CM is not a good indicator of the priority ratios estimation process quality and may mislead in further actions towards acceptance or rejection of the derived priority ratios vector.An examination of the research problem depicts Figures1 and 2 which present the performance of KoczkodajK(A) (Plots A–H). In each case hereafter a horizontal axis represents values of examined PCM-CM, and a vertical axis denotes particular estimation errors.Figure 1
Performance ofK(A) as PCM-CM. The plots present the relation between mean values ofK(A) within a given interval (VRCMn) and values of MAEs and quantiles of a given order for MAEs distribution concerning estimated and true PV for the case. The results are generated with application of LLSM as the PP andgeometric scale, i.e., f(n)=cn/2forn∈-8,…,8∧n∈I as the preference scale with its most popular parameterc=2. Plots are based on 10,000 random reciprocal PCMs forn=4 andn=6. MSRC denotes Mean Spearman Rank Correlation Coefficient between analyzed variables and describes their relation’s strength.Figure 2
Performance ofK(A) as PCM-CM. The plots present the relation between mean values ofK(A) within a given interval (VRCMn) and values of MREs and quantiles of a given order for MREs distribution concerning estimated and true PV for the case. The results are generated with application of LLSM as the PP andgeometric scale, i.e., f(n)=cn/2forn∈-8,…,8∧n∈I as the preference scale with its most popular parameterc=2. Plots are based on 10,000 random reciprocal PCMs forn=4 andn=6. MSRC denotes Mean Spearman Rank Correlation Coefficient between analyzed variables and describes their relation’s strength.Noticeably, when the quality of priority ratios estimation in a pairwise comparisons based process is taken into consideration, the presented relations indicate that the performance of selected PCM-CM varies from the assumption presented earlier. The phenomenon was thus far only examined in [5]. It may seem disturbing because the relation between priority ratios estimation errors and analyzed PCM-CM indicates that the analyzed index may sometimes misinform DMs about their judgments’ quality, affecting the vector of priorities which best converge with the true vector.As seen similarly in the example provided earlier in this paper (Tables1 and 2), taking the particular index as the measure of PCM consistency, especially when errors are small and inconsistency is not negligible, one can expect both, i.e., the improvement of priority ratios estimation quality (increase of the estimation accuracy) together with the increase of the particular CI (decrease of PCM consistency) and, inversely, the deterioration of priority ratios estimation quality (decrease of the estimation accuracy) together with the descent of the particular CI (improvement of PCM consistency).This problem seems very troublesome especially when differences among derived priority ratios are insignificant. Then, it may occur that a ranking order of alternatives in the estimated priority vector can drastically differ from the true one because of estimation errors. From that perspective, a necessity of controlling that issue seems paramount. In order to evaluate that problem more thoroughly, detailed statistical characteristics for the examinedK(A) are provided in Tables 3–6. It can be concluded that the performance of the presented PCM-CM varies (compare, e.g., [9]).Table 3
Performance of theK(A) index. Statistical characteristics of the MAEs distribution in relation to various VRCMi for i=1,…,15 of K(A) values. The results were generated for n=4 on the basis of SA|K| as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and Saaty’s preference scale.
i V R C M i for K(A) MeanK(A) in VRCMi p–quantiles of MAEs among w and w∗(LLSM) Average MAEs amongw and w∗(LLSM) p = 0.05 p = 0.1 p = 0.5 p = 0.9 p = 0.95 1 [ 0.00,0.293 ) 0.123175 0.003832 0.005244 0.013085 0.031933 0.044036 0.017332 2 [ 0.293,0.343 ) 0.298548 0.004178 0.005586 0.014531 0.039371 0.057014 0.020352 3 [ 0.343,0.394 ) 0.367621 0.007113 0.009216 0.020061 0.055438 0.077366 0.027509 4 [ 0.394,0.444 ) 0.418743 0.008836 0.011492 0.024425 0.068702 0.089414 0.033247 5 [ 0.444,0.494 ) 0.468803 0.010929 0.014093 0.031328 0.080087 0.097035 0.040523 6 [ 0.494,0.544 ) 0.505368 0.006643 0.009028 0.025446 0.076131 0.099659 0.035366 7 [ 0.544,0.595 ) 0.570415 0.014658 0.019158 0.044794 0.086016 0.104479 0.049809 8 [ 0.595,0.645 ) 0.621502 0.018761 0.023914 0.047224 0.082449 0.099615 0.051410 9 [ 0.645,0.695 ) 0.660045 0.012166 0.016648 0.042915 0.089487 0.109722 0.048732 10 [ 0.695,0.746 ) 0.721332 0.020838 0.025263 0.046327 0.085889 0.101378 0.051714 11 [ 0.746,0.796 ) 0.765848 0.017353 0.022895 0.049089 0.093422 0.110678 0.054968 12 [ 0.796,0.846 ) 0.822092 0.020521 0.026287 0.054130 0.093059 0.109786 0.058010 13 [ 0.846,0.897 ) 0.871055 0.020469 0.026452 0.053883 0.095374 0.114356 0.058507 14 [ 0.897,0.947 ) 0.921011 0.019656 0.026032 0.056166 0.111384 0.132945 0.063610 15 [ 0.947 , o o ) 0.969329 0.020548 0.027742 0.070211 0.142947 0.168917 0.078853Table 4
Performance of theK(A) index. Statistical characteristics of the MREs distribution in relation to various VRCMi for i=1,…,15 of K(A) values. The results were generated for n=4 on the basis of SAK as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and Saaty’s preference scale.
i V R C M i for K(A) MeanK(A) in VRCMi p–quantiles of MREs among w and w∗(LLSM) Average MREs amongw and w∗(LLSM) p = 0.05 p = 0.1 p = 0.5 p = 0.9 p = 0.95 1 [ 0.00,0.293 ) 0.122019 0.021006 0.028171 0.061569 0.146142 0.218195 0.096958 2 [ 0.293,0.343 ) 0.298608 0.023159 0.029979 0.071009 0.192293 0.314784 0.117319 3 [ 0.343,0.394 ) 0.367711 0.041275 0.049211 0.098279 0.296609 0.400347 0.155107 4 [ 0.394,0.444 ) 0.418546 0.049447 0.060524 0.119874 0.366732 0.524257 0.271243 5 [ 0.444,0.495 ) 0.469288 0.064006 0.075775 0.162260 0.402196 0.521439 0.282957 6 [ 0.495,0.545 ) 0.505474 0.036757 0.048691 0.127198 0.403209 0.575937 0.238749 7 [ 0.545,0.596 ) 0.571548 0.086091 0.109329 0.227951 0.454215 0.609114 0.354275 8 [ 0.596,0.646 ) 0.622419 0.123841 0.152670 0.226833 0.451058 0.627101 0.348887 9 [ 0.646,0.697 ) 0.660911 0.070502 0.096137 0.214043 0.511722 0.707955 0.349459 10 [ 0.697,0.747 ) 0.722936 0.135092 0.147302 0.217057 0.425921 0.548016 0.303217 11 [ 0.747,0.798 ) 0.767036 0.112457 0.134261 0.256527 0.514438 0.715117 0.377223 12 [ 0.798,0.848 ) 0.823550 0.122843 0.149805 0.262036 0.490951 0.662803 0.363241 13 [ 0.848,0.899 ) 0.872625 0.125868 0.158338 0.257350 0.518942 0.714696 0.377362 14 [ 0.899,0.949 ) 0.922412 0.123110 0.152628 0.284100 0.655613 0.988531 0.471624 15 [ 0.949 , o o ) 0.970250 0.123223 0.157524 0.362451 1.033090 1.822590 0.672675Table 5
Performance of theK(A) index. Statistical characteristics of the MAEs distribution in relation to various VRCMi for i=1,…,15 of K(A) values. The results were generated for n=6 on the basis of SAK as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and the geometric preference scale.
i V R C M i for K(A) MeanKA in VRCMi p–quantiles of MREs among w and w∗(LLSM) Average MREs amongw and w∗(LLSM) p = 0.05 p = 0.1 p = 0.5 p = 0.9 p = 0.95 1 [ 0.00,0.500 ) 0.309970 0.003138 0.003923 0.007291 0.013229 0.015591 0.008080 2 [ 0.500,0.537 ) 0.501271 0.004210 0.005171 0.010393 0.019240 0.023050 0.011567 3 [ 0.537,0.573 ) 0.554137 0.006699 0.008198 0.016071 0.029390 0.035490 0.017867 4 [ 0.573,0.610 ) 0.592713 0.007459 0.009298 0.018158 0.030929 0.035619 0.019454 5 [ 0.610,0.646 ) 0.630157 0.009699 0.011333 0.019057 0.030303 0.034814 0.020245 6 [ 0.646,0.683 ) 0.650693 0.006237 0.007816 0.015671 0.028959 0.034565 0.017455 7 [ 0.683,0.719 ) 0.702542 0.009187 0.010726 0.018481 0.030391 0.035464 0.019892 8 [ 0.719,0.756 ) 0.745895 0.008338 0.010188 0.019239 0.036328 0.043535 0.021702 9 [ 0.756,0.792 ) 0.774678 0.009249 0.010954 0.019613 0.034921 0.041489 0.021766 10 [ 0.792,0.829 ) 0.816861 0.010058 0.012214 0.023462 0.043198 0.051295 0.026023 11 [ 0.829,0.865 ) 0.847194 0.010501 0.012745 0.022585 0.038486 0.044885 0.024501 12 [ 0.865,0.902 ) 0.880103 0.011555 0.013988 0.026175 0.049286 0.058456 0.029391 13 [ 0.902,0.938 ) 0.921940 0.013008 0.016109 0.032457 0.059978 0.069519 0.035730 14 [ 0.938,0.975 ) 0.958146 0.014460 0.018227 0.037434 0.067413 0.078266 0.040733 15 [ 0.975 , o o ) 0.985451 0.019837 0.024678 0.049174 0.085952 0.097939 0.052645Table 6
Performance of theK(A) index. Statistical characteristics of the MREs distribution in relation to various VRCMi for i=1,…,15 of K(A) values. The results were generated for n=6 on the basis of SAK as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and the geometric preference scale.
i V R C M i for K(A) MeanK(A) in VRCMi p–quantiles of MAEs amongw and w∗(LLSM) Average MAEs amongw and w∗(LLSM) p=0.05 p=0.1 p=0.5 p=0.9 p=0.95 1 [ 0.00,0.500 ) 0.310089 0.023352 0.028073 0.048709 0.085736 0.100416 0.054715 2 [ 0.500,0.537 ) 0.501135 0.031293 0.037867 0.070485 0.130644 0.163143 0.146845 3 [ 0.537,0.573 ) 0.554356 0.053930 0.063208 0.116827 0.235816 0.318104 0.172693 4 [ 0.573,0.610 ) 0.592078 0.069405 0.084870 0.136276 0.226008 0.293496 0.249541 5 [ 0.610,0.646 ) 0.630342 0.087466 0.096573 0.132412 0.216063 0.282887 0.401898 6 [ 0.646,0.683 ) 0.650498 0.048560 0.059216 0.113700 0.235798 0.330528 0.226842 7 [ 0.683,0.719 ) 0.701886 0.078667 0.086119 0.125897 0.231228 0.292786 0.173261 8 [ 0.719,0.756 ) 0.746012 0.067538 0.079147 0.139081 0.326062 0.481375 0.227734 9 [ 0.756,0.792 ) 0.774741 0.073180 0.082412 0.139194 0.274548 0.366040 0.196296 10 [ 0.792,0.829 ) 0.817070 0.079380 0.094561 0.169309 0.395980 0.580617 0.273966 11 [ 0.829,0.865 ) 0.847198 0.088182 0.101460 0.153473 0.300013 0.418148 0.224248 12 [ 0.865,0.902 ) 0.880101 0.093595 0.108170 0.189377 0.449450 0.667343 0.305017 13 [ 0.902,0.938 ) 0.921891 0.105242 0.125400 0.246040 0.605379 1.034870 0.459726 14 [ 0.938,0.975 ) 0.958183 0.118728 0.145412 0.276527 0.581686 0.861373 0.433898 15 [ 0.975 , o o ) 0.985442 0.157070 0.190925 0.372889 0.888862 1.638710 0.686757The motivation for this research was to develop a PCM-CM which could depict a more credible relation between the consistency of pairwise judgments and the priority ratios estimation quality, i.e., whose priority ratios estimation error, reflected by SRC, would be very close or equal to 1 for all their quantiles (the most desirable situation).Successfully, a solution of the problem was generated and is presented as follows. On the basis of triad inconsistency measure introduced in [57], the following PCM-CM was devised:(16)TSLA=∑i=1n-2∑j=i+1n-1∑k=j+1nln2⁡aik/aijajkn31+maxi<j<k⁡lnp2⁡aik/aijajkp=1,…,n3The proposed PCM-CM is denoted asthe Triads Squared Logarithm Corrected Mean and an examination of its performance on the basis of simulation algorithm  SAK proposed earlier in this paper was carried out (Figures 3 and 4).Figure 3
Performance of the new PCM-CM:TSL(A). The plots present the relation between a mean value ofTSL(A) within a given interval (VRCMn) and quantiles of order 0.1 (Plot A), 0.5 (Plot B), and 0.9 (Plot C) of MAEs distribution as well as MAEs average values (Plot D) for estimated and true PVs. The results were generated with application of LLSM as the PP and Saaty’s scale. Plots are based on 10,000 random reciprocal PCMs forn=4.Figure 4
Performance of the new PCM-CM:TSL(A). The plots present the relation between a mean value ofTSL(A) within a given interval (VRCMn) and quantiles of order 0.1 (Plot A), 0.5 (Plot B), and 0.9 (Plot C) of MREs distribution as well as MREs average values (Plot D) for estimated and true PVs. The results were generated with application of LLSM as the PP and geometric scale. Plots are based on 10,000 random reciprocal PCMs forn=4.As can be noticed, the proposedTSL(A) performs credibly from the perspective of the relation among the consistency of pairwise judgments and the priority ratios estimation quality. It is undeniably a positive piece of information which opens a new chapter in pairwise judgments based priority ratios estimation process embedded in many methodologies of decision making such as AHP. It behooves us to mention thatTSL(A) is suitable for both reciprocal and nonreciprocal PCMs which prospectively may improve the pairwise judgments based priority ratios estimation quality when nonreciprocal PCMs are accepted.Tables7 and 8 provide detailed characteristics data forTSL(A) with application of LLSM (as the most popular alternative to REV), Saaty’s scale, and geometric scale as the most popular preference scales. Results for other PPs are similar; thus they are not presented in order to conserve the length of this paper. However, it is stressed that other PPs and preference scales were also tested and examined during the research and their results are not depicted in this paper because they coincide with results herein presented.Table 7
Performance of theTSL(A) index. Statistical characteristics of the MAEs distribution in relation to various VRCMi for i=1,…,15 of TSL(A) values. The results were generated for n=4 on the basis of SA|K| as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and Saaty’s preference scale.
i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MAEs among w and w∗(LLSM) Average MAEs amongw and w∗(LLSM) p = 0.05 p = 0.1 p = 0.5 p = 0.9 p = 0.95 1 [ 0.00,0.0934 ) 0.049049 0.005253 0.007007 0.017709 0.047828 0.072271 0.024607 2 [ 0.0934,0.12 ) 0.108368 0.006543 0.008965 0.022188 0.064636 0.089647 0.030783 3 [ 0.120,0.147 ) 0.131977 0.007560 0.010195 0.025401 0.073557 0.096264 0.034682 4 [ 0.147,0.173 ) 0.161289 0.009281 0.012408 0.032397 0.084857 0.106224 0.041842 5 [ 0.173,0.200 ) 0.186567 0.010605 0.014283 0.039235 0.087242 0.107040 0.046369 6 [ 0.200,0.227 ) 0.213651 0.012731 0.017479 0.044342 0.092117 0.112116 0.050310 7 [ 0.227,0.253 ) 0.240868 0.017165 0.022310 0.047478 0.093918 0.112928 0.053405 8 [ 0.253,0.280 ) 0.267645 0.018953 0.024106 0.048903 0.095909 0.112627 0.055422 9 [ 0.280,0.307 ) 0.293803 0.020081 0.025244 0.052348 0.097503 0.114723 0.058089 10 [ 0.307,0.333 ) 0.319702 0.020174 0.025936 0.054471 0.101461 0.120842 0.060164 11 [ 0.333,0.360 ) 0.345876 0.021179 0.027149 0.055049 0.104366 0.126714 0.061558 12 [ 0.360,0.387 ) 0.372744 0.021740 0.027979 0.056325 0.107628 0.130267 0.063053 13 [ 0.387,0.413 ) 0.400500 0.022274 0.028479 0.057966 0.110502 0.132659 0.064974 14 [ 0.413,0.440 ) 0.425325 0.021991 0.028264 0.060355 0.116391 0.137831 0.067429 15 [ 0.440 , o o ) 0.509413 0.022479 0.029361 0.063909 0.122018 0.144845 0.071126Table 8
Performance of theTSL(A) index. Statistical characteristics of the MREs distribution in relation to various VRCMi for i=1,…,15 of TSL(A) values. The results were generated for n=4 on the basis of SA|K| as the simulation algorithm and are based on 10,000 perturbed random reciprocal PCMs. The scenario assumed LLSM as the PP and geometric scale for preferences.
i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MREs among w and w∗(LLSM) Average MREs amongw and w∗(LLSM) p = 0.05 p = 0.1 p = 0.5 p = 0.9 p = 0.95 1 [ 0.00,0.063 ) 0.038493 0.021222 0.027957 0.063897 0.162196 0.250369 0.101603 2 [ 0.063,0.092 ) 0.075288 0.043098 0.049606 0.093880 0.275593 0.404725 0.159070 3 [ 0.092,0.120 ) 0.108927 0.030409 0.041118 0.100685 0.327138 0.441926 0.170932 4 [ 0.120,0.149 ) 0.131428 0.038427 0.049762 0.121698 0.379335 0.529179 0.199342 5 [ 0.149,0.177 ) 0.165031 0.054076 0.070318 0.180633 0.449114 0.626229 0.285351 6 [ 0.177,0.206 ) 0.194473 0.051717 0.069238 0.199856 0.462612 0.640247 0.301912 7 [ 0.206,0.234 ) 0.220825 0.089354 0.120756 0.223531 0.469640 0.630734 0.320947 8 [ 0.234,0.263 ) 0.250293 0.108335 0.138489 0.218002 0.465311 0.629102 0.324638 9 [ 0.263,0.291 ) 0.277319 0.114172 0.135641 0.246244 0.504231 0.682016 0.356697 10 [ 0.291,0.320 ) 0.306276 0.120962 0.143374 0.258090 0.507218 0.698669 0.361393 11 [ 0.320,0.348 ) 0.333741 0.122310 0.147515 0.260439 0.552231 0.775952 0.390815 12 [ 0.348,0.377 ) 0.361859 0.123757 0.156182 0.263475 0.579019 0.842248 0.425032 13 [ 0.377,0.405 ) 0.390045 0.133595 0.160033 0.267660 0.619568 0.920619 0.443646 14 [ 0.405,0.434 ) 0.416203 0.126431 0.153950 0.289740 0.699219 1.141290 0.523514 15 [ 0.434 , o o ) 0.506589 0.129256 0.162990 0.328123 0.834585 1.430670 0.632685It is noted that all statistical characteristics of the MAEs and MREs distribution in relation to various VRCMi fori=1,…,15 ofTSL(A) values, with few exceptions, monotonically grow. This examination ascertains that the proposed PCM-CM is a suitable measure of relation among pairwise comparisons consistency and the priority ratios estimation quality. The paramount position of the proposedTSL(A) is that it performs better than the other, evaluated here, PCM-CM, i.e.,K(A). Its position is additionally strengthened by the fact that its performance is similar and independent from the applied PP and improves significantly for higher numbers of alternatives without regard to which PP is selected.It should be noted that all characteristics presented herein are of great importance in the priority ratios estimation process, because one has to consider the potential of rejecting a “good” PCM, and vice versa, i.e., the possibility of accepting a “bad” PCM, as in the classic statistical hypothesis testing theory. However, for the first time in the course of pairwise judgments based prioritization development history, the possibility of selecting the level of certainty and basing decisions on statistical facts has been demonstrated.For instance, considering some hypothetical PCM forn=4, with its meanTSL(A)≈0.319702 for LLSM as the PP (Table 7), one can expect with 95% confidence that the MAE should not exceed the value of 0.1208420. At the same time, one can expect with 95% confidence that it will be higher than 0.0201740 (Table 7). Whether one decides to accept such a PCM or reject it obviously depends on the quality requirements of the priority ratios estimation and the attitude regarding these errors. Indeed, the outcome of the research finally creates the potential for true consistency control in an unprecedented way, i.e., directly related to the priority ratios estimation quality.For example, the following PV is considered asw=[0.27,0.26,0.24,0.23] denoting DM preferences for alternatives A1, A2, A3, and A4, respectively. Taking into consideration the earlier assumed level ofTSL(A)≈0.319702, the order of alternatives ranks A1=1, A2=2, A3=3, and A4=4 can be very deceptive and is rather meaningless. In such a situation one can expect with 95% confidence that the MAE>0.0201740 which makes one aware that the true rank order of examined preferences may appear otherwise, due to estimation errors related to DM inconsistency, e.g.,  w∗=[(0.27–0.025), (0.26–0.025), (0.24+0.025), (0.23+0.025)] = [0.242, 0.231, 0.269, 0.258], with MAE=0.025, which designates a different order for alternatives ranks, A1=3, A2=4, A3=1, and A4=2. On the other hand, consider PV as  w=[0.49,0.33,0.16,0.02] of DM preferences for alternatives A1, A2, A3, and A4 consecutively, as previously. Assuming this timeTSL(A)≈0.049049, it can be anticipated with 95% certainty that the MAE<0.072271 which rather insures that the order of alternatives ranks should remain unchanged.For similar but even more detailed calculation, MRE can be applied. It is a more accurate measure of priority ratios deviation; however, its straightforward application for calculation of discrepancies within normalized priority vectors is problematic.In order to enable other researchers to make similar analysis concerning different numbers of alternatives, the exemplary characteristics ofTSL(A) performance are provided forn>4 in the Appendix of this article, computed with application of commonly known PP, i.e., LLSM and the most common Saaty's scale (Table 9) and geometric scale (Table 10) as prospective preferences scales selected by decision makers.Table 9
Performance ofTSL(A) index under the action of LLSM as the PP. Statistical characteristics of the MAEs distribution in relation to various levels of TSL(A) within a given VRCMi for i=1,…,15. The results are based on 10,000 perturbed random reciprocal PCMs with application of Saaty’s scales and were generated on the basis of SA|K| as the simulation algorithm. The table contains results for n∈{5,6,7,8,9}, presented consecutively.
i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MAEs among w and w∗(LLSM) Average MAEs amongw and w∗(LLSM) p=0.05 p=0.1 p=0.5 p=0.9 p=0.95  1 [ 0,0.0899 ) 0.057912 0.0039186 0.0049954 0.0109799 0.0221753 0.0274887 0.0127898 2 [ 0.0899,0.107 ) 0.099124 0.0056158 0.0073876 0.0157136 0.0324243 0.0398139 0.0183201 3 [ 0.107,0.124 ) 0.116088 0.0063140 0.0079525 0.0184299 0.0389673 0.0490687 0.0214159 4 [ 0.124,0.142 ) 0.133907 0.0075132 0.0102233 0.0230429 0.0443459 0.0539668 0.0258898 5 [ 0.142,0.159 ) 0.151127 0.0099921 0.0129535 0.0261046 0.0486044 0.0581258 0.0290851 6 [ 0.159,0.176 ) 0.167911 0.0113191 0.0142543 0.0289546 0.0558904 0.0682777 0.0328936 7 [ 0.176,0.193 ) 0.184671 0.0125612 0.0158052 0.0320054 0.0594491 0.0730399 0.0357402 8 [ 0.193,0.211 ) 0.201896 0.0136853 0.0171375 0.0339101 0.0640703 0.0789391 0.0380755 9 [ 0.211,0.228 ) 0.219329 0.0142803 0.0178080 0.0361548 0.0711273 0.0839402 0.0408705 10 [ 0.228,0.245 ) 0.236371 0.0150518 0.0185369 0.0380656 0.0762136 0.0919801 0.0435024 11 [ 0.245,0.262 ) 0.253302 0.0161087 0.0208189 0.0405464 0.0789105 0.0929572 0.0462684 12 [ 0.262,0.280 ) 0.270523 0.0160427 0.0205586 0.0431223 0.0821647 0.0965329 0.0482168 13 [ 0.280,0.297 ) 0.288211 0.0165698 0.0209757 0.0457022 0.0865715 0.100490 0.0504072 14 [ 0.297,0.314 ) 0.305099 0.0177870 0.0226112 0.0455671 0.0859316 0.100544 0.0507868 15 [ 0.314 , o o ) 0.357080 0.0186614 0.0241816 0.0493007 0.0932224 0.107664 0.0547348  i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MAEs among w and w∗(LLSM) Average MAEs amongw and w∗(LLSM) p=0.05 p=0.1 p=0.5 p=0.9 p=0.95  1 [ 0,0.0901 ) 0.0618775 0.0036042 0.0044511 0.0090509 0.0175099 0.0212066 0.0102634 2 [ 0.0901,0.102 ) 0.096694 0.00584585 0.0072472 0.0147877 0.0255157 0.0297767 0.0158427 3 [ 0.102,0.115 ) 0.109186 0.0071783 0.0088408 0.0167774 0.0304119 0.0360603 0.0186253 4 [ 0.115,0.127 ) 0.121228 0.00831565 0.0102091 0.0192100 0.0349865 0.0420209 0.0214601 5 [ 0.127,0.139 ) 0.133028 0.0088771 0.0109435 0.0208206 0.0393504 0.0481357 0.0236802 6 [ 0.139,0.151 ) 0.144977 0.0097898 0.0118208 0.0225534 0.0439163 0.0538868 0.0259512 7 [ 0.151,0.163 ) 0.156874 0.0101678 0.0126009 0.0248914 0.0500528 0.0613696 0.0288113 8 [ 0.163,0.176 ) 0.169306 0.0113233 0.0138144 0.0274455 0.0552421 0.0656847 0.0317599 9 [ 0.176,0.188 ) 0.181783 0.0120341 0.0147276 0.0297646 0.0587297 0.0700824 0.0339487 10 [ 0.188,0.200 ) 0.193745 0.0124796 0.0157621 0.0317564 0.0613300 0.0720410 0.0356610 11 [ 0.200,0.212 ) 0.205758 0.0137977 0.0167981 0.0329443 0.0622977 0.0721443 0.0368687 12 [ 0.212,0.225 ) 0.218204 0.0140878 0.0175574 0.0347152 0.0652521 0.0774105 0.0386492 13 [ 0.225,0.237 ) 0.230723 0.0140705 0.0177333 0.0369638 0.0672822 0.0764555 0.0402684 14 [ 0.237,0.249 ) 0.242818 0.0146810 0.0186397 0.0381558 0.0692375 0.0786225 0.0413928 15 [ 0.249 , o o ) 0.279499 0.0168309 0.0207854 0.0401272 0.0721349 0.0829652 0.0439267  i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MAEs among w and w∗(LLSM) Average MAEs amongw and w∗(LLSM) p=0.05 p=0.1 p=0.5 p=0.9 p=0.95  1 [ 0,0.07975 ) 0.061626 0.00329141 0.0040063 0.0079292 0.0153184 0.017980 0.0089902 2 [ 0.07975,0.09 ) 0.085354 0.00558449 0.0066781 0.0124836 0.0217916 0.0254877 0.0136394 3 [ 0.09,0.10 ) 0.095128 0.00622084 0.0074651 0.0136346 0.0241580 0.0288301 0.0151410 4 [ 0.10,0.11 ) 0.105046 0.00677146 0.0081844 0.0150571 0.0277432 0.0343089 0.0170348 5 [ 0.11,0.12 ) 0.114884 0.00728075 0.0089529 0.0164708 0.0329745 0.0408782 0.0192642 6 [ 0.12,0.13 ) 0.124902 0.00792417 0.0097471 0.0185168 0.0378364 0.0464765 0.0217170 7 [ 0.13,0.14 ) 0.134949 0.00851189 0.0104389 0.0202075 0.0415434 0.0507830 0.0236614 8 [ 0.14,0.15 ) 0.144883 0.00952136 0.0115606 0.0224145 0.0446314 0.0531641 0.0257116 9 [ 0.15,0.161 ) 0.155416 0.0101888 0.0121602 0.0241178 0.0465694 0.0553538 0.0275101 10 [ 0.161,0.171 ) 0.165845 0.0110535 0.0132394 0.0261677 0.0499157 0.0583309 0.0293786 11 [ 0.171,0.181 ) 0.175874 0.0116123 0.0139639 0.0273006 0.0515428 0.0596329 0.0304575 12 [ 0.181,0.191 ) 0.185981 0.0121824 0.0150547 0.0293308 0.0532065 0.0613544 0.0320030 13 [ 0.191,0.201 ) 0.195819 0.0122294 0.0152015 0.0299135 0.0553010 0.0642011 0.0330142 14 [ 0.201,0.211 ) 0.205937 0.0132402 0.0164008 0.0321805 0.0552598 0.0636846 0.0343310 15 [ 0.211 , o o ) 0.235348 0.0147413 0.0179580 0.0321805 0.0586515 0.0682411 0.0363445  i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MAEs among w and w∗(LLSM) Average MAEs amongw and w∗(LLSM) p=0.05 p=0.1 p=0.5 p=0.9 p=0.95  1 [ 0,0.06861 ) 0.056493 0.0029930 0.0036359 0.0071723 0.0129253 0.0151100 0.0079193 2 [ 0.06861,0.078 ) 0.073616 0.0047668 0.0056647 0.0098201 0.0165545 0.0197820 0.0107659 3 [ 0.078,0.087 ) 0.082558 0.0051148 0.00615425 0.0108293 0.0189720 0.0232764 0.0121106 4 [ 0.087,0.095 ) 0.090957 0.0054815 0.0067644 0.0120701 0.0230822 0.0289636 0.0139455 5 [ 0.095,0.104 ) 0.0995085 0.0062208 0.0074045 0.0134360 0.0267488 0.0338094 0.0157422 6 [ 0.104,0.113 ) 0.108507 0.0065308 0.0079148 0.0148310 0.0307300 0.0379708 0.0175495 7 [ 0.113,0.122 ) 0.117503 0.0073636 0.0087983 0.0166204 0.0342287 0.0402093 0.0192815 8 [ 0.122,0.131 ) 0.126447 0.0077367 0.00920785 0.0182781 0.0366835 0.0432579 0.0209778 9 [ 0.131,0.140 ) 0.135467 0.0081883 0.0099817 0.0200944 0.0391024 0.0463982 0.0227669 10 [ 0.140,0.149 ) 0.144406 0.00893715 0.0109052 0.0215995 0.0404294 0.0465999 0.0240918 11 [ 0.149,0.158 ) 0.153395 0.0096365 0.0118788 0.0228543 0.0420224 0.0488816 0.0252208 12 [ 0.158,0.167 ) 0.162379 0.0105213 0.0128739 0.0250637 0.0441591 0.0509963 0.0270496 13 [ 0.167,0.176 ) 0.171319 0.0109917 0.0133182 0.0253654 0.0446033 0.0525163 0.0275815 14 [ 0.176,0.185 ) 0.180246 0.0120041 0.0144395 0.0266159 0.0464516 0.0529339 0.0289197 15 [ 0.185 , o o ) 0.205854 0.0127740 0.0155662 0.0283804 0.0479564 0.0549310 0.0304352  i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MAEs among w and w∗(LLSM) Average MAEs amongw and w∗(LLSM) p=0.05 p=0.1 p=0.5 p=0.9 p=0.95  1 [ 0,0.059795 ) 0.051197 0.0026372 0.0031677 0.0061278 0.0107141 0.0122502 0.0066588 2 [ 0.05979,0.068 ) 0.064092 0.0040166 0.0047872 0.0079722 0.0133870 0.0158901 0.0087678 3 [ 0.068,0.076 ) 0.072019 0.0044127 0.0052033 0.0089180 0.0154595 0.0189767 0.0099818 4 [ 0.076,0.085 ) 0.080495 0.0046625 0.0055923 0.0098746 0.0183837 0.0234965 0.0114040 5 [ 0.085,0.093 ) 0.089047 0.0052813 0.0062378 0.0110158 0.0221043 0.0279174 0.0129826 6 [ 0.093,0.101 ) 0.097017 0.0056575 0.0067669 0.0124636 0.0261396 0.0326188 0.0147615 7 [ 0.101,0.109 ) 0.105051 0.0061505 0.0074036 0.0138920 0.0290254 0.0358010 0.0164984 8 [ 0.109,0.118 ) 0.113488 0.0066692 0.0079686 0.0153474 0.0308319 0.0365922 0.0177312 9 [ 0.118,0.126 ) 0.122009 0.0073133 0.0087907 0.0171076 0.0330852 0.0388189 0.0193438 10 [ 0.126,0.134 ) 0.129857 0.0076181 0.0092912 0.0186416 0.0343317 0.0401595 0.0204982 11 [ 0.134,0.142 ) 0.137970 0.0083801 0.0102174 0.0199779 0.0355818 0.0416818 0.0216939 12 [ 0.142,0.151 ) 0.146298 0.0091112 0.0107807 0.0212040 0.0376109 0.0430078 0.0229256 13 [ 0.151,0.159 ) 0.154883 0.0097330 0.0118942 0.0219635 0.0378245 0.0435528 0.0237785 14 [ 0.159,0.167 ) 0.162793 0.0102563 0.0125995 0.0228089 0.0390630 0.0443591 0.0244409 15 [ 0.167 , o o ) 0.184864 0.0115601 0.0138072 0.0242891 0.0403012 0.0468790 0.0259996Table 10
Performance ofTSL(A) index under the action of LLSM as the PP. Statistical characteristics of the MREs distribution in relation to various levels of TSL(A) within a given VRCMi for i=1,…,15. The results are based on 10,000 perturbed random reciprocal PCMs with application of geometric scale and were generated on the basis of SAK as the simulation algorithm. The table contains results for n∈{5,6,7,8,9}, presented consecutively.
i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MREs among w and w∗(LLSM) Average MREs amongw and w∗(LLSM) p=0.05 p=0.1 p=0.5 p=0.9 p=0.95  1 [ 0,0.0899 ) 0.057816 0.025386 0.031105 0.063148 0.125229 0.157531 0.078369 2 [ 0.0899,0.107 ) 0.099189 0.037543 0.046524 0.094170 0.210311 0.285465 0.134037 3 [ 0.107,0.124 ) 0.116009 0.041496 0.051655 0.111830 0.248076 0.333749 0.162180 4 [ 0.124,0.141 ) 0.133320 0.052169 0.066257 0.141688 0.280316 0.388208 0.192453 5 [ 0.141,0.159 ) 0.150737 0.069557 0.088451 0.153061 0.322629 0.435671 0.228918 6 [ 0.159,0.176 ) 0.167818 0.083038 0.099089 0.176675 0.380074 0.531411 0.254632 7 [ 0.176,0.193 ) 0.184601 0.089335 0.106115 0.189599 0.418380 0.584780 0.282962 8 [ 0.193,0.21 ) 0.201376 0.094216 0.114126 0.200407 0.457435 0.650873 0.321145 9 [ 0.21,0.227 ) 0.218386 0.100531 0.122845 0.211782 0.509984 0.738768 0.357627 10 [ 0.227,0.244 ) 0.235167 0.104517 0.126347 0.229201 0.550244 0.802552 0.405093 11 [ 0.244,0.262 ) 0.252630 0.110690 0.133916 0.259418 0.623605 0.914232 0.454293 12 [ 0.262,0.279 ) 0.270118 0.114260 0.139642 0.275259 0.633504 0.926234 0.472063 13 [ 0.279,0.296 ) 0.287110 0.119073 0.144920 0.285331 0.656276 0.937927 0.463537 14 [ 0.296,0.313 ) 0.304180 0.124614 0.150476 0.293591 0.674574 0.982104 0.481352 15 [ 0.313 , o o ) 0.355480 0.130391 0.159726 0.311994 0.714442 1.105550 0.530692  i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MREs among w and w∗(LLSM) Average MREs amongw and w∗(LLSM) p=0.05 p=0.1 p=0.5 p=0.9 p=0.95  1 [ 0,0.0905 ) 0.063152 0.026955 0.032403 0.061404 0.122756 0.150047 0.091929 2 [ 0.0905,0.103 ) 0.097447 0.047882 0.058260 0.106461 0.193601 0.263458 0.169359 3 [ 0.103,0.115 ) 0.109447 0.055524 0.068321 0.119658 0.237508 0.340020 0.188758 4 [ 0.115,0.127 ) 0.121265 0.065842 0.078555 0.135479 0.288241 0.416876 0.228675 5 [ 0.127,0.14 ) 0.133603 0.071783 0.085975 0.147552 0.336650 0.492428 0.236603 6 [ 0.14,0.152 ) 0.145988 0.078850 0.093398 0.160596 0.399013 0.598307 0.288555 7 [ 0.152,0.164 ) 0.157828 0.083621 0.100037 0.181155 0.460463 0.713230 0.328321 8 [ 0.164,0.176 ) 0.169764 0.088879 0.106790 0.204747 0.509118 0.799108 0.392186 9 [ 0.176,0.189 ) 0.182294 0.095569 0.114805 0.224679 0.539696 0.816097 0.387082 10 [ 0.189,0.201 ) 0.194851 0.101796 0.122377 0.241424 0.569156 0.840488 0.403202 11 [ 0.201,0.213 ) 0.206810 0.106787 0.130048 0.253098 0.584213 0.886693 0.414521 12 [ 0.213,0.225 ) 0.218759 0.111142 0.135331 0.263625 0.597492 0.909918 0.427723 13 [ 0.225,0.238 ) 0.231244 0.115692 0.142985 0.274482 0.629127 1.011140 0.458373 14 [ 0.238,0.25 ) 0.243689 0.119538 0.145306 0.282161 0.629213 1.016580 0.492177 15 [ 0.25 , o o ) 0.281051 0.131983 0.160474 0.301961 0.725361 1.305030 0.569886  i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MREs among w and w∗(LLSM) Average MREs amongw and w∗(LLSM) p=0.05 p=0.1 p=0.5 p=0.9 p=0.95  1 [ 0,0.07935 ) 0.061389 0.027453 0.033069 0.064786 0.121032 0.148877 0.091901 2 [ 0.07935,0.089 ) 0.084618 0.048742 0.058162 0.097221 0.180521 0.258818 0.150911 3 [ 0.089,0.1 ) 0.094726 0.055028 0.064762 0.108760 0.227929 0.354122 0.188055 4 [ 0.1,0.11 ) 0.105058 0.060689 0.072055 0.119836 0.288762 0.451518 0.223222 5 [ 0.11,0.12 ) 0.114936 0.066323 0.077914 0.135084 0.360043 0.586682 0.258300 6 [ 0.12,0.13 ) 0.124948 0.071119 0.084561 0.156093 0.418538 0.673318 0.310323 7 [ 0.13,0.14 ) 0.134942 0.076207 0.091276 0.176892 0.459282 0.723317 0.339038 8 [ 0.14,0.15 ) 0.144913 0.082067 0.099388 0.197851 0.491089 0.760550 0.360014 9 [ 0.15,0.16 ) 0.154903 0.089571 0.107351 0.214104 0.510660 0.781063 0.370044 10 [ 0.16,0.17 ) 0.164889 0.094907 0.113971 0.227829 0.527074 0.821481 0.390846 11 [ 0.17,0.181 ) 0.175348 0.103233 0.124254 0.244012 0.543260 0.818923 0.403747 12 [ 0.181,0.191 ) 0.185853 0.109529 0.131299 0.251883 0.574974 0.893017 0.417672 13 [ 0.191,0.201 ) 0.195779 0.115457 0.138375 0.261156 0.587231 0.940269 0.442622 14 [ 0.201,0.211 ) 0.205771 0.122449 0.144794 0.267270 0.591900 0.908632 0.452321 15 [ 0.211 , o o ) 0.235657 0.132645 0.158647 0.289569 0.697684 1.246370 0.540476  i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MREs among w and w∗(LLSM) Average MREs amongw and w∗(LLSM) p=0.05 p=0.1 p=0.5 p=0.9 p=0.95  1 [ 0,0.0687 ) 0.056815 0.028298 0.033736 0.064678 0.120167 0.158884 0.090449 2 [ 0.0687,0.078 ) 0.073695 0.047574 0.055250 0.089089 0.176783 0.305679 0.164938 3 [ 0.078,0.087 ) 0.082568 0.050834 0.059422 0.099121 0.231914 0.395970 0.195522 4 [ 0.087,0.096 ) 0.091492 0.055367 0.065374 0.113529 0.317872 0.516773 0.237766 5 [ 0.096,0.105 ) 0.100445 0.060858 0.072058 0.129920 0.373219 0.601663 0.264844 6 [ 0.105,0.114 ) 0.109477 0.066252 0.078744 0.146265 0.410744 0.653367 0.295777 7 [ 0.114,0.123 ) 0.118453 0.072271 0.085780 0.167583 0.439425 0.692378 0.307610 8 [ 0.123,0.132 ) 0.127465 0.077733 0.092503 0.185631 0.463697 0.714777 0.319999 9 [ 0.132,0.141 ) 0.136439 0.083953 0.100942 0.204941 0.498413 0.736361 0.332616 10 [ 0.141,0.15 ) 0.145370 0.090884 0.108899 0.219291 0.513393 0.782983 0.349392 11 [ 0.15,0.159 ) 0.154363 0.098619 0.118239 0.232536 0.541734 0.833077 0.377437 12 [ 0.159,0.168 ) 0.163371 0.104881 0.126765 0.240974 0.550401 0.870980 0.403649 13 [ 0.168,0.177 ) 0.172341 0.111783 0.134349 0.252118 0.561672 0.855287 0.415888 14 [ 0.177,0.186 ) 0.181318 0.117385 0.140871 0.257608 0.593642 0.961143 0.434641 15 [ 0.186 , o o ) 0.206773 0.130867 0.155858 0.280862 0.679144 1.287050 0.519607  i V R C M i for TSL(A) MeanTSL(A) in VRCMi p–quantiles of MREs among w and w∗(LLSM) Average MREs amongw and w∗(LLSM) p=0.05 p=0.1 p=0.5 p=0.9 p=0.95  1 [ 0,0.0599 ) 0.051235 0.028109 0.032911 0.061763 0.117622 0.182188 0.097775 2 [ 0.0599,0.068 ) 0.064180 0.044045 0.050531 0.079861 0.180490 0.351457 0.153068 3 [ 0.068,0.076 ) 0.072026 0.047540 0.054987 0.090918 0.249092 0.452849 0.184488 4 [ 0.076,0.085 ) 0.080534 0.050473 0.059381 0.104907 0.316722 0.541229 0.212364 5 [ 0.085,0.093 ) 0.089006 0.056571 0.066786 0.122096 0.368507 0.60405 0.249676 6 [ 0.093,0.101 ) 0.096996 0.061878 0.073179 0.138248 0.411438 0.664728 0.271985 7 [ 0.101,0.109 ) 0.104957 0.066931 0.079049 0.157497 0.442368 0.692257 0.283826 8 [ 0.109,0.118 ) 0.113442 0.072436 0.085955 0.178618 0.466070 0.725716 0.307943 9 [ 0.118,0.126 ) 0.121941 0.077338 0.093199 0.196075 0.486246 0.762465 0.324215 10 [ 0.126,0.134 ) 0.129922 0.084719 0.101779 0.210156 0.505687 0.795012 0.344586 11 [ 0.134,0.142 ) 0.137902 0.091598 0.110772 0.222684 0.521883 0.834840 0.354677 12 [ 0.142,0.151 ) 0.146341 0.098879 0.120136 0.232111 0.531432 0.860653 0.375085 13 [ 0.151,0.159 ) 0.154800 0.107414 0.129619 0.241998 0.548790 0.897229 0.385119 14 [ 0.159,0.167 ) 0.162810 0.113641 0.136610 0.248387 0.574881 0.955015 0.375036 15 [ 0.167 , o o ) 0.185022 0.128570 0.152627 0.272902 0.706691 1.342150 0.542536


## 5. Conclusions

In this research, the performance of the selected PCM-CM from the perspective of its relation between pairwise judgments consistency and the quality of the priority ratios estimation process was examined with application of the most popular PP, i.e., LLSM, preference scales, and number of alternatives. The research findings can be stated as follows:(i)
It is possible to improve the quality of pairwise judgments based priority ratios estimation whenTriads Squared Logarithm Corrected Mean (TSL(A)) is applied as the PCM-CM(ii)
From the perspective ofTSL(A) performance, it is stable and its results differ insignificantly from the perspective of applied prioritization procedure (i.e., LLSM or any other selected PP)(iii)
When the number of alternatives increases, the performance of the examinedTSL(A) improves independently from the applied PP(iv)
The examinedTSL(A) can be applied to both reciprocal and nonreciprocal PCMs broadening their applications and perfecting the quality of decision makers’ preferences estimation processCertainly, there is a place for further research in the field; e.g., advantages ofTSL(A) in comparison to other consistency measures can be examined theoretically (which was beyond the scope of this research), and the performance ofTSL(A) can be examined when nonreciprocal PCMs are applied what encourage further development of pairwise judgments methodology.



---

*Source: 3574263-2019-02-03.xml*