# Identification of a Multicriteria Decision-Making Model Using the Characteristic Objects Method

Authors: Andrzej Piegat; Wojciech Sałabun
Journal: Applied Computational Intelligence and Soft Computing
 (2014)

---

## Abstract

This paper presents a new, nonlinear, multicriteria, decision-making method: the characteristic objects (COMET). This approach, which can be characterized as a fuzzy reference model, determines a measurement standard for decision-making problems. This model is distinguished by a constant set of specially chosen characteristic objects that are independent of the alternatives. After identifying a multicriteria model, this method can be used to compare any number of decisional objects (alternatives) and select the best one. In the COMET, in contrast to other methods, the rank-reversal phenomenon is not observed. Rank-reversal is a paradoxical feature in the decision-making methods, which is caused by determining the absolute evaluations of considered alternatives on the basis of the alternatives themselves. In the Analytic Hierarchy Process (AHP) method and similar methods, when a new alternative is added to the original alternative set, the evaluation base and the resulting evaluations of all objects change. A great advantage of the COMET is its ability to identify not only linear but also nonlinear multicriteria models of decision makers. This identification is based not on a ranking of component criteria of the multicriterion but on a ranking of a larger set of characteristic objects (characteristic alternatives) that are independent of the small set of alternatives analyzed in a given problem. As a result, the COMET is free of the faults of other methods.


---

## Body

## 1. Introduction

The subject of criteria is often misunderstood, both by laypeople and scientists. Expert criterions exist in the mind of the decision maker for use in the evaluation of alternatives, allowing for determination of the attractiveness of a considered object (absolute evaluation) or relative attractiveness of two or more objects (comparative evaluation). The alternatives evaluated can include anything (e.g., internet pages, films, cars, machines and technical devices, bridges, homes, motorbikes, horses, dogs, exam projects of students, other people, feminine beauty, the choice of a firm for a task realization, etc.) and each individual problem has various mental multicriteria models that are applied during evaluation. These individual criteria are usually called subjective criteria [1–3], which can often mistakenly evoke a negative interpretation akin to fuzzy or unclear. Therefore, rather than subjective criteria, the term individual multicriteria will hereafter be used to describe these criteria [4, 5]. As well as person-specific multicriteria, objective multicriteria criteria also exist. However, these objective criteria are mostly simple, one or two component multicriteria. For instance, cars can be compared objectively and independently of human opinions if we compare their attractiveness only in respect to one criterion, for example, to fuel consumption; a car which consumes less fuel is more attractive. If we evaluate two enterprises in respect of profit, then the firm which produces greater profit is objectively more attractive. However, if we want to evaluate attractiveness of cars taking into account two attributes, for example, fuel consumption and acceleration, then no objective criterion exists because different people assign different significance to each attribute; thus multicriteria are mostly specific to an individual and are not objective (with few exceptions). However, if the individual multicriteria of all people in a group are known, the average multicriterion can be determined and considered as representative multicriterion of that group. The group can also elaborate representative multicriteria through discussion and opinion exchange [6, 7]. It should be noted that such known and popular methods as AHP, ANP, and so forth identify individual, subjective, not-objective multicriteria models of experts (people). For example, in the AHP method an expert is asked which component criterion is more important and which is less important. Then, he/she is asked to rank evaluated alternatives (objects) according to each component-criterion, and so forth. In this way, AHP method identifies subjective, individual preferences of only this particular expert. In Simple Additive Weighing method, an expert is asked to give weight-coefficients for particular component-criteria. These coefficients also express only individual, subjective preferences of the expert. If coefficients are delivered by group of experts, then they represent also only this particular group and are specific and subjective for this group. Another experts’ group (committee, party) may give different coefficients, specific for it. Thus, we have mostly to do with subjective multicriteria.


## 2. Are Expert-Criteria Linear or Nonlinear?

In this section, the problem of multicriteria nonlinearity will be analyzed using examples of individual multicriteria (representing preferences of a single person). Group multicriteria can be achieved by aggregation of individual criteria, requiring special methods and mathematical formulation [8]. Between individuals, multicriteria are generally quantitatively and qualitatively distinct. Thus, it would be unreasonable to assume the same type of mathematical model (e.g., the linear model) to represent criteria of different people. Despite this, in the case of multicriteria, the linear model is the most frequently applied one [9–12] (1):
(1)
K
=
ω
0
+
ω
1
K
1
+
ω
2
K
2
+
⋯
+
ω
n
K
n
,
where K is multicriteria function and ω
i is weight (significance) coefficient of the component criterion K
i, i
=
1
,
…
,
n, ∑
i
=
1
n
ω
i
=
1.Equation (1) shows that the component-criteria (K
i) are aggregated linearly. The linear criterion-function is represented in 2D space by a straight line, in 3D space by a plane (Figure 1), and in nD-space by a hyperplane. However, it should be noted that in the linear multicriteria function K particular component-criteria K
i influence the global multicriterion K in a mutually independent and uncorrelated manner. Therefore, the linear multicriterion is unable to contain information about correlations between component-criteria existing in the human mind. Additionally, in this representation the influence strength of particular K
i components is global, constant, and unchanging in the full criterion-domain. These features greatly disadvantage the linear models of multicriteria; real human multicriteria are in most cases nonlinear (linearity is a special ideal feature of dependencies in systems) and the significance of K
i components is variable and usually dependent on other components. Indeed, an individual component-significance varies in local subdomains of the global multicriteria domain. Unfortunately, linear or linear-like multicriteria models are used in many internationally known methods for decision making. The following examples illustrate the following: the SAW (Simple Additive Weighing) [13–16], the very well-known and widely used AHP (Analytic Hierarchy Process) [12, 17–25], and ANP (Analytic Network Process) [26–33]. Other well-known methods, such as TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution) [34–40], ELECTRE (ELimination and Choice Expressing REality) [41–45], and PROMETHEE (Preference Ranking Organization Method for Enrichment Evaluations) [46–50] are not strictly linear; however, they assume and use global weight coefficients ω
i that are constant in the full multicriteria domain, and in certain steps of their algorithms they use the linear, weighted aggregation of alternatives. There are many convincing examples which demonstrate that human criterion-functions are nonlinear, a point often best expressed through use of easily visualized examples. One such example can be the criterion-function representing human evaluation of coffee taste (CT
∈
[
0,1
]) in dependence of sugar quantity (S
∈
[
0,5
]), which is expressed in sugar spoons (Figure 2).A linear criterion function in 2D space (a) and in 3D space (b).
(a)
(b)Figure 2
Plot of the individual criterion function for coffee taste CT versus the number of teaspoons of sugarS (this criterion function is from one of the authors, Andrzej Piegat).The criterion function for coffee taste can be identified by interviewing a given person (declarative identification) or experimentally, by giving the person coffees containing a range of sugar concentrations and asking her/him to evaluate the coffee taste (or to compare tastes of various coffee pairings). The obtained taste-evaluations can be processed using the characteristic objects method proposed here, which enables criterion-function identification. Yet even without such scientific rigor, it should be evident that the criterion-function shown in Figure2 is qualitatively correct. In fact, this function represents the individual preferences of an author (Andrzej Piegat) of this paper, who dislikes coffee containing more than 3 spoons of sugar (which is evaluated as CT = 0). Coffee without sugar is also evaluated as unpleasant (S
=
0), whereas coffee containing two spoons of sugar is optimum (S
opt
=
2). As the optimal sugar amount will differ between individuals, the criterion-function from Figure 2 is not objective; it is specific to the author Andrzej Piegat and does not represent the whole population.Figure3 demonstrates another example of a one-component criterion, K
1
=
f
(
T
), where K
1 represents body-comfort feeling and T is the room temperature. Function K
1 can also be interpreted as the attractiveness of temperature for the human body (specifically for the body of the co-author Wojciech Sałabun in this example). Again, the body-comfort function can be identified through interview (declarative method) or experimentally, by introducing the person into an air-conditioned room at various temperatures. Figure 4 is a further example of a one-component criterion where function K
2
(
H
) represents the comfort-feeling (of co-author Wojciech Sałabun) in dependence of the relative humidity in the room H[%], which can be called the function of humidity attractiveness. Both the function K
1
=
f
(
T
) (Figure 3) and function K
2
=
f
(
H
) (Figure 4) are nonlinear and have maxima of comfort. It is therefore not difficult to imagine the general qualitative form of the two-component human-criterion K
3
=
f
(
T
,
H
), expressing the body-comfort feeling dependent on both temperature T[°C] and humidity H[%].Figure 3
Plot of the individual criterion-functionK
1
=
f
(
T
) for personal comfort depending on temperature T[°C] in a room. This criterion-function is from author Wojciech Sałabun.Figure 4
Plot of the individual criterion-functionK
2
=
f
(
H
) for personal comfort depending on relative humidity H[%] in a room. This criterion-function is from author Wojciech Sałabun.An approximate picture of this multicriteria model is shown in Figure5, although it should be noted that in most cases such a two-component criterion will not be a simple composition of two one-component-criteria and that knowledge of two separate one-component-criteria does not allow exact construction of the two-component multicriteria model. The duel influences of temperature and humidity will result in a maximum function for K
3
=
f
(
T
,
H
) with coordinates different than maxima of functions K
1 and K
2.Figure 5
Surface of the individual multicriteriaK
3
=
f
(
T
,
H
) for personal comfort depending on temperature in a room T[°C] and the relative humidity H[%] of the air. This model is from author Wojciech Sałabun.This raises the question of how to identify the two-component (orn-component) human multicriteria function for the attractiveness of objects. To address this we have developed the method of characteristic objects (ChO) (conceived by Andrzej Piegat and presented in Section 4).


## 3. Decision-Making Paradox

Although many multicriteria determination methods have been described, particular methods may yield different results to the same problem and data set, making it difficult to decide which method to apply to a given decision problem [51]. This question was first formulated by Triantaphyllou and Mann [52] and is recognized as a fundamental paradox in multicriteria decision making and challenge for reliable decision making [51]. Issues of rank reversals and “nonexistent” rank are important factors in understanding this paradox. For example, when an unacceptable alternative has a nonzero priority, the “nonexistent” rank of alternatives will occur. Let us suppose that a set of alternatives have been ranked by a person (which we will call original rank) using method X. If we add or remove alternatives from the original set, a new ranking may be determined by method X. If the original and new rankings have different sequences determined by method X then it is called a rank reversal problem. Although the AHP method (and others) is sensitive to rank reversal, it remains one of the most popular methods in multicriteria decision making and is frequently applied in scientific and commercial projects. As a demonstration of rank reversal caused by the AHP method we can consider the problem of buying a new car using two component-criteria: K
1, purchase price [$] and K
2, horsepower [hp]. Three alternatives are taken into account: A—Mazda 5, B—Chevrolet Orlando, and C—Peugeot 5008 with a description of each alternative given in Table 1.Table 1
The purchase price (K
1) and horsepower (K
2) of car models (A—Mazda 5, B—Chevrolet Orlando, and C—Peugeot 5008).
A
B
C
K
1
$30,000
$24,000
$22,000
K
2
150 hp
141 hp
120 hpCriteria weights are first computed with the first eigenvector providing information achieved from a person on the relative importance of the criteria purchase price and horsepower (Table2).Table 2
Matrix comparing attributes.
Criteria
K
1
K
2
Weights
K
1
1
1
0.5
K
2
1
1
0.5Priorities of alternatives for each component-criterion are then computed using the same technique as in the previous step (Table3). Finally, values of the original rank as a linear combination of criteria weights and alternative priorities were computed (2), arriving at the ranking A
>
C
>
B, meaning that the Mazda 5 is the best alternative (and the Chevrolet Orlando is the worst one):
(2)
0.0974
0.3331
0.5695
0.5
+
0.6267
0.2797
0.0936
0.5
=
0.3621
0.3064
0.3316
A
B
C
.Table 3
Matrices comparing three cars A, B, and C with respect to criteriaK
1, K
2, AHP method.
K
1
A
B
C
Priorities
A
1
1/4
1/5
0.0974
B
4
1
1/2
0.3331
C
5
2
1
0.5695
K
2
A
B
C
Priorities
A
1
3
5
0.6267
B
1/3
1
4
0.2797
C
1/5
1/4
1
0.0936However, when fourth alternative (D, Renault Megane) with a purchase cost $30,000 and 150 hp is added, although the matrix comparing criteria will be unchanged, priorities of alternatives for each criterion do change (Table4). Finally, values of the new rank were computed:
(3)
0.0899
0.3182
0.5039
0.0899
0.5
+
0.3861
0.1647
0.0632
0.3861
0.5
=
0.2375
0.2414
0.2836
0.2375
.Table 4
Matrices comparing fourth cars A, B, C, and D with respect to the criteriaK
1, K
2, AHP method.
K
1
A
B
C
D
Priorities
A
1
1/4
1/5
1
0.0899
B
4
1
1/2
4
0.3182
C
5
2
1
5
0.5039
D
1
1/4
1/5
1
0.0899
K
2
A
B
C
D
Priorities
A
1
3
5
1
0.3861
B
1/3
1
4
1/3
0.1647
C
1/5
1/4
1
1/5
0.0632
D
1
3
5
1
0.3861The new ranking produces a sequence of alternatives:C
>
B
>
A
=
D. The original most preferred alternative (Mazda 5) has now become the weakest (a total replacement of the original ranking order), thus demonstrating the important shortcoming of the AHP method and similar multicriteria approaches. This phenomenon is partly the result of using one and the same type of mathematical formula for a criterion representing different people and different problems. The multicriteria model of different people will be qualitatively different and therefore require the application of qualitatively different formulas. However, many known methods use one and the same type (mostly linear) of mathematical formula. Weights of component-criteria are often determined separately and independently on the basis of information achieved from a decision maker. Then, the so-determined weights have sense of average significances, average for the full, and global domain of the multicriteria model and are independent of local specificities. They are of global character and they are the same and constant for all considered objects (alternatives). For example, doctors accept 36.8°C as a normal human body temperature. This value is used for all possible objects (patients) and thus is of global character. However, the “normal” temperature differs between men and women and even varies across a day. Therefore, we should not define a globally correct (normal) body temperature for all genders throughout a 24-hour day. Rank reversals are thus the result of the use of global and constant weights for all component-criteria. Moreover, the significance level of one component-criterion dependents on levels of all other criteria. Frequently component-criteria are one with another correlated. In the previous example, a considerable correlation exists between body-temperature and gender, but if component-criteria are considered separately identification of correlations becomes impossible. Unfortunately, in many multicriteria methods component-criteria are considered separately. It leads to rank reversals. Rank reversals are also caused by determining absolute evaluations of particular considered objects directly on the basis of all objects themselves, and not on the basis of an independent, constant, and unchangeable set of other different objects. Despite much effort invested into improving the AHP and ANP methods [53–56], important shortcomings remain. The development of reliable decision-making methods requires the creation of a special, independent base and scale of evaluations. The next chapter describes a rank reversal free multicriteria method, with several important advantages over current methods.


## 4. The Characteristic Objects Method for Elicitation and Modeling of Human (Expert) Multicriteria Function

The essence of the COMET will be explained using an example of a two-component multicriteria problem. The presented version of the method is not the final one, because the method is continually being investigated and developed as its authors work on various versions. As the example, we present the problem of attractivenessK of the male body depending on two component-criteria K
1 (height [cm] of the body) and K
2 (weight [kg]). The identified multicriteria model represents the preferences of co-author Andrzej Piegat. Each man possesses an individual ideal pattern of his body, the body he would like to have. This pattern depends not only on height K
1 and weight K
2, but also on a number of other body attributes such as the mutual proportion of different body parts, hair color, eye color, skin color, and facial features. Thus, the global attractiveness K of the male body depends on n component-criteria K
i. However, in this example, only a simplified version of the attractiveness will be investigated to permit an easy understanding of the essence of the method and to enable visualization of the multicriteria model. The attractiveness of the male body is a multicriteria problem, which is easy to understand for any man or woman and does not require a special expert knowledge. However, it should be noted that the COMET can be applied to the identification of any decision problem from any area, for example, from economics, management, technology, engineering, politics, medicine, environmental protection, and so on. The COMET can be used for elicitation both from ordinary people and from experts. In the identification process of human multicriteria preferences, the COMET uses fuzzy modeling. Thus, it can be included to the method group called “fuzzy multicriteria decision making.” This direction of research is very important, because it tries to take into account the uncertainty of the data occurring in human decision-making problems. This uncertainty cannot be passed by or overlooked. Decisional problems cannot be solved as deterministic ones. One of the last and the most important books that summarizes and presents actual achievements of fuzzy investigations in decision making is the book by Pedrycz et al. [8]. Although this book presents a lot of methods, none of them however seems to use an identical approach to the multicriterion identification as the COMET. In the book [8] methods were presented for comparison of alternatives and for choice of the best alternative. Examples of the methods are a method based on construction of preference relation, interactive decision-making system for multicriteria analysis of alternatives in a fuzzy environment, multicriteria analysis of alternatives with fuzzy ordering of criteria, with the concept of fuzzy majority, a method based on outranking approach (fuzzy Promethee), and other methods. These methods are certainly valuable, because they were accepted by scientific community. However, as we have written, they differ considerably from the Characteristic Objects method (ChO). In the above methods alternatives under discussion are directly used for construction of the evaluating multicriteria models, which next, mostly with use of matrices, make choice of the best alternative and make ranking of alternatives. If the number n of evaluated alternatives is changed to, for example, n
+
1, then the resulting ranking also can change (rank reversal similarly as in the AHP method) because the rank is determined by an inner multicriteria model that was constructed on the basis of the same analyzed alternatives. If the number n of analyzed alternatives changes, the multicriteria model based on them changes also. The approach of ChO method is completely different. Here also few (n) alternatives are to be evaluated (e.g., n
=
5 used cars), but this n alternatives are not applied for construction of the multicriteria model. To construct this model special m characteristic objects (alternatives) are assumed (in the human-body evaluation example this number was equal to m
=
48; see Table 8), which are independent of the n analyzed alternatives. Only when the multicriteria model has been identified, then the n alternatives under discussion are separately, one after another, evaluated by this model, and their global attractiveness K is calculated and compared to achieve their ranking. Introducing a new (n
+
1)th alternative to the set of the previous n alternatives does not have any influence on the multicriteria model, because it has been evaluated on the bases of the different set of m characteristic objects. This approach is according to our knowledge fully new.Step 1 (general analysis of the multicriteria problem).
A person whose individual multicriteria model is to be identified should consider and analyze which particular component-criteriaK
i have a general significance for him and influence his global model. In the considered problem, the person should consider his preferences in relation to the body height K
1 and weight K
2. In the case of the attractiveness of a vehicle, the considered parameters might include the maximal velocity, motor power, fuel consumption, the visual attractiveness of the car, and so on. Which attribute is the most important for me and why? Which attribute is the second most important, and the third most importtant, and so on? This introductory ranking of component-criteria K
i is not used later for mathematical calculations with the COMET. However, for the accuracy of the resulting model, K
=
f
(
K
1
,
K
2
,
…
,
K
n
) is general necessary in order for the investigated person to correctly understand the meaning and significance of particular criterion components. If the components are not understood, the identification accuracy will be low. For example, if we identify the individual model of vehicle attractiveness and the investigated person is a woman (women usually have limited knowledge about the technical aspects of cars), then she should be informed about the practical importance of component-criteria motor power for the car, for example, of each. Similarly, investigated people should be informed about the meaning of more difficult components of specific problems that require advanced expert knowledge. After explaining the meanings of particular components to the investigated person, an introductory training of this person in the comparison and evaluation of attractiveness of a certain number of object pairs should be conducted. For example, in the case of body attractiveness, the person should consider and decide which of the presented objects (
K
1
,
K
2
) = (height, weight) are personally the most attractive to him:
(4)
O
1
=
173
cm
,
65
kg
or
O
2
181
cm
,
85
kg
?
O
3
=
179
cm
,
96
kg
or
O
4
(
175
cm
,
83
kg
)
?
and
so
forth
.
This introductory training is very important because, as practical elicitation experiments have shown, many people make false, inconsiderate decisions in first evaluating of objects. However, after few preliminary examples and evaluations, they begin to better understand the problem and the accuracy of their evaluations improves.Step 2 (determining domains and characteristic values of particular component-criteriaK
i).
Since the domain of the component criterionK
1 height was assumed to be over the interval [150, 200] cm, the following characteristic values were chosen: 150, 160, 170, 180, 185, 190, and 200 cm. In the subinterval [180, 190] cm, the value 185 cm was introduced, because in the opinion of the investigated man, this height would be optimal for him. Since the domain of the component K
2 weight was assumed to be over the interval [50, 120] kg, the following characteristic values were chosen: 50, 60, 70, 80, 85, 90, 100, 110, and 120 kg. In the subinterval [80, 90] kg, the value 85 kg was introduced because the investigated man regards the combination of (185 cm, 85 kg) as optimal body parameters. Figure 6 shows all characteristic objects (crossing points) resulting from the above assumptions.Figure 6
Two-dimensional domain of the multicriteria problem of male body attractiveness, with marked characteristic objectsO
i
j = (height, weight).Each of the objects represents one possible body version of a man; for example,O = (190, 80) means a male body height of 190 cm and a weight of 90 kg. The similarity of any object O
K = (
K
1
,
K
2
) to other characteristic objects that encircle it can be determined using the membership functions μ
i
(
K
1
) and μ
i
(
K
2
) that define the characteristic values of the attributes K
1 height and K
2 weight of the male body, Figures 7 and 8. The characteristic object O
i
j = (
K
i
,
K
j
) is defined as the logical product of two fuzzy sets (Near K
i) AND (Near K
j); for example, the object O
67 = (190, 100) is defined as the product (Near 190 cm) AND (Near 100 kg). The operation AND is realized with the operator PRODUCT [57–59]. The number of all characteristic objects O
i
j is equal to 7 × 9 = 63 and is fairly high. However, as will be shown in the next step, this number can and should be considerably reduced.Figure 7
Membership functionsμ
i
(
K
1
), defining the similarity in height [cm] to particular characteristic values of the first component-criterion K
1 (height).Figure 8
Membership functionsμ
i
(
K
2
), defining the similarity in weight [kg] to particular characteristic values of the second component-criterion K
2 (weight).Step 3 (reduction of the number of characteristic objects).
To reduce the number of characteristic objects, the investigated person should determine which objects have in his/her opinion zero attractiveness; that is, they are not at all acceptableK
=
0. Next, the investigated person should determine all such objects that are completely acceptable K
=
1 (the number of optimal objects can be greater than 1). In the problem of male body attractiveness, the investigated person rejected the objects O
i
j = (height, weight) given below as not acceptable because the man certainly would not like to have the height, weight combinations:(150, 50), (150, 60), (150, 70), (150, 80), (150, 85), (150, 90), (150, 100), (150, 110), (150, 120), (160, 50), (160, 60), (160, 70), (160, 80), (160, 85), (160, 90), (160, 100), (160, 110), (160, 120), (170, 50), (170, 90), (170, 100), (170, 110), (170, 120), (180, 50), (180, 60), (180, 100), (180, 110), (180, 120), (185, 50), (185, 60), (185, 110), (185, 120), (190, 50), (190, 60), (190, 70), (190, 110), (190, 120), (200, 50), (200, 60), (200, 70), (200, 80), (200, 85), (200, 90), (200, 100), (200, 110), (200, 120).
All of the above characteristic objects achieve attractivenessK
=
0 (zero objects). The investigated person determined the object O
55 = (185, 85) to be the most attractive body version. This characteristic object (ChO) achieves attractiveness K
=
1 (the optimal object). No additional objects that the investigated person evaluated were equal in attractiveness as the object O
55. All of the remaining ChOs, apart from the objects with K
=
0 and K
=
1, will have fractional attractiveness values of 0
<
K
<
1 (fractional objects). How much attractiveness the fractional ChOs have in the opinion of the investigated person should be elicited with an appropriate method. Figure 9 shows all fractional ChOs. Table 5 contains values of height K
1 and weight K
2 characterizing all CHOs.Table 5
Fractional characteristic objects ChO (0
<
K
<
1) with the optimal object (K
=
1).
O
i
j
O
32
O
33
O
34
O
35
O
43
O
44
O
45
O
46
O
53
O
54
O
55
O
56
O
57
O
64
O
65
O
66
O
67
K
1
170
170
170
170
180
180
180
180
185
185
185
185
185
190
190
190
190
K
2
60
70
80
85
70
80
85
90
70
80
85
90
100
80
85
90
100Figure 9
Domain of the considered problem of male attractiveness with fractional characteristic objects0
<
K
<
1 marked.Step 4 (determining values of the multicriteriaK for fractional objects).
Determining attractiveness is realized with a comparative tournament-like method in which the investigated person compares the attractiveness of all possible pairs of fractional objects. The comparative method presented in this section has many advantages. It is simple and decreases the mental burden of the investigated person. People usually avoid greater burdens; if the elicitation method used in the research process burdens them too much, part of the people may cease to carefully analyze the alternatives presented to them and may begin to give hasty, imprecise answers. In the tournament version presented here, the person is given a pair of objects and has to decide which object is more attractive in his or her opinion. For example, the person has to analyze a pair of male bodies (170, 85) and (180, 80). The object chosen as more attractive receives 1 point and the less attractive object receives 0 points. If the investigated person has difficulties with the decision, then it means that the compared two objects are approximately equally attractive to him or her and both objects receive 0.5 points (a draw). In the tournament process, first the fractional objectO
32 from Table 5 undergoes a tournament with all other remaining objects. The process is then repeated with object O
33 and so on. Each object fights with all remaining objects and wins or loses points. After all tournaments finished, the score of each object is calculated. Table 6 shows example results of tournaments realized by the first object O
32 = (170, 60). Values 1 or 0 or 0.5 given on the right-hand side of each object is the result of the attractiveness comparison of both objects by the investigated person. In the lower part of Table 6, the aggregated object, which was called “all not acceptable objects,” was placed. The number of objects that were not acceptable to the investigated person is equal to 46. The weakest fractional object has to achieve attractiveness K
>
0. To satisfy this condition, each fractional object has to be compared with the full set of not acceptable objects with K
=
0. Similarly, if the number of the optimal objects with K
=
1 is greater than 1, then the set of all optimal objects has to be created and compared as one set with all fractional objects. In the analyzed example, only one object (185, 85) composes the set of optimal objects. All fractional objects have to win the tournament with the set of not acceptable objects and lose only to the set of optimal objects.
As Table6 shows, object O
32 = (170, 60) represents only a marginally attractive body to the investigated person. This body type won only three points in comparison with the other 17 fractional objects. Because of the paper volume constraints, the tournament results of the other 17 objects are not shown here. Only summarized results with rank places and the normalized scores are shown. The “normalized score” of an object is understood as the sum of points won by the object divided by the greatest score won by the best (optimal) object with K
=
1. In this case, the optimal object is O
55 = (185, 85). This object won 17 points. The normalized score of the object O
i
j refers to its relative attractiveness K
(
O
i
j
). The final results are shown in Table 7. As can be seen in Table 7, some objects won equal numbers of points. For example, two objects won eleven points each and two objects six points each. Each column in Table 7 means one rule of the fuzzy multicriteria model. For example, the second column generates the rule “IF (K
1 near 185 cm) AND (K
2 near 90 kg) THEN (attractiveness K near 0.941), where the quantifiers “near 185 cm” and “near 90 kg” are defined by corresponding membership functions shown in Figures 7 and 8. The full rule base has 17 rules with nonzero conclusions and 31 rules with zero-conclusions and is shown in Table 8.Table 6
Tournament results of the characteristic objectO
32 = (170, 60) with the remaining, fractional, characteristic objects. Object O
32 won three points in all tournaments.
height
(
cm
)
weight
(
kg
)
(
score
)
170
60
(
0
)
170
70
(
1
)
170
60
(
1
)
170
80
(
0
)
170
60
(
0
)
170
85
(
1
)
170
60
(
0
)
180
70
(
1
)
170
60
(
0
)
180
80
(
1
)
170
60
(
0
)
180
85
(
1
)
170
60
(
1
)
180
90
(
0
)
170
60
(
0
)
185
70
(
1
)
170
60
(
0
)
185
80
(
1
)
170
60
(
0
)
185
85
(
1
)
170
60
(
0
)
185
90
(
1
)
170
60
(
0
)
185
100
(
1
)
170
60
(
0
)
190
80
(
1
)
170
60
(
0
)
190
85
(
1
)
170
60
(
0
)
190
90
(
1
)
170
60
(
0
)
190
100
(
1
)
170
60
(
1
) all not acceptable objects (0)Table 7
Final results of tournaments of characteristic objects.K is the identified object attractiveness (the multicriteria value), the number of points won in comparison with all remaining objects (fractional objects plus the set of not acceptable objects). Each column of the table generates one rule of the type IF (K
1 near…) AND (K
2 near…) THEN (K near…), where K means attractiveness of the characteristic object.
O
i
j
O
55
O
56
O
54
O
66
O
65
O
64
O
67
O
44
O
57
O
45
O
53
O
46
O
43
O
33
O
32
O
34
O
35
K
1
185
185
185
190
190
190
190
180
185
180
185
180
180
170
170
170
170
K
2
85
90
80
90
85
80
100
80
100
85
70
90
70
70
60
80
85
S
17
16
15
13
12
11
11
9
8.5
8
7
6
6
4
3
2.5
1
K
1
.941
.882
.765
.705
.647
.647
.529
.500
.471
.412
.353
.353
.235
.176
.147
.059
Rank
1
2
3
4
5
6
6
7
8
9
10
11
11
12
13
14
15Table 8
Rule base of the individual multicriteriaK = f(K
1, K
2) representing male body attractive preferences of the investigated person.
K
2 [kg] weight
K
1 [cm]  height
Near 160 cm
Near 170 cm
Near 180 cm
Near 185 cm
Near 190 cm
Near 200 cm
Near 50 kg
0
0
0
0
0
0
Near 60 kg
0
0.176
0
0
0
0
Near 70 kg
0
0.235
0.353
0.412
0
0
Near 80 kg
0
0.147
0.529
0.882
0.647
0
Near 85 kg
0
0.059
0.471
1.000
0.705
0
Near 90 kg
0
0
0.353
0.941
0.765
0
Near 100 kg
0
0
0
0.500
0.647
0
Near 110 kg
0
0
0
0
0
0Figure10 shows how the attractiveness value increases depending on the rank place won by particular objects. As Figure 10 shows, an increase in rank place translates to an increase in multicriteria value K. This increase is not drastic but rather gradual, which supports the stability of the applied method of the attractiveness elicitation from the human mind.Figure 10
Illustration of the increase in object attractivenessK with changing rank place of the characteristic object.Figure11 shows a visualization of the functional dependence of the multicriteria K on the component-criteria K
1 (height) and K
2 (weight). Figure 11 allows us to conclude that the functional surface of the multicriteria model is regular. It increases or decreases gradually, has no drastic jumps, and is rather smooth. These features testify for the correctness of the identified model because the human mind does not rapidly changes attractiveness of objects after only small changes of attribute values. Similar results and conclusions from investigations of smoothness of human membership functions were found by Schwab (Schwab axioms) and are described in [4, 59]. It is very interesting that the functional surface of the attractiveness in Figure 11 is placed aslant in the domain of the components K
1 and K
2. This fact testifies to a correlated influence of the height K
1 and the weight K
2 on the global attractiveness evaluation of the human body. This correlation could not be detected if, for example, the linear criteria model was used for modeling body attractiveness. However, the applied fuzzy modeling method allows these correlations to be detected.Figure 11
Surface of the individual multicriteriaK
=
f
(
K
1
,
K
2
) for male body attractiveness depending on height (K
1) and weight K
2. This model is from author Andrzej Piegat.Step 5 (fuzzy-model construction for the multicriteriaK on the basis of the tournament results).
The multicriteria modelK
=
f
(
K
1
,
K
2
) elicited from the mind of the investigated person can be used for predicting his or her evaluation of new objects that are not characteristic ones. The ChOs are reference objects possessing known values of the multicriteria K, which enable interpolative evaluation of new objects. However, attractiveness calculations for new objects are only then possible when the rule base representing the tournament results has been defined. The rule base given in Table 8 was elaborated on the basis of Figure 9 and Table 7. The rule base shown in Table 8, together with membership functions of the component-criteria K
1 and K
2, which were shown in Figures 7 and 8, enables the attractiveness calculations for new evaluated objects (alternatives). If we want to calculate attractiveness K for any object (human body) with values different from characteristic objects given in Table 8, for example, for the object (K
1 = 177 cm, K
2 = 93 kg), then we make it similarly as in any fuzzy model using fuzzification, inference, and defuzzification, which is described in any book of fuzzy modeling, for example, in [58, 59]. It is recommended to use operator Product for realization of conjunction operation AND, Mamdani implication operator for conclusion determining of single rules, MAX operator for aggregation of single-rule conclusions, and singleton method for defuzzification [59].
In the final result, the rank reversal phenomenon is not observed in the COMET method because evaluation of each particular alternative is functional dependent to all characteristic objects and their preferences. It means that COMET method identifies a fuzzy function, which determines values of evaluations for all alternatives from the domain. Therefore, if we evaluate an alternative by using a particular model, the evaluating value is constant for this alternative. This fact guarantees that method is rank reversal free method. If, for example, three alternatives are considered A1(165 cm, 75 kg), A2(182 cm, 70 kg), and A3(185 cm, 95 kg), then with use of the identified multicriteria model (Table 8) attractiveness of each alternative can be calculated. Results of calculations are K
A
1 = 0.044, K
A
2 = 0.377, and K
A
3 = 0.721. It allows determining ranking of alternatives: A
3
>
A
2
>
A
1. If the fourth alternative A
4(182.5 cm, 75 kg) had been added, the model calculated its attractiveness K
A
4 = 0.544. For four alternatives the ranking A
3
>
A
4
>
A
2
>
A
1 was determined, but the ranking of the three previous alternatives A
1, A
2, and A
3 has not changed (no rank reversal).


## 5. Method of Testing Human Multicriteria Models

To our knowledge there is no reported testing method for elicited multicriteria model from the human mind or accuracy testing methods that can be applied to known decision-making methods such as AHP, ANP, TOPSIS, ELECTRE, or PROMETHEE. Instead reports are limited to descriptions of the methods and explanations of how their use can identify “optimal” decisions or how “optimal alternatives” can be determined. For example, Saaty addresses the problem of which drink is consumed more in the USA, with only one criterion and seven options (coffee, wine, tea, beer, soda, milk, and water) [22, 24, 29]. Saaty applied the AHP method to estimate the proportion for each alternative and proposed the actual consumption (from statistical sources) as the standard reference for evaluation. For each object we calculated values of absolute and relative error, which ranged from 0.91 to 90% (Table 9). However, it is not possible to accurately assess the precision of particular multicriteria methods or the goodness of their “optimal” decisions. This is due to the impossibility of determining the precise value of the multicriteria function K
=
f
(
K
1
,
K
2
,
…
,
K
n
), that an investigated person assigns in his/her mind to the considered object. If this precise value is unknown, then calculation of the multicriteria model error Δ
K
=
K
-
K
M is not possible (K
M is the multicriteria value assigned to the considered object by the multicriteria model). In fact, it is impossible to determine the absolute error of the multicriteria model. However, it is possible to investigate model accuracy by a comparative approach, for example, with tournament method or a theoretical decision-making agent. For example, in a simple problem of AHP method with three alternatives and one criterion, we can suppose that a theoretical decision-making agent knows the searched attractiveness values of objects K
(
A
) = 0.28, K
(
B
) = 0.78, and K
(
C
) = 0.39. With the AHP method, the best fit matrix (5) for the following judgment was achieved:
(5)
1
1
4
1
2
4
1
3
2
1
3
1
.
On the basis of (5) the following K
M values were obtained: K
M
(
A
) = 0.1365, K
M
(
B
) = 0.6250, and K
M
(
C
) = 0.2385, with estimation of the model error between 19 and 51%. As an alternative to the theoretical decision-making agent, the survey approach can be applied. It is much easier for the investigated person to compare the attractiveness of two objects than to determinate the numerical value of multicriteria function from the human mind. If the elaborated multicriteria model is precise it should be able to arrive at the same winner objects as the investigated person for pairs of testing objects, and this idea forms the basis of the “method of comparative testing.” According to this method the investigated person compares two objects (O
1 and O
2) indicating which is more attractive (with a draw also possible). The identical task is also performed by the multicriteria model under assessment. If both comparison results are identical we can draw positive conclusion about the model precision. In this way, without determining the absolute value of the multicriteria function, precision of the multicriteria model can be assessed. This approach should not be limited by low numbers of testing pairs. Credible model testing should include many object pairs. Furthermore, in investigations of attractiveness comparisons logically inconsistent answers have been observed; that is, if the same pairs repeat two or more times amongst a number of compared pairs, a person can report inconsistent preferences. This occurs mostly in cases where objects are of similar attractiveness and difficult to differentiate. Thus, human evaluations are not always repeatable. Such inconsistent evaluations occur in both learning object pairs and testing pairs and cannot be avoided. Therefore, a multicriteria human model of 100% accuracy can rather never be achieved. Some of the best models have precision of 82% (for nine characteristic objects).Table 9
Drink consumption in the USA: comparison statistical data and result from the AHP method.
Water
Sodas
Coffee
Milk
Beer
Tea
Wine
AHP
0.327
0.190
0.177
0.129
0.116
0.042
0.019
Statistical
0.330
0.180
0.180
0.140
0.120
0.040
0.010
Abs error
0.003
0.010
0.003
0.011
0.004
0.002
0.009
R-e error
0.91%
5.56%
1.67%
7.86%
3.33%
5.00%
90.00%


## 6. Investigations of Multicriteria Models

Because of paper-volume limitations, it is not possible to describe all interesting investigations of eliciting human multicriteria functions that have been carried out by the authors and their co-workers. Author Wojciech Sałabun identified group multicriteria models for the attractiveness of the color achieved by mixing three simple colors at different ratios: percent of red color (R), percent of green color (G), and percent of blue color (B). The aim was to detect which mixture would be the most attractive for people. The investigation involved 302 subjects. As a result, a four-dimensional model of the group multicriteria function was identified: the global, visual attractiveness of the color mixturef
(
R
,
G
,
B
). Because the multicriteria model is four-dimensional it cannot be visualized directly. However, it can be visualized indirectly by its three-dimensional sections, as shown in Figure 12.Figure 12
Six 3-dimensional sections of the group (302 people) multicriteria of attractiveness of the 3-colour-mixture (red, green, and blue). The sections were made for the constant %-share of the red color equal to 50%.As Figure12 shows, the human models of color-mixture attractiveness are nonlinear. In other experiment, author Andrzej Piegat identified the individual multicriteria K of the attractiveness of gambling, depending on the winnings K
1 [PLN] and on the loss K
2 [PLN]. The identification was made with a few groups of students. The individual multicriteria functions of different students were usually similar. The differences between students were not considerable. Figure 6 shows the functional surface of the typical multicriteria K
=
f
(
K
1
,
K
2
). The results are described in [11].As can be seen in Figure13, the multicriteria surface for small stakes and losses is strongly nonlinear, and in any case it should not be modeled by the linear model K
=
a
0
+
a
1
K
1
+
a
2
K
2. For large stakes and losses, the nonlinearity of the model is still greater. Apart from the criteria presented in this section, the authors and their co-workers have identified other multicriteria models, such as(i)
attractiveness of the Internet domainK, depending on the registration continuity K
1 [months] of the domain, and the length of the domain name K
2, measured as number of letters in the domain name;
(ii)
effectivenessK of the parallel code in the standard OPENMP, depending on the coefficient BUR (Bus Utilization Ratio) K
1, the coefficient LCMI (L2 Chache Miss Impact) K
2, and the coefficient MDSR (Modified Data Sharing Ratio) K
3;
(iii)
evaluationK of the quality of candidates for the position of ward head at a hospital, depending on the length of the doctor’s practice [years] K
1, the number of operations and interventions K
2, the amount of time spent doing additional work outside the hospital K
3;
(iv)
qualityK of human sleep, depending on the time of sleep K
1, the moment of falling asleep K
2, and the interruptions of the sleep K
3;
(v)
evaluation of the effectiveness of a production systemK of integrated circuits, depending on the waiting time K
1 of the intermediate products, on the level K
2 of intermediate products in the system, and on the number of products K
3 waiting before the printing stand;
(vi)
attractivenessK of a new car, depending on the motor power K
1, the fuel consumption K
2, and the price K
3;
(vii)
attractivenessK of financial investment products, depending on the profit K
1 from the investment, the investment time K
2, and the investment risk K
3.Figure 13
Surface of the individual multicriteriaK
=
f
(
K
1
,
K
2
) for attractiveness of gambling depending on possible winnings K
1 [PLN] and losses K
2 [PLN]. This model is from author Andrzej Piegat for case of small stakes K
1 < 10 PLN and small losses K
2 < 10 PLN.The above examples are only a sample from a much greater set of investigated problems. The examples were three- or four-dimensional because the doctoral candidates achieved the task of the problem visualization, which also in the case of four-dimensional problems is partly realizable. However, the COMET can be used without limitations for problems of higher dimension. Independently of the problem dimension, an interesting question can be asked: how strongly nonlinear is the determined model? For evaluation of the multicriteria model, a special nonlinearity index was proposed in [11, 60]. Here it is presented as
(6)
N
-
Ind
K
=
∑
i
=
1
m
K
i
-
K
L
i
0.5
K
max
⁡
-
K
min
⁡
.
This nonlinearity index (
N
-
Ind
K
) is based on the absolute difference of points with the denominator (6) realizing normalization of the indicator to interval [
0,1
]. Nonlinearity of the fuzzy model approximating the criterion-function K
=
f
(
K
1
) will be the smaller; the smaller is the difference sum K
i
-
K
L
i of corresponding points lying on the fuzzy model and on the linear approximation of the criterion function.


## 7. Conclusions

This paper presents the principal shortcomings of known methods for elicitation of multicriteria evaluations from the human mind. Firstly, the instability of current models after the introduction of new objects (alternatives) to the original set produces the rank reversal phenomenon. Additionally, there is a lack of quantitative testing methods to assess the reliability of multicriteria models. In addition to highlighting these challenges we proposed the characteristic objects method that offers several important advantages over current methods. The characteristic objects method delivers stable evaluations of objects that are not subject to change by the introduction of new objects to the original object set. The COMET enables identification of nonlinear human multicriteria functions as well as detection of correlations between particular component-criteria. A novel method for assessing reliability of multicriteria models also was described. Multicriteria modeling is a continually evolving science. Although a number of advances have been described here, one can anticipate that they will provide only a platform for further methodological development.



---

*Source: 536492-2014-11-27.xml*