[image] [image] Ressource protégée - IMT Atlantique Accéder directement au contenu

*[image] [image]*

• Documentation

• FR
Français (FR)

• Anglais (EN)
Se connecter

• Portail HAL IMT Atlantique

• Recherche
Loading...

## Recherche avancée

Information de documents

Titres

○ Titres

○ Sous-titre

○ Titre de l'ouvrage

○ Titre du volume (Série)

○ Champ de recherche par défaut (multicritères) + texte intégral des PDF

○ Résumé

○ Texte intégral indexé des documents PDF

○ Mots-clés

○ Type de document

○ Sous-type de document

○ Tous les identifiants du document

○ Identifiant HAL du dépôt

○ Langue du document (texte)

○ Pays (Texte)

○ Ville

○ À paraître (true ou false)

Ajouter

Auteur
A regret-based query selection strategy for the
incremental elicitation of the criteria weights in an
SRMP model

Arwa Khannoussi, Alexandru-Liviu Olteanu, Patrick Meyer, Nawal Benabbou

► To cite this version:

Arwa Khannoussi, Alexandru-Liviu Olteanu, Patrick Meyer, Nawal Benabbou. A regret-based query selection strategy for the incremental elicitation of the criteria weights in an SRMP model. Operational Research, 2024, 24 (2), pp.12. 10.1007/s12351-024-00823-y. hal-04519903

HAL Id: hal-04519903

https://imt-atlantique.hal.science/hal-04519903v1

Submitted on 28 Mar 2024

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
## Auteur (multicritères)

○ Auteur (multicritères)

○ Auteur : Nom complet

○ Auteur : Nom de famille

○ Auteur : Prénom

○ Auteur : Complément de nom, deuxième prénom

○ Auteur : Organisme payeur

○ Auteur : IdHal (chaîne de caractères)

○ Auteur : Fonction

○ Auteur : personID (entier)

○ Auteur : Identifiant de l'organisme payeur

○ Auteur : Identifiant de la structure

○ Directeur de thèse

○ Éditeur

○ Éditeur scientifique

○ Éditeur de la série

## Ajouter

### Structure

### Structure (multicritères)

○ Structure (multicritères)

○ Structure : Acronyme

○ Structure : Nom

○ Structure : Code

○ Structure : Pays

○ Structure : Type

○ Structure : État dans le référentiel

○ Structure : Identifiant HAL de la structure

○ Structure/équipe de recherche : Pays

○ Structure/regroupement d'équipes : Pays

○ Structure/laboratoire : Pays

○ Structure/regroupement de laboratoires : Pays

○ Structure/institution : Pays

○ Structure/regroupement d'institutions : Pays

## Ajouter

## Autres
# A regret-based query selection strategy for the incremental elicitation of the criteria weights in an SRMP model

Arwa Khannoussi¹, Alexandru-Liviu Olteanu², Patrick Meyer³* and Nawal Benabbou⁴

¹*IMT Atlantique, LS2N, UMR CNRS 6004, F-44300 Nantes.

²Lab-STICC, UMR 6285, CNRS, Equipe Decide, Université Bretagne Sud.

³IMT Atlantique, Lab-STICC, UMR CNRS 6285, F-29238 Brest, France.

⁴Sorbonne Université, LIP6, Paris.

*Corresponding author(s). E-mail(s):
patrick.meyer@imt-atlantique.fr;

Contributing authors: arwa.khannoussi@imt-atlantique.fr; alexandru.olteanu@univ-ubs.fr; nawal.benabbou@lip6.fr;

## Abstract

SRMP, which stands for “Simple Ranking Method using Reference Profiles”, is a Multi-Criteria Decision Aiding model which aims at ranking alternatives according to the preferences of a Decision Maker (DM), according to the principles of outranking techniques. Determining the preference parameters of SRMP can be tiring for the DM, who is asked to compare several alternatives pairwisely during a preference elicitation process. It has been proposed in the literature to use an incremental elicitation process which selects informative pairs of alternatives which are submitted to the DM in sequence. The goal in such a process it to refine the SRMP model at each iteration, until a robust recommendation is determined. In this research, using a regret-based elicitation approach, we present a new heuristic for choosing the pairs of alternatives sequentially submitted for evaluation to the DM. We also provide a mixed-integer linear program for an efficient computation of regret values in practice. We limit our solution to the elicitation of the criteria weights, a subset of the SRMP model’s
Champ de recherche par défaut (multicritères)

○ <u>Champ de recherche par défaut (multicritères)</u>

○

○ <u>Revues (multicritères)</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Titre abrégé</u>

○ <u>Revue : Titre</u>

○ <u>Revue : Identifiant interne</u>

○ <u>Revue : Date de début de publication</u>

○ <u>Revue : ISSN électronique</u>

○ <u>Revue : ISSN</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Couleur dans SHERPA/RoMEO</u>

○ <u>Revue : État dans le référentiel</u>

○

○ <u>Colloque (multicritères)</u>

○ <u>Colloque : Titre</u>

○ <u>Colloque : Organisateur</u>

○ <u>Colloque : date de début (Année)</u>

○ <u>Colloque : date de fin (Année)</u>

○ <u>Conférence invitée (oui/non)</u>

○

○ <u>Projets ANR (multicritères)</u>

○ <u>Projet ANR : Acronyme</u>

○ <u>Projet ANR : Acronyme du programme</u>

○ <u>Projet ANR : Code décision (référence)</u>

○ <u>Projet ANR : Nom</u>

○ <u>Projet ANR : Identifiant interne</u>

○ <u>Projet ANR : État dans le référentiel</u>

○

○ <u>Projets européens (multicritères)</u>

○ <u>Projet européen : Acronyme</u>

○ <u>Projet européen : Identifiant de l'appel à projet</u>

○ <u>Projet européen : Référence</u>

○ <u>Projet européen : Nom</u>

○ <u>Projet européen : Date de fin</u>

○ <u>Projet européen : Financement</u>

○ <u>Projet européen : Date de début</u>

○ <u>Projet européen : État dans le référentiel</u>
2 *A regret-based query selection strategy SRMP parameters elicitation*

parameters, and we demonstrate that in this setting, the suggested
heuristic outperforms previously examined query selection algorithms.

**Keywords:** Multi-Criteria Decision Aiding, incremental preference elicitation, query selection strategy, regret-based approach

MSC Classification: 91B06, 91B08, 65L12, 90C11

# 1 Introduction

Multi-Criteria Decision Aiding (MCDA) (Roy, 1996) aims at helping Decision Makers (DMs) to solve discrete decision problems involving multiple, often conflicting criteria. The goal can be to *choose* an appropriate solution among a set of decision alternatives, *sort* alternatives into preferentially ordered categories, or *rank* them from the “best” one to the “worst” one. MCDA models are usually classified into three approaches (Bouyssou et al, 2006): (i) Multi-Attribute Value Theory (MAVT) (Keeney and Raiffa, 1976), (ii) outranking-based approaches (Figueira et al, 2005b) and (iii) rule-based models (Greco et al, 2001). In this paper, we focus on the Simple Ranking Method using Reference Profiles (SRMP) (Bouyssou and Marchant, 2013; Rolland, 2013), which is based on the outranking paradigm. This ranking method can be applied in various real-world applications, because of its capacity to deal with heterogeneous evaluation scales, and because it produces a preorder of the alternatives. It can also be used to explain the decision recommendation, as it is inspired by normative outranking-based sorting approaches (Bouyssou and Pirlot, 2009).

In MCDA an important step consists in the elicitation (or learning) of the DM’s preferences. An indirect elicitation approach is usually recommended, in which the DM is asked to provide so-called “holistic” judgments, such as assignment examples in the sorting context, or a partial pre-order on the alternatives or pairwise comparisons of alternatives in a ranking or choice context. Classically, two types of indirect elicitation approaches are considered (Mousseau, 1995): “batch elicitation”, in which the learning data is given all at once to the learning algorithm (Bouyssou et al, 2006), and “incremental elicitation”, in which the learning data arrives sequentially and the model is improved iteratively (Boutilier et al, 2006; Fürnkranz and Hüllermeier, 2011; Chajewska et al, 2000).

In this paper, we study the potential of incremental elicitation to learn some
of the preference parameters of the SRMP model, namely the criteria weights.
Khannoussi et al (2021) have already studied this elicitation paradigm with
the SRMP model, but in this work, we propose algorithms which outperform
their results.

In the *batch* setting, the elicitation of the preference parameters of the
SRMP model (criteria weights, reference profiles and a corresponding lexi-
cographic order) are studied in Olteanu et al (2022), where the preference
○ Projet européen : Identifiant interne

○

○ Date de production : année

○ Date de mise en ligne : année

○ Date de publication : année

○ Date d'écriture : année

○ Date de modification du dépôt : année

○ Date de dépôt : année

○ Date de publication électronique : année

○

○ Collection HAL (multicritères)

○ Collection HAL : catégorie

○ Collection HAL : Code

○ Collection HAL : Nom

○ Collection HAL : Identifiant interne

○

○ Identifiant interne du contributeur/déposant

○ Nom complet du contributeur/déposant

○

○ Domaines

○ Domaine primaire

○ Domaine racine

○ Sous-domaine niveau 1

○ Sous-domaine niveau 2

○ Sous-domaine niveau 3

○

○ Statut du document

○ Version du document

○ Type de dépôt

○ Type de document

○ ISBN

○ Numéro - référence

○ Identifiant DOI

○

○ Classification

○ Audience

○ Vulgarisation

○ Comité de lecture - texte (oui ou non)

○ Actes de colloque

○ Référence interne
parameters are inferred by solving a mixed integer linear optimization problem (MILP). Another approach which consists in solving a Boolean satisfiability (SAT) problem was proposed by Belahcène et al (2018). This approach is faster than the previous one and can handle larger sets of pairwise comparisons of alternatives. Liu et al (2014) suggest to learn the parameters of an SRMP model through a metaheuristic. On the one hand, the advantage is that this approach is faster than the MILP approach, but on the other hand, no guarantee is given that the generated model fits perfectly the input pairwise comparisons.

More recently, Khannoussi et al. Khannoussi et al (2018, 2021) proposed to learn the parameters of the SRMP model via an *incremental* elicitation process aimed at decreasing the cognitive fatigue of the DM by reducing the number of required learning examples (pairwise comparisons of alternatives). At every step of the process, a pair of alternatives is selected using a “heuristic” and presented to the DM in order to augment the information used to infer the SRMP model. The proposed heuristic (denoted by $H_{mp}$ hereafter) consists in selecting a pair of alternatives that requires a maximal number of profiles in order to be discriminated by the current model. In this paper, we propose a new query selection heuristic for the SRMP model which is inspired by regret-based incremental elicitation approaches studied in the context of MAVT. We show that this new heuristic dominates $H_{mp}$ by generating models whose generalization power is way higher with a lower number of input comparisons.

In MAVT methods, it has been proposed to use the *minimax regret decision criterion* within an incremental elicitation approach in order to make robust decisions under preference imprecision and generate informative preference queries (White et al, 1984; Boutilier et al, 2006). The idea is to save preference queries by identifying the part of preference information that is necessary to solve the instance under consideration without seeking to precisely specify the decision model. This approach, sometimes referred to as *regret-based incremental elicitation*, was efficiently used in various decision contexts, such as multicriteria decision making (Benabbou et al, 2017), collective decision making (Lu and Boutilier, 2011), and decision making under risk (Regan and Boutilier, 2009). The adaptation of minimax regret elicitation strategies to outranking methods is not straightforward as regrets are usually defined using direct comparisons of solutions (without considering any reference profile). In this paper, we propose a new definition of regrets which takes into account the reference profiles of the SRMP model and that facilitates generating informative preference queries during the elicitation process.

The rest of this article is structured in the following way: First, we recall some useful background on the SRMP model in Section 2. Then, in Section 3 we introduce a new notion of regret for the SRMP model together with a regret-based query selection heuristic which efficiently selects pairs of alternatives to be presented to the DM at every step of the incremental elicitation procedure. We also propose a mixed-integer linear programming formulation to solve the corresponding regret-based optimization problems at every step
○ Financement

○ Collaborations

Ajouter

Lancer la recherche
Recherche experte (SolR)

Recherche experte (SolR)

Lancer la recherche
Vers la recherche avancée

• Déposer

Archives ouvertes Hal IMT
Atlantique

• Accueil

• Consulter

○ Par auteur

○ Par type de publication

○ Par année

○ Les brevets IMT Atlantique

○ Tous les brevets des chercheurs d'IMT Atlantique

○ Les thèses

○ Les HDR

○ La collection des rapports de recherche IMT Atlantique

○ Par laboratoire

■ GEPEA (UMR)

■ IRISA (UMR)

■ Lab-STICC (UMR)

■ LATIM (UMR)

■ LEGO (Equipe d'accueil)

■ LS2N (UMR)

■ MARSOUIN (GIS)
4 *A regret-based query selection strategy SRMP parameters elicitation*

of the procedure. Finally, we report the results of numerical tests on gener-
ated data in Section 4, showing that our procedure is more efficient than other
heuristics proposed in the literature.

## 2 The SRMP model

In the outranking paradigm, alternatives are compared pairwisely through an “at least as good as” relation. This binary relation, $\succsim$ is called the “outranking relation” (Roy, 1991). The general idea behind this relation is that an alternative $a$ outranks another one, say $b$, if there are enough arguments in favor of the statement “$a$ is at least as good as $b$” and if there is no strong opposition against it. However, comparing all alternatives pairwisely according to such an outranking relation can lead to a non-transitive relation, with the possible appearance of cycles. A complete ranking is therefore difficult to achieve through such a relation (Figueira et al, 2005a). To avoid this problem, it has been proposed in Rolland (2013) to use of a so-called reference point when comparing two alternatives. Intuitively, $a$ is said to be preferred to be $b$ if and only if $a$ outranks this reference point “stronger” than $b$ does. In this paper, we use the SRMP method which makes use of several reference points, lexicographic ordered, as well as criteria weights to define the “strength” of those outranking arguments. Let us now recall the basics of SRMP, inspired from the presentation made in Khannoussi et al (2021).

Let $\mathcal{A}$ be the set of $n$ alternatives that are evaluated with respect to a set of $m$ criteria $M = \{1, \dots, m\}$. For a given alternative $a \in \mathcal{A}$, let $a_j$ denote the evaluation of $a$ on criterion $j \in M$. Consequently, $a$ can be identified with its performance vector $a = (a_1, \dots, a_m)$. On each criterion $j \in M$, the evaluation scale generates a preorder $\succsim_j$ such that $a \succsim_j b$ if $a$ is at least as good as $b$ on criterion $j$.

An SRMP model is characterized by multiple preference parameters, whose
precise values depend on the considered DM:

* $\mathcal{P}$: a set of $k$ reference profiles (the reference points from above) denoted by $p^h \equiv (p_1^h, \dots, p_m^h)$, with $h \in \{1, \dots, k\}$. These profiles are such that $p^h \succsim_j p^{h+1}$ for all $h \in \{1, \dots, k-1\}$, which means that $p^h$ dominates $p^{h+1}$ for all $h \in \{1, \dots, k-1\}$.

* $\sigma$: a permutation of $\{1, \dots, k\}$, defining a lexicographic order on the profiles, indicating the order in which the profiles will be used when comparing two alternatives.

* $w = (w_1, \dots, w_m)$: a vector of weights attached to criteria such that $w_j > 0$ for all $j \in M$ and $\sum_{j \in M} w_j = 1$.

For any alternative $a \in \mathcal{A}$ and any profile $p^h \in \mathcal{P}$, we define the weight $f_w(a, p^h)$ representing the strength of arguments supporting the statement “a
■ SUBATECH (UMR)

○ Par chaire

■ PRACOM

■ RESOH

• Par département

○ DAPI - Département Automatique, productique & informatique

○ DELMA - Langues, Médiation scientifique, Activités physiques, sportives et d'expression

○ DI2S - Département interdisciplinaire de sciences sociales

○ DSD - Département de Science des Données

○ DSEE - Systèmes énergétiques & environnement

○ INFO - Informatique

○ MEE - Mathematical and electrical engineering

○ MO - Micro-ondes

○ OPT - Optique

○ SRCD - Systèmes réseaux, cybersécurité & droit du numérique

○ SUBATECH - Physiques subatomiques et technologies

○ Hors département

• Par équipe

○ 2AI - Algorithm Architecture Interactions (Lab-STICC)

○ ADOPNET - Technologies avancées pour les réseaux d'opérateurs (IRISA)

○ ARCAD - Architectures matérielles et outils de CAO (Lab-STICC)

○ ASMP - Architectures et Systèmes opto-Microondes et Photoniques (Lab-STICC)

○ ASTRO - Astroparticules (SUBATECH)

○ Autre (SUBATECH)

○ BRAIN (Lab-STICC)

○ CODES (Lab-STICC)

○ CODEX - Commande, Observation, Diagnostic et Expérimentation (LS2N)
is at least as good as $p^h$" as follows:

$$f_w(a, p^h) = \sum_{j \in C(a, p^h)} w_j \quad (1)$$

where $C(a, p^h) = \{j \in M : a \succsim_j p^h\}$. When comparing two alternatives $a, b \in A$ through a profile $p^h \in \mathcal{P}$, three cases can be distinguished:

* if $f_w(a, p^h) > f_w(b, p^h)$, then $a$ is said to be strictly preferred to $b$ with respect to $p^h$, which is denoted by $a \succ_{p^h} b$.

* if $f_w(a, p^h) = f_w(b, p^h)$, then $a$ is said to be indifferent to $b$ with respect to $p^h$, which is denoted by $a \sim_{p^h} b$.

* if $f_w(a, p^h) < f_w(b, p^h)$, then $b$ is said to be strictly preferred to $a$ with respect to $p^h$, which is denoted by $b \succ_{p^h} a$.

The ranking generated by SRMP, representing the DM's preferences, is obtained by considering sequentially the profiles $p^{\sigma(1)}, p^{\sigma(2)}, \dots, p^{\sigma(k)}$ according to the lexicographic order $\sigma$. More precisely, $a$ is strictly preferred to $b$ (resp. $b$ is strictly preferred to $a$) as soon as a profile in the lexicographic order states that $a$ is strictly preferred to $b$ (resp. $b$ is strictly preferred to $a$). Otherwise, when no profile was able to discriminate between $a$ and $b$, both alternatives are considered as indifferent. More precisely, according to the SRMP model:

* $a$ is strictly preferred to $b$ ($a \succ b$), if and only if:

$$\exists h \in \{1, \dots, k\} \text{ s.t. } a \succ_{p^{\sigma(h)}} b \text{ and } \forall \ell < h, a \sim_{p^{\sigma(\ell)}} b \quad (2)$$

* $a$ is indifferent to $b$ ($a \sim b$), if and only if:

$$\forall h \in \{1, \dots, k\}, a \sim_{p^{\sigma(h)}} b \quad (3)$$

## 3 A Regret-Based Incremental Elicitation Process

The SRMP method presented in the previous section involves a weighting vector $w = (w_1, \dots, w_m)$ representing the importance of criteria according to the DM's preferences. In this paper, we assume that this parameter $w$ is initially not known, and our aim is to propose an incremental elicitation procedure for its assessment. The remaining preferential parameters are supposed to be known.

Similarly to Khannoussi et al (2018, 2021), at each iteration step of the incremental elicitation process, a heuristic is used to select a pair of alternatives that will be presented to the DM. She is then asked to express her preferences on these alternatives, either in the form of a strict preference or an indifference. This information is then added to the set of preference statements obtained so far, which is then used to restrict the set of admissible parameters for the SRMP model. The process stops after a certain number of iterations, or when
○ CoSyDe - Communication System Design (Lab-STICC)

○ CPS3 - Conception, Pilotage, Surveillance et
Supervision des systèmes (LS2N)

○ DECIDE (Lab-STICC)

○ DH - Dispositifs Hyperfréquences (Lab-STICC)

○ EASE - Enabling Affordable Smarter Environment
(IRISA)

○ ERMINE - mEasuRing and ManagIng Network operation
and Economic (IRISA)

○ ETIC - Evaluation des dispositifs TIC (LEGO)

○ Gallinette (LS2N)

○ INUIT - Immersive Natural User Interaction team (Lab-
STICC)

○ IRIS - SécurIté et Résilience des Systèmes d'Information
(Lab-STICC)

○ KM3NET - Sous-thématique Neutrinos (SUBATECH)

○ M3 - Marine Mapping & Metrology (Lab-STICC)

○ MATHNET - Math & Net (Lab-STICC)

○ MATRIX - Modèles et Algorithmes pour le TRaItement
et l'eXtraction de l'information (Lab-STICC)

○ MODELIS - Modélisation, Optimisation et DEcision pour
la Logistique, l'Industrie et les Services (LS2N)

○ MOTEL - Modèles et outils pour soutenir l'apprentissage
(Lab-STICC)

○ NaoMod - Nantes Software Modeling Group (LS2N)

○ Neutrino (SUBATECH)

○ NSF - Nanostructures Fonctionnelles (Lab-STICC)

○ OCIF - Objets communicants et Internet du Futur
(IRISA)

○ ODYSSEY - Océan Dynamique Observations Analyse

○ OGRE - Optimisation Globale et Résolution Ensembliste
(LS2N)

○ OSE - Optimisation - Système - Energie (GEPEA)

○ OSE - Observations Signal & Environnement (Lab-
STICC)

○ P4S - Processes for Safe and Secure Software and
Systems (Lab-STICC)

○ PACCE - Perception, Action, Cognition pour la
Conception et l'Ergonomie (LS2N)

○ PIM (Lab-STICC)
6 A regret-based query selection strategy SRMP parameters elicitation

the DM considers that the current SRMP model (it can be constructed at any point in the process) is faithful to her preferences. We propose here a new query selection heuristic that is inspired by the minimax regret decision criterion which is commonly used in the MAVT setting. Our heuristic helps in reducing the number of preference queries that are needed for converging towards a good enough model in practice, and allows to stop the elicitation process sooner using a regret threshold that is acceptable for the DM.

3.1 Query Selection Heuristic

At any step of the elicitation procedure, we are given a (possibly empty) set $\mathcal{L}_P$ of pairs $(a, b) \in \mathcal{A} \times \mathcal{A}$ such that $a$ is known to be preferred to $b$ by the DM (which is filled iteratively during the elicitation steps). We are also given a (possibly empty) set $\mathcal{L}_I$ of pairs $(a, b) \in \mathcal{A} \times \mathcal{A}$ such that $a$ is known to be indifferent to $b$. Let $W$ be the set of weighting vectors $w$ which are compatible with the available preference statements, i.e.

• $w \in [0, 1]^m$

• $\sum_{j=1}^{m} w_j = 1$

• $a \succ b$ for all $(a, b) \in \mathcal{L}_P$

• $a \sim b$ for all $(a, b) \in \mathcal{L}_I$

Now the problem consists in identifying a pair of alternatives $(a, b) \in \mathcal{A} \times \mathcal{A}$ that enables to reduce the parameter imprecision in an efficient way when presented to the DM. Note that some pairs $(a, b)$ will be more informative than others. For instance, asking the DM to compare two alternatives $a, b$ such that $a \succ b$ for all admissible weights $w \in W$ will provide no value as no weighting vectors will be eliminated after collecting the DM's answer. On the other hand, if the statements $a \succ b$ and $b \succ a$ are supported by two different admissible weighting vectors, then $(a, b)$ will possibly constitute a good preference query. Note that an answer of type “$a$ is indifferent to $b$” is generally much more informative than a strict preference as it amounts to imposing equality constraints of type $f_w(a, p^h) = f_w(b, p^h)$, with $h \in \{1, \dots, k\}$.

In this work, we propose to use a regret-based approach that consists in evaluating the relevance of a query by considering the worst-case loss (regret) induced when deciding whether $a$ is ranked before $b$ or not, given the current parameter imprecision. The worst-case regret of ranking $a$ before $b$ is given by the strongest support of the opposing assertion $b \succ a$ over all the admissible weighting vectors. More precisely, we use the following definition of regrets:

**Definition 1** The profile-based Pairwise Max Regret of $a \in \mathcal{A}$ outranking $b \in \mathcal{A}$ according to profile $p^{\sigma(h)}$, denoted by $PMR^h(a, b, W)$, is defined by:

$$PMR^h(a, b, W) := \max_{w \in W'} \{f_w(b, p^{\sigma(h)}) - f_w(a, p^{\sigma(h)})\}$$

where $W' := \{w \in W : \forall \ell < h, a \sim_{p^{\sigma(\ell)}} b\}$.
○ Plasma (SUBATECH)

○ Prisma (SUBATECH)

○ Radiochimie (SUBATECH)

○ RAMBO - Robot interaction, Ambient system, Machine
learning, Behaviour, Optimization (Lab-STICC)

○ ReV - Robotique Et Vivant (LS2N)

○ Scénario (SUBATECH)

○ SEN - Structure et Energie Nucléaires (SUBATECH)

○ Services techniques (SUBATECH)

○ SI3 - Sécurité, Intelligence et Intégrité de l'Information
(Lab-STICC)

○ SMART (SUBATECH)

○ SOTERN - Self-prOtecting The futurE inteRNet (IRISA)

○ Stack - Software Stack for Massively Geo-Distributed
Infrastructures (LS2N)

○ T2O - Technologies et µ-Technologies Optiques (Foton-
SP-OGC)

○ TASC - Théorie, Algorithmes et Systèmes en Contraintes
(LS2N)

○ TEAM - Traitement Eau Air Métrologie (GEPEA)

○ Théorie (SUBATECH)

○ TIM - Traitement de l'Information Médicale (LaTIM)

○ VERTE - Valorisation Energie-matière des Résidus et
Traitement des Emissions (GEPEA)

○ Xénon (SUBATECH)

X

X

X

Loading...

X

Le document que vous demandez est en access restreint
jusqu'au 2024-09-28 !

Revenir a la notice

Retourner à l'accueil

Contact
$PMR^h(a, b, W)$ is simply equal to $-\infty$ when $W' = \emptyset$. By definition, the profile-based pairwise max regret $PMR^h(a, b, W)$ is the worst-case loss induced by the assertion $a$ outranks $b$ when profile $p^{\sigma(h)}$ is the discriminating profile. Next, we define the worst-case loss of the assertion $a$ outranking $b$ (considering all possible discriminating profiles) as follows:

**Definition 2** The Pairwise Max Regret $PMR(a, b, W)$ of $a \in \mathcal{A}$ outranks $b \in \mathcal{A}$ is defined by:

$$PMR(a, b, W) := \max_{h \in \{1, \dots, k\}} PMR^h(a, b, W)$$

When $PMR(a, b, W) \le 0$, we have $f_w(b, p^{\sigma(h)}) \le f_w(a, p^{\sigma(h)})$ for all possible discriminating profiles $p^{\sigma(h)} \in \mathcal{P}$ and all admissible weighting vectors $w \in W$. In that case, we know that $a$ is at least as good as $b$ for all admissible weighting vectors $w \in W$, and therefore asking the DM to compare these two alternatives is not informative. If instead we have $PMR(a, b, W) > 0$ and $PMR(b, a, W) > 0$, then we need more preference information in order to determine whether $a$ outranks $b$ or not. We define the worst-case loss of a pair $(a, b)$ as follows:

**Definition 3** The Min Pairwise Max Regret $MPMR(\{a, b\}, W)$ of a pair $(a, b) \in \mathcal{A} \times \mathcal{A}$ is defined by:

$$MPMR(\{a, b\}, W) := \min \{PMR(a, b, W), PMR(b, a, W)\} \quad (4)$$

Note that $MPMR(\{a, b\}, W) \le 0$ means that the ranking between $a$ and $b$ is known and no query pertaining to them is needed. Otherwise, the larger this value, the larger the imprecision on their ranking, and asking the DM to compare $a$ to $b$ will lead to $MPMR(\{a, b\}, W)$ dropping below 0 upon updating $W$ using the answer. For this reason, we propose the following query selection heuristic:

**The Regret-Based Query Selection Strategy:** At each iteration step of the elicitation procedure, select the pair of alternatives $(a, b) \in \mathcal{A} \times \mathcal{A}$ with the maximum value of $MPMR(\{a, b\}, W)$.

Ideally, we would like to ask queries until $MPMR(\{a, b\}, W) \le 0$ for all $a, b \in \mathcal{A}$, which corresponds to the identification of the complete ordering of the alternatives according to the DM's preferences. To save preference queries, we could alternatively ask queries until $MPMR(\{a, b\}, W) \le \nu$ for all $a, b \in \mathcal{A}$, where $\nu > 0$ is a given positive threshold, and then use an inference algorithm to generate a SRMP model using the available preference data. Another alternative is to generate an SRMP model at every iteration step, and stop whenever the DM is satisfied with it.
•

## Ressources

• Documentation

• FAQ

• API

• OAI-PMH

• AuréHAL

## Informations

• À propos

• Données personnelles

• Mentions légales

• Accessibilité

• Conformité RGAA

## Questions juridiques

• Je publie, quels sont mes droits ?

• Loi pour une République numérique

• Stratégie de non-cession des droits

• SHERPA/RoMEO

## Portails

• Portails institutionnels

• HAL

• HAL SHS

• HAL Thèses

• MédiHAL

## CCSD

• CCSD

• Episciences

• Sciencesconf
## 3.2 Regret-optimization

In the procedure presented in the previous section, we have to compute $PMR^h(a, b, W)$ for all ordered pairs $(a, b) \in \mathcal{A} \times \mathcal{A}$ at each iteration step in order to determine the next preference query. To compute these values, we use an exact approach using a Mixed-Integer Linear program (MIP). The formulation of the MIP is given in Table 3 for a given pair $(a, b)$, and its parameters and variables are respectively given in Tables 1 and 2.

<table><caption>**Table 1** Parameters of the MIP</caption><tr><td><i>A</i></td><td>the set of alternatives</td></tr><tr><td><i>M</i></td><td>the set of criteria (m in total)</td></tr><tr><td>(<i>a</i>, <i>b</i>)</td><td>the current pair of alternatives</td></tr><tr><td><i>k</i></td><td>the number of reference profiles</td></tr><tr><td><i>h</i></td><td>the current profile index</td></tr><tr><td><i>σ</i></td><td>a permutation of {1, ..., <i>k</i>}</td></tr><tr><td>δ<sub><i>j</i></sub><sup><i>x</i><sup>ℓ</sup></sup></td><td>1 if alternative <i>x</i> outranks profile <i>p</i><sup>σ(<i>ℓ</i>)</sup> on criterion <i>j</i> and 0 otherwise</td></tr><tr><td><i>L</i><sub><i>P</i></sub></td><td>a set of pairs (<i>x</i>, <i>y</i>) ∈ <i>A</i> × <i>A</i> where <i>x</i> is preferred to <i>y</i> by the DM</td></tr><tr><td><i>L</i><sub><i>I</i></sub></td><td>a set of pairs (<i>x</i>, <i>y</i>) ∈ <i>A</i> × <i>A</i> where <i>x</i> and <i>y</i> are considered as indifferent by the DM</td></tr><tr><td><i>γ</i></td><td>a small constant used to model strict inequalities</td></tr></table>

In Table 3, the objective function represents the PMR of Definition (1), which has to be maximized. Constraints (i) specify that all the weights have to be strictly positive and sum up to 1, while constraints (ii) model an indifference between alternatives *a* and *b* according to the first *h* − 1 profiles. Constraints (iii) (resp. (iv)) are used to represent indifference (resp. preference) judgments of the DM w.r.t. the pairs of alternatives (*x*, *y*) which were queried during the previous iterations of the incremental inference process. They restrict the possible values of the weighting vector *w* to subsets that are compatible with the information previously given by the DM.

## 4 Empirical validation

In this section, we compare the performances of our query selection heuristic (denoted by $\mathcal{H}_{mmr}$) to that of Max Profiles of Khannoussi et al (2018) (denoted by $\mathcal{H}_{mp}$) which consists in selecting a pair $(a, b)$ that requires the highest number of profiles to be discriminated using the current SRMP model. In our experiments, the DM is replaced by a randomly generated SRMP model

<table><caption>**Table 2** Variables of the MIP</caption><tr><td>$w_j$</td><td>continuous</td><td>: the weights of the criteria, $\forall j \in M$</td></tr><tr><td>$s_{\ell}^{xy}$</td><td>binary</td><td>: 1 if $x$ is strictly preferred to alternative $y$ w.r.t. $p^{\sigma(\ell)}$ and 0 if $x$ is indifferent to $y$ w.r.t. $p^{\sigma(\ell)}$<br>$\forall (x, y) \in \mathcal{L}_P, \forall \ell \in \{1, \dots, k\}$</td></tr></table>
$$
\begin{array}{l@{\quad}l}
\begin{array}{@{}l@{\quad}l}
\max & \displaystyle\sum_{j=1}^{m} w_j \delta_j^{bh} - \sum_{j=1}^{m} w_j \delta_j^{ah} \\
\text{s.t.} & \\
& \displaystyle\sum_{j=1}^{m} w_j = 1 \\
& w_j \ge \gamma \\
& \sum_{j=1}^{m} w_j \ge \gamma \\
& \forall j \in M
\end{array}
&
\left.
\begin{array}{@{}l@{}}
\text{(i)} \\
\text{(ii)} \\
\text{(iii)}
\end{array}
\right\} \\
\\
& \displaystyle\sum_{j=1}^{m} w_j \delta_j^{x\ell} = \sum_{j=1}^{m} w_j \delta_j^{y\ell} \\
& \forall (x, y) \in \mathcal{L}_I, \forall \ell \in \{1, \dots, k\} \\
\\
& \displaystyle\sum_{j=1}^{m} w_j \delta_j^{x\ell} \ge \sum_{j=1}^{m} w_j \delta_j^{y\ell} - s_{\ell}^{xy} - s_{\ell-1}^{xy} \\
& \forall (x, y) \in \mathcal{L}_P, \forall \ell \in \{1, \dots, k\} \\
\\
& \displaystyle\sum_{j=1}^{m} w_j \delta_j^{x\ell} \le \sum_{j=1}^{m} w_j \delta_j^{y\ell} + s_{\ell}^{xy} + s_{\ell-1}^{xy} \\
& \forall (x, y) \in \mathcal{L}_P, \forall \ell \in \{1, \dots, k\} \\
\\
& \displaystyle\sum_{j=1}^{m} w_j \delta_j^{x\ell} \ge \sum_{j=1}^{m} w_j \delta_j^{y\ell} + \gamma \\
& -(1 - s_{\ell}^{xy} + s_{\ell-1}^{xy}) \times (1 + \gamma) \\
& \forall (x, y) \in \mathcal{L}_P, \forall \ell \in \{1, \dots, k\} \quad \text{(iv)}
\end{array}
\right\} \\
\\
& s_0^{xy} = 0 \\
& s_k^{xy} = 1 \\
& \forall (x, y) \in \mathcal{L}_P \\
& \forall (x, y) \in \mathcal{L}_P
\end{array}
$$

<caption>**Table 3** MIP to compute the PMR(a, b, p<sup>(h)</sup>, W)</caption>

(denoted by $M_{\text{DM}}$), which is used to compare pairs of alternatives at every step of the incremental elicitation procedures.

## 4.1 Design of experiments

To evaluate the performances of the two considered heuristics, we use two different sets of alternatives:

* a training dataset $\mathcal{D}$ of 100 alternatives, from which we will extract pairwise comparisons that will be used for the identification of the SRMP parameters,

* and a test dataset $\mathcal{D}_{\text{test}}$ composed of 5000 alternatives, which is used to evaluate the performances of the heuristics.
10

A regret-based query selection strategy SRMP parameters elicitation

The performance vectors attached to alternatives are randomly drawn as floats
(with 2 digits after the decimal point) within the interval [0, 100] using a
uniform distribution.

In a first step, we generate all possible (unordered) pairs of alternatives
from D. Here we obtain 4950 pairs with 100 alternatives. They correspond to
all the preference queries which could be presented to the DM regarding the
alternatives of D. Let Q denote the set of all (unordered) pairs of alternatives
in set D.

The $H_{mp}$ heuristic is evaluated as follows: at every iteration step, the heuristic selects a pair of alternatives from $Q$ which is then compared using the model corresponding to the DM, i.e. . Using this information and that from the previous iterations, we generate an SRMP model (denoted by $M_{cur}$) using a mixed-integer linear programming approach detailed in Section 4.2. Two rankings of the alternatives in $D_{test}$ are then computed using both $M_{DM}$ and $M_{cur}$ models and the *Kendall's* rank correlation ($\tau$) is used to compare them. This measure is used as a similarity indicator for the two rankings, and varies between -1 and 1 (*Kendall*, 1938). If both rankings are identical then $\tau = 1$, while if they are completely reversed then $\tau = -1$. The incremental elicitation process stops after 100 iteration steps, to limit the total elicitation time. We repeat this process 100 times (using different sets of alternatives and generated SRMP ($M_{DM}$) and report the averaged results in Section 4.3.

$\mathcal{H}_{mmr}$ is evaluated using a similar protocol. It only differs in the very beginning, where we restrict the set of pairs from which the heuristic makes its selection to a random subset $Q'$ of $Q$ of size 50. The reason for this is that the computation time for selecting a query in a set of pairs, at a given iteration, linearly grows with the cardinality of this set. Preliminary tests have shown that sets of pairs larger than 50 render the waiting time between the iterations of the protocol impractical. Once a query is selected from this subset, it is immediately replaced by a new one, randomly selected from $Q$. As a consequence, the cardinality of $Q'$ remains constant during the 100 iterations of the incremental elicitation process. The remaining experimental setting is the same as for $\mathcal{H}_{mmr}$.

## 4.2 Implementation details

The calculations are performed on multiple servers configured with 20 CPUs (which allows reaching a parallelism of 20 when using the CPLEX v12.6 solver on the proposed MIP approach) and 30 GB of RAM each. To generate a compatible SRMP model $M_{cur}$ at every step of the incremental elicitation procedures, we use an exact approach consisting in solving a MILP. Its formulation is identical to the one used to compute $PMR^h$ values (see Table 3), except that the objective function is removed as well as constraints (ii) and their corresponding $\delta_j^{al}$ parameter values. These elements correspond to a pair of alternatives (a, b) for which the $PMR^h$ needs to be computed, however they are not needed for the construction of an SRMP model compatible with the collected preference information.
## 4.3 Results

We begin by illustrating the use of the $H_{mmr}$ heuristic on one randomly chosen instance with 2 profiles and 7 criteria. In Figure 1 we show, for each iteration step, the value of Kendall’s tau for the generated SRMP model using the test dataset (left), and the MPMR value (see Equation 4) of the selected query (right).

<figure><img src="image_1.png" /><figcaption>**Fig. 1** Kendall’s tau for 1 instance with 2 profiles and 7 criteria and MPMR for the same instance (right)</figcaption></figure>

The plot on the left has an increasing trend with the number of pairs of alternatives being used to infer the model parameters, and tends towards 1. This is natural, as increasing the number of preference queries leads to a more accurate model, which has a higher generalization power on the 5000 unseen alternatives.

Regarding the plotted MPMR value, it is important to understand that it corresponds to the maximal value of MPMR in set $Q'$, as it is the MPMR value of the selected pairs. We can observe that this value has a decreasing trend over the 100 iteration steps. This can be explained by the fact that, as we remove the pair from $Q'$ holding the highest MPMR, the remaining pairs tend to have smaller MPMR values. In addition, the pair that is used to replace the selected one also tends to have a lower probability to have a better MPMR value than all of the unselected pairs. However, as the process goes on, this probability decreases and we observe slight increases in the MPMR measure. We can also observe that starting from the 35th iteration step, the MPMR value drops to, or below, 0. On the left plot, this corresponds to the iteration
12 *A regret-based query selection strategy SRMP parameters elicitation*

where Kendall’s tau on the test set hits a local maximum, which stays the highest obtained value for a certain number of iterations. This means that after this point, the alternatives remaining in $Q'$ are no longer informative. We can see that around iteration 75, we have a slight increase in the maximal value of Kendall’s tau, which corresponds in the MPMR plot to the moment where a new pair is added to $Q'$ which contains new valuable information for the elicitation (and therefore has an MPMR value above 0).

<figure><img src="image_2.png" /><figcaption>**Fig. 2** Mean kendall’s tau for 2P 7C using the two heuristics, on $D_{test}$</figcaption></figure>

Let us now switch to the results of the experimental protocol. Figure 2 illustrates the mean value for the Kendall’s tau measure for each iteration step of the incremental procedure, when problems with 2 profiles and 7 criteria (denoted by (2P 7C)) are considered, and using the two query selection heuristics $\mathcal{H}_{mp}$ and $\mathcal{H}_{mmr}$.

As expected, the accuracy of the inferred models (given by the *Kendall’s tau* values) increases with the number of pairs of alternatives being used to infer them (given by the iteration number) and tends towards 1. We also notice that heuristic $\mathcal{H}_{mmr}$ leads to more accurate SRMP models faster than $\mathcal{H}_{mp}$, and leads to a higher accuracy after 100 iterations. Both heuristics start rather poorly during the first iterations since fewer queries naturally lead to poor models.

In Figure 3 we show the standard deviation around the mean values, while in Figure 4 we show Kendall’s tau value of the 5th percentile of the 100 datasets. This latter plot shows that, for a given query selection heuristic, in 95% of the studied cases, the obtained ranking had a similarity with the sought
<figure><img src="image_3.png" /><figcaption>Fig. 3 Standard deviation (and mean) of Kendall’s tau for 2P 7C, on $D_{test}$</figcaption></figure>

ranking above the corresponding curve. In any case, we can see that $\mathcal{H}_{mmr}$ performs better than $\mathcal{H}_{mp}$ for this size of problems.

Figure 5 depicts the mean values of Kendall’s tau for different problem sizes. More precisely, we vary the number of profiles ($k$) from 2 to 3 and the number of criteria ($m$) among 3, 5 and 7. The corresponding problems are denoted by ($k$P $m$C) with $k \in \{2, 3\}$ and $m \in \{3, 5, 7\}$.

The results show the same upward trend in accuracy when increasing the number of queries as well as the fact that the proposed heuristic is significantly better than the reference one. We also observe that when we have more criteria, the top accuracy we can reach using the 100 pairs of alternatives decreases, potentially indicating the this set is not large or representative enough for reaching a higher accuracy. The same observation is true to a smaller degree when increasing the number of profiles.

Figure 6 shows the mean maximal MPMR at each iteration step through the 100 instances. As expected, this value decreases, and drops below 0 at one moment during the executions. For a fixed number of profiles, these negative values, appear later in the iterations for higher number of criteria. This was expected, as when the number of criteria increases, more pairs are needed to restore good model parameters.

The computation time for selecting a query is also important during an incremental elicitation process, therefore we now focus on it. Figure 7 depicts the mean time for the selection of a pair at each iteration step (in seconds) for
14

A regret-based query selection strategy SRMP parameters elicitation

<figure><img src="image_4.png" /><figcaption>Fig. 4 5th percentile Kendall's tau for 2P 7C using the two heuristics, on $D_{test}$</figcaption></figure>

the two selection heuristics, for problems containing 2 profiles and 5 criteria. For the $H_{mmr}$ heuristic we also show the standard deviation of those selection times. For the $H_{mp}$ heuristic, the mean time (dashed line) decreases slightly with the number of steps, however this increase cannot be perceived when looking at Figure 7. Computation time for one iteration of $H_{mp}$ is on the order of a millisecond. It decreases slightly, because the size of the set of available pairs decreases at each iteration.

The mean time for the $H_{mmr}$ heuristic (continuous line) is significantly larger than that of $H_{mp}$. This can be explained by the fact that $H_{mmr}$ executes multiple MIPs (Table 3) for each remaining pair of alternatives in the selection dataset $Q'$. This execution time also increases with the number of iteration steps. This is due to the fact that the MILPs to be solved become more and more constrained at each iteration.

Figure 8 depicts the mean time for the selection of a pair at each iteration in seconds for the two selection heuristics $H_{mp}$ and $H_{mmr}$ for different problem sizes. We observe first that the different curves have the same general behavior independently of the problem sizes (i.e. a very steep increase during the first iterations). We also observe that obviously the more the problem is complex in terms of number of criteria and of number of profiles, the more the computation time increases. For larger problems (with higher number of profiles and criteria), the variation of the execution time becomes very large, as shown by the standard deviation. After the steep increase of the first iterations, selection time seems to
<figure><img src="image_5.png" /><figcaption>Fig. 5 Mean Kendall’s tau for problems of different sizes using the two heuristics, on $D_{test}$</figcaption></figure>

# 5 Discussion and conclusion

In this paper, we propose a new query selection heuristic, based on the min-
imax regret strategy, for the incremental elicitation of a ranking model using
reference points (SRMP) and show, using artificial data, that it significantly
outperforms existing approaches. Our aim is to be able, using this strategy, to
reduce the number of queries necessary for converging to a SRMP model that
accurately represents the perspective of the DM, therefore indirectly helping
in easing this process.

Since the computational effort for finding the query that minimizes the
regret (as defined in this paper) can become significant, we have limited the
learning process to only inferring the criteria weights of the SRMP model
(assuming that for the reference profiles and their lexicographic order, a direct
A regret-based query selection strategy SRMP parameters elicitation

<figure><img src="image_6.png" /><figcaption>Fig. 6 Mean maximal MPMR for problems of different sizes, on $D_{test}$</figcaption></figure>

elicitation approach has been used beforehand). Such an approach is plausible, as the profiles define independent ordinal performance intervals that should have a clear significance for the DM and therefore can be directly expressed by him/he more easily.

The experimental results using the proposed approach show that it outperforms a previously published approach in terms of model convergence. In order to reach a 90% accuracy, it requires approximately 27 (against 50) queries for problems containing 7 criteria and 2 reference profiles and around 30 (against 45) queries for problems containing 7 criteria and 3 reference profiles. The
<figure><img src="image_7.png" /><figcaption>**Fig. 7** Mean selection time and standard deviation for 2P 7C at each iteration</figcaption></figure>

number of required queries when considering smaller problems is also reduced while at the same time being lower than when using the query selection strategy from the literature. The computational time for finding a query at each iteration, however, can grow up to around 3 minutes, however this can be mitigated, for example, by preemptively launching in parallel the calculations for finding the next query while the DM considers the current one. These calculations would need to consider all three possible outcomes for the current query (a preference in favor of any of the two alternatives, or an indifference). As soon as the DM expresses his/her preference on the current pair of alternatives, we can stop the calculations for the two no longer relevant outcomes.

All of these conclusions, however, depend on the number of pairs of alternatives to be considered, which we currently fixed to 50. Having more or fewer pairs of alternatives will naturally impact the required computational time at each iteration step. It is also worth noting that at one moment, the maximal value of MPMR drops below zero. This could be the moment where the set $Q'$ may be totally renewed, instead of continuing the process by only adding one new alternative at a time. The topics of reducing calculation time at each iteration, while increasing the generalization power of the inferred model will be explored in the future.
<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /><figcaption>**Fig. 8** Mean selection time for the different problem sizes</figcaption></figure>

References

Belahcène K, Mousseau V, Ouerdane W, et al (2018) Ranking with Multiple reference Points: Efficient Elicitation and Learning Procedures. p 8, in Proceeding from multiple criteria Decision aid to Preference Learning (DA2PL)

Benabbou N, Perny P, Viappiani P (2017) Incremental elicitation of Choquet capacities for multicriteria choice, ranking and sorting problems. Artificial Intelligence 246:152–180
Boutilier C, Patrascu R, Poupart P, et al (2006) Constraint-based optimization and utility elicitation using the minimax decision criterion. Artificial Intelligence 170(8):686 – 713. https://doi.org/https://doi.org/10.1016/j.artint. 2006.02.003

Bouyssou D, Marchant T (2013) Multiattribute preference models with reference points. European Journal of Operational Research 229(2):470 – 481. https://doi.org/https://doi.org/10.1016/j.ejor.2013.02.046

Bouyssou D, Pirlot M (2009) An axiomatic analysis of concordance–discordance relations. European Journal of Operational Research 199(2):468 – 477. https://doi.org/https://doi.org/10.1016/j.ejor.2008.11.011

Bouyssou D, Marchant T, Pirlot M, et al (2006) Evaluation and decision models with multiple criteria: Stepping stones for the analyst, 1st edn. International Series in Operations Research and Management Science, Volume 86, Boston

Chajewska U, Koller D, Parr R (2000) Making Rational Decisions Using Adaptive Utility Elicitation. AAAI Press / The MIT Press

Figueira J, Greco S, Ehrgott M (2005a) Multiple Criteria Decision Analysis: State of the Art Surveys. Springer Verlag, Boston, Dordrecht, London

Figueira J, Mousseau V, Roy B (2005b) ELECTRE methods. In: Figueira J, Greco S, Ehrgott M (eds) Multiple Criteria Decision Analysis: State of the Art Surveys. Springer Verlag, Boston, Dordrecht, London, p 133–162

Fürnkranz J, Hüllermeier E (2011) Preference Learning and Ranking by Pairwise Comparison, Springer Berlin Heidelberg, Berlin, Heidelberg, pp 65–82

Greco S, Matarazzo B, Slowinski R (2001) Rough sets theory for multicriteria decision analysis. European Journal of Operational Research 129(1):1 – 47. https://doi.org/https://doi.org/10.1016/S0377-2217(00)00167-3

Keeney R, Raiffa H (1976) Decisions with Multiple Consequences: Preferences and Value Tradeoffs. New York, NY: John Wiley & Sons

Kendall MG (1938) A new measure of rank correlation. Biometrika 30(1/2):81–93

Khannoussi A, Olteanu AL, Dezan C, et al (2018) Incremental Learning of Simple Ranking Method Using Reference Profiles Models. In: DA2PL2018: from Multiple Criteria Decision Aid to Preference Learning, Poznan, Poland
20

*A regret-based query selection strategy SRMP parameters elicitation*

Khannoussi A, Olteanu AL, Labreuche C, et al (2021) Simple ranking method using reference profiles: incremental elicitation of the preference parameters. 4OR: A Quarterly Journal of Operations Research https://doi.org/10.1007/s10288-021-00487-w

Liu J, Ouerdane W, Mousseau V (2014) Metaheuristic approach for preference learning in multicriteria ranking based on reference points, in Proceeding of the 2nd wokshop from multiple criteria Decision aid to Preference Learning (DA2PL)

Lu T, Boutilier C (2011) Robust approximation and incremental elicitation in voting protocols. In: Proceedings of IJCAI’11, pp 287–293

Mousseau V (1995) Eliciting information concerning the relative importance of criteria. In: Pardalos P, Siskos Y, Zopounidis C (eds) Advances in Multicriteria Analysis. Kluwer Academic, Dordrecht, p 17–43

Olteanu AL, Belahcène K, Mousseau V, et al (2022) Preference Elicitation for a Ranking Method based on Multiple Reference Profiles. 4OR-Q J Oper Res (20):63 – 84. https://doi.org/10.1007/s10288-020-00468-5

Regan K, Boutilier C (2009) Regret-based reward elicitation for Markov decision processes. In: UAI, pp 444–451

Rolland A (2013) Reference-based preferences aggregation procedures in multi-criteria decision making. European Journal of Operational Research 225(3):479–486. https://doi.org/10.1016/j.ejor.2012.10.013

Roy B (1991) The outranking approach and the foundations of electre methods. Theory and Decision 31(1):49–73

Roy B (1996) Multicriteria Methodology for Decision Aiding. Kluwer Academic, Dordrecht

White CC, Sage AP, Dozono S (1984) A model of multiattribute decisionmaking and trade-off weight determination under uncertainty. IEEE Transactions on Systems, Man, and Cybernetics 14(2):223–229