PCMC-N ET : F EATURE - BASED P AIRWISE C HOICE M ARKOV C HAINS
A BSTRACT
Pairwise Choice Markov Chains (PCMC) have been recently introduced to overcome limitations of choice models based on traditional axioms unable to express empirical observations from modern behavior economics like context effects occurring when a choice between two options is altered by adding a third alternative.
The inference approach that estimates the transition rates between each possible pair of alternatives via maximum likelihood suffers when the examples of each alternative are scarce and is inappropriate when new alternatives can be observed at test time.
In this work, we propose an amortized inference approach for PCMC by embedding its deﬁnition into a neural network that represents transition rates as a function of the alternatives’ and individual’s features.
We apply our construction to the complex case of airline itinerary booking where singletons are common (due to varying prices and individual-speciﬁc itineraries), and context effects and behaviors strongly dependent on market segments are observed.
Experiments show our network signiﬁcantly outperforming, in terms of prediction accuracy and logarithmic loss, feature engineered standard and latent class Multinomial Logit models as well as recent machine learning approaches.
I NTRODUCTION
Choice modeling aims at ﬁnding statistical models capturing the human behavior when faced with a set of alternatives.
Classical examples include consumer purchasing decisions, choices of schooling or employment, and commuter choices for modes of transportation among available options.
Traditional models are based on different assumptions about human decision making, e.g. Thurstone’s Case V model (Thurstone, 1927) or Bradley-Terry-Luce (BTL) model (Bradley & Terry, 1952).
Nevertheless, in complex scenarios, like online shopping sessions presenting numerous alternatives to user-speciﬁc queries, these assumptions are often too restrictive to provide accurate predictions.
Formally, there is a universe of alternatives U , possibly inﬁnite.
In each choice situation, some ﬁnite choice set S ⊆
U is considered.
A choice model is a distribution over the alternatives of a given choice set S , where the probability of choosing the item i among S is denoted as P S ( i ) .
These models can be further parameterized by the alternatives’ features and by those of the individual making the choice.
An important class of choice models is the Multinomial Logit (MNL), a generalization of the BTL model—deﬁned for pairwise choices only—to larger sets.
MNL models satisfy Luce’s axiom also known as independence of irrelevant alternatives (Luce, 1959), which states that the probability of selecting one alternative over another from a set of many alternatives is not affected by the presence or absence of other alternatives in the set.
Moreover, any model satisfying Luce’s axiom is equivalent to some MNL model (Luce, 1977).
Equivalently, the probability of choosing some item i from a given set S can be expressed as P S ( i ) =
w i / (cid:80) j ∈ S w j where w i is the latent value of the item i .
Luce’s axiom implies stochastic transitivity i.e. if P ( a (cid:66) b ) ≥ 1 / 2 and P ( b (cid:66) c ) ≥ 1 / 2 , then P ( a (cid:66) c ) ≥ max ( P ( a (cid:66) b ) , P ( b (cid:66) c )) where P ( i (cid:66) j ) ≡ P { i,j } ( i ) (Luce, 1977).
Stochastic transitivity implies the necessity of a total order across all elements and also prevents from expressing cyclic preference situations like the stochastic rock-paper-scissors game described in Section 3.2.
Thurstone’s Case V model exhibits strict stochastic transitivity but does not satisfy Luce’s axiom (Adams & Messick, 1958).
Luce’s axiom and stochastic transitivity are strong assumptions that often do not hold for empirical choice data (see (Ragain & Ugander, 2016) and references therein).
For example, Luce’s axiom prevents models from expressing context effects like the attraction effect (also know as asymmetric dominance or decoy effect), the similarity effect and the compromise effect .
The attraction effect occurs when two alternatives are augmented with an asymmetrically dominated one (i.e., a new option that is inferior in all aspects with respect to one option, but inferior in only some aspects and superior in other aspects with respect to the other option) and the probability of selecting the better, dominant alternative increases (Huber et al., 1982).
The similarity effect arises from the introduction of an alternative that is similar to, and competitive with, one of the original alternatives, and causes a decrease in the probability of choosing the similar alternative (Tversky, 1972).
The compromise effect occurs when there is an increase in the probability of choosing an alternative that becomes the intermediate option when a third extreme option is introduced (Simonson, 1989).
Examples of these effects are visualized in Section 4.
A larger class of models is the one of Random Utility Models (RUM)
(Block & Marschak, 1960;
Manski, 1977), which includes MNL but also other models satisfying neither Luce’s axiom nor stochastic transitivity.
This class afﬁliates with each i ∈ U a random variable X i and deﬁnes for each subset S ⊆
U the probability P S ( i ) =
P ( X i ≥ X j , ∀ j ∈ S ) .
RUM exhibits regularity i.e. if A ⊆ B then P A ( x ) ≥ P B ( x ) .
Regularity also prevents models from expressing context effects (Huber et al., 1982).
The class of Nested MNL (McFadden, 1980) allows to express RUM models but also others that do not obey regularity.
Nevertheless, inference is practically difﬁcult for Nested MNL models.
Recently, a more ﬂexible class of models called Pairwise Choice Markov Chains has been introduced in Ragain & Ugander (2016).
This class includes MNL but also other models that satisfy neither
Luce’s axiom, nor stochastic transitivity, nor regularity.
This class deﬁnes the choice distribution as the stationary distribution of a continuous time Markov chain deﬁned by some transition rate matrix.
Still, it satisﬁes a weakened version of Luce’s axiom called uniform expansion stating that if we add “copies” (with no preference between them), the probability of choosing one element of the copies is invariant to the number of copies.
Although the ﬂexibility of this class is appealing, the proposed inference is based on maximizing the likelihood of the rate matrix for the observed choices which is prone to overﬁtting when the number of observations for each possible alternative is small and is inappropriate when new alternatives can be seen at test time.
Alternatives and individuals making choices can be described by a set of features that can be then used to understand their impact on the choice probability.
A linear-in-features MNL assumes that the latent value is given by a linear combination of the parameters of the alternatives and the individual.
Features of the individual can be taken into account by these models but inference suffers from scarcity and is inappropriate when new alternatives can be seen at test time.
The latent class MNL (LC-MNL) model (Greene & Hensher, 2003) takes into account individual heterogeneity by using a Bayesian mixture over different latent classes—whose number must be speciﬁed—in which homogeneity and linearity is assumed.
A linear-in-features parameterization of PCMC, for features in R d , was suggested in (Ragain & Ugander, 2016, Appendix) but requires ﬁtting a weight matrix of size | U | × d , which makes it scale poorly and does not allow to predict unseen alternatives.
In this work, we propose an amortized inference approach for PCMC in the sense that the statistical parameters are reused for any pair of alternatives and their number is thus independent of the size of the universe.
In addition, we allow non-linear modeling by using a neural network.
In complex cases like airline itinerary choice, where the alternatives are strongly dependent on an individual-speciﬁc query and some features, like price, can be dynamic, the previous approaches have limited expressive power or are inappropriate.
Two recently introduced methods allow complex feature handling for alternatives and individuals.
Mottini & Acuna-Agost (2017) proposes a recurrent neural network method consisting in learning to point, within a sequence of alternatives, to the chosen one.
This model is appealing because of its feature learning capability but neither its choice-theoretic properties have been studied nor its dependence on the order of the sequence.
Lhéritier et al.
(2019) proposes to train a Random Forest classiﬁer to predict whether an alternative is going to be predicted or not independently of the rest of the alternatives of the choice set.
This approach does not take into account the fact that in each choice set exactly one alternative is chosen.
For this reason, the probabilities provided by the model are only used as scores to rank the alternatives, which can be interpreted as latent values—making it essentially equivalent to a non-linear MNL.
To escape this limitation and make the latent values dependent on the choice set, relative features are added (e.g. the price for i -th alternative price i is converted to price i / min j ∈ S price j ).
The non-parametric nature of this model is appealing but its choice-theoretic properties have not been studied either.
In this work, we propose to enable PCMC with neural networks based feature handling, therefore enjoying both the good theoretical properties of PCMC and the complex feature handling of the previous neural network based and non-parametric methods.
This neural network parameterization of
PCMC makes the inference amortized allowing to handle large (and even inﬁnite) size universes as shown in our experiments for airline itinerary choice modeling shown in Section 5.
B ACKGROUND : P AIRWISE C HOICE M ARKOV C HAINS 2. D EFINITION
A Pairwise Choice Markov Chain (PCMC) (Ragain & Ugander, 2016) deﬁnes the choice probability
P S ( i ) as the probability mass on the alternative i ∈ S of the stationary distribution of a continuous time Markov chain (CTMC) whose set of states corresponds to S .
The model’s parameters are the off-diagonal entries q ij ≥ 0 of a rate matrix Q indexed by pairs of elements in U .
Given a choice set S , the choice distribution is the stationary distribution of the continuous time Markov chain given by the matrix Q S obtained by restricting the rows and columns of Q to elements in S and setting q ii = − (cid:80) j ∈ S \ i q ij for each i ∈ S .
Therefore, the distribution P S is parameterized by the | S | ( | S | − 1) transition rates of Q S .
The constraint
[Equation]
is imposed in order to guarantee that the chain has a single closed communicating class which implies the existence and the unicity of the stationary distribution π S (see, e.g., Norris (1997)) obtained by solving
[Equation]
where 0 and 1 are row vectors of zeros and ones, respectively.
Since any column of Q s is the opposite of the sum of the rest of the columns, it is equivalent to solve
[Equation]
where Q (cid:48)
S ≡ (cid:104) (( Q S ) ij ) 1 ≤ i ≤| S | , 1 ≤ j< | S | | 1 T (cid:105) .
P ROPERTIES
In Ragain & Ugander (2016), it is shown that PCMC allow to represent any MNL model, but also models that are non-regular and do not satisfy stochastic transitivity (using the rock-scissor-paper example of Section 3.2).
In the classical red bus/blue bus example (see, e.g., Train (2009)), the color of the bus is irrelevant to the preference of the transportation mode “bus” with respect to the “car” mode.
Nevertheless, MNL models reduce the probability of choosing the “car” mode when color variants of buses are added, which does not match empirical behavior.
PCMC models allows to model this kind of situations thanks to a property termed contractibility , which intuitively means that we can “contract” subsets
A i ⊆ U to a single “type” when the probability of choosing an element of A i is independent of the pairwise probabilities between elements within the subsets.
Formally, a partition of U into non-empty sets A 1 , . . .
, A k is a contractible partition if q a i a j = λ ij for all a i ∈
A i , a j ∈ A j for some Λ = { λ ij } for i, j ∈ { 1 , . . . , k } .
Then, the following proposition is shown.
Proposition 1 (Ragain & Ugander (2016)) .
For a given Λ , let A 1 , . . .
, A k be a contractible partition for two PCMC models on U represented by Q, Q (cid:48) with stationary distributions π, π (cid:48) .
Then, for any
A i ,
[Equation]
Then it is shown, that contractibility implies uniform expansion formally deﬁned as follows.
Deﬁnition 1 (Uniform Expansion) .
Consider a choice between n elements in a set S [-
-](1)
= { i 11 , . . .
, i n 1 } , and another choice from a set S ( k ) containing k copies of each of the n elements:
S ( k ) =
{ i 11 , . . .
, i 1 k , i 21 , . .
, i 2 k , . .
, i n 1 , . .
, i nk } .
The axiom of uniform expansion states that for each m ∈ { 1 , . . . , n } and all k ≥ 1 ,
[Equation]
2. I NFERENCE
Given a dataset D , the inference method proposed in Ragain & Ugander (2016) consists in maximizing the log likelihood of the rate matrix Q indexed by U
[Equation]
where P QS ( i ) denotes the probability that i is selected from S as a function of Q and C iS ( D ) denotes the number of times in the data that i was chosen out of set S .
This optimization is difﬁcult since there is no general closed form expression for P QS ( i ) and the implicit deﬁnition also makes it difﬁcult to derive gradients for log L with respect to the parameters q ij .
The authors propose to use Sequential Least Squares Programming (SLSQP) to maximize log L ( Q ; D ) , which is nonconcave in general.
However, in their experiments, they encounter numerical instabilities leading to violations ( q ij + q ji = 0 ) of the PCMC deﬁnition, which were solved with additive smoothing at the cost of some efﬁcacy of the model.
In addition, when the examples of each alternative are scarce like in the application of Section 5, this inference approach is prone to severe overﬁtting and is inappropriate to predict unseen alternatives.
These two drawbacks motivate the amortized inference approach we introduce next.
PCMC-N ET
We propose an amortized inference approach for PCMC based on a neural network architecture called
PCMC-Net that uses the alternatives’ and the individual’s features to determine the transition rates and can be trained using standard stochastic gradient descent techniques.
3. A RCHITECTURE
Input layer For PCMC, the choice sets S were deﬁned as a set of indices.
For PCMC-Net, since it is feature-based, the choice sets S are deﬁned as sets of tuples of features.
Let S i be the tuple of features of the i -th alternative of the choice set S belonging to a given feature space F a
and I be the tuple of the individual’s features belonging to a given feature space F 0 .
The individual’s features are allowed to be an empty tuple.
Representation layer The ﬁrst layer is composed of a representation function for the alternatives’ features
[Equation]
and a representation function for the individual’s features
[Equation]
where w 0 and w a are the sets of weights parameterizing them and d 0 , d a ∈ N are hyperparameters.
These functions can include, e.g., embedding layers for categorical variables, a convolutional network for images or text, etc., depending on the inputs’ types.
Cartesian product layer
In order to build the transition rate matrix, all the pairs of different alternatives need to be considered, this is accomplished by computing the cartesian product
[Equation]
S =   ...
S i ...
S j ...
  ρ wρ w a   ... ρ w a ( S i ) ..
ρ w a ( S j ) .
  × ...
R ij ...
(cid:15)li n e a r s o l v e r ˆ π   ...
ˆ q ij − (cid:80) j (cid:54) =
i ˆ q ij ...
...  (cid:2)...
0 1 (cid:3) ⊕ w q
I ρ w 0 ( I )
The combinations of embedded alternatives are concatenated together with the embedded features of the individual, i.e.
[Equation]
8) where ⊕ denotes vector concatenation.
Transition rate layer
The core component is a model of the transition rate ( Q S ) ij , i (cid:54) =
[Equation]
where f w q consists of multiple fully connected layers parameterized by a set of weights w q and (cid:15) >is a hyperparameter.
Notice that taking the maximum with 0 and adding (cid:15) guarantees non-negativity and the condition of Eq.
The transition rate matrix ˆ Q is then obtained as follows:
[Equation]
Stationary distribution layer
The choice probabilities correspond to the stationary distribution ˆ π that is guaranteed to exist and be unique by the condition of Eq.
and can be obtained by solving the system
[Equation]
by, e.g., partially-pivoted LU decomposition which can be differentiated with automatic differentiation.
The whole network is represented in Fig.
3. P ROPERTIES
Non-regularity As shown in Ragain & Ugander (2016), non-regular models can be obtained by certain rate matrices.
For example, the stochastic rock-paper-scissors game can be described by a non-regular model obtained with the following transition rate matrix with 12 < α ≤ 1 :
[Equation]
PCMC-Net can represent such a model by setting the following design parameters.
In this case, the individual’s features correspond to an empty tuple yielding an empty vector as representation.
By setting ρ w a to a one-hot representation of the alternative (thus d a = 3 ), a fully connected network f w q consisting of one neuron (i.e. six coefﬁcients and one bias) is enough to represent this matrix since six combinations of inputs are of interest.
Non-parametric limit More generally, the following theorem shows that any PCMC model can be arbitrarily well approximated by PCMC-Net.
Theorem 1.
If ρ w a and f w q are given enough capacity, PCMC-Net can approximate any PCMC model arbitrarily well.
Proof.
A PCMC model jointly speciﬁes a family of distributions π S for each S ∈ 2 U obtained by subsetting a single rate matrix Q indexed by U .
Therefore, it is sufﬁcient to prove that PCMC-Net can approximate any matrix Q since ˆ q ij , with i (cid:54) =
j , does not depend on S .
PCMC-Net forces the transition rates to be at least (cid:15) , whereas the PCMC deﬁnition allows any q ij ≥
as long as q ij + q ji > 0 .
Since multiplying all the entries of a rate matrix by some c > 0 does not affect the stationary distribution of the corresponding CTMC, let us consider, without loss of generality, an arbitrary PCMC model given by a transition rate matrix Q (cid:63) , whose entries are either at least (cid:15) or zero.
Let π (cid:63) be its stationary distribution.
Then, let us consider the matrix Q ( (cid:15), c ) obtained by replacing the null entries of Q (cid:63) by (cid:15) and by multiplying the non-null entries by some c > 0 , and let π ( (cid:15), c ( δ )) be its stationary distribution.
Since, by Cramer’s rule, the entries of the stationary distribution are continuous functions of the entries of the rate matrix, for any δ > 0 , there exist c ( δ ) > 0 such that | π ( (cid:15), c ( δ )) − π (cid:63) | < δ .
Since deep neural networks are universal function approximators (Hornik et al., 1989), PCMC-Net allows to represent arbitrarily well any Q ( (cid:15), c ) if enough capacity is given to the network, which completes the proof.
Contractibility Let Q, Q (cid:48) be the rate matrices obtained after the transition rate layer of two different
PCMC-Nets on a ﬁnite universe of alternatives U .
Then, Proposition 1 can be applied.
Regarding uniform expansion, when copies are added to a choice set, their transition rates to the other elements of the choice set will be identical since they only depend on their features.
Therefore, PCMC-Net allows uniform expansion.
3. I NFERENCE
The logarithmic loss is used to assess the predicted choice distribution ˆ π given by the model parameterized by w ≡ w 0 ∪ w a ∪ w q on the input ( I, S ) against the index of actual choice Y S , loss( w, I, S, Y S ) ≡ log ˆ π Y S .
Training can be performed using stochastic gradient descent and dropout to avoid overﬁtting, which is stable unlike the original inference approach.
E XPERIMENTS ON SYNTHETIC DATA WITH CONTEXT EFFECTS
To illustrate the ability of PCMC and PCMC-Net models of capturing context effects we simulate them using the multiattribute linear ballistic accumulator (MLBA) model of Trueblood et al.
(2014) that is able to represent attraction, similarity and compromise effects.
MLBA models choice as a process where independent accumulators are associated to each alternative and race toward a threshold.
The alternative whose accumulator reaches the threshold ﬁrst is selected.
The speed of each acculumator is determined by a number of parameters modeling human psychology notably, weights that determine the attention paid to each comparison and a curvature parameter determining a function that gives the subjective value of each alternative from its objective value.
We consider the example of (Trueblood et al., 2014, Figure 7) reproduced in Figure 2(i) where a choice set of two ﬁxed alternatives { a, b } is augmented with a third option c and the preference of a over b , i.e. [Equation] [Equation] ) , is computed.
The model considers two attributes such that higher values are more preferable.
The two original alternatives are a = (4 , 6) , b = (6 , 4) .
For example, these can correspond to two different laptops with RAM capacity and battery life as attributes, and c is a third laptop choice inﬂuencing the subjective value of a with respect to b .
In order to generate synthetic choice sets, we uniformly sample the coordinates of c in [1 , 9] 2 .
Then, we compute the choice distribution given by the MLBA model 1 that is used to sample the choice.
We instantiate PCMC-Net with an identity representation layer with d a = 4 and d 0 = 0 and a transition rate layer with h ∈ { 1 , 2 , 3 } hidden layers of ν = 16 nodes with Leaky ReLU activation
1 2 3 4 5 6 7 8 9        a b c
A c C c S (i) Ground truth.
1 2 3 4 5 6 7 8 9        (ii) MNL 1 2 3 4 5 6 7 8 9        (iii) PCMC 1 2 3 4 5 6 7 8 9        (iv) PCMC-Net 3 ×
( slope = 0 . 01 ) and (cid:15) = 0 .
In order to use the original PCMC model 3 , we discretize the attributes of the third option using 8 bins on each attribute, obtaining 64 different alternatives in addition to identiﬁed a and b .
We also compare to a linear-in-features MNL model 4 .
Figure 2 shows how the different models represent the preference for a over b .
Table 1, shows a Monte Carlo estimation of the expected Kullback-Leibler divergence comparing each model ˆ P to the true MLBA model P
[Equation]
As expected, MNL is unable to represent context effects due the independence of irrelevant alternatives.
After discretization, the original PCMC provides a good approximation of the MLBA model.
Nevertheless, it is difﬁcult to further reﬁne it since the number of statistical parameters grows biquadratically with the number of bins for each feature.
As shown in Table 1, the amortized inference approach of PCMC-Net allows a better approximation of the original model with signiﬁcantly fewer statistical parameters than the discretized PCMC.
[Table]
E XPERIMENTS ON AIRLINE ITINERARY CHOICE MODELING
In this section [Table] we instantiate PCMC-Net for the case of airline itinerary choice modeling.
As shown in Babutsidze et al.
, this kind of data often exhibit attraction effects, calling for more ﬂexible models such as PCMC.
Nevertheless, in the considered dataset, alternatives rarely repeat themselves, which makes the original inference approach for PCMC inappropriate.
5. D ATASET
We used the dataset from Mottini & Acuna-Agost (2017) consisting of ﬂight bookings sessions on a set of European origins and destinations.
Each booking session contains up to 50 different itineraries, one of which has been booked by the customer.
There are 815559 distinct alternatives among which 84% are singletons and 99% are observed at most seven times.
In total, there are 33951 choice sessions of which 27160 were used for training and 6791 for testing.
The dataset has a total offeatures, both numerical and categorical, corresponding to individuals and alternatives (see Table 4).
5. I NSTANTIATION OF PCMC-N ET
PCMC-Net was implemented in PyTorch (Paszke et al., 2017) {+
+}
During training, a mini-batch is composed of a number of sessions whose number of alternatives can be variable.
Dynamic computation graphs are required in order to adapt to the varying session size.
Stochastic gradient optimization is performed with Adam (Kingma & Ba, 2015).
In our experiments, numerical variables are unidimensional and thus are not embedded.
They were standardized during a preprocessing step.
Each categorical input of cardinality
c i is passed through an embedding layer, such that the resulting dimension is obtained by the rule of thumb
d i := min( (cid:100)
c i / 2 (cid:101) , 50) .
We maximize regularization by using a dropout probability of 0 .
5 (see, e.g., Baldi & Sadowski (2013)).
The additive constant (cid:15) was set to 0 .
The linear solver was implemented with torch.solve , which uses LU decomposition.
Table 2 shows the hyperparameters and learning parameters that were optimized by performingiterations of Bayesian optimization (using GPyOpt authors (2016)).
Early stopping is performed during training if no signiﬁcant improvement (greater than 0 .
01 with respect to the best log loss obtained so far) is made on a validation set (a random sample consisting of 10% of the choice sessions from the training set) during 5 epochs.
Using the hyperparameters values returned by the Bayesian optimization procedure and the number of epochs at early stopping (66), the ﬁnal model is obtained by training on the union of the training and validation sets.
[Table]
5. R ESULTS
We compare the performance of the PCMC-Net instantiation against three simple baselines:
• Uniform: probabilities are assigned uniformly to each alternative.
• Cheapest (non-probabilistic): alternatives are ranked by increasing price.
• Shortest (non-probabilistic): alternatives are ranked by increasing trip duration.
We also compare against the results presented in Lhéritier et al.
• Multinomial Logit (MNL): choice probabilities are determined from the alternatives’ features only, using some feature transformations to improve the performance.
• Latent Class Multinomial Logit (LC-MNL): in addition to the alternatives’ features, it uses individual’s features which are used to model the probability of belonging to some latent classes whose number is determined using the Akaike Information Criterion.
Feature transformations are also used to improve the performance.
• Random Forest (RF): a classiﬁer is trained on the alternatives as if they were independent, considering both individual’s and alternatives’ features and using as label whether each alternative was chosen or not.
Some alternatives’ features are transformed to make them relative to the values of each choice set.
Since the classiﬁer evaluates each alternative independently, the probabilities within a given session generally do not add to one, and therefore are just interpreted as scores to rank the alternatives.
And, ﬁnally, we compare to
• Deep Pointer Networks (DPN) (Mottini & Acuna-Agost, 2017): a recurrent neural network that uses both the features of the individual and those of the alternatives to learn to point to the chosen alternative from the choice sets given as sequences.
The results are dependent on the order of the alternatives, which was taken as in the original paper, that is, as they were shown to the user.
We compute the following performance measures on the test set T of choice sets and corresponding individuals:
• Normalized Log Loss (NLL): given a probabilistic choice model ˆ P , NLL ≡ − 1 |T | (cid:80) ( S,I ) ∈T log ˆ P S
( Y S | I ) .
• TOP N accuracy: proportion of choice sessions where the actual choice was within the top N ranked alternatives.
In case of ties, they are randomly broken.
We consider N ∈ { 1 , 5 } .
Table 3 shows that PCMC-Net outperforms all the contenders in all the considered metrics.
It achieves a 21.3% increase in TOP-1 accuracy and a 12.8% decrease in NLL with respect to the best contender for each metric.
In particular, we observe that the best in TOP N accuracy among the contenders are LC-MNL and RF, both requiring manual feature engineering to achieve such performances whereas PCMC-Net automatically learns the best representations.
We also observe that our results are signiﬁcantly better than those obtained with the previous deep learning approach DPN, showing the importance of the PCMC deﬁnition in our deep learning approach to model the complex behaviors observed in airline itinerary choice data.
[Table]
C ONCLUSIONS
We proposed PCMC-Net, a generic neural network architecture equipping PCMC choice models with amortized and automatic differentiation based inference using alternatives’ features.
As a side beneﬁt, the construction allows to condition the probabilities on the individual’s features.
We showed that
PCMC-net is able to approximate any PCMC model arbitrarily well and, thus, maintains the ﬂexibility (e.g., allowing to represent non-regular models) and the desired property of uniform expansion.
Being neural network based, PCMC-Net allows complex feature handling as previous machine learning and deep learning based approaches but with the additional theoretical guarantees.
We proposed a practical implementation showing the beneﬁts of the construction on the challenging problem of airline itinerary choice prediction, where attraction effects are often observed and where alternatives rarely appear more than once—making the original inference approach for PCMC inappropriate.
As future work, we foresee investigating the application of PCMC-Net on data with complex features (e.g. images, texts, graphs ...) to assess the impact of such information on preferences and choice. A CKNOWLEDGMENTS
Thanks to María Zuluaga, Eoin Thomas, Nicolas Bondoux and Rodrigo Acuña-Agost for their insightful comments and to the four anonymous reviewers whose suggestions have greatly improved this manuscript.
A F EATURES OF THE AIRLINE ITINERARY CHOICE DATASET
[Table]