PCMC-Net: Feature-based Pairwise Choice Markov Chains

1 INTRODUCTION . Choice modeling aims at finding statistical models capturing the human behavior when faced with a set of alternatives . Classical examples include consumer purchasing decisions , choices of schooling or employment , and commuter choices for modes of transportation among available options . Traditional models are based on different assumptions about human decision making , e.g . Thurstone ’ s Case V model ( Thurstone , 1927 ) or Bradley-Terry-Luce ( BTL ) model ( Bradley & Terry , 1952 ) . Nevertheless , in complex scenarios , like online shopping sessions presenting numerous alternatives to user-specific queries , these assumptions are often too restrictive to provide accurate predictions . Formally , there is a universe of alternatives U , possibly infinite . In each choice situation , some finite choice set S ⊆ U is considered . A choice model is a distribution over the alternatives of a given choice set S , where the probability of choosing the item i among S is denoted as PS ( i ) . These models can be further parameterized by the alternatives ’ features and by those of the individual making the choice . An important class of choice models is the Multinomial Logit ( MNL ) , a generalization of the BTL model—defined for pairwise choices only—to larger sets . MNL models satisfy Luce ’ s axiom also known as independence of irrelevant alternatives ( Luce , 1959 ) , which states that the probability of selecting one alternative over another from a set of many alternatives is not affected by the presence or absence of other alternatives in the set . Moreover , any model satisfying Luce ’ s axiom is equivalent to some MNL model ( Luce , 1977 ) . Equivalently , the probability of choosing some item i from a given set S can be expressed as PS ( i ) = wi/ ∑ j∈S wj where wi is the latent value of the item i. Luce ’ s axiom implies stochastic transitivity i.e . if P ( aB b ) ≥ 1/2 and P ( bB c ) ≥ 1/2 , then P ( a B c ) ≥ max ( P ( aB b ) , P ( bB c ) ) where P ( i B j ) ≡ P { i , j } ( i ) ( Luce , 1977 ) . Stochastic transitivity implies the necessity of a total order across all elements and also prevents from expressing cyclic preference situations like the stochastic rock-paper-scissors game described in Section 3.2 . Thurstone ’ s Case V model exhibits strict stochastic transitivity but does not satisfy Luce ’ s axiom ( Adams & Messick , 1958 ) . Luce ’ s axiom and stochastic transitivity are strong assumptions that often do not hold for empirical choice data ( see ( Ragain & Ugander , 2016 ) and references therein ) . For example , Luce ’ s axiom prevents models from expressing context effects like the attraction effect ( also know as asymmetric dominance or decoy effect ) , the similarity effect and the compromise effect . The attraction effect occurs when two alternatives are augmented with an asymmetrically dominated one ( i.e. , a new option that is inferior in all aspects with respect to one option , but inferior in only some aspects and superior in other aspects with respect to the other option ) and the probability of selecting the better , dominant alternative increases ( Huber et al. , 1982 ) . The similarity effect arises from the introduction of an alternative that is similar to , and competitive with , one of the original alternatives , and causes a decrease in the probability of choosing the similar alternative ( Tversky , 1972 ) . The compromise effect occurs when there is an increase in the probability of choosing an alternative that becomes the intermediate option when a third extreme option is introduced ( Simonson , 1989 ) . Examples of these effects are visualized in Section 4 . A larger class of models is the one of Random Utility Models ( RUM ) ( Block & Marschak , 1960 ; Manski , 1977 ) , which includes MNL but also other models satisfying neither Luce ’ s axiom nor stochastic transitivity . This class affiliates with each i ∈ U a random variable Xi and defines for each subset S ⊆ U the probability PS ( i ) = P ( Xi ≥ Xj , ∀j ∈ S ) . RUM exhibits regularity i.e . if A ⊆ B then PA ( x ) ≥ PB ( x ) . Regularity also prevents models from expressing context effects ( Huber et al. , 1982 ) . The class of Nested MNL ( McFadden , 1980 ) allows to express RUM models but also others that do not obey regularity . Nevertheless , inference is practically difficult for Nested MNL models . Recently , a more flexible class of models called Pairwise Choice Markov Chains has been introduced in Ragain & Ugander ( 2016 ) . This class includes MNL but also other models that satisfy neither Luce ’ s axiom , nor stochastic transitivity , nor regularity . This class defines the choice distribution as the stationary distribution of a continuous time Markov chain defined by some transition rate matrix . Still , it satisfies a weakened version of Luce ’ s axiom called uniform expansion stating that if we add “ copies ” ( with no preference between them ) , the probability of choosing one element of the copies is invariant to the number of copies . Although the flexibility of this class is appealing , the proposed inference is based on maximizing the likelihood of the rate matrix for the observed choices which is prone to overfitting when the number of observations for each possible alternative is small and is inappropriate when new alternatives can be seen at test time . Alternatives and individuals making choices can be described by a set of features that can be then used to understand their impact on the choice probability . A linear-in-features MNL assumes that the latent value is given by a linear combination of the parameters of the alternatives and the individual . Features of the individual can be taken into account by these models but inference suffers from scarcity and is inappropriate when new alternatives can be seen at test time . The latent class MNL ( LC-MNL ) model ( Greene & Hensher , 2003 ) takes into account individual heterogeneity by using a Bayesian mixture over different latent classes—whose number must be specified—in which homogeneity and linearity is assumed . A linear-in-features parameterization of PCMC , for features in Rd , was suggested in ( Ragain & Ugander , 2016 , Appendix ) but requires fitting a weight matrix of size |U | × d , which makes it scale poorly and does not allow to predict unseen alternatives . In this work , we propose an amortized inference approach for PCMC in the sense that the statistical parameters are reused for any pair of alternatives and their number is thus independent of the size of the universe . In addition , we allow non-linear modeling by using a neural network . In complex cases like airline itinerary choice , where the alternatives are strongly dependent on an individual-specific query and some features , like price , can be dynamic , the previous approaches have limited expressive power or are inappropriate . Two recently introduced methods allow complex feature handling for alternatives and individuals . Mottini & Acuna-Agost ( 2017 ) proposes a recurrent neural network method consisting in learning to point , within a sequence of alternatives , to the chosen one . This model is appealing because of its feature learning capability but neither its choice-theoretic properties have been studied nor its dependence on the order of the sequence . Lhéritier et al . ( 2019 ) proposes to train a Random Forest classifier to predict whether an alternative is going to be predicted or not independently of the rest of the alternatives of the choice set . This approach does not take into account the fact that in each choice set exactly one alternative is chosen . For this reason , the probabilities provided by the model are only used as scores to rank the alternatives , which can be interpreted as latent values—making it essentially equivalent to a non-linear MNL . To escape this limitation and make the latent values dependent on the choice set , relative features are added ( e.g . the price for i-th alternative pricei is converted to pricei/minj∈S pricej ) . The non-parametric nature of this model is appealing but its choice-theoretic properties have not been studied either . In this work , we propose to enable PCMC with neural networks based feature handling , therefore enjoying both the good theoretical properties of PCMC and the complex feature handling of the previous neural network based and non-parametric methods . This neural network parameterization of PCMC makes the inference amortized allowing to handle large ( and even infinite ) size universes as shown in our experiments for airline itinerary choice modeling shown in Section 5 . 2 BACKGROUND : PAIRWISE CHOICE MARKOV CHAINS . 2.1 DEFINITION . A Pairwise Choice Markov Chain ( PCMC ) ( Ragain & Ugander , 2016 ) defines the choice probability PS ( i ) as the probability mass on the alternative i ∈ S of the stationary distribution of a continuous time Markov chain ( CTMC ) whose set of states corresponds to S. The model ’ s parameters are the off-diagonal entries qij ≥ 0 of a rate matrix Q indexed by pairs of elements in U . Given a choice set S , the choice distribution is the stationary distribution of the continuous time Markov chain given by the matrix QS obtained by restricting the rows and columns of Q to elements in S and setting qii = − ∑ j∈S\i qij for each i ∈ S. Therefore , the distribution PS is parameterized by the |S| ( |S| − 1 ) transition rates of QS . The constraint qij + qji > 0 ( 1 ) is imposed in order to guarantee that the chain has a single closed communicating class which implies the existence and the unicity of the stationary distribution πS ( see , e.g. , Norris ( 1997 ) ) obtained by solving { πSQS = 0 πS1 T = 1 ( 2 ) where 0 and 1 are row vectors of zeros and ones , respectively . Since any column of Qs is the opposite of the sum of the rest of the columns , it is equivalent to solve πSQ ′ S = [ 0 | 1 ] ( 3 ) where Q′S ≡ [ ( ( QS ) ij ) 1≤i≤|S|,1≤j < |S| | 1 T ] . 2.2 PROPERTIES . In Ragain & Ugander ( 2016 ) , it is shown that PCMC allow to represent any MNL model , but also models that are non-regular and do not satisfy stochastic transitivity ( using the rock-scissor-paper example of Section 3.2 ) . In the classical red bus/blue bus example ( see , e.g. , Train ( 2009 ) ) , the color of the bus is irrelevant to the preference of the transportation mode “ bus ” with respect to the “ car ” mode . Nevertheless , MNL models reduce the probability of choosing the “ car ” mode when color variants of buses are added , which does not match empirical behavior . PCMC models allows to model this kind of situations thanks to a property termed contractibility , which intuitively means that we can “ contract ” subsets Ai ⊆ U to a single “ type ” when the probability of choosing an element of Ai is independent of the pairwise probabilities between elements within the subsets . Formally , a partition of U into non-empty sets A1 , . . . , Ak is a contractible partition if qaiaj = λij for all ai ∈ Ai , aj ∈ Aj for some Λ = { λij } for i , j ∈ { 1 , . . . , k } . Then , the following proposition is shown . Proposition 1 ( Ragain & Ugander ( 2016 ) ) . For a given Λ , letA1 , . . . , Ak be a contractible partition for two PCMC models on U represented by Q , Q′ with stationary distributions π , π′ . Then , for any Ai , ∑ j∈Ai PU ( j ) = ∑ j∈Ai P ′U ( j ) . Then it is shown , that contractibility implies uniform expansion formally defined as follows . Definition 1 ( Uniform Expansion ) . Consider a choice between n elements in a set S ( 1 ) = { i11 , . . . , in1 } , and another choice from a set S ( k ) containing k copies of each of the n elements : S ( k ) = { i11 , . . . , i1k , i21 , . . . , i2k , . . . , in1 , . . . , ink } . The axiom of uniform expansion states that for each m ∈ { 1 , . . . , n } and all k ≥ 1 , PS ( 1 ) ( im1 ) = k∑ j=1 PS ( k ) ( imj ) .

Summary: This paper enables a feature-based parametrization and amortized inference of Pairwise Choice Markov Chains (PCMCs), a model for decisions in the face of a set of alternative choices (e.g. the rock-paper-scissors game).  Previous approaches to fitting PCMCs have leveraged sequential least squares programming, making optimization unstable, the model prone to overfitting, and test-time inference difficult.  The authors propose parametrizing PCMCs with neural networks to fix these issues.  Relying on universal function approximation results, the authors show that their PCMC-Net can represent arbitrary transition matrices.  The experiments report results on a dataset of airline booking behavior, comparing PCMC-Net with four other baselines from the literature.