Learning Mixed Multinomial Logits with Provable Guarantees
Anonymous Author(s)
AffiliationAddress email
Abstract
A mixture of multinomial logits (MMNL) generalizes the single logit model,which is commonly used in predicting the probabilities of different outcomes.
While extensive algorithms have been developed in the literature to learn MMNLmodels, theoretical results are limited.
Built on the Frank-Wolfe (FW) method, wepropose a new algorithm that learns both mixture weights and component-specificlogit parameters with provable convergence guarantees for an arbitrary numberof mixtures.
Our algorithm utilizes historical choice data to generate a set ofcandidate choice probability vectors, each being close to the ground truth with ahigh probability.
We further provide a sample complexity analysis to show that onlya polynomial number of samples is required to secure the performance guarantee ofour algorithm.
Finally, we conduct simulation studies to evaluate the performanceand demonstrate how to apply our algorithm to real-world applications
Introduction
Multinomial logit models (MNL) are widely used in a variety of settings to predict the probabilitiesof different outcomes.
It is termed as multinomial logistic regression in statistics and its formulafor choosing the j -th option,
[Equation]
i exp( v i ) , is well known as the softmax function for multi-classclassification in machine learning.
In choice modelling, it is a type of discrete choice model used toanalyze and predict people’s decisions.
Mixed multinomial logit (MMNL) is a mixture model consisted of multiple individual MNL components and can be adopted when the target data is heterogeneous.
For instance, Yang et al.
[2018]demonstrates that using a single softmax does not have enough capacity to model natural languageand claims this as a bottleneck.
To alleviate this issue, they proposed a new method called Mixtureof Softmaxes (MoS), which has higher expressiveness and can better incorporate the contextualinformation.
Later, new techniques are proposed on top of this architecture to further boost theefficiency, such as the Mixtape from Yang et al.
On the other hand, McFadden and Train[2000] shows that MMNL models can lift the restrictive “independence of irrelevant alternatives”(IIA) assumption present in single MNL models as well as approximate any random utility-baseddiscrete choice models to arbitrary degree of accuracy under appropriate assumptions.
Despite its importance, the study on the learning theory of MMNL is limited.
A common approachin the literature is to assume a parametric family of the mixing distribution (cf.
Train [2009] and
Berry et al.
But this approach suffers from the model misspecification issue, i.e., there couldbe systematical errors in the estimation if the assumed parametric family is different from the trueone.
Expectation-Maximization (EM) algorithm is a widely used non-parametric method for mixturemodels.
But it does not guarantee the estimators’ convergence.
Recent inspiring work by Jagabathulaet al.
[2020] creatively applies the Frank-Wolf (FW) algorithms to estimate the MMNL model and establish the sub-linear convergence of the proposed method.
However, the major issue with theirapproach is that the convergence result only applies to the aggregate predictors. and cannot recovereach individual mixture.
In this paper, we propose a new Stochastic Subregion Frank-Wolfe ( SSRFW )algorithm, which can generate each individual mixture within an ϵ -ball to one of the ground truthvectors with a high probability.
We summarize the main contributions of this paper as follows.
First, we provide a learning algorithmfor MMNL models that comes with provable convergence guarantees of the estimators.
In particular,each generated mixture from our algorithm is shown to be within an ϵ -ball to one of the ground truthwith a high probability.
To the best of our knowledge, this is the first paper to prove these results for
MMNL models with an arbitrary number of mixture types and minimal assumptions on the modelparameters.
Second, we further quantify the sample complexity of the proposed algorithm.
We showthat only a polynomial number of samples is required to achieve a high probability convergenceguarantee.
1.1 Preliminaries
Consider a setting where the decision makers need to choose among a set of [ M ] = { 1 , . .
, M } items,where each alternative has a feature vector z j ∈ R d .
Furthermore, we assume the population consistsof K different components (types), each associated with a unique “preference vector” β k ∈ R d anda mixture weight α k , where (cid:80)
Kk
=
α k = 1 .
In other words, β k ’s are not drawn from a parametricmixing distribution and we consider MMNL in this paper as a type of latent class models, which iswhy we claim our method as non-parametric .
This is consistent with the definitions used in otherliterature such as Train [2009] and Jagabathula et al.
[2020].β k ’s can also be used as user embeddings with high interpretability for decision makers who belong tothe same type, which represent the corresponding type’s taste on different aspects of the alternatives.
Under linear utility assumption, the probability of choosing j -th item for a particular mixture k is
[Equation]
and the aggregated counterpart for the entire population is
[Equation]
To simplify notation, we use q k to denote the logit vector q ( β k ) =
[ q 1 ( β k ) , · · · , q M ( β k )] ⊤ ∈ R M ,and q kj for q j ( β k ) , j ∈ [ M ] , ∀ k .
We will also refer to these logit vectors as choice probabilityvectors .
1.2 Related Work
Learning MMNL models remains an open-research problem.
One common approach in the literatureis to presume a parametric family of distributions on the parameters and apply parametric estimationmethods such as the maximum likelihood estimation (MLE) method (c.f. Train [2009]) or the leastsquare regression model to compute the parameters.
A well-known work in this regime is by Berryet al.
They assume the component-specific parameters are normal distributed and proposed atwo-step estimation method to learn the parameters using an aggregated market share data.
A potentialissue for this stream of methods is model misspecification, which leads to inaccurate predictions.
Inparticular, if the assumed parametric family is different from the true one, there could be systematicalerrors in the estimation.
Non-parametric estimation is also widely recognized when learning MMNL models, with the mostcommonly known and used method being the Expectation-Maximization (EM) algorithm (c.f.
Dempster et al.
Train [2008] studied three different types of EM algorithms using historical choicedata from each individual decision maker.
The two main restrictions for the EM algorithm are 1) thealgorithm can get stuck in local optimum (hence no guarantee on the convergence of the estimators)and 2) the number of mixture types needs to be pre-specified (additional heuristics are needed if suchinformation is not known in advance, which may also introduce undesired noises).
Jagabathula et al.
[2020] recently developed an estimation method for mixture models also from a nonparametric perspective based on the Frank-Wolfe (FW) algorithm (cf.
[Frank and Wolfe, 1956], Jaggi[2013], [Jaggi and Sulovsk`y, 2010], [Bach, 2013], Harchaoui et al.
[2015],Lacoste-Julien and Jaggi[2015]), which is an iterative method originally designed to solve constrained quadratic optimization.
They manage to establish a sublinear convergence rate of the proposed estimation method for MMNLmodels.
But the convergence result only applies to the aggregate choice probabilities.
In contrast, weprovide the convergence guarantee for each individual choice probability (more details about theirapproach and the comparisons will be discussed in Section 2).
There is also another stream of work related to MMNL models from a theoretical learning perspective.
The key idea is to solve a system of equations in terms of all the parameters and prove that there isone and only one solution (identifiability).
Due to the entanglement of the individual logit parametersand mixture weights, as well as the high non-linearity in the system of equations, only 2-MNL models(MMNL with two mixture types) have been studied so far.
Chierichetti et al.
[2018] considers thesetting where the mixture weights are equal (i.e. each mixture represents 50% of the population)and Tang [2020] recently studied 2-MNL with unknown weights.
This paper enriches the theoreticallearning literature for MMNL models by generalizing the analysis to an arbitrary number of mixturetypes and with minimal assumptions on the model parameters.
1.3 Main Results
The main contribution of this paper is to provide a learning algorithm for MMNL models withprovable convergence guarantees of the estimators.
Specifically we will show that each of thegenerated choice probability vectors from our algorithm is within an ϵ -ball to one of the ground truthvectors with a high probability.
Subsequently, we provide the sample complexity analysis of thealgorithm.
We name our algorithm the Stochastic Subregion Frank-Wolfe ( SSRFW ) algorithm, whileis inspired by the Frank-Wolfe framework by Jagabathula et al.
but has a carefully constructedstochastic feasible region that is learned from the data.
The SSRFW framework assumes to work with repeated choice data, such as consumer panel or surveydata.
This is a reasonable assumption in many real-world scenarios (cf.
Revelt and Train [1998]and Brownstone et al.
We assume the data consists of historical choice decisions from thetarget population of size N .
Specifically, for each time period t = 1 , . .
, T , we assume each decisionmaker
i ’s choice X ( t ) i are i.i.d categorical random variables with support x ∈ [ M ] and probabilitymass function (PMF) specified by q k if he/
she is of type k .
For the MMNL to be learnable, we need to assume that none of the individual MNL is identical to therest.
Let F and G be the cumulative distribution functions (CDF) for two different choice probabilitydistribution q k and q k ′ respectively.
Define the Kolmogorov-Smirnov (KS) distance between the twodistributions as D KS ( q k , q k ′ )
=
sup x | F ( x ) − G ( x ) | .
Definition 1 below is adapted from a similarassumption introduced in Kalai et al.
[2010] to impose some regularity in the MMNL model.
Definition 1.
We call a mixed multinomial logit model (MMNL)
g ϵ -standard if1.
min k α k ≥ ϵ2.
β kd ≤ 1 ϵ , ∀ k, ∀ d3.
D [Equation] [Equation] [Equation]
Definition 1 imposes the following properties on the MMNL model: 1) none of the mixtures has zeroweight; 2) all parameters in the model are bounded; and 3) each individual logit model has somelevel of separation from others.
We consider this as a very mild assumption in our setting, as later wewill see that we will not predetermine the number of mixtures ( K ) and if a mixture/component is notpresent in the observed data (since it has zero weight), the algorithm will simply not learn it.
This isalso one of the advantages of our algorithm over the traditional approaches, such as EM.
The main result is as follows:
Theorem 1.
Let g = (cid:80)
Kk
=1 α k q k be a mixed multinomial logit (MMNL) model over a set of M items.
Assume M ≥ K .
For any ϵ > 0 , 0 < δ < 1 , Algorithm 1 outputs an MMNL ˆ g = (cid:80) K ′ k
=
ˆ α k ˆ q k where K ′ ≥ K such that, with probability ≥ 1 − δ , there exists a many-to-one mapping π : k ′ (cid:55)→ k, k ′ ∈
[ K ′ ] , k ∈
[ K ] such that
[Equation]
and
[Equation]
The number of samples required by Algorithm 1 is O ( 1 ϵ 2 log( 1 δ )) .
The outline of the paper is as follows: Section 2 and Section 3 will introduce the main algorithms,namely the Stochastic Subregion Frank-Wolfe ( SSRFW ) and the Q -construction algorithm, where theoutput of the latter serves as an input to the former.
In Section 4, we will discuss the theoreticalproperties of the SSRFW algorithm.
We conduct simulation studies in Section 5.
All the detailedproofs and parts of simulation studies are included in the appendix.
Stochastic Subregion Frank-Wolfe Algorithm
Recall that we observe each decision maker i ’s choice at time t : x ( t ) i (as realization of X ( t ) i ).
Let
Y ti =
[0 , · · · , 1 , · · · , 0] ⊤ ∈ R M , where Y tij
=
if x ( t ) i =
j .
The observed aggregated choiceprobability vector for period t can be expressed asy
[Equation]
N (cid:88) i =
Y ti where y t ∈ R M .
Define P = { q ( β ) ∈
R M | β ∈ R d } , the set of all valid logit vectors for the givenchoice set and their features.
Denote P as its closure.
Since g = (cid:80) Kk
=1 α k q k , by definition
we haveg ∈
Conv ( P ) .
Our main learning objective is to minimize the difference between the observed (from data) andtheoretical aggregated choice probability, as follows
[Equation]
Note that problem (1) is a non-convex problem in β k and is in general hard to solve.
We adoptthe conditional gradient approach proposed by Jagabathula et al.
[2020], which we will refer to asthe original FW algorithm.
Each iteration k is consisted of two steps: 1) the supporting findingstep which searches for a new direction represented by a choice probability vector q ( k ) (as a newlatent class k ) via solving a linear optimization subproblem in the search space Conv ( P ) , and 2)the proportion update step which updates the mixture proportion α i , i = 0 , . .
k , assigned to allchoice probability vectors obtained so far.
The final learning outcome ˆ g is (cid:80) k
i =
α k q ( k ) whereq (0) =
g (0)
∈ P is a random initialization.
This setup corresponds to a fully-corrective variant of thegeneric Frank-Wolfe algorithm, and is guaranteed to find the optimal solution to the optimizationproblem
[Jaggi, 2013] for the decision variable g .
The problem with the original FW approach is that the feasible region Conv ( P ) contains the completeset of all logit vectors and their limiting points.
The former corresponds to non-boundary types ,whose choice model can be characterized by a standard MNL choice model, and the latter correspondsto boundary types , whose standard MNL parameters become unbounded, resulting infinite utilityfor some options and zero for the rest.
Such broad search space Conv ( P ) in FW generates mixturecompositions with a considerable number of the boundary types.
More specifically, this problem isrooted to the linear subproblem solved at each support finding step , whose optimal solution always lieson the extreme point of the feasible region which contains many of these limiting choice probabilityvectors ∈ P \ P as its extreme points.
Often, each limiting choice probability vector corresponds toan unbounded β k and cause the learning outcome unusable for downstream applications.
Thoughsignificant effort is made to justify the legitimacy of these boundary types in the paper, the convergenceresult only applies to the population’s aggregate choice probability, i.e. ˆ g → (cid:80) k
α k q k , which is an Frank-Wolfe property.
In other words, their method fails to recover each of the individual logitmodels ( β k ) and the corresponding mixture weights ( α k ) accurately.
To remedy this issue and seek convergence guarantees for the general MMNL estimation problem,we designed the Stochastic Subregion Frank-Wolfe ( SSRFW ) algorithm, as shown in Algorithm 1.
Algorithm 1: Stochastic Subregion Frank-Wolfe
Input: data y , Q from Algorithm
Initialization: k = 0 ; α (0) =
[1] , a random g (0) =
q (0) chosen from Q 1 while stopping condition not met do k ← k +
Compute q ( k ) =
arg min v ∈ Conv ( Q ) ⟨∇L (cid:16) g ( k
[Equation]
1) (cid:17) ,
[Equation] [Equation]
Compute α ( k ) =
arg min
[Equation]
(cid:32)
[Equation] [Equation]
) (cid:33)
Update g ( k ) :
= α ( k )
g (0)
+
[Equation]
(cid:88) s
= α ( k ) s q ( s ) 6 end
Output : choice prob.
q (0) , . .
. , q ( k ) mixture weights.
α ( k ) ∈ ∆ k ⊂ R k +
Note in the above algorithm, after k iterations, we have generated k choice probability vectors q ( s ) ,s = 1 , . . .
, k , all of which will be the output of the learning algorithm.
The mixture weight estimatoris slightly different.
α ( k ) ∈
R k represents the mixture weight vector for the k choice probabilityvectors we have generated so far, where α ( k ) s refers to the s -th element in the vector, correspondingto the mixture associated with q ( s ) .
In other words, the vector length of α increases by 1 for eachiteration and only the final vector α ( k ) is outputted from the algorithm.
The key different in SSRFW is that it takes in an additional input Q from the Q Construction Algorithm(discussed in detail in Section 3), which is the key to elimiate the possibility of producing boundarytypes .
In particular, the new feasible region Conv ( Q ) is a subset of Conv ( P ) , where each elementq ∈ Q will be learned from data and is guaranteed to be within an ϵ -ball of the true choice probabilityvector for some mixture type with high probability.
Subsequently, this also ensures the SSRFWalgorithm to recover the mixture weight for each latent class.
Note that the extreme points of
Conv ( Q ) by definition is a subset of Q .
Since we only care about the extreme points of the feasibleregion, we can safely ignore any points ∈ Conv ( Q ) \ Q and replace Conv ( Q ) with Q in Algorithm 1.
Since Frank-Wolfe will converge to the optimal solution [Jaggi, 2013], and as we will see in Section4 that the optimal solution to problem (1) is an interior point of the feasible region Conv ( Q ) , thestopping condition can simply be set as (cid:13)(cid:13) g ( k ) − y (cid:13) (cid:13) ≤ ϵ .
Q Construction Algorithm
Our primary goal in this section is to learn the input set Q for SSRFW (Algorithm 1).
In particular forthis set Q = { ˆ q ℓ } l =1 ,...,L , we require that ∀ ˆ q ℓ , there exists some mapping π : [ L ] → [ K ] such that(cid:13)(cid:13) ˆ q ℓ − q π ( ℓ ) (cid:13)(cid:13) ≤ ϵ , where q π ( ℓ ) is one of the ground truth logit vectors.
Once this property holds, eachof the SSRFW ’s q outputs will also be ϵ -close to some true choice probability vector as the algorithmalways selects an extreme point of Conv ( Q ) in each iteration.
The two major components for the Q Construction Algorithm is first computing a distance scorematrix (Section 3.1) and then creating subsamples that contain decision makers from only one mixturetype through random seeding and non-uniform sampling (Section 3.2).
Section 3.3 presents thealgorithm and discusses the properties of the learning outcomes.
[Equation]
3.1 Distance Score Matrix
We first define a distance score matrix S , whose element s ij measures the dissimilarity between anytwo decision makers i and j in term of their choice decisions.
Recall that each x ( t ) i are realizations ofi.i.d categorical random variables with PMF q k if i is from mixture k .
Denote its associated empiricalcumulative distribution function (CDF) as F T ( x ; i ) when a total of T decisions are observed.
Inparticular,
[Equation]
Define the distance score function for each pair of decision makers as
[Equation]
where || · || ∞ represents the infinity norm.
3.2 Subsample Construction
We now want to create a number of subsamples such that each subsample of decision makers containsonly one mixture type with high probability.
Each of these subsamples is generated following aprocedure that contains a random seeding step, and a subsampling step.
These steps are repeated Ltimes to create a Q set of size L .
Random Seeding.
This step randomly samples a decision maker i ∈
[ N ] from the population andcan be done with simple uniform sampling technique.
The selected decision maker is called a seed .
We will generate L seeds and later investigate how large L needs to be in order to cover all mixturetypes in Q .
Subsampling Strategy.
This step generates an index set I ( i ) for a seed i such that, with highprobability, decision makers whose indices belonging to this set are of the same type as i .
I ( i ) iscalled the subsample originated from seed i .
Given a selected seed i , we first calculate an accepting probability p
j | i which determines the likelihoodof another decision maker j being accepted to the corresponding subsample
I ( i ) .
It is designed to bea monotonically decreasing function with respect to the distance score we defined in Section 3.1:
[Equation] [Equation]
A simple example can be p j | i = 1 − s
( i, j ) .
Intuitively, when s ( i, j ) is small, seed i and decisionmaker j ’s empirical CDF is close to each other, indicating there is a higher chance that they are of thesame type and sharing the same choice probability q .
Subsequently, p j | i will be larger compared toa large value of s ( i, j ′ ) for another person j ′ , resulting we accept j with higher probability than j ′ .
This is consistent with our objective that we want each subsample to be composed of decision makersof the same type as the seed.
For implementation, we first initiate an empty index set I ( i ) for the selected seed i .
Then we repeatthe following two steps until we reach a desired subsample size n : 1) draw a random sample from set[ N ] \ I ( i ) , 2) accept this sample into the index set with probability p
j | i and reject with 1 − p
j | i .
3.3 Algorithm and Properties
The remaining step is to obtain the set Q whose elements ˆ q ℓ are computed as the average of historicalchoice decisions from decision makers in the subsamples I ℓ , i.e. ˆ q ℓ = 1 nT (cid:88) i ∈ I ℓ
T (cid:88) t =
Y ( t ) i .
Finally, wepresent the Q Construction Algorithm in Algorithm 2.
Discussion We next discuss a few properties of the two algorithms.
SSRFW returns the logit vectors q instead of the parameters β .
We can simply perform maximumlikelihood estimation under the single MNL setting where, if the extreme point corresponding to
Input : score matrix S , number of subsamples L , subsample size n
Initialization: Q = set () 1 for ℓ ← 1 to L do
Choose seed: i ∼ U (0 , N )
Initiate: I ℓ = set () while | I ℓ | ̸ = n do j ← random_sample ([ N ] \ I ℓ )
Generate u ∼ U (0 , 1) if u < p
j | i then
I ℓ .
add ( j ) end end
Compute ˆ q ℓ = 1 nT (cid:88) i ∈ I ℓ
T (cid:88) t =
Y ( t )
i
Q . add (ˆ q ℓ ) 13 end
Output: Q subsample ℓ is selected during an SSRFW iteration, the log-likelihood is
[Equation]
and we have ˆ β ℓ = arg max β l ℓ ( β ) .
With the assumption of linear utility in β , we can obtain theoptimal solution using MLE (c.f. McFadden [1974]).
In general, we prefer learning β k ’s compared to q k ’s.
This makes the algorithm more robust whenitem attributes change over time which results in change in the q k ’s whereas β k values persist.
Toachieve this, we just need to add two additional steps after the index set is created:
• MNL paramter estimation: apply the MLE step outlined above; create B = { ˆ β ℓ } ℓ =1 ,...,L• Map B to Q : fix item attributes of current interest, calculate Q = {
ˆ q | ˆ q = q
( ˆ β ℓ ) , ˆ β ℓ ∈ B}
Next, stochasticity in our algorithm comes from two parts, namely the realization of each decisionmakers’ choices, as well as the randomness in the process of generating subsamples.
Yet we willshow that under appropriate assumptions, SSRFW can still recover all individual mixture parameterswith high probability.
In addition, we do not impose any restriction on the subsamples created during Q construction to bemutually exclusive.
As long as they only contain one single mixture type (i.e. with high purity), itcreates a legitimate estimation of ˆ q or ˆ β with MLE.
Last but not least, this is a more robust strategy than clustering algorithms to segment the population.
Not only do we not require the number of mixtures K as a hyper-parameter, each decision maker’sdata being used more than once for the estimation can be thought of as a special bootstrap mechanismthat has custom weights tailored to our objective in creating homogeneous subsamples.
Theory of the SSRFW Algorithm
Theorem 1 is the main result of this paper.
To prove it, we will break down this section into twoparts, where the first part will examine the provable convergence properties of the estimators, and thesecond part will focus on the sample complexity.
We will outline the proof sketch but all detailedproofs can be found in the Appendix.
Property 1: (cid:12) (cid:12) g FW − (cid:80) k
α k q k (cid:12) (cid:12) ≤ ϵ
Property 2: ∀ k ′ , ˆ q k ′ ∈ E ( Conv ( P ))
Property 3: (cid:80) k
α k q k ∈ Conv ( Q )
Property 4: ∀ q ∈ E ( Conv ( Q )) , ∃ k s.t.
∥ q − q k ∥ ≤ ϵ (cid:12)(cid:12) g SSRFW − (cid:80) k
α k q k (cid:12)(cid:12) ≤ ϵ ∀ k ′ , ∃ k
= π ( k ′ )
s.t.
| ˆ q k ′ − q k
|
≤ ϵ (cid:12)(cid:12) (cid:12)(cid:12)(cid:12)(cid:12) (cid:88) k ′ :
π ( k ′ )
=
k α k ′ − α k (cid:12) (cid:12) (cid:12)(cid:12)(cid:12)(cid:12) ≤
ϵ , ∀ k
FW properties SSR properties Provable Convergence. Assume K ′ is the total number of iterations that the algorithm has performed before reaching the stopping criteria.
To simplify notation, we denote the generated outcomefrom k -th iteration q ( k ) in SSRFW as ˆ q k to indicate they are the estimators and the final updatedmixture weight corresponding to q ( k ) , α ( K ′ )
k , as ˆ α k .
In other words, we have g SSRFW
=
(cid:80) K ′ k ′
=
1 ˆ α k ′
ˆ q k ′ ,
We use a similar notation g FW for the original FW approach.
Figure 1 describes the necessary components and steps of proving the result.
In particular, Property 1and 2 (purple boxes) are adapted from existing results of the Frank-Wolfe framework [Jaggi, 2013].
In Appendix A.1.1, we prove Property 1 in Lemma 2.
Property 2 follows from the fact that each ofthe ˆ q k ′ , obtained by solving a linear optimization problem, has to be an extreme point (denoted by
E ( · ) ) of the feasible region.
On the other hand, Property 3 and 4 (red boxes) hold as a consequence of the stochastic subregion weconstructed in Algorithm 2.
Property 3 claims that with high probability, the ground truth aggregatedchoice probability vector is still an interior point of the shrunken feasible region Conv ( Q ) (proofin Appendix A.1.2).
Together with Property 1, this tells us that | g SSRFW − (cid:80) k
α k q k | ≤ ϵ .
That isto say the aggregated choice probability vector of SSRFW also converges to the ground truth.
Toachieve Property 4 from the Q construction algorithm, we need to supply enough of data, the detailsof which will be discussed in the sample complexity part.
Combining Property 2 and 4, we have∀ k ′ , ∃ k
= π ( k ′ )
s.t.
| ˆ q k ′ − q k
| ≤ ϵ .
Finally, by doing some algebraic manipulation, we can obtain
[Equation]
Sample Complexity.
Recall we use T as the number of choice data records for each decision maker.
Denote T min as the minimum number required and T min ≥ 1 .
Define in-types to be the decision makers who share the same type as the seed, and out-types to bethe ones of different types.
We focus on the following three properties:
• probability of accepting the in-types,
• probability of rejecting the out-types,
• probability of covering all mixture types.
Consider a seed i , with type k i
= k .
Assume there is a total of m k decision makers in the populationthat is of type k .
Accepting In-Types.
Denote the empirical choice probability for the seed i as F n and that foranother decision maker j as G m , with n and m representing the number of independent priceexperiments from i and j , respectively.
Recall that s ( i, j ) =
|| F n ( x ) − G m ( x ) || ∞ = sup x | F n ( x ) − G m ( x ) | simplicity use F and G to denote F ( x ) and G ( x ) in the subsequentpresentation
Theorem 2 (Sample Complexity I) .
Assume i and j are of the same type.
Define p
j | i = 1 − s ( i, j ) .∀
δ > 0 , ϵ > 0 , we can achieve p j | i > 1 − ϵ with probability at least 1 − δ with sample size
T min =
O ( 1 ϵ 2 log (cid:0) 1 δ (cid:1) ) .
Theorem 2 indicates that with sufficient amount of data, there is a high probability to accept ain-type—who shares the same type as the seed—into the subsample if it is chosen from the populationafter the random draw.
Rejecting Out-Types.
Assume a decision maker j is of a different type from the seed i .
We use Fand G to denote their CDFs.
Let F n and G m represent the corresponding empirical CDFs based onsamples drawn from the two different distributions with sample size n and m , respectively.
s ( i, j ) isdefined the same as before.
Theorem 3 (Sample Complexity II) .
Assume F and G correspond to the choice CDFs of two differenttypes and sup x | F ( x ) − G ( x ) | ≥ ξ .
Let F n and G m be independent empirical distribution basedon m and n i.i.d.
samples drawn from F and G , respectively.
Denote T min = min { m, n } .
Definep j | i = 1 − s ( i, j ) .
∀ δ > 0 , ϵ > 0 , we can achieve p j | i < 1 − ξ + ϵ with probability at least 1 − δwith T min =
O ( 1 ϵ 2 log 1 δ ) .
Theorem 3 complements Theorem 2 to rule out the out-types from being selected with a highprobability.
The more separable (larger ξ ) their underlying distributions are, the lower acceptingprobabilities are.
Lemma 1.
For ϵ > 0 and 0 < δ < 1 , with minimum of choice records T required by Theorem2 and Theorem 3, ∀ q ℓ ∈ Q , (cid:13)(cid:13) q ℓ − q π ( ℓ ) (cid:13)(cid:13) ≤ ϵ with probability at least 1 − δ , where ˆ q ℓ =1 nT (cid:80)
[Table]
Lemma 1 states that with enough data samples, the Q -construction algorithm achieves Property 4, asshown in Figure 1, which SSRFW ’s desired provable convergence builds upon.
With Lemma 1, weconclude the proof for Theorem 1.
The complete proof can be found in Appendix A.4.
As a final remark, we discuss the number of subsamples (i.e. L ) needed in the Q -construction process.
We distinguish this from sample complexity as we are not requesting more data points with a larger L .
Instead, we should view L as a computation complexity factor.
Recall that each moving directionderived from the support finding step is viewed as a learned logit vector of one mixture component.
Therefore, we want the Q -construction algorithm to include at least one ˆ q ℓ for each mixture k in orderfor it to be picked in SSRFW .
This impose a requirement on L .
Theorem 4 states that the expectednumber of the number of subsamples is controlled by the smallest mixture weight.
Theorem 4.
Assume α 1 ≤ α 2 ≤ · · · α K .
The expected number of subsamples L we need to constructis bounded by 1 α 1 log 1 α 1 .
Intuitively, this means if we have one mixture component that is very under-represented, then weneed to create more subsamples to ensure that one of the seeds belongs to this mixture.
Conclusion
In this work, we propose the SSRFW algorithm, which provides an end-to-end solution for learning
MMNL models using historical choice data.
This novel approach utilizes a carefully designedsampling method to construct a meaningful search space.
Not only does it resolves the drawback ofthe original Frank-Wolfe approach with boundary-type issues but also enables us to obtain provableguarantees for the model estimates and sample complexity.
Another major advantages of ouralgorithm is that we do not asusme prior knowledge of the number of mixtures K in the model.
It isalso more robust than traditional unsupervised approaches such as clustering method and EM.
In marketing science and operations research, MMNL provides a fundamental tool for demandprediction, which many downstream applications such as pricing and inventory position reply on.
For instance, consider a pricing model that tries to maximize the revenue by solving the followingoptimization problem: max p (cid:80) Mj
=
p j (cid:80) Kk =1 α k q j ( β k ; p ) where p j ≥ 0 , ∀ j ∈ [ M ] .
This requiresan accurate estimation of each individual MNL components.
Due to page limit, we include oursimulation studies and application in real-world setting in the appendix.
We will also briefly discussthe requirements on the datasets in order to apply our algorithm.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .•
Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes] See Appendix B.(c)
Did you discuss any potential negative societal impacts of your work?
[N/A](d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 1,Section 4, and Appendix A.(b)
Did you include complete proofs of all theoretical results?
[Yes] See Section 4, andAppendix A.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See AppendixB.(b)
Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Appendix B.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Appendix B.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[N/A]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Appendix B.
(b) Did you mention the license of the assets?
[Yes] See Appendix B.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[-Kk =
-]
[-=
-]
[-T = 
-]
[-For any δ > 0 , we have P ( H L ) ≥ 
-]
[-=
-]
spent on participant compensation?
[N/A]