Title: Tractable Multinomial Logit Contextual Bandits with Non-Linear Utilities
Venue: NeurIPS 2025
Accepted: yes
Keywords: multinomial logit choice model

contextual bandit

non-linear function approximation

regret analysis
Abstract: We study the _multinomial logit_ (MNL) contextual bandit problem for sequential assortment selection. Although most existing research assumes utility functions to be linear in item features, this linearity assumption restricts the modeling of intricate interactions between items and user preferences. A recent work (Zhang & Luo, 2024) has investigated general utility function classes, yet its method faces fundamental trade-offs between computational tractability and statistical efficiency. To address this limitation, we propose a computationally efficient algorithm for MNL contextual bandits leveraging the upper confidence bound principle, specifically designed for non-linear parametric utility functions, including those modeled by neural networks. Under a realizability assumption and a mild geometric condition on the utility function class, our algorithm achieves a regret bound of $\tilde{\mathcal{O}}(\sqrt{T})$, where $T$ denotes the total number of rounds. Our result establishes that sharp $\tilde{\mathcal{O}}(\sqrt{T})$-regret is attainable even with neural network-based utilities, without relying on strong assumptions such as neural tangent kernel approximations. To the best of our knowledge, our proposed method is the first computationally tractable algorithm for MNL contextual bandits with non-linear utilities that provably attains $\tilde{\mathcal{O}}(\sqrt{T})$ regret. Comprehensive numerical experiments validate the effectiveness of our approach, showing robust performance not only in realizable settings but also in scenarios with model misspecification.


Peer reviews:
comments: SUMMARY:
The paper studies multinomial logistic bandits where the action is a set of $N$ items and the feedback is the item chosen. The choice model is a logistic model where the utility of an item depends on the context and the item. The utility function is assumed to be a non-linear parametric function and the learner has access to the parametric function class. The paper proposes a two-stage algorithm where the first state does uniform exploration and the second stage uses the principle of optimism to select the most optimistic set of items. A naive explore-then-commit type of algorithm gives suboptimal regret bound. The paper introduces novel regularization and accompanying analysis to get the optimal $\sqrt{T}$ regret bound.

STRENGTHS AND WEAKNESSES:
Strengths:

The paper is clearly written and the algorithm is well-motivated by its accompanying analyses.

Weaknesses:

There are some parts of the results presented that raise some questions. (See below).

QUESTIONS:
Q1) What if the parameter $\kappa$ in the regret bound is very small? Is there a trivial lower bound on $\kappa$? What is the intuition for the dependence on $\kappa^{-1}$? For example, why would the presence of an item that is rarely chosen make the problem harder?

Q2) In Line 194, the paper says the context vectors after Phase I may be chosen adversarially. This questions the role of Phase I. What if the context distribution is only along one direction, and the contexts are chosen adversarially to be orthogonal to that direction? In this case, wouldn't the data in Phase I be obsolete?

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-23

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the multinomial logit bandit problem with parameterized non-linear utility functions. Unlike prior works on this problem, computational and statistical efficiency are simultaneously achieved; the authors propose a new algorithm that achieves the order-optimal $\\sqrt{T}$ rate while allowing tractable implementation. The algorithm has two phases: the first is a random exploration phase where an initial estimate of the latent parameter defining the utility function is learned, while in the second phase, a UCB approach is adopted. The estimated parameter vector is refined in the second phase via minimizing at each round a regularized negative log-likelihood function that employs a linear relaxation of the utility function, while the predictions (assortments) are chosen so as to maximize the expected reward when using optimistic estimates for the utility values. The achieved regret bound does not depend on the number of arms nor the cardinality constraint, but depends on a problem-specific constant $\\kappa$.\n\nSTRENGTHS AND WEAKNESSES:\nOn the positive side, by proposing a provably statistically efficient and tractable algorithm, the paper seems to bring forth a significant contribution to a challenging but practically relevant problem that combines a combinatorial structure with complex utility function and feedback mechanism. The paper also seems to provide new technical contributions like employing a linearized MNL model and establishing a new concentration result. The presentation of the paper is mostly clear and the writing is of good quality.\n\nOn the other hand, maybe I missed this, but it does not seem that the authors have established the tractability of solving the optimization problem through which the initial estimate is obtained. In fact, it has been mentioned on two occasions that the negative log-likelihood function lacks important properties here compared to the linear case, but only its effect on the analyses is highlighted. Moreover, it is not entirely clear if the dependence of the regret on $\\kappa$ is in any way natural, it also seems that Zhang and Luo [34] argue that this could result in an exponential dependence on some problem parameters. This limitation is mentioned in the conclusion, but elsewhere it is asserted that this is preferred over depending on the number of arms or the cardinality constraint as in [34]. Maybe the authors can elaborate more on why this is the case. It is also worth mentioning that in obtaining their results, the authors employ extra assumptions compared to [34], even as the relative mildness of which is extensively argued for. Lastly, it seems that a similar 'reverse Lipschitz' property has already been established in [34, Lemma 3]; hence, some credit should be given. Maybe the paper could benefit from a more explicit comparison with the techniques in [34].\n\nQUESTIONS:\n* Can you argue for the tractability of solving the optimization problem through which the initial estimate is obtained?\n* Why is the dependence on $\\kappa$ preferred over a dependence on $K$ or $N$? Are there interesting cases that can motivate this?\n* Do you believe that assumption 4 is necessary for obtaining a sharp regret rate efficiently?\n\nLIMITATIONS:\nThe authors acknowledge that the dependence of the regret on (the reciprocal of) $\\kappa$ can be a limitation. No other limitations/shortcomings seem to be highlighted in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the contextual multinomial logistic (MNL) bandit problem with non-linear utility function. Based on upper confidence bound (UCB) principle, a novel computationally efficient algorithm for contextual MNL is provided. Such algorithm is applicable for utility functions modelled by neural network, and achieves a near-optimal regret $O(\\sqrt{T})$ without relying on over-parameterized neural tangent kernel approximations. Numerical experiments are conducted to validate the effectiveness and robustness of the proposed algorithm, in both the realizable setting and model misspecification scenarios.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n(1)\tThis paper proposes for multinomial logistic bandit problem a novel UCB-based algorithm, which is computationally tractable and statistically efficient. The regret bound of $\\tilde{O}(\\sqrt{T})$ is near optimal under non-linear utility settings.\n\n(2)\tThe algorithm uses two-phase optimization: in the first phase it runs uniform exploration and empirical risk minimization strategy to output a pilot estimator as the initial utility parameter for the second phase; in the second phase, the algorithm adopts a sophisticated strategy, uses the estimated parameter $\\hat{\\boldsymbol{w}} _{t}$ to construct a tight confidence set and the optimistic utility estimate, then construct the optimistic reward based on the optimistic utility estimate, finally chooses an assortment set by maximizing the optimistic expected reward.\n\n\n(3)\tIn the design of negative log-likelihood in MNL model, the authors apply a first-order Taylor approximation of the non-linear utility function to circumvent the non-linear challenges.\n\n(4)\tThe proofs are technical. I check the proofs as far as I can, the crucial technique is to leverage Lemma D.1 in Zhang and Luo [34] and the quadratic growth condition in Assumption 4 to derive high-probability convergence rate $O(1/n)$ (via induction) for $\\min _{\\boldsymbol{w} \\in \\mathcal{W}^{\\star}} \\lVert \\hat{\\boldsymbol{w}} _{t} - \\boldsymbol{w}\\rVert _{2}^{2}$, use it to set the upper confidence bound on $\\min _{\\boldsymbol{w} \\in \\mathcal{W}^{\\star}} \\lVert \\hat{\\boldsymbol{w}} _{t} - \\boldsymbol{w}\\rVert _{\\mathbf{V} _{t}}^{2}$, and eliminate the dependence on the number $N$ of items. Finally, use this upper confidence bound to derive the near-optimal regret $O(\\sqrt{T\\log{T}})$.\n\n\n(5)\tExperiments on the realizable and misspecification settings further validate the effectiveness and lower regret of the proposed algorithm.\n\n**Weaknesses**\n\n(1)\tThe major concern lies in whether the used assumptions are reasonable enough to derive improved results. For example, the boundedness of loss function/its gradient/its Hessian in Assumption 2 is a very strong assumption, especially in the non-convex deep neural network settings. It is not very clear whether such assumption is used in the related MNL works, or holds in the neural network-based bandit setting.\n\n(2)\tThe comparisons with related works are not fair enough. It is true that this work obtains sharper regret bound than that in Zhang and Luo [34], but in referent [34] their theoretical results do not rely on the very strong boundedness assumption as in Assumption 2 of this work. Besides, [34] does not use generalized geometric condition in Assumption 4 either (although I know Assumption 4 is the key assumption to derive improved regret bound of this paper). Therefore, I would suggest authors to list a table (like Table 1 in [34]) in the main paper or appendix, to explicitly compare the necessary assumptions and regret bounds of existing works and this work. \n\n(3)\tThe quadratic growth condition in Assumption 4 is the key to derive tighter confidence bound. However, more concrete examples (especially in the non-convex deep neural network bandit setting) should be added to explain the reasonableness of this assumption.\n\n(4)\tIn Theorem 1, $\\kappa=\\min _{\\boldsymbol{w}, X,S,i}p(0|X,S,\\boldsymbol{w})p(i|X,S,\\boldsymbol{w})$ is the minimum of many probability, and its value affects the number of iteration $n/T$. Will it be close to zero, and more experiments for the value of this hyperparameter should be added.\n\nQUESTIONS:\n(1)\tCould you please give more examples to show the reasonableness of Assumptions 2 and 4 ?\n\n(2)\tCould you please give more fair comparisons (i.e. the assumptions used to derive regret bounds) with related works (e.g. [34]) in the discussion of Theorem 1 ?\n\n(3)\tCould you please show empirically the value of the hyperparameter $\\kappa$ ?\n\nLIMITATIONS:\n(1)\tIt seems that the used Assumption 2 is very strong, and more examples of neural network-based bandit model (except logit bandit with negative loss) are missing.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the contextual multinomial logistic (MNL) bandit problem, a key model for sequential assortment selection in applications like online recommendation and retail. Most prior work assumes linear utility functions, which limits expressiveness. The authors propose a new algorithm—ONL-MNL—that:\n\n1. Handles general non-linear parametric utility functions (e.g., neural networks),\n2. Achieves a regret bound of the square root of T. \n3. Is computationally tractable (unlike prior work that trades off tractability for statistical efficiency),\n4. Avoids reliance on neural tangent kernel (NTK) assumptions or overparameterization.\n\nThe algorithm uses a two-phase strategy: uniform exploration followed by optimistic exploration using a novel confidence set construction. Theoretical guarantees are supported by a new generalized geometric condition and a concentration inequality. Empirical results validate the method under both realizable and misspecified settings.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The problem is relevant and well-motivated.\n2. The regret bound is great. However, I did not check the math in great detail.\n\n**Weaknesses:**\n\n1. Can you provide some real-world numerical evaluations? Experiments are limited to synthetic settings with fixed N, K, and d.\nThere is no exploration of scalability (e.g., larger N or deeper networks), nor any real-world datasets.\n2. As there can be multiple global minima, the convergence rate of the algorithm may depend on which global minima the algorithm finds. This may affect the regret bound. Did you consider this in the design and analysis of your algorithm?\n3. The paper is dense and highly technical. Some sections (e.g., the regret analysis) could benefit from more intuition.\n4. The regret bound depends on a problem-specific parameter $\\epsilon$, which may be small in worst-case scenarios.\n5. The baselines are mostly linear or from a single prior work on general utilities. Recent advances in neural bandits (e.g., NeuralUCB, NeuralTS) are not included.\n\nQUESTIONS:\nRefer to weaknesses.\n\nLIMITATIONS:\nRefer to weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}