DeepHalo: A Neural Choice Model with Controllable Context Effects
Authors: Shuhan Zhang
Zhi Wang
Rui Gao
Shuang Li
Venue: NeurIPS 2025 spotlight ()
Keywords: Choice Model
Preference Learning
Deep Learning
Primary area: machine_learning_for_sciences

Abstract: Modeling human decision-making is central to applications such as recommendation, preference learning, and human-AI alignment. While many classic models assume context-independent choice behavior, a large body of behavioral research shows that preferences are often influenced by the composition of the choice set itself---a phenomenon known as the context effect or Halo effect. These effects can manifest as pairwise (first-order) or even higher-order interactions among the available alternatives.
Recent models that attempt to capture such effects either focus on the featureless setting or, in the feature-based setting, rely on restrictive interaction structures or entangle interactions across all orders, which limits interpretability. 
In this work, we propose DeepHalo, a neural modeling framework that incorporates features while enabling explicit control over interaction order and principled interpretation of context effects.
Our model enables systematic identification of interaction effects by order and serves as a universal approximator of context-dependent choice functions when specialized to a featureless setting.
Experiments on synthetic and real-world datasets demonstrate strong predictive performance while providing greater transparency into the drivers of choice.

DeepHalo: A Neural Choice Model with Controllable Context Effects
Shuhan Zhang
Zhi Wang
The Chinese University of Hong Kong. Shenzhen shuhanzhang@link cuhk. edu. en
University of Toronto zhiss wangoutoronto.co
Rui Gao
Shuang Li"
The University of Texas at Austin rui madi.aoo@nccmmss.. utexas.ee... edu
The Chinese University of Hong Kong, Shenzhen lishuang@cuhle www.huu...cce...........o.c.ee.. cn
Abstract
Modeling human decision-making is central to applications such recommenda- tion, preference learning, and human-A AI alignment. While many classic models assume context-independee
choice behavior, a large body of behavioral research shows that preferences are often influenced by the composition of the choice set itself-a phenomenon known the context effect Halo effect. These effects can manifest as pairwise (first-order) or even higher-order interactions among the available alternatives. Recent models that attempt to capture such effects either focus on the featureless setting or, the feature-based setting, rely on restrictive interaction structures or entangle interactions across all orders, which limits in- terpretability. In this work, we propose DeepHalo, a neural modeling framework that incorporates features while enabling explicit control over interaction order and principled interpretation of context effects. Our model enables systematic identification of interaction effects by order and serves as a universal approximator of context-depende choice functions when specialized to a featureless setting. Experiments on synthetic and sal-world datasets demonstrate strong predictive performance while providing greater transparency into the drivers of choice. Code available at: htttps://githeii com Asimov -awma/os--CuunnnDeeeeeeeee
Introduction
Choice modeling [MeFadden] 2001] provides a principled framework for capturing and predict- ing human preferences, making central numan-in-the--lle systems. It enables personalized recommendations in online platforms [Resnick and Varian 1997], preference inference in inverse reinforcement learning [Zeng et al.) 2005], and reward alignment in large language models [Rafailov et al. 2023). Across these applications, choice models offer structured way to integrate human decision patterns into machine learning systems.
Traditional choice models often assume that each alternative's utility is independent of the others, implying stable preferences unaffected by context. However, human decisions frequently exhibit the context-dependent effect, also known as the Halo effect TTorndike et al.) [920) in cognitive science, where the composition of the choice influences preferences. For example, dominated options can increase the appeal of dominating ones (decoy effect [Huber et al.) 1982). extreme options can shift preference toward intermediates (compromise effect [Simonson and Tversky 1992). ). and similar alternatives can cannibalize each other's appeal (similarity effect [Tversky and Simonson 1993)) )
*Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Such effects violate core assumptions of random utility models and call for more flexible approaches that capture interactions among alternatives.
While there have heen choice models aiming to capture context effects, they often face structural limi- tations. Many rely on one-hot encodings and ignore rich feature information, and restrict interactions
to first-order (pairwise) terms [Maragheh et al. [0188] Seshadri et al. [019 Yousefi Maraghch et al. 2020b Seshadri al. 2020, Ko and Li 2024]. On the other hand, Neural models offer flexibility and can incorporate features Wong and Faroog [021 Wang et al. 2023). but their complex architectures hinder interpretability by entangling lower- and higher-order effects. As a result, existing methods often face a trade-off off between expressiveness and transparency in modeling context-sensitive choice.
To bridge this gap, propose modeling framework that is both expressive enough to capture complex higher-order interactions among alternatives with features, and structured enough to system- atically disentangle and interpret these effects. We begin by decomposing utility into interpretable, feature-based components- base utility, pairwise interactions, and higher-order effects- charac- terize the associated permutation equivariance properties. This decomposition informs the design of a neural architecttre with inductive biases aligned with the combinatorial structure of context effects. When specialized to the classic featureless setting (i.e., alternatives without access to their features), our model serves as a universal approximator of context-dependent choice functions. Additionally, we introduce principled method for identifying interaction effects by order. We evaluate the model on hypothetical, synthetic, and real-worle datasets wwith and without alternative features- and show that it matches or exceeds the predictive performance of state-of oftee-of-the-rr baselines while providing greater interpretability and insight into the underlying drivers of choice behavior.
1.1 Literature Review
A substantial body of early work in marketing and cognitive science has examined context-dependent
effects [Simonson] 1989 Tversky and Simonson 1993 Simonson and Tversky 1992 Tversky and
Kahneman 1991 Huber et al. 1982]. Rieskamp et al. [2006] provides a comprehensive review of empirical findings that challenge the assumptions of rational choice theory. The universal logit model-also known as the mother logit-was introduced by McFadden et al. [1977] and shown to approximate any discrete choice probability function [McFadden] [984] by allowing for context- dependent utilities. [Batsell and Polking [995a]] further specified this framework as a high-order interaction utility model. Besides, various context-dependent models have been proposed. These include models with menu-dependent consideration sets [Brady and Rehbeck 2016). stochastic preference structures [Berbeglia and Venkataraman 2018), , pairwise utility formulations [Ragain and U Ugander] 2016). contextual ranking models [Seshadri et al., [0200] [Bower and Balzano [2220] Makhijani and Ugander) 2019), and welfare-based frameworks that capture substitutability and complementarity [Feng et al. 2018))
This work takes [Batsell and Polking 1985a as a conceptual starting point. While theoretically expressive, this model presents significant estimation challenges, as the number of interaction terms can grow exponentially with the size of the choice set. To address tractability, much of the existing literature focuses on first-order context effects, which offer a compromise between interpretability and computational efficiency. These models are typically applied in featureless settings with small item universes, as their complexity still scales quadratically with the number of alternatives. The first-order interaction model also known as the context logit or Halo model has been explored in Maragheh et al. [2018). [eshadri et al. [2019]. and Yousefi Maragheh et al. (2020b). To capture low-rank structures, Li et al. [222] and Ko and Li [2024] incorporate self-attention mechanisms into the Halo model. Separately, column generation approaches have been proposed for estimating generalized stochastic preference models [Berbeglia and Venkataraman 2018), as in et al. (2022). Tomlinson and Benson 202 extends the Halo model to incorporate fcatures, but its reliance linear context structure limits its capacity to capture richer interactions in high-dimensional settings.
To address scalability and capture richer nonlinear context effects, recent work has explored deep learning architectures. Deep Sets [Zaheer et al. [2018). which ensure permutation invariance, have been adapted to choice modeling through set-dependent aggregation mechanisms [Rosenfeld et al.) 2020). though primarily in featureless settings. [ffannschmidt et al.] [2022] propose two neural architectures that incorporate features, one limited to first-order effects and the other entangling all higher-order interactions, making interpretation difficult. Residual net-based [Wong and Farooq
2021] and transformer-basee Wang et al. 2023 Peng et al. 2024 Yang et al. 2025 models integrate
2

---

residual connection and/or attention mechanisms, improving predictive performance but often at the expense of interpretability, given that all-order interactions are entangled. To address this, we omit softmax normalization and further observe that the dot-product attention mechanism is not essential for modeling decomposable utility functions. This insight motivates a simplification of the architecture. While our model draws conceptual inspiration from Transformer-style aggregation, it diverges in architectural design to retain deep architectures' expressiveness while enabling explicit control over interaction order, yielding transparent and interpretable framework for context-dependent choice.
Several related approaches address choice modeling from different angles. Context-indepente models such as TasteNet [Han et al., 2020], RUMnet [Aouad and Désir 2023], and NN-Mixed Logit [Wang et 2024) assume stable alternative utilities across choice sets. Graph-base besph-bseeddd models Tomlinson and Benson [0224] Zhang et al.] [024]] capture alternative interactions using Graph Neural
Network. Context-dependent models have also been explored in tasks like choice-set optimization
[Tomlinson and Benson 2020)
Preliminaries
2.1 Choice Model
Let S denote finite universe of alternatives. A choice set is any non-empty subset S E S, and choice model specifies a probability distribution P, (S) over the elements of S. That is, for any j E S, (S) denotes the probability that alternative j is chosen from the set S.
general and expressive framework for modeling such probabilities is McFadden's universal logit model [McFadden et al., 1977. McFadden 1984]. For each j E S, define a utility function (S),. which represents the utility of alternative j when the offered set is S. Given choice set S, the probability of choosing alternative j E S is defined as
exp (U)(
PS(S) = ΣKES exp (S))
This formulation is called the universal logit model because it can approximate any choice function by defining the utility function (S) as the logarithm of the probability of choosing j from the set S. It generalizes the Multinomial Logit model, which assumes that uy uy(S) = Uý is constant and independent the choice set.
2.2 Context Effects
In general, the utility of an alternative S can depend not only its own characteristics but also on the specific composition of the choice set C S which appears. That is, the presence or absence of other alternatives in S may influence the perceived attractiveness of This phenomenon, known as the Halo effect or context effect [Terssy and Simonson, 1993], represents a departure from classical assumptions in discrete choice modeling, such as the Independence of Irrelevant Alternatives, which posit that utility is context- tnntexttiddepedden..
In the literature, context effect is typically discussed in the featureless setting, where each alternative has no features and represented by a one-hot vector. To formally characterize context effects, we consider the following ttluusinn-xellsionn Seshadri et al. 2019]:
decomposition of the utility function uy (S) [Btsell
and Polking 1985a
uy US(S) (2( + Σ my((ir))
(2(( +... (s (i)), S
Σ
(1)
SIESMA
{i} where the set function vy(-) represents the marginal contribution of a subset T' S S Si to the utility of alternative j. Here T' is a subset that exclude the item itself and thus emphasizes the context information. In this decomposition, v (ø) denotes the intrinsic context-independeer utility of j: 2y((1)) }) captures first-order (pairwise) effect, such as how the presence of a single other alternative influences the utility of j; (f(i..j22) captures second-order effects, reflecting how pairs of alternatives jointly influence the utility of j: and so on up to e \ (j)), which encodes the full high-order effect of all remaining alternatives.
The decomposition makes possible to disentangle the utility impact of different interaction levels, allowing to model subtle behavioral phenomena that cannot be captured by additive or
3

---

context- context-indepents independent utility specifications. If all first-order and above terms vanish (i.e., (T) = 0 for all [T]] 2 1), the model reduces context-inde taneext-indeeendent utility framework. Many well-documented behavioral choice anomalies can be understood as manifestations of non-zero context effects. We follow the framework of Yousefi Maragheh et 2020b) to reinterpret common effects:
Decoy effect: The presence of clearly dominated alternative k increases the utility of a dominat- ing option j. reflected in uy((k)) > 0.
Similarity effect: The utility of alternative j decreases due to the presence of a similar option k indicated by negative first-order term uy((k) O.
Compromise effect: The presence extreme options and enhances the attractiveness of an
intermediate option j. modeled by a positive second-order interaction Ui ()((k, ( > 0.
3 Modeling the Context Effects in the Presence of Features
We now formalize modeling framework of context effects in the settings where each alternative is associated with feature vector. Let S be the universe of all alternatives, and suppose that each item assume the size of any choice set S
j E S is described by feature vector 2j E Rd Throughout, is upper bounded by J. To maintain a consistent domain for the utility function u, (-), we assume the choice S is always padded (if necessary) Equivalently. when S|| < J, one may define U -oo for j = |S| + 1,............... 1,...,JJ J. implying that the choice probability is zero for these alternatives. The value of the utility function is taken from the extended real line R- = IR U }. In actual implementation, we apply binary mask p E {0, 1) ensure numerical stability. Our goal is to model the utility of each alternative j E S as a function of its feature vectors, in way that accounts for context-dependent effects among alternatives.
fixed size J using placeholder or null alternatives.
3.1 Utility Decomposition and Permutation Equivariance
Observe that the summation (T) over unordered subsets S, thereby the utility function (y(S) invariant permutations of the other alternatives in S. Extending to the feature-based setting, this means the utility assigned to alternative j depends only on the feature vectors S \ {i} and not the order in which the alternatives appear. On the other hand, to facilitate neural network parameterization that will be discussed in the next subsection, we treat the choice set S as an ordered tuple of indices (1...., (S), and represent the corresponding input feature matrix
Xs= = [-1,-- x S), Os-1s]] E TRdx.
where O, is d - zero matrix to pad the feature matrix to fixed size d X J. This ordering enables the use standard neural net architectures that operate over fixed-format inputs. However, since the true choice behavior is inherently invariant the ordering of alternatives, is critical that the model respects this symmetry. This motivates the explicit incorporation of such invariance into the model design.
To formalize this idea, with slight abuse of notation, we denote the utility function as u, (Xx) to emphasize that it is function of the feature matrix Xs. which is ordered by the indices of the alternatives S. We require that the utility function uy (Xx) satisfies the property of permutation equivariance with respect to the choice set S. Recall a function f _dxJ R. is permutation equivariant [Zaheer et al. 2018 if
(I( ... ( fn())
vj=1....,.
for any permutation of the indices 1....,J. to (Ia(I),..., (*(())) the output of the function is permuted accordingly. We have the following result.
J. This means that if the input feature vectors are permuted
from
Proposition 1. Every utility function RdexJ decomposed as
RJ that is permutation equivariant can be
(Xss)
Σ vy(XTu()).
TCS(())
where vi function over subsets of feature vectors that includes Iy and is itself permutation equivariant in its arguments.

---

Here, XTuO) E Rd, denotes the matrix formed from the feature matrix Xs by replacing the feature vectors not in the subset T U {i} with zero. Proposition I shows that any permutation- equivariant utility function admits a representation of subsets of feature vectors. This formulation mirrors the featureless decomposition in ① but with utility now defined functions of feature vectors. In particular, if each alternative is represented by one-hot vector, then (-) becomes table over discrete subsets and (2 reduces to the original context-effect decomposition ( without features. Intuitively, the function y (XTu(j) captures the contribution to utility from the interaction between 2j and each subset T of the remaining alternatives. In our constructive proof, we define f/ as
sum over permutation--quivaaian functions
uy (y(XTu)))
(( (X(n()))
RCT
where the factor (-1)171-18) refers to the inclusion-exclusion exclusion inversion coefficient the Boolean lattice of subsets. This provides an explicit decomposition form for the Tirs--Fyalaatee-eee--grrre Then-Aggregate (FETA) approach in |Pfannschmidt et al. 2022]. which only focuses on the first-order effect due scalability limitations in modeling higher-order interactions; whereas their First-Aggregate-T11e Then- Evaluate (FATE) architecture entangles all higher-order effects. contrast, our explicit decomposition allows us to construct neural architectures that capture higher-order effects and systematically identify them, as detailed in the sections that follow.
3.2 Neural Net Parameterization
Based on the discussion above, we can now construct a neural network architecture, termed as DeepHalo, to parameterize the utility function uy (Xxs in a way that captures context effects of varying orders. Using (2) we rewrite the utility function as
181-1 Σ D= ICSUGHTE
My uy(Xs)
Σ
(y(X))0)).
The inner sum is over all subsets T of size p from the S {i}, capturing the contribution of p-th order interactions. The outer sum is over all possible orders of interactions, from 0 to |S| We now describe neural architecture that parameterizes uy (X as a sum of interaction terms of increasing order.
Pairwise Interactions
We start with the first-order interaction:
u/ (Xs)
Σ (y((Ton)))
Σ (y(()))))).
TESVGLITHI
RESVE)
One way to model the pairwise terms (1y(X ((()))))))ssss is use multi-layer perceptron (MLP) that maps the concatenation [يy; my] ] to a vector in (s iPfannschmidt et al. 2022]. While expressive, this approach can be computationally expensive and does not scale well to higher-order interactions. To address this, we propose more efficient form.
Let 20 := [x(T,)) where X de + dd is a shared nonlinear embedding function applied to each alternative. The use of shared transformation ensures permutation equivariance. We deñin context summary vector by linearly aggregating the embedded representations of all alternatives:
ž z²:=
=
1
Σ RES
W
S
where W11 E H is shared linear projection across alternatives. Here, H denotes the number of interaction heads, analogous to the number of channels or attention heads, which can be interpreted as H different consumer types with different tastes. Thereby, H controls the diversity of the interaction patterns. Let o) Rdd + Rd, h = I,..., H, be nonlinear transformations applied to each embedded alternative. For each j e S. define
1 H
2) = 2
Σ 2h oh()))
H 1=1
5

---

where the aggregated context 2,, is modulated by nonlinear transformation of the base embedding =0. This operation introduces first-order interactions, allowing the representation of alternative j to 2j.
depend on the presence of other alternatives in the set. The resulting utility u " (Xs) = =] can be interpreted the utility of alternative j when considering the first-order interactions with all other alternatives in the choice set.
Higher-order Interactions We now extend the above structure to capture higher-order interactions. For each / = .,..., TA, we recursively define
z =
W
- J-1 k
(4)
S
RES
2/ : 2j
Σ
Z/ (2))) h=1
(5)
where W E #xxd and of, R + Rd are head-specific nonlinear transformations applied to the base embedding. This recursive formulation incrementally increases the order of interactions. The context summary Z' aggregates global information from the previous layer, while the nonlinear transformations of the base embedding modulate how this summary influences each alternative. As result, each layer introduces an additional higher-order interaction, while the residual connection preserves the lower-order effects accumulated from previous layers. Finally, we set
" (Xs) = 3 of-
Permutation equivariance is maintained at each layer through symmetric aggregation in Zaheer et al. 2018). The layered residual structure (5) builds the utility representation step by step, with the 7-th layer capturing interaction effects up to order 1. Compared to prior neural choice models Wong and Farooq (202T] PPfannschmitt et al. [0222] Wang et al. 2023) Peng al. (2024). which entangle all interaction orders through deeply nested nonlinearities, our architecture incrementally and explicitly controls the interaction order each layer. This yields both interpretability and architectural modularity, allowing practitioners to tailor the model's complexity to the degree of the context effects.
4 Discussions
In this section, we present alternative architectures for DeepHalo, establish its universal approximation property in the classic featureless setting, and discuss the identification of context effects; additional details are provided in the supplemental material.
4.1 Residual Connection for Large Choice Sets
The recursive structure defined in uses exactly layers to capture (-th order interactions. When the choice set S contains a large number of alternatives, the maximum interaction order |S] - may
be large, leading very deep architectures. To improve computational efficiency while retaining expressiveness, we can consider a polynomial aggregation in place (4):
Σ W'o(s
(6)
S
EES
where a is an element-wise polynomial activation, and consider a more flexible residual function:
=
t (2 (% (=9)),
H
where $/, is a polynomial function on R x R. For example, setting a to be quadratic and $ to be a linear transformation of the second argument, we get a ResNet-like structure
1
=
W (
(8)
5

---

4.2 Specialization to the Featureless Setting
special case of our framework arises in the featureless setting, where each alternative is identified solely by unique index in the finite universe S = (1...., J). In this case, the feature vector of the j-th alternative, denoted by 15, is one-hot vector of dimension J with a one in the j-th position if j E S and zero elsewhere. Denote by ej the j-th unit vector in R', and by Es Σ jest Ej be the indicator vector of the set S. Identifying es with the set S, the utility decomposition ① can be viewed as degree- (J 1) multivariate polynomial uy(es) Ep=0 up(es), where wp(es) is degree-p polynomial of es. We now show how our recursive architecture naturally expresses such polynomials.
Suppose the base embedding function is the identity map and let the interaction modulator (1f) o if j E S and zero otherwise, where q' == 1dylhs E R HxJ is learnable parameter matrix. Under this setup, the recursion (5) becomes
2) 2 HS -==1
EKES Whi) &
(9)
2 = 2 + HS E EEES W/, =1, ohzej, 1= 22.,..., L,
where WA RX denotes the h-th row of the matrix W RHHJ
Define matrix O ==
HS E (cl..) JW,, RRXJ, which has rank at most H. Let y' = 1== 2) Rd denote the aggregate representation at layer 1. Then, with yd = es. the recursion (9) implies
y
eg), I=
where 2 denotes the Hadamard product, and the utility of alternative is given by uz(S) - v5-
Thus, our model reduces to an L-layer residual network with residual connections between the input layer and each subsequent layer, where each layer applies rank-H linear transformation to masked (context-aware) version of the previous layer's output. By induction, each y' an (1 + 1d-degree polynomial of es. When I. 1, this model reduces to the lower-rank context-dependent random utility model of Seshadri et al. (2019) of the low-rank Halo MNL model [Ko and Li, 2024]. When L= and H J. it recovers the full-rank context-dependen random utility model of [Seshadri ct al. 2019] or the contextual MNL (CMNL) model of Yousefi Maragheh et al. [2020b).
More generally, following Section 4.1 the recursion becomes
if we use an element-wise quadratic activation af-) = ()),
-= -=---++''0g/71 O'o(s'-1)
1= 1=
(10)
Under this formulation, takes at most (J=2( layers to capture all orders of interactions of the J alternatives.
4.3 Estimating the Context Effects
Observe that due to the translation invariance of the softmax function, the context effect coefficients (T), T C S) {f}, in ① are not directly identifiable from the choice probabilities unless additional linear constraints are imposed these coefficients (Seshadri et al. 2019]. Nonetheless, the following quantity, which we call the relative context effect, is identifiable (see also Park and Hahn [1998a]):
+ ark (T) (T) + BT (+)) ( (TU
(11)
where j,k S \ T. It captures the marginal effect of the subset T on the utility difference between alternatives j and k. To estimate (T'), J, it suffices to first evaluate our neural network on subsets R U k) for all R T, and then use the formula 3 to compute the four terms in (11) to obtain the desired quantity.
5 Empirical Study
In this section, we conduct experiments to evaluate the proposed model. Detailed experimental settings are provided in the supplementary material.
7

---

5.1 Featureless Datasets
Hypothetical Data with Halo Effects To demonstrate our model's efficacy capturing and recovering context effects, we hypothetical beverage market share dataset, originally employed by Batsell and Polking (1985a) Park and Hahn [1998a]. Suppose we have four products- Coke, 7-Up, and Sprite-Ti in the universe, denoted shares for all possible choice sets are as listed in Table la and follow the hypothetical behavioral rules: (a) consumers view the products primarily as choice between cola and non-cola; (b) they almost always choose Pepsi over Coke when both are available and cola is desired; and (c) they almost always choose 7-Up over Sprite when both are available and a non-cola desired.
2,3,3, 3, 4). respectively. Assume the market
Choice Set
2
3
4
1,2
(1,2)
0.98
0.02
1,3-
(1,3)
0.50
0.50
(1,4)
0.50
0.50
(2,3)
0.50
0.50
Competing Product
1.4-
(2,4)
0.50
0.50
(3,4)
0.90
0.10
2.3
(1,2,3)
0.49
0.01
0.50
(1,2,4)
0.49
0.01
0.50
2,4
(1,3,4)
0.50
0.45
0.05
(2,3.4)
0.50
0.45
0.05
(1,2,3,4)
0.49
0.01
0.45
0.05
Effect Source Set
(a) Beverage market share
(b) Relative Halo effect
Figure 1: Market-share table (left) and relative Halo effect visualization (right)
For each conceivable choice set, generate 2000 samples, where each choice outcome is drawn according to the market share as the choice probability. We then cuudratic-activaan DeepHalo model with depth L 2. specified in Section 4.2 Subsequently.
recover the identifiable relative context effects following the procedure in Section 4.3 and visualize them as a heatmap in Figure 1b Each colored cell represents the marginal influence of source 'T' on the relative utility between product pair (j, k). The column labeled by the empty set (2) corresponds to the zero-order relative Halo effect between alternatives and k.
This visualization highlights the interpretability of our model. For example, the first column, the red cells for pairs (2,3) and (2, 4) indicate that, the presence of product 1 (Pepsi) lowers the utility of product 2 (Coke) relative to products and (7-Up and Sprite), making Coke less likely to be chosen. This exactly reflects the "Pepsi-over-Coke" preference captured by logic rule (b) above. similar pattern appears the top-right corner the heatmap, where product consistently has higher zero-order utility than product 2, reaffirming that Pepsi is inherently more preferred than Coke, regardless of context.
Synthetic Data with High-order Effects To empirically assess how model depth influences expressiveness under fixed parameter budget, we construct a synthetic dataset with universe of alternatives S = (1..... .......0)) 20) and choice sets of fixed cardinality 15. For each such choice set, we sample a choice probability vector uniformly from the probability simplex on 15 and generate 80 i..dd choice observations. This results in a dataset with 1,240,320 training samples and can implicitly incorporate higher-order interactions up to 14-th order. We report the training root mean squared error (RMSE) of the predicted choice probabilities in Figure
Recall from Section 4.2 that with quadratic activations, an L-layer network can represent interactions up to 2 -th order. In Figure 2a we observe that when the number of parameters is fixed (either 200k or 500k), RMSE decreases significantly as depth increases up to 5, beyond which 25-1 exceeds the choice set size of 15. For deeper models (L > 6). further performance gains are marginal. At fixed depth, models with more parameters perform slightly better, as expected, but increasing width alone does not compensate for insufficient depth. These results confirm that model expressiveness scales

---

200k
500k
200k Dep
0.640
200k
0.65
Dep
200k Dep
Training RMSE
500k Dep
0.030
500k
Dep
500k Dep
0.025
0.35
Depth
Epochs
Depth RMSE
(b) Training curve
Figure 2: Effect of model depth approximation error.
exponentially with depth and that depth the dominant factor for capturing high-order interactions in choice behavior.
Real Data To investigate the necessity of modeling high-order context effects, we evaluate the empirical performance of our featureless model on three real-words datasets: Hotel [Bodea et al. 2009), SFOwork and SFOshop [Seshadri et al.) 2019). The Hotel dataset records bookings from five continental hotels. Following the preprocessing steps in [erbbglia et al. [2022]. we focus on the second hotel, where each assortment includes up to alternatives (including "leave" option). We use 1,845 observations for training and 465 for testing; due to the limited sample size, no separate validation split held out. The SFOwork and SFOshop datasets contain travel mode choices in the San Francisco Bay Area, with the former focused SFOwork provides 5,029 observations with up to 6 alternatives per choice set, and SFOshop provides 3,157 observations with up to 8 alternatives. Both datasets are partitioned into training, validation, evaluated using the negative log-likelihood
work trips and the latter shopping trips.
and test sets in an 8:1:1 ratio. All models are trained (NLL), and results are reported in Table [ alongside baseline methods including Multinomial Logit (MNL), Multilayer Perceptron (MLP), and Contextual MNL (CMNL) (Yousefi Maragheh et al.) 2020b Across all three datasets, DeepHalo achieves the lowest NLL among the compared models. This consistent performance highlights the value of modeling deep Halo effects: MNL and MLP do not account for context effects, and CMNL captures only first-order interactions.
Table 1: Negative likeliiiddodd (NLL) on the Hotel, SFOshop, and SFOwork datasets
Hotel
SFOshop
SFOwork
Model
Train
Test
Train
Test
Train
Test
MNL
0.7743
0.7743
1.7281
1.7262
0.9423
0,9482
MLP
0.7569
0.7523
1,5556
1.5523
0.8074
0.8120
CMNL
0.7566
0.7561
1.5676
1.5686
0.8116
0.8164
DeepHalo (Ours)
0.7479
0.7483
1.5385
1.5263
0.8040
0.8066
5.2 Interpretable Low-Order Modeling vs. Post Hoc Truncation
To assess the effect explicit order control, we compare two models on controlled synthetic dataset and the real SFOshop dataset: (1) DeepHalo(k==2) constrained to second-orde interactions, and (2) MLP+Trunc(k), a parameter-matched MLP whose interaction terms are extracted and truncated to order k at evaluation. Model expressiveness is measured by in-sample negative lig-likelihood (NLL).
DeepHalo achieves consistently lower NLL by directly constraining interaction order, while Mr.P+Trunc matches its performance only when all higher-order terms are included. Increasing k beyond the true interaction order can even worsen fit, as higher-order effects co-adapt with lower- order ones and their removal distorts predictions. On SFOshop. the NLLs of DeepHalo(k==2) and

---

Table 2: In-sample NLL (lower better) for MLP with truncation to order A and DeepHalo constrained to second order.
Dataset
Truncation order
MLP++Tuncck) NLL DeepHalo((==)) NLL
0.7643
1.6429
0,8172
0.6719
Synthetic (true order S 2)
0.0922
0.3280
0.1502
0.0958
0,0910
1.6065
1.6168
1.6834
SFOshop (human data)
1.5339
1.5806
1.5406
1.5340
full-order DeepHalo are nearly identical (1.5339 vs. 1,5331), revealing that in real-world scenarios, human choice behavior can be captured by low-order interactions.
5.3 Datasets with Features
We experiment on two
datasets of different scales: the smaller LPMC transportation dataset [Hillel et al. 2018] with 81,086 observations and the larger Expedia Hotel Choice dataset [Adam et al. 2013] with 275,609 transactions. The LPMC dataset, based the London Travel Demand Survey, captures travel mode choices among walking, cycling, public transport, and driving, with each alternative described by 8 item-specific features and 17 shared features (e.g., cost, travel time, gender, and age). The Expedia dataset contains hotel search and booking records. After preprocessing, each choice set includes up to 38 hotels, with 35 item-specific and 56 shared features (e.g., price, star rating, destination).
Table 3: Negative log likelihood (NLL) on the LPMC and Expedia datasets.
Datuset
Split
MNL
MLP
TasteNet
RUMnet
DLCL
ResLogit
FateNet
TCNet
DeepHalo (Ours)
Train
0.8816
0,7062
0.7240
0,7041
0.7137
0,7032
0.6936
0.6645
0.6444
LPMC
Test
0.8643
0.6857
0.7001
0.6830
0.6977
0.6833
0.6774
0.6487
0.6430
Train
2.6272
2.5637
2.5529
2.5440
2.5606
2.5462
2.5419
2.4865
2.5059
Expedia
Test
2.6244
2.5758
2.5776
2.5776
2.5601
2.5715
2.5515
2.5340
2.5263
We compare model against range of representative feature-based baselines, including classic models such as MNL and MLP, and recent neural approaches like TasteNet (Han et al., 2020). RUM- net[Aouad and Désir 2023). ). ResLogit [Wong and Farooq [222]] DLCL [oomlinson and Benson 2021]. FateNet (Pfannschmidt et al., [022], and TCNet [Wang et al., 2023]. Expressive models that account for context effects, including DeepHalo, FateNet, and TCNet, consistently outperform context-independent approaches (MNL, MLP, TasteNet, and RUMnet) and the remaining less expres- sive depennttde ones on both datasets, highlighting the importance modeling contextual interactions with sufficient flexibility in choice behavior. Among them, DecpHalo generalizes the best consistently while offering favorably improved interpretability compared other deep neural network models.
Conclusion
We proposed neural framework for context-dependent choice modeling that enables controlled modeling of interaction effects by order. While we focused on one specific parameterization of higher-order effects, the broader framework admits many alternatives. Exploring such variants offers a promising direction for developing interpretable high-capacity models with downstream applications in building AI systems that reason about and adapt to human behavior.
10

---

Acknowledgments and Disclosure of Funding
This work in part supported by the Key Program of the NSFC under grant No. 72495131, NSFC under grant No. 62206236, Shenzhen Stability Science Program 2023, Shenzhen Science and Technology Program DDSSS$023062001100000000 and Longgang District Key Laboratory of Intelligent Digital Economy Security.
References
Adam, Ben Hamner, Dan Friedman, and SSA Expedia. Personalize expedia hotel searches iedm 2013. htttp://kagglu. com////mttte personalized -sor 2013, Kaggle.
Ali Aouad and Antoine Désir. Representing Random Utility Choice Models with Neural Networks, July 2023. httptttp//aaawv.orr///aaaa
12877 arXiv:2207.18777 [es, math, stat].
Ali Aouad, Vivek Farias, and Retsef Levi. Assortment Optimization Under Consider- Chen-Chhose Choice Models. Management Science, 67(6):3368-3336 June 2021. Publisher: INFORMS.
Richard R Batsell and John C Polking. new class of market share models. Marketing Science, 4(3): 177-198, 1985a.
Richard R. Batsell and John C. Polking. A New Class of Market Share Models. Marketing Science, 4(3):177-198, 1985b. ISSN 0732-2399. URL Publisher: INFORMS.
Gerardo Berbeglia and Ashwin Venkataraman. The generalized stochastic preference choice model. arXiv preprini arXiv: 1803.04244, 2018.
Gerardo Berbeglia, Agustín Garassino, and Gustavo Vulcano. Comparative Empirical Study of Discrete Choice Models in Retail Operations. Management Science, 686)):4005-4003,, June 2022. ISSN 0025- 1909, 1526-5501. doi: 2887 URL /ttp/ inf orms. org /o1//00111111////// 1287 /wsc//nccccc/////////////////m 0021.4066
Tudor Bodea, Mark Ferguson, and Laurie Garrow. Data cet-choice-barte revenue management: Data from a major hotel chain. Manufacturing & Service Operations Management, 11(2):356-31,,
2009.
Amanda Bower and Laura Balzano. Preference modeling with context- salient features. In International Conference on Machine Learning, pages 1067-1077. PMLR, 2020.
Richard L Brady and John Rehbeck. Menu-dependent stochastic feasibility. Econometrica, 84(3): 1203-1223, 2016.
Guiyun Feng. Xiaobo Li, and Zizhuo Wang. On substitutability and complementarity in discrete choice models. Operations Research Letters, 46(1):141-146, 2018
Yafei Han, Francisco Camara Pereira, Moshe Ben-Akiva, and Christopher Zegras. A neural-embedded choice model: Tastenet-mul modeling taste heterogeneity with flexibility and interpretability. arXiv preprint arXiv:2002.00922 00922, 2020.
Tim Hillel, Mohammed ZEB Elshafic, and Ying Jin. Recreating passenger mode choice-sets for transport simulation: A case study of london, uk. Proceedings of the Institution of Civil Engineers- Smart Infrastructure and Construction, 171(1):29-44 42, 2018
Joel Huber, John W. Payne, and Christopher Puto. Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis. Journal of Consumer Research, 9(1):90 06 1982. ISSN 0093-5301. doi: 10. (0.1086/208899 URL / 1tt.
Sanjay Dominik Jena, Andrea Lodi, and Claudio Sole. On the estimation discrete choice models to capture irrational customer behaviors. INFORMS Journal Computing, 1466)):666-1125, 2022.
Joohwan Ko and Andrew A. Li. Modeling Choice via Self-Attention, February 2024. URL U // //axiiv.. org/ abs/ 22211/601107 arXiv:2311.07600 [cs].

---

Frank Koppelman and Chandra Bhat. A self instructing course in mode choice modelling: Multino- mial and nested logit models. 06 2006.
Mofei Li, Yutaka Nakamura, and Hiroshi Ishiguro. Choice modeling using dot-product attention mechanism. Artificial Life and Robotics, 26:116-11 16-12,, 2021.
Rahul Makhijani and Johan Ugander. Parametric models for intransitivity in pairwise rankings. In The World Wide Web Conference, pages 3056-3062, 2019.
Reza Yousefi Maragheh, Alexandra Chronopoulou, and James Mario Davis. A customer choice model with halo effect. arXiv preprint arXiv:1805.01660 2018
Daniel McFadden. Economic Choices. The American Economic Review, 91(3)331 2001. ISSN 0002-8282. URL. https::///// http:://wwww.sttt/ org/ http//aabee226776666 Publisher: American Economic Association.
Daniel McFadden, William Tye, and Kenneth Train. An application of diagnostic tests for the independence from irrelevant alternatives property of the multinomial logit model. Institute of Transportation Studies, University of California Berkeley, 1977.
Daniel L. McFadden. Chapter 24 econometric analysis of qualitative response models. vol- ume of Hanndbook of Econometrics, pages 1395-1457. Elsevier, 1984. doi: https://oii. org/ ar:icieepp1//11177541188000000
URL htt.h:tp//// /ww. http:://wwwwscccncedeteoo sciencedirect.......m.....e wwt.:occccccooooooe com/science/
Sang-June Park and Minhi Hahn. Direct Estimation of Batsell and Polking's ' Model. Marketing Science, (7(2):100-173 1998a. doi: 10. 1tt.//mksc..1....11.. URL https://ideas.. repec
htt////inmm/ommeee http://iimmmmmkkkk////1/111222/
178. h7m.
Sang-June Park and Minhi Hahn. Direct Estimation of Batsell and Polking's Model. Marketing
Science, 17(2):170-178, 1998b. ISSN 0732-2399. URL BRRLhhtts://wwwt www.w.sstrroooo ptt.sss..oooog//tbllll.....
193165 Publisher: INFORMS.
Zhenkang Peng, Ying Rong, and Tianning Zhu. Transformer-based choice model: A tool for
assortment optimization evaluation. Naval Research Logistics (NRL), 7166):85-877, 2024. ISSN 1520-6750. doi: 10. 1002/nav.22183. URL htttps://oomiiiellibaaa wwwprrywwwwwl.o.o.o....mco. how.l//y..co//o//abaaaaaaaaaa / 10. 10.2/aa//av... 22183
eprint: http://oooelllllo..wwoooo/oooood
Karlson Pfannschmidt, Pritha Gupta, Björ Haddenhorst, and Eyke Hüllermeier. Learning context- dependent choice functions. International Journal of Approximate Reasoning, 140:116-155: 2022.
Rafacl Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, 36:53728-53777 2023.
Stephen Ragain and Johan Ugander. Pairwise choice markov chains. Advances in neural information processing systems, 29, 2016.
Paul Resnick and Hal R Varian. Recommender systems. Communications of the ACM, 40(3):56-58, 1997.
Jörg Rieskamp, Jerome R Busemeyer, and Barbara A Mellers. Extending the bounds of rationality: Evidence and theories of preferential choice. Journal of Economic Literature, 44(3):631-66 2006.
Nir Rosenfeld, Kojin Oshiba, and Yaron Singer. Predicting choice with set-dependent aggregation. In International Conference on Machine Learning, pages 8220-8229. PMLR, 2020.
Arjun Seshadri, Alex Peysakhovich, and Johan Ugander. Discovering context effects from raw choice data. In International conference on machine learning, pages 6669-5669. PMLR, 2019.
Arjun Seshadri, Stephen Ragain, and Johan Ugander. Learning rich rankings. Advances Neural Information Processing Systems, 33:9435-9446 2020.
12

---

Itamar Simonson. Choice based on reasons: The case of attraction and compromise effects. Journal of Consumer Research, 16(2):158 174, 09 1989. ISSN 0093-5301. doi: 10. 10.1006/209205 URL https://doi. org/10. 10. 10086/00200
Itamar Simonson and Amos Tversky. Choice in context: Tradeoff contrast and extremeness aversion.
Journal of marketing research, 29(3):281-295 1992.
Edward L Thorndike et al. constant error in psychological ratings. Journal applied psychology, 4(1):25-29, 1920.
Kiran Tomlinson and Austin Benson. Choice optimization under discrete choice models of group decisions, In International Conference on Machine Learning, pages 9514-9525, PMLR, 2020.
Kiran Tomlinson and Austin R Benson. Learning interpretable feature context effects in discrete choice. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery data mining, pages 1592-1592, 2021.
Kiran Tomlinson and Austin R Benson. Graph-based methods for discrete choice. Network Science, 12(1))21-40, 2024.
Amos Tversky and Daniel Kahneman, Loss aversion in riskless choice: A seference-dependeen model. The Quarterly Journal of Economics, 103(4)1009--161, 1991. ISSN 00335533, 15314650. URL http:/ /twp:///www... jstor orgooog//ao. httg//tabeee/99799//
Amos Tversky and Itamar Simonson. Context-dependent preferences. Management science, 39(10):
1179-11899 1189, 1993.
Hanzhao Wang, Xiaocheng Li, and Kalyan Talluri. Transformer Choice Net: A Transformer Neural Network for Choice Prediction, October 2023. URL http::/tttt/iii http://aaxvv..rrga/////1333 2010 08716 arXiv:2310.08716 [es].
Zhi Wang, Rui Gao, and Shuang Li. Neural-network mixed logit choice model: Statistical and optimality guarantees. Available at SSRN 5118033, 2024.
Melvin Wong and Bilal Farooq. Reslogit: residual neural network logit model for data-driven choice modelling. Transportation Research Part C: Emerging Technologies, 126:103050) 2021.
Yiqi Yang, Zhi Wang, Rui Gao, and Shuang Li. Reproducing kernel hilbert space choice model. Available SSRN 5267975, 2025.
Reza Yousef Maragheh, Xin Chen, James Davis, Jason Cho, and Sushant Kumar. Choice Mod- eling and Assortment Optimization in the Presence of Context Effects. SSRN Electronic Jour- nal, 2020a. ISSN 1556-5068.
doi: 2221399ssmm.7473354 URL https://wws http///wwws..nnnn/////////// sern, com/
abstract=3747354
Reza Yousefi Maragheh, Xin Chen, James Davis, Jason Cho, Sushant Kumar, and Kannan Achan. Choice modeling and assortment optimization the presence of context effects. Available at SSRN 3747354, 2020b.
Manzil Zaheer. Satwik Kottur, Siamak Ravanbakhsh, Baraabas Poczos, Ruslan Salakhutdinov, and Alexander Smola. Deep Sets, April 2018. URL arXiv:1703.061 [cs, stat].
06114
Siliang Zeng, Mingyi Hong, and Alfredo Garcia. Structural estimation markov decision processes in high-dimensional state space with finite-time guarantees. Operations Research, 73(2):720-737, 2025.
Liang Zhang, Guannan Liu, Junjie Wu, and Yong Tan. Modeling reference-dependdet choices with graph neural networks. arXiv preprint arXiv:2408. 13302, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction accurately reflect the paper's contributions. It clearly states the model's key advantages- expressiieeeessseeeeeess.se interpretability. and empirical cerformance-wichh are all substantiated in the main text.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes, the paper discusses its limitations in the conclusion, noting that the proposed parameterization of higher-order interactions is not unique, and that other model exploration.
variants may also satisfy the same design maneiplee--hghlighhinn opportunities for future
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Yes, all theoretical results in the paper are either fully proved correctly cited from prior work. Assumptions are clearly stated, and the main results are rigorously supported.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Thcorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Yes, the paper provides sufficient details to reproduce the main experimental results relevant its core claims, including dataset descriptions, model architectures, training objectives, and evaluation metrics. Further implementation details are clearly deferred to the appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
15

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The datasets used in the paper are all publicly available. Our model imple- mentation will be open-sourced upon publication, with accompanying instructions to ensure faithful reproduction of the main results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7/ //iips.cc/ppll wttpccpppppppppeeeeeeeeeeeeeeeee
http:///ggu.eee//oeSStbiiiissss sonoooooocss for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Yes, the paper specifies key training and test details, including data splits, model architecture, loss functions, and optimizers. Hyperparameter choices and additional training settings are documented in the appendix for clarity and completeness.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
16

---

Yes, detailed experimental results, including error metrics and significance- related information, are provided in the appendix support transparency and reproducibility. Guidelines:
Justification:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, the information on computational resources-sude as hardware specifica- tions, memory, and training time-is provided in the appendix support reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: respects.
Yes, the research fully conforms with the NeurlPS Code of Ethics in all
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
17

---

Question:
Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes, the paper discusses potential societal impacts by empirically validating the presence of halo effects in human decision-making, which can inform the design of more transparent and fair decision-support systems.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g.. pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: N/A - the paper does not involve high-risk models or data such as pretrained language models, image generators, or scraped datasets, so no special safeguards are necessary.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
18

---

Yes, all models and datasets used in the paper are properly cited, with licenses and terms of use duly respected.
Justification:
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperssii thoode com/ com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- e-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: N/A - the paper does not introduce new assets that require separate documen- tation.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: N/A - the paper does not involve crowdsourcing experiments research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
19

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: N/A the paper does not involve human subjects, no risks, disclosures, IRB approvals are applicable.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
The core methods in this research do not utilize large language models (LLMs); any of LLMS was limited writing and editing, which does affect the methodology or originality.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non standaad components.
Please refer to LLM http:p////eerreeecccc/o/ooerreee for what should or should not be described.
20

---

Appendix
Appendix Contents
A Proofs and Derivations
A.1 Proof of Proposition
A.2 Derivation in Section 4.2
A.2.1 Recursion for y'
A.2.2 Universal Approximation
A.2.3 Quadratic Activation
A.3 Derivation in Section 4.3
B Implementation Details
25
B.1 Featureless Models
B.2 Feature-based Models
C Empirical Study Details
26
C.1 Featureless Experiments
26
C.I.1 Synthetic Data Experiments
C.1.2 Real Dataset Experiments
C.2 Feature-based Experiments
27
C.2.1 Datasets Details
C.2.2 Experiments Details
21

---

A Proofs and Derivations
A.1 Proof of Proposition
!
Proof. Consider the following form of u:
U (XTut))) = -- (X(n))))).
(12)
RCT
Thanks the permutation equivariance uj. the function by given by (3) is permutation equivariant. Indeed, for every permutation on T {5}. it holds that
Σ (-1) QCx(T)
u (XQui())
= =Σ (-1) (m) (X
RCT
where the second equality follows from the change of variable R = (Q)- Next, let us verify that the function v given in (3) can express the utility function (x(s)). Indeed, for every S and j = 1, I...., (S], using (3) we write LTCMV u(I, Xr) as
Σ
(eluu(RRRRR (XRutt))
TCS RCT RCT
By switching the order of summation, this equals
Σ
u(Xnor))
E (-1)
RCS
TORTCRN
When R = S {i}. the inner sum is 1. Otherwise, the inner equals
[8| 11-1-RR
[
Σ
(-1)*
01 0.
k
k=
This shows that
Σ
v(z,, = (X())).
TCSMU
which completes the proof.
A.2 Derivation in Section 4.2
A.2.1 Recursion for y'
Let us first write of = E 21 with yd = es where 2 is defined in 9 Since y/ = es
Σ/= Eg = Σi=1 2g. it implics x(x)= = æ being an identity mapping such that 29 = ej. Recall
H
0¹ HS h=1
Σ
Twh.
1
Σ
Ing W
nk.
HS h=1
We write
H
23 =e ej +
W1
engej ej + Σ O3eez.
1
HS
1=1
KES
kES
It follows that
y
O
Σ
E
Ofeeze (esees = +0 Oes).
j=1
kes
KES
1=1
j=1
kES
22

---

For 1 = 2,...,L L.
H
y'
Σ
Σ
Wh h. *
ghges
HS
j=1
1=1
KES
W Wh t Bjj
HS
b=1k=
ΣΣ 1=
0 StatEtee
=y² O'(y 3 es)
as desired.
A.2.2 Universal approximation
We consider a flexible model by setting y' E Rd with J' > J. Let O² 'RJ ER' for 1 = ,, and W E J'X""
yy = O'es,
yf = yf + O'(''to 1 2
We show inductively that there exists W J such that W'y' can represent any up I-th order interactions.
We first consider the base case 1 = 1. By definition of y².
J
Σ
T(k 5).
k=1
For j E S, we can further write
ッソ TO)
Σ
Tl E S)
k=1dofj
-0)(0) S)
Σ ()(()))((K e S),
k=ikkfj
where 0)((0) =: }) (]())) = 0/ For j > J, it can be set to any function value parameterized
by e), provided that S is not empty, which is independent of other 5/ - with j < J. Without loss of generality, we can simply set W = [T E J' to output w'y = {v) hiss, which represent any first-order interaction model.
Now suppose Wi-1y/ can represent any interaction model up to (1 1)-th order with Wi-1- [];;0] e J'x"" which means for any j = 1.....J.
Σ
(T)
II like S).
TS
KET
and for any j > J. 1/ 1-1
can be set to any function value independent of other - with j s J.
For j = I,...,J', J'
J
1/1 1 +
I E S)
jj
==1
k=
y'
-
1-1 +
Σ
(T)
[[k E S) Σ
J ES)
Jf
==1
TC( (E........ M----11
KET
#==1
-
Σ
Σ
0 θ Jf (T)
II KETu(K))
I(W E E).
[TIg(L F.EE (e 1.TUU)==
23

---

where B/j is another function that can capture any to I - 1-th order interaction by combining
original W with the terms the summation where E T. Since 6 j is independent of o/ (T), we can identify (T U {k}) = Σ @ jyy jr ( (T) for any TU {k} = I, which use basis
to approximate any I-th order interaction. Thus
Jラ 1=
Σ PC
l(kk E S). (T) II RET
With W [[=; RJxJ' Why Wylles approximate any up to I-th order interactions. By induction, v can capture up to L-th order interaction with sufficiently large J', where W can be set to [T; 0] E JJJ''' without loss generality.
A.2.3 Quadratic activation
Here we explain why the formulation 100 needs at most [1 + loga - 1)1 layers to capture all orders of interactions of the J alternatives.
Starting / 1, given y6 = es.
y =s + Oes Oes) es + O'ess
thus
0 = I(E
Σ
0 I[k E) =: e):=) E) + Σ
((+)))) e S),
km
R= 1:bppt)
which contains up to first-order interaction.
Now consider / = .,....,.. L. Assume y' contains up to p-th order interaction such that
- = of E ES
Σ
of (Lin)iEEE ES) +
Σ
(ti.........
I I E S).
6 11
=1
Then the term
v/ = 1/ + Σ 0 / (v) 222
can contain high-order term for oí + + 1/ + +1.
Σ o ( 1 J,) 1DD II (IG e S)I(i E
which is 2p-th order interaction. By induction, with recursion, the model contains up to 24-1-th order interactions. Therefore, to represent the full order interaction for // products with up to J - 1-th order interaction, we need at most [1 + log, (J 1)] recursions.
A.3 Derivation in Section 4.3
In this section, derive the identifiability property for the parameter defined in (1). Recall denotes the full set of products. Suppose for a fixed product pair {i,}} } C S, the dataset contains observed market shares for all choice sets in the collection
E ((i.j) B S (i,i))
We claim that the system of equations defined by (1) is square and full-rank such that the relative halo effect parameters (T)) uniquely identifiable from the observed data.
24

---

To prove this claim, we need to prove that the number of new variables, denoted as Na is the same as the number of equations associated with them (from the choice probability of each choice set), denoted as Nep, and
Nep Σ 2
x (g 1).
(13)
Note that the summation Σq= (n-4) - (g) is the number of subset of all items whose size smaller than 2. These sets be the source set T of halo effect difference a, since there is least one pair of items out of these sets. Suppose the number of items in the subsets is q. We only need q halo effect differences to calculate all pair-wise differences within q items. Therefore,
No Nep Σ
()-
(14)
q
Furthermore, prior work such as Batsell and Polking [1885 and Park and Hahn (1998b has shown that the corresponding coefficient matrix full-rank, as the system represents saturated model. This guarantees the linear independence of equations, and hence, the parameters a are identifiable.
B
Implementation Details
B.1
Featureless Models
Model Width In our formulation (Equation and Equation 10), we initially constrain the model width to match the size of the item universe J, setting the shape of O' to RJxJ_ This aligns our model with prior work, such as the low-rank Halo MNL model [Ko and Li 2024 and contextual MNL Yousef Maragheh et al., 2020aa].
While a width of J suffices to capture zeroth- and first-order Halo effects, it is inefficient for higher- order interactions. As the number of possible interactions grows exponentially with order, deeper layers must activate and compose increasingly complex patterns from lower-order terms. How- ever, under fixed width, scaling model capacity necessitates increasing depth, which complicates optimization and training stability.
To alleviate this limitation, we expand the width of the first layer to θ E R''' and all subsequent layers to O' E J''xJ' xJ' for 2 2. After the final layer, we project o back to dimension J using a linear transformation L E JxJ", By choosing J' > J, we effectively increase model capacity. We refer J' the model width. In essence, this over-parametrizes lower-order representations to better facilitate the modeling of higher-order effects. We refer the readers to more details about this in Appendix A.2.2
B.2 Feature-based Models
Non-Linear Embedding In our implementation, the non-linear embedding function is defined as three-layer multilayer perceptron (MLP) with ReLU activations after each hidden layer and final layer normalization. The network maps the input x e IR to an embedding space of dimension d:
x(x)= LayerNorm (Wy- ReLU (W₂ ReLU (W1x b1) b2) by).
Rd
(15)
where Wi E [Rdxd, W2, Wa E IRd and b, b2, by E Rd. This embedding function is shared by all items in the choice set.
Head-specific Nonlinear Transformations o% We implement the head-specific non-linear trans- formation of, as a two-layer MLP. The first layer is specific to each head h, while the second (output) layer is shared across all heads. A LayerNorm is applied after the second layer to stabilize training.
H (29) LayerNorm (Wa ReL.U(W,, + bb,) Ushaas):
#I ddd
(16)
where 29 = x(x;) E I w e gaxd w Wh E _dxd and 6/,, bruees E Idd.
25

---

Dummy Items To enable parallel computation and efficient model training, all choice sets are padded to uniform size. We pad dummy items using zero vectors in the input space, i.e., 0 and explicitly enforce their latent representations to remain inactive throughout the network. Specifically, after each residual layer, we reset Z = 0 for any dummy item j. ensuring that dummy items do not accumulate or propagate any signal across layers. Since is defined as an element-wise polynomial activation function, it satisfies (0) O, ensuring that the aggregation process Equation ( unaffected by dummy items. In the final softmax mapping. we set all dummy items' utility value to be -inf. This mechanism guarantees that dummy items do not affect contextual aggregation or final predictions, thereby preserving the integrity of the learned representations.
C Empirical Study Details
C.1 Featureless Experiments
C.I.1 Synthetic Data Experiments
Data Generation In this experiment, we directly sample the choice probability vector uniformly from the 15-dimensional probability simplex R rather than sampling each individual halo effect from distribution-succ as the standard normal-and then computing the corresponding choice probabilities. While the latter approach may seem more intuitive realistic, it would require evaluating the choice probabilities exponentially many times due to the complex structure of the choice sets, making it computationally inefficient. Importantly, any choice data can be represented by an appropriate system of halo effects [Batsell and Polking 1985b]. implying that it is not necessary to explicitly specify the underlying halo effects. This justifies the feasibility and practicality of our direct sampling approach.
Experiments Setup We conduct two sets of experiments under parameter budgets of 200k and 500k, respectively. For each budget, we vary the model depth from 3 to 7. The batch size and learning rate fixed to be 1024 and 1 x 10-4. All models are trained for 500 epochs. The detailed configurations and corresponding training results are summarized below (see Table All experiments, including hypothetical data experiments, are conducted single Google Colab T4 GPU with Adam optimizer.
Table Model configurations with depth, width, parameter count, and in-sample training RMSE.
Group
Depth
Width
Param Count
Training RMSE
3
306
200.736k
0.0434
4
251
200.298k
0.0243
200k
5
218
200.124k
0.0202
6
195
199,290k
0.0188
7
179
200.838k
0.0183
3
489
499.758k
0.0419
4
401
500.448k
0.0156
500k
5
348
500,424k
0,0140
6
312
501.384K
0.0131
7
285
501.030k
0.0130
In these experiments, we use the mean squared error (MSE) between the predicted choice probabilities and the ground-truth one-hot choice vectors as the training objective. This allows to use training RMSE as a proxy for evaluating the model's 's approximation error. Note that the training RMSE reported in Table 4 and Figure is not computed directly from the loss. Instead, for each choice set, we first compute the empirical choice frequency vector and then measure the RMSE between this vector and the model's predicted choice probabilities. A perfect fit would result in an RMSE zero This evaluation method highlights the expressiveness of the model, as it directly reflects how well the predicted distributions capture the underlying variability in observed choices, beyond merely fitting to individual samples.
26

---

C.1.2 Real Dataset Experiments
From our experimental results, the contextual effects are more pronounced in the second hotel, suggesting that user choices in this setting are more strongly influenced by the composition of the choice set. result, we select the second hotel as case study to compare our method against baseline models, in order to better evaluate their ability to capture such contextual dependencies.
Hotel
SFOwork and SFOshop The SFOwork dataset includes six transportation modes: driving alone, shared ride (2), shared ride (3+), bike, transit, and walk. The SFOshop dataset comprises eight transportation modes: SharedRide (2+) and DriveAlone, SharedRide (2/3+), DriveAlone, SharedRide (3+), walk, transit, SharedRide (2), and bike. These datasets originate from Koppelman and Bhat (2066), and we use the preprocessed versions provided by /Seshadri et al. [2019]
Table 5: DeepHalo hyper- papeetpprameters and training settings for experiments on Hotel, SFOshop, and SFOwork datasets.
Hyper-parameter
Hotel
SFOshop
SFOwork
Model Width J'
20
20
Layers
Polynomial Activation o Batch size
Quadratic
Quadratic 256
Quadratic
Full Batch
256
Learning Rate
1 10-3
1 10
1 x 10-4
Experiments Setup For the HOTEL dataset, due to its small size and the absence of a dedicated validation split, we train the model using full-batch setting for 300 epochs without early stopping. To mitigate overfitting, we adopt a compact model configuration with reduced width and fewer parameters. In contrast, for experiments on the SFO datasets (SFOsHOP and SFOWORK), we tune the number of layers L E (4,5) and the intermediate width J' E {10, 20} based on validation performance. Early stopping is applied with patience of 10 epochs to prevent overfitting. All experiments are conducted on single Google Colab T4 GPU with Adam optimizer.
Reproducibility Remark We observe that across all three datasets, the experimental results are highly stable: the standard deviations of both NLL and accuracy over runs consistently close to zero. Morcover, top-1 accuracy, while intuitive, shows limited sensitivity in reflecting model performance differences due to the relative simplicity of the tasks. Therefore, for brevity and clarity, we omit the full multi-run result tables from the appendix. We have, however, ensured that all reported findings representative and reproducible.
C.2 Feature-based Experiments
C.2.1 Datasets Details
Expedia Dataset For data preprocessing, we generally follow the procedures outlined in Aouad et al.] [02T]. Specifically, we one-hot encode the categorical features site_id id, disitor_locae cattaaiiiooooootntioeee country_id, prop_coun arop_country_id and arch_destination _d, grouping all categories with fewer than 1,000 occurrences into a special category labeled -1. Continuous features such as price_ust uuceeddddddeee and srch_bo bechhbookiigg_iin
window are filtered remove unrealistic values: we retain only searches with hotel prices between $10 and $1,000 and booking windows shorter than one year. logarithmic transformation is applied both features reduce skewness. Missing values across the dataset are imputed using the placeholder value -1
We further filter out all records where no hotel modeling discrete choice and constitute a large portion of the raw data. To handle dummy items introduced for padding, we obtain a dataset consisting of 275,609 transaction observations, each described by 35 item-specific features and 56 shared features.
chosen, such cases
informative for
assign all input features of these items zero vectors. After preprocessing,
LPMC Dataset
The LPMC dataset is preprocessed by constructing both item-specific and item- shared features for the four available transportation modes: walk, cycle, public transit, and drive. Each alternative is represented by 4-dimensional item-specific feature vector that includes duration
27

---

(e.g., walking time, cycling time, transit access/ail/bus/iterccaagg time for public transit), cost (such fuel prices, transit fare, or congestion charges), number of interchanges (for transit), and congestion level indicator (for driving).
In addition, we incorporate set of shared features that common across all items within choice set. These include numerical variables such as straight-line distance, user age, gender (binary), driver's license status (binary), and the number of cars owned. Categorical variables like day of the week and trip purpose are encoded using one-hot representations. After preprocessing, the final dataset includes item-specific features and 17 shared features for each sample.
C.2.2 Experiments Details
We tune the hyperparameters of DeepHalo based on validation performance. Specifically, we search the number of layers L E {4,5), embedding dimensions d E {32, 64), and hidden dimensions H {8, 16). For all configurations, we adopt early stopping with a patience of 10 epochs prevent overfitting. On the LPMC dataset, we further employ learning rate scheduling strategy: training is first conducted with a learning rate of 10-3, and then fine-tuned using smaller rate of 10-4 This two-phase training helps improve convergence stability and final model performance, More details are shown Table 6 All experiments are conducted on a single Google Colab T4 GPU with Adam optimizer.
Experiments Setup
Table 6: DeepHalo hyper-parameters and training settings on Expedia and LPMC Experiments.
Hyper-parameter
Expedia
LPMC
Emhedding dimension d
32
32
Hidden size H
8
Layers
5
4
Polynomial Activation
Hadamard product
Hadamard product
Batch size
256
256
Learning Rate
x 10
1 x 10 1/003/1
/11 x 10-4
Baseline Information We summarize below the key configurations and model architectures for each baseline used in our experiments.
TCNer adopts a single-layer Transformer encoder-decoder architecture with multi-head self attention and feedforward layers. Both the source and target inputs are linear projections of item features. For the LPMC dataset, we use an embedding dimension of d = 36 and K attention heads; for the Expedia dataset, we set d = 64 and K = 8. The decoder output for each item is passed through final linear layer to obtain utility scores.
RUMnet models sequential utility construction using a GRU that processes items in their presented order within each choice set. At each timestep, the GRU updates latent preference state, from which the utility for the current item is computed using linear transformation. Additionally, an item-specific bias term, learned from raw features, added to the final score. The hidden size of the GRU is set to 96 for LPMC and 128 for Expedia to accommodate the respective feature complexities.
TasteNer implements latent factor model with learned item and user representations. Both the item encoder and the user encoder are two-layer multilayer perceptrons (MLPs) with hidden size of 128 and output dimension 128. The embedding is obtained by averaging valid item feature vectors within a choice set. Final utilities are computed as dot products between item and user embeddings, with an item-specific bias added.
FATEnet uses a DeepSet architecture where the embedding dimension matches the input size dy- Both the embedding network and the pairwise utility network are implemented as five-layer MLPs with 64 hidden units per layer and ReLU activations. The pairwise utility network outputs a scalar utility score for each item-context pair.
MLP a simple thyee-layer MLP with ReLU activations and a default hidden width of dembed = 128. It maps each item feature vector to a scalar utility, followed by masked softmax normalization over available alternatives. The model does not include any residual connections or context aggregation mechanisms.
28

---

ResLogit encodes each item through three-layer MLP with embedding dimension d = 32, using ReLU activations and dropout. The encoded features are passed through layer normalization layer and then linearly projected via a learned coefficient vector 3 E IR The resulting utility scores are refined through 10-layer residual network consisting of nonlinear residual blocks based on softplus activations, followed by masked softmax.
DLCL a context-aware linear model in which item utilities are computed via learned matrix B modulated by context-dependen adjustments from a second matrix A, based on the average feature vector of the entire choice set. The outputs from different feature dimensions are combined using a set of learnable mixture weights to produce the final utility distribution.
Detailed Results We run each experiment with different random seeds and report the mean and standard deviation of the evaluation metries from repetitions. This improves statistical robustness and accounts for variance due to initialization and stochastic training dynamics. We evaluate model performance using negative log- likelihood (NLL) and top-1 accuracy, where accuracy defined as the proportion of cases where the model assigns the highest predicted probability to the actually chosen item. Unless otherwise specified, the main results reported in the paper correspond to a single representative run. The summary of detailed results is shown in Table 7 and Table g
Table 7: Expedia Detailed Results
Train
Validation
Test
Model
Lass
Acc
Loss
Acc
Loss
Daep
2.5086 :: 0.0051
0.2501 0.0011
2.5216 0.0025
0.2474 = 0.0007
2,5288 a: 0.0026
0.2442 =: 0.0009
TCnet
2.4955 : 0.0125
0.2519
0.0034
2.5260 0.0051
0,2476 0.0006
2.5343 0.0041
0.2423
MLP
2,5693 =: 0.0030
0,2362 a 0.0008
2.5695
0.0011
0.2371 =: 0.0009
2.5805 a: 0.0015
0.230M a: 0,0011
BATEaet
2.5460 = 0.0067
0.2407
0.0015
2.5440
0.2421 + 0.0008
2.5548 0.0057
0.2365
ResLogit
2.5540 + 0.0044
0,2401
0.0015
2.5577 0.0015
0.2396 + 0.001 I
2.5692 0.0028
0.2342 0,0008
RUMaet
2.5565 = 0.0041
0,2397 1 0.0016
2.5685 0.0011
0,2358 = 0.0009
2,5782 : 0.0026
0.2308
0.0021
TasteNet
2.5623 = 0.0081
0,2383 0.0016
2.5675 0.0036
0.2374 =: 0,0007
2.5766 0.0040
0.2326 0.0008
DLCI
2.5624 0.0009
0.2357 + 0.0002
2.5546
0.0010
0.2391 # 0.0005
2,5621 # 0.0010
0.2307 0.0002
MNI.
2.6272 + 0.0001
0,2260 0.0002
2.6160 0.0001
0.2275 0.0002
2.6245 + 0.0001
0.2210 0.0003
Table 8: LPMC Detailed Results
Train
Validation
Tost
Model
Loss
Acc
Loss
Acc
Loss
Acc
DeepHalo
0.6427 = 0.0122
0.7527 0.0037
0.6412 0.0053
0.7576 = 0.0028
0.6407 0,0045
0,7552 0.0022
TCnet
0.6744 + 0.0098
0.7422 + 0.0036
0.6581 + 0.0096
0.7500 + 0.0038
0.6577 + 0.0076
0.7468
0.0023
MLP
0.7036 0.0034
0.7333 0.0019
0.6855 0.0029
0.7402 0.0019
0,6870 0.0049
0.7367 0.0030
EATEnet
0.6765 = 0.0042
0.7413 1 0.0015
0.6583 2 0.0046
0.7508 :: 0.0018
0.6619 2 0.0043
0.7457 2 0.0029
ResLogit
0.7080 = 0.0047
0.7293
0.0025
0.6926
0.7348 : 0.0022
0.0052
0.7370
RUMnet
0.6923 + 0.0074
0,7378
0.0017
0.6754 0.0079
0.7460 + 0.0025
0.6779 + 0.0067
0,7411 + 0,0020
TastoNet
0.7140 = 0.0056
0.7292
0.0031
0.6976 0.0056
0.7356 = O. 0.0032
0.6963 0.0068
0.7348 0.0035
DLCL
0.7148 = 0.0027
0.7151
0.0013
0.6978 0.0016
0.7266 : 0.0018
0,6988
0.0026
0,7218 0.0011
MNL
0.8813 + 0.0000
0.63 + 0.0002
0.8621 + 0.0000
0.6426 + 0.0003
0,8637 + 0.0001
0.6441 = 0.0002
C,3 Additional Experiments
C.3.1 Data Efficiency on the SFOshop Dataset
To evaluate data efficiency, we subsample the SFOshop dataset- for strong context effects ratios from 10% to 100%, training all neural models under strict parameter budget of approximately IK. Across all settings, DeepHalo consistently achieves the lowest test NLL, showing strong general- ization even with limited data. Its test NLL steadily improves with more data, while other models (TCNet, MLP, FATE) either plateau early or fluctuate.
C.3.2 Training Time Comparison
Table T0 reports total training time (until early stop) in representative LPMC setting. DeepHalo re- quires longer training due to explicit order-decomoositionn,, but remains within the same computational scale as other deep baselines.
29

---

Table Data efficiency comparison on SFOshop (test NLL). Lower is better.
Ratio (%)
CMNL
MNL
FATE
MLP
MixedMNL.
TCNet
DeepHalo
10
1.574
1.898
1,577
1.571
1,674
1,589
1,566
20
1.567
1.869
1.576
1.563
1.617
1.573
1.558
30
1.564
1.844
1.574
1.557
1,590
1.569
1.551
40
1.562
1.820
1.574
1.554
1.576
1.568
1.548
50
1.561
1.779
1.576
1.550
1,566
1.570
1.544
60
1.562
1.760
1.576
1.545
1.563
1,551
1.537
70
1.555
1.742
1.576
1.542
1.560
1.539
1.532
80
1.544
1.726
1.576
1.537
1,558
1.560
1.528
90
1,550
1.699
1.576
1.539
1,555
1.549
1.526
100
1,550
1.726
1.560
1.537
1,558
1.538
1.528
Table 10: Training time comparison under the LPMC setting.
Model
Total Training Time (s)
MNL
96.5
MLP
32.3
TasteNet
45.5
RUMNet
102.2
DL.CI
298.1
ResLogit
143.3
FATE
61.3
TCNet
141.6
DeepHalo
202.0
C.3.3 Featureless Dataset Baselines
We additionally evaluate FATE, TCNet, and MixedMNL. in featureless settings ensure fair parison. All neural models are controlled to approximately 1K parameters to focus on architectural differences.
Table 11: Results on featureless datasets (Train/Test NLL). Lower is better.
Model
Hotel
SFOshop
SFOwork
MNL
0.7743 0.7743
1.7281 / 1.7262
0.9423 0,9482
0.7569 / 0.7523
1,5556 / 1.5523
0.8074 0.8120
CMNL
0.7566 / 0.7561
1.5676 1.5686
0.8116 0.8 616
MixedMNL
0.7635 / 0.7613
1.5599 / 1.5577
0.8092 / 0.8153
FATE
0.7575 0.7568
1.5726 / 1.5765
0.8133 / 0.8167
TCNet
0.7467 0.7670
1.5694 / 1.5742
0.8114 0.8145
DeepHalo (ours)
0.7479 / 0.7483
1.5385 / 1.5263
0.8040 0.8066
30


Reviews:
[{"review_id": "NhHXMmbPIB", "reviewer": "Reviewer_Uum8", "summary": "Human revealed preferences are context-sensitive: Human choices are influenced by the context (made up of the set of alternatives) within which they are taken. These context effects cannot be explained by humans rationally choosing the alternative with the highest utility. One potential explanation for this phenomenon is that humans maximize utility, but that each alternatives’ utility is context dependent (dependent on the other items in the set of alternatives).\n\nAs this requires utility functions which maps arbitrary-size sets to real utility values, the learning problem becomes exponentially more complex. Prior work has attempted to solve this by decomposing the utility function, and has attempted to make computation cheaper by limiting calculations to lower-order interactions between alternatives. In line with this prior work, this paper proposes a similar decomposition of utility. It then proposes a new neural network architecture, DeepHalo, that can tractably calculates the decomposed utility up to arbitrarily high orders of interaction. This is done using a recursive formulation of context embeddings, where higher-order embeddings are produced from lower-order ones.\n\nExperiments on synthetic data show the benefit of modeling higher-order effects with DeepHalo. On real choice data, the model’s predictions are shown to outperform baselines in terms of negative log-likelihood.", "questions": "Equation (3) includes the term $(-1)^{|T|-|R|}$, making each summand where the sizes of $R$ and $T$ differ by an uneven amount negative. I do not understand what this is supposed to achieve. What is the purpose of this?\n\nIn (3) you use $u_j(x_j, X_R)$. Here $u_j$ takes different arguments than in its previous uses. Is this the same function as defined in (2) (might I suggest writing it then as $u_j(X_{K \\cup \\\\{j\\\\}})$, or is this a different function $u_j$?\n\nFor Tables 1 and 2, are the differences in test-set Negative log-likelihood (statistically) between DeepHalo and TCNet statistically significant? If I am reading Tables 5 and 6 in the appendix correctly then their mean NLLs are within 2 standard errors.\n\nHow sensitive is DeepHalo to the amount of training data? Would you say it is more of less data efficient than the other baselines?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 1, "strengths_and_weaknesses": "The basic problem this paper tackles, modeling context effects, remains a very important problem in learning from human preferences. The amount of work that has been done in this area — and the paper does a very good job of covering the relevant literature here — shows how important it is. The writing of the paper is very clear, making it easy to understand the proposed model. The experiments are also excellent, covering a variety of datasets and comparing to a large set of baselines.\n\nMy main concern with this paper is novelty. As far as I can see, the specific decomposition of the utility function which DeepHalo uses mirrors FETA’s almost exactly. The proposed neural network architecture is a fairly straightforward variation on DeepSets. The authors claim that the model offers an increased level of interpretability. However, the notion of relative context effect in (10) used to interpret the model can be calculated with any model (including prior ones) that decomposes the utility of a choice set into contributions of subsets; the interpretability is therefore not derived from the DeepHalo architecture specifically.\n\nFurthermore, I’m not convinced that the proposed network architecture replicates the utility decomposition the paper aims for (line 189). It seems correct for pairwise interactions, but I do not see how the recursive structure proposed replicates this utility decomposition.\n\nMinor points:\n- Line 79: duplicate instance of “Deep Sets [Zaheer et al., 2018]\n- Lines 180-182. Misspelling of FETA and FATE. In the original paper the “E” stands for “evaluate” not “evaluation”.\n- At various points in the paper, the authors use $S$ both to indicate the set $S$ and the size of that set. This is needlessly confusing. It would be better to use $|S|$ for the size of $S$ consistently.", "was_revised": true, "final_justification": "The paper is technically solid but the contribution and novelty are fairly thin for NeurIPS. Nevertheless, this is compensated by the quality of the experiments and the value of being able to control interaction orders within the proposed method, pushing my judgement over the edge towards acceptance.", "rebuttal": "We sincerely appreciate your recognition of our problem modeling, methodological soundness, real-world validation, and thoughtful baseline selection. \n\nTo address the concerns, we provide detailed point-wise responses below.\n\n\n> **Q: As far as I can see, the specific decomposition of the utility function which DeepHalo uses mirrors FETA’s almost exactly...**\n \nA: The key methodological contribution of DeepHalo lies in introducing a **structured and order-controllable framework** that balances expressiveness and interpretability—a capability that is lacking in existing context-dependent choice models. Among context-dependent models, \n\n- First-order models such as CMNL (Yousefi Maragheh et al., 2020), low-rank Halo MNL (Ko and Li, 2024), and FETA (Pfannschmidt et al., 2022) are **inherently restricted to pairwise interactions**. For instance, FETA proposes a decomposition that resembles the halo effect, but in practice, it is limited to first-order pairwise interactions. The architecture is not scalable to higher-order effects, as doing so would require exponentially large tensor storage and computational resources, rendering it impractical for real-world applications.\n\n- Other higher-order models, such as FATE (Pfannschmidt et al.,2022) and TCNet (Wang et al., 2023), attempt to learn context effects by **directly entangling all item features** within the offer set. While their outputs can be retrospectively interpreted as halo effects via Eq. (10), the interaction order is not explicitly controlled. Recovering a meaningful decomposition in these models would require computational complexity on the order of $O(2^{∣S∣−1})$, due to the exponential number of possible interaction terms. This limitation stems from their architectural design: context effects are not modeled in a structured or decomposable way. Specifically, FATE aggregates all item features into a set embedding, which is then combined with individual item features through complex nonlinear transformations, obscuring the origin and order of interactions. TCNet, similarly, uses attention mechanisms where the softmax denominator implicitly mixes information from all items in the offer set, making it difficult to isolate individual effects. As a result, neither model can recover a truly decomposable utility function. \n\nIn contrast, DeepHalo is explicitly designed to address this gap by enabling **precise control over the maximum order of halo interactions**. To the best of our knowledge, it is the first model that supports flexible trade-offs between model complexity and interpretability in this context.\n\nIn e-commerce applications such as recommendation systems and bundle pricing, practitioners are often primarily interested in lower-order context effects, which are more stable, interpretable, and practically useful. While expressive models can theoretically recover such effects, they do so implicitly as part of an entangled representation involving many higher-order interactions. Consequently, low-order effects cannot be meaningfully isolated. DeepHalo offers a principled approximation that explicitly models limited-order halo effects. It allows users to control the maximum interaction order $K$, with computational complexity only $O(2^K)$, offering a **flexible trade-off between efficiency and expressiveness**.\n\nOur contribution goes beyond what Deep Set offers. While DeepSets provides permutation invariance, DeepHalo integrates domain-specific insights from discrete choice theory and the halo literature, along with architectural elements like residual connections and polynomial activations. This enables interpretable, high-order interaction modeling in a principled and controllable way.\n\n\n> **Q: Furthermore, I’m not convinced that the proposed network architecture replicates the utility decomposition the paper aims for (line 189)...**\n\nA: Thank you for raising this point. The key intuition is that every recursion depth l in Eqs. (4)–(5) adds **exactly one additional order of interaction** while preserving all lower‑order terms via the residual connection. The process is the same as Appendix A 2.2 and A 2.3. To better facilitate the understanding, we provide a simple example below.\n\nConsider an offer set ${j, k, l}$ and let $\\phi$ be an identity mapping. We focus on the change of $z_{j}$ in DeepHalo.\n1. **Pairwise (1‑st order) layer.**  \n   The first layer aggregates the linear summary vector $\\bar Z^{1}$ from raw embedding vector $z^{0}$ and modulates them with $z_{j}^{0}$. This yields  \n   $z_{j}^{1}=z_{j}^{0} + \\frac1H\\sum_{h}\\bar Z_{h}^{1} \\cdot z_{j}^{0}$, which contains only **pairwise interactions** between the target alternative $j$ and every other item $k, l$ in the set.\n   If we denote $f_j$ as any function containing only item j's feature information, the above output can be formulated as,\n   $z_{j}^{1} = f_j + \\frac1H\\sum_{h}(f_j + f_k + f_l) \\cdot f_j$, thus it contains zero order term $f_j$ and first order term $f_k \\cdot f_j$ and $f_l \\cdot f_j$, second order term $f_j \\cdot f_k \\cdot f_l$ won't appear. For simplicity, we use $f_{ab}$ to represent any term like $f_a \\cdot f_b$ and $f_b \\cdot f_a$, which only contains the features of item a and item b.\n\n2. **Second‑order layer.**  \n   After the first layer, $z_{k}^{1}$ and $z_{l}^{1}$ already embed their pairwise effects with every other item.  \n   The second layer builds\n    \n   $\\bar Z^{2}=\\tfrac1S\\sum_{m}W^{2}z_{m}^{1}$, which contains term $f_j, f_k, f_l, f_{jk}, f_{jl}, f_{kl}$.\n   \n   $z_{j}^{2}=z_{j}^{1}+\\tfrac1H\\sum_{h} \\bar Z_{h}^{2} \\cdot z_{j}^{0}$ which is analogous to $(f_j + f_{jk} + f_{jl}) + \\sum_h(f_j + f_k + f_l + f_{jk} + f_{jl} + f_{kl}) \\cdot f_j$,\n   which brings about second order term $f_{jkl} = f_{kl} \\cdot f_{j}$. It can be interpreted as the effect of ${k,l}$ on $j$.\n\nThis example illustrates how we can add up orders by increasing the number of layers and get a decomposable utility representation. If we change the $\\cdot$ in layers to some other polynomial function, the term order will grow faster, like in the first layer we change to $(f_j + f_k + f_l)^2 \\cdot f_j$, in the sum will directly bring about the second order term $f_{jkl}$. The quadratic activation naturally brings a $log_2$ form depth requirement.\n\n> **Q: The purpose of $(-1)^{|T|-|R|}$.**\n\nA: The $(−1)^{∣T∣−∣R∣}$ factor in Eq. (3) is the **inclusion–exclusion inversion coefficient** on the Boolean lattice of subsets. Our $v_j(⋅)$ must invert the cumulative‑sum relation $u_j(X_S)=\\sum_{T\\subset S\\setminus\\{j\\}} v_j(X_{T\\cup\\{j\\}})$; effects contributed to many supersets are otherwise over‑counted. Alternating signs guarantee that terms appearing an even number of times cancel and those appearing an odd number of times net to the correct single contribution, yielding a unique decomposition. \n\n> **Q: About Performance Difference between DeepHalo and TCnet...**\n\nA: Thanks for the insightful observation. We acknowledge that similar performance between DeepHalo and TCNet is expected; however, we wish to emphasize a key difference in their modeling philosophies.\n\nWhile TCNet can approximate utility decompositions (maximum order), its Transformer architecture lacks control over the maximum interaction order. Attention layers aggregate across the full set, leading to large quantities of high-order effects that are difficult to interpret. In contrast, DeepHalo explicitly controls interaction order through its recursive structure, enabling better interpretability and identifiability.\n\nGiven that low-order interactions often dominate in real-world settings, it is not surprising that TCNet matches DeepHalo's accuracy. However, DeepHalo achieves this with controlled expressiveness, offering a more interpretable alternative. Rather than outperforming black-box models in accuracy, our main goal is to provide a principled, order-aware design that balances predictive performance with transparency.\n\n> **Q: About data efficiency**\n\n| Ratio (%) | CMNL  | MNL   | FATE   | MLP   | MixedMNL | TCnet | **DeepHalo** |\n|-----------|-------|-------|--------|-------|-----------|--------|--------------|\n| 10        | 1.574 | 1.898 | 1.577 | 1.571 | 1.674     | 1.589 | **1.566**     |\n| 20        | 1.567 | 1.869 | 1.576 | 1.563 | 1.617     | 1.573 | **1.558**     |\n| 30        | 1.564 | 1.844 | 1.574 | 1.557 | 1.590     | 1.569 | **1.551**     |\n| 40        | 1.562 | 1.820 | 1.574 | 1.554 | 1.576     | 1.568 | **1.548**     |\n| 50        | 1.561 | 1.779 | 1.576 | 1.550 | 1.566     | 1.570 | **1.544**     |\n| 60        | 1.562 | 1.760 | 1.576 | 1.545 | 1.563     | 1.551 | **1.537**     |\n| 70        | 1.555 | 1.742 | 1.576 | 1.542 | 1.560     | 1.539 | **1.532**     |\n| 80        | 1.544 | 1.726 | 1.576 | 1.537 | 1.558     | 1.560 | **1.528**     |\n| 90        | 1.550 | 1.699 | 1.576 | 1.539 | 1.555     | 1.549 | **1.526**     |\n| 100       | 1.550 | 1.726 | 1.560 | 1.537 | 1.558     | 1.538 | **1.528**     |\n\nTo evaluate data efficiency, we subsample the SFOshop dataset—known for strong context effects—at ratios from 10% to 100%, training all neural models under a strict ~1k parameter budget.\n\nAcross all settings, **DeepHalo consistently achieves the lowest Test NLL**, showing strong generalization even with limited data. Its Test NLL steadily improves with more data, while other models (TCNet, MLP, FATE) either plateau early or fluctuate.\n\n\n### **Clarification**\nThe expression $u_j(x_j, X_R)$ in Eq. (3) refers to the same function as $u_j(X_{R \\cup \\{j\\}})$ in Eq. (2), and we will revise the notation for consistency. We will also correct the duplicate citation of Deep Sets [Zaheer et al., 2018], fix the misspelling of FETA and FATE (“E” stands for “evaluate”), and use $|S|$ consistently to denote set size. These issues will be addressed in the final version. We appreciate the reviewer’s careful reading."}, {"review_id": "ey7EzN2cmR", "reviewer": "Reviewer_B9Eq", "summary": "The authors propose a neural discrete choice model with context effects that handles item features. They use a Batsell-Polking style decomposition of context effects, but parameterize that decomposition with neural networks. The model applies iterative layers to capture higher-order context effects. By using a quadratic activations, the authors also provide an approach that captures higher-order context effects with only logarithmically many layers. In experiments on synthetic data, the authors demonstrate how their approach can recover planted similarity effects and learn very high-order interactions with low depth. On real datasets, they demonstrate better held-out performance than existing context effect models.", "questions": "1. Section 4.1 should explicitly say that (8) lets you capture $\\ell$th order interactions with only $log_2 \\ell$ layers, and should explain why.\n2. Do you recommend using (8) in general? Are there settings where we should use (5) instead? Adding some guidance to the paper here would be helpful.\n3. The intro says DeepHalo is more interpretable than the approaches of [Wong and Farooq, 2021], [Pfannschmidt et al., 2022], [Wang et al., 2023, Peng et al., 2024]. Similarly, the results state \"while offering favorably improved interpretability compared to other deep neural network models.\" But using (10), can't we interpret context effects from any of the existing neural context effect models in the same way? What makes DeepHalo more interpretable?\n4. The experiments of [Rosenfeld et al., 2020] use item features too---despite 1.1 saying they focus on the featureless case. What's the advantage of DeepHalo over SDA? DeepHalo seems like a Set-Dependent Embedding approach [Rosenfeld et al., 2020; eq. (2)], right? \n\nQuestions 3 and 4 are why I gave a 4 instead of a 5, and I'm very interested to hear the authors' response.", "limitations": "The discussion of limitations is limited. How does the sample complexity and need for hyperparameter tuning compare to existing approaches?", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Clearly written paper\n2. Interesting approach, combining the Batsell-Polking approach with a neural network parametrization and an idea for efficiently capturing higher-order effects\n3. Solid experiments\n\nWeaknesses:\n1. Could be clearer what the advantages are of the proposed method vs existing neural context effect models\n\nQuality: good. Background and related work are thorough. Motivation and methods are good, baselines are pretty thorough and experiments are well done. \n\nClarity: There are a few points that can be clarified (see questions). Overall, though, the paper is clearly written. The title, though, is a bit generic, and could apply to several of the cited papers; I would suggest something that highlights the distinction of this approach vs those of Rosenfeld et al., Pfannschmidt et al, etc.\n\nSignificance: this is an important problem that has received significant attention, and the proposed method performs well in experiments.  \n\nOriginality: reasonable, applies a different approach to neural modeling of context effects. I have some questions about what distinguishes DeepHalo from existing neural context effect models (see below)\n\nMinor:\n- Deep Sets double cite in Line 79", "was_revised": true, "final_justification": "Thanks for the responses. I think this is a nice paper and I’d be happy to see it accepted. I’ve updated my score to a 5.", "rebuttal": "We truly appreciate Reviewer B9Eq’s positive feedback. To address the questions and concerns about our method, we provide point-wise responses as follows.\n\n### Weakness and Limitation\n\n> Q: Could be clearer what the advantages are of the proposed method vs existing neural context effect models\n\nA: The key advantage of DeepHalo lies in its ability to **explicitly control the maximum order of context (halo) effects**, enabling a flexible trade-off between model expressiveness and interpretability—something existing neural models lack.\n\n- First-order models (e.g., CMNL [Yousefi Maragheh et al., 2020], FETA [Pfannschmidt et al., 2022]) are interpretable but limited to pairwise interactions and cannot scale to higher-order effects due to computational constraints.\n- High-capacity models (e.g., FATE [Pfannschmidt et al., 2022], ResLogit [Wong and Farooq, 2021], and TCNet [Wang et al., 2023]) are expressive but entangle all item features, making it infeasible to control interaction order. Recovering meaningful low-order effects from these models requires evaluating up to $O(2^{|S|-1})$ terms.\n\nIn contrast, DeepHalo models context effects in a **structured and decomposable** way. It allows practitioners to specify the maximum interaction order $K$, with a complexity of $O(2^K)$ to recover the effect terms, making it both **efficient and interpretable**. This is especially useful in real-world settings like recommendation and bundle pricing, where lower-order effects are often sufficient and desirable.\n\n> Q: The title, though, is a bit generic, and could apply to several of the cited papers; ...\n\nA: Thanks for your suggestion! We will consider adding some unique elements to our title that help distinguish it from other context-dependent model papers.\n\n> Q: How does the sample complexity and need for hyperparameter tuning compare to existing approaches?\n\nA: Regarding sample complexity and data efficiency, we refer the reviewer to our response to Reviewer Unm8 (Q: About data efficiency), where we provide an experiment evaluating model performance under varying data proportions.\n\nAs for the hyperparameter tuning, we find empirically that DeepHalo is relatively easy to tune. **Once the maximum effect order is specified, the overall effect complexity is fixed**, which is the main tuning difference compared to existing approaches. Additional parameters are then introduced only to improve the approximation accuracy of these effects, rather than to increase model capacity arbitrarily. \n\n#### 1. Clarifying Section 4.1 — why Equation (8) needs only $\\log_2l$ layers to model $l$‑th‑order interactions\n\nWe agree that Section 4.1 should state this property explicitly. Currently, this is mentioned in Section 4.2, and the explanation is deferred in the appendix. \nIn the revision, we will:\n\n1) **Add a sentence to Section 4.1**  \n   > “Because each Equation (8) layer doubles the maximum attainable interaction order, an *l*‑th‑order effect can be represented with at most ⌈log₂ l⌉ stacked layers.”\n\n2) **Provide an intuitive explanation**  \n   - A single layer (Equation (8)) captures up to second‑order effects.  \n   - Stacking two layers composes these interactions, yielding up to 4‑th‑order effects, and so on.  \n   - After *k* layers the highest order is $2^{k}$, giving the logarithmic bound.\n\n\n#### 2. Practical guidance on when to use Equation (5) vs Equation (8)\n\nWe appreciate this insightful question. Generally, we recommend:\n\n- **Using Eq. (5) when training stability is a concern or when higher-order context effects are not considered.**\n- **Using Eq. (8) to reduce network depth when higher-order interactions are considered.**\n- Combining them for a tunable trade-off.\n\nHere we provide some details for further intuition. One of the strengths of the DeepHalo architecture is its structural flexibility: the activation function σ can be instantiated as any polynomial function. (5) and (8) can be used interchangeably within the same model. In practice, they can be freely mixed—for example, stacking layers with linear and quadratic activations can achieve effective interaction orders such as $((1+1+1)\\times2+1)\\times2$, offering nuanced control over expressivity.\n\n- Higher-degree activations (like Eq. (8)) allow *faster growth* in representational power with depth, efficiently capturing higher-order effects.\n- However, very high-degree activations in deep networks may introduce *training instability*, such as exploding gradients.\n- Empirically, we observe that for the same representable order, both variants achieve similar accuracy. Yet, first-order activations (Eq. (5)) tend to be more stable, especially on noisy or real-world datasets.\n\nThis modularity makes DeepHalo a unified and controllable framework for context-dependent choice modeling, in both feature-rich and featureless settings. We will add a guidance paragraph in the revision to clarify these trade-offs.\n\n#### 3. Clarifying what makes DeepHalo more interpretable than prior neural context-effect models\nWe appreciate the reviewer’s thoughtful question. While many prior models can interpret the effects using Eq. (10), this is under two premises:\n- The model is expressive enough to include all interaction orders up to $(|S|-1)$ instead of some specific orders (which first-order models like CMNL and FETA cannot achieve)\n- Existing expressive models like FATE and TCNet require a computational complexity of $O(2^{|S|})$ for recovering all halo effects in the model.\n\nIn contrast, **our expressive DeepHalo model can capture up to a specific order and only needs $O(2^K)$ computational time to obtain $k$-order interactions**, which provide more interpretability. For example, modeling third-order effects over a catalog of 100 items without such control would require handling over 160,000 subsets, which is computationally prohibitive and interpretively infeasible.\n\nBesides, **the exact-interaction-order control inherited in our model is of practical interest.** As a concrete example, consider a choice set $\\{A, B, C, D\\}$. Then, estimating how the subset $\\{A, B\\}$ influences the utility of item $C$, the function must depend only on the features of $A$, $B$, and $C$, and must not involve irrelevant items like $D$. This subset-specific conditioning is critical for interpretability, but is not enforced in existing context-dependent ML models. \n\nThe reason the existing models cannot capture exactly up to $k$-order interactions is that they attempt to learn context by directly entangling all item features. For instance, FATE combines global context with individual features but cannot restrict interaction depth. TCNet’s attention mechanism inherently mixes all items, effectively modeling full-order interactions by design. As a result, these models do not formulate context effects in a structured, decomposable way, making interpretation difficult.\n\nOn the contrary, **our model achieves more interpretability with a modular design**:\n- Each layer introduces exactly one additional interaction order or $\\times 2$ using quadratic activation.\n- The maximum effect order is bounded by the network depth $L$.\n- Residual and polynomial structures guarantee halo-decomposable utility functions. We will give a simple example later for better illustration.\n\n\n#### 4. Clarifying the distinction between DeepHalo and SDA [Rosenfeld et al., 2020]\n\nThank you for the insightful observation. While DeepHalo shares some high-level similarities with set-dependent embedding (SDA) approaches such as [Rosenfeld et al., 2020, Eq. (2)], we would like to emphasize several key distinctions in both motivation and design, some are similar to Point 3.\n\nDeepHalo is fundamentally grounded in halo effect theory, which prioritizes **interpretable higher-order interactions**. While simply incorporating context information (e.g., through contextual functions like $\\phi$) is straightforward, interpretability requires controlling the maximum order of interactions—a key principle not enforced by SDA.\n\nAs the reviewer rightly noted, if Eq. (10) were applied to arbitrary context-based architectures (e.g., SDA), the number of halo terms would grow exponentially with the size of the offer set. This would result in a combinatorial explosion of interaction terms that are computationally hard and practically uninterpretable.\n\nIn contrast:\n\n- SDA and related models aggregate item features in a fully entangled way.\n  - This makes it virtually impossible to recover the halo effect without enumerating all subsets—an exponentially large and ill-posed task.\n- DeepHalo, by design:\n  - Uses residual connections and polynomial activations to explicitly control and construct the interaction order.\n  - Ensures that utilities are halo-decomposable and that item features remain disentangled.\n  - Caps model expressiveness to low-order interactions, preserving interpretability and generalization.\n\nIn short, while SDA enables generic set-dependent context modeling, DeepHalo provides a theoretically-grounded and architecturally-enforced approach to **interpretable, order-controlled context effects**, which is essential for practical discrete choice applications. \n\nFor points 3 & 4, we refer the reviewer to Appendix A2.2 and A2.3, as well as the illustrative example in Question 2 of our response to reviewer Unm8, for a detailed explanation of how DeepHalo supports structured, decomposable modeling of context effects."}, {"review_id": "HJ02fNOIHm", "reviewer": "Reviewer_6ydQ", "summary": "This paper introduces an ML based context-dependent choice model. The issue is that the classical IIA assumption of choice models is often not met/questionable, and one perspective on addressing it is through modeling the “halo” effect. In this approach the authors consider using information from all alternatives when evaluating each single one, and considering their interactions. The paper shows the derivations of these interactions in a permutation equivariant deep neural network framework. Results show that the model generally outperforms the baselines.", "questions": "I enjoyed reading the paper, and this seems lile an interesting research direction. However, it’s very unclear to me why this fits in the NeurIPS context. The methodological ML contribution lies practically exclusively in the modeling of interactions and equivariance, and in both cases, the contributions are not very novel. If the contribution relates to DCM behaviour modeling for the ML community, then some aspects should be considered (e.g. why not consider nested logit, mixed logit, LCCM? why classical MNL as baseline, which is the same for DCM literature as fully connect MLP is for deep learning (the most vanilla of vanilla…). \nDon’t take me wrong, I do think this is an interesting paper, but I’d see it as more appropriate for Journal of Choice Modeling for example…", "limitations": "Some clarifications:\n\n- The intro is good but some things are a bit unclear. For example, “featureless settings”?? What is that (I know what it is, but I did have my doubts in the beginning, and it would be nice to spare the reader of such trivial doubts)? \n\n- “For each j ∈S, define a utility function uj(T), which represents the utility of alternative j when the context is the subset T ⊆S\\{j}.” This is quite confusing. You are introducing the classical MNL, but also introduce the concept of context at the same time. Fine, but then, you refer to uj(T) (utility of j within the context?) in the text, while the MNL presented has uj(S). What’s the relationship here? \n\n- “Here, XT∪{j}∈Rdx ×J denotes the matrix formed by replacing the feature vectors not in the subset\nT ∪{j}with zero.” —> rephrase. At first sight, it almost implies that this matrix is only zeros (it’s not mentioned what to do with the features that ARE in the subset). I know this is obvious, but the cognitive load of a reader would thank the authors…\n\n- This sentence got me a bit confused: “Here, H denotes the number of interaction heads, analogous to the number of channels or attention heads, and thereby controls the diversity of the interaction patterns. “ It seems as though W^1 is projecting the z embeddings vectors to a (lower?) space, and that W^1 is shared by all alternatives (there is ONE W^1). Then, why not train the z with dimensionality h directly? What’s the benefit of W^1?\n\n-Section “4.1 Residual Connection for Large Choice Sets” is quite confusing: in what sense does polynomial aggregation improve computational efficiency? In fact, with larger S, then you will get extremely high degree polynomials, which will be prone to numerical errors. Perhaps the benefit is then that you can use polynomials to reduce number of layers (i.e. trade-off linearity with many layers with polynomials with less layers), but this is not clear in the explanation nor on the notation. \n\n- For section 4.2., if I understand well, e_S is a vector of 1s. If that is true, then u_j(e_S) is a sum of ones irrespeective of the polynomials, and thus totally S. This makes no sense, implying I’m misunderstanding something here… I do understand that, later on, given that we learn the q values (eq 9), this no longer applies (i.e. we get polynomials), and it makes intuitive sense.  \n\n\n- There’s something confusing with the meaning of \\union in equation 3. Classical interpretation implies that T \\union {j} is just the set T plus the element j. But It sort of implies that j is treated differently to the rest. Ok, you already index v_j (on the left of the equation), thus allowing you to differentiate j on the right hand side. At first time, that was totally fine. But then we get to equation 10, in particular v_j(T) + vj_(T \\union {k}), where you make vj_(T \\union {k}), which becomes ambiguous. I mean, I understand that you want to estimate the context effect of T and also consider the context role of k in interaction with T, but then what would be expansion (according to 3)? \n\n- About number of parameters in Fig 2, what are we talking about specifically? Dimensionality of embeddings? Of the phi functions? \n\n- \\bar{Z}_1 := 1/S…  I believe it should be |S|, since S is a set. \n\n\nThe beverage experiment is not very intuitive. The strongest effect seems to be that, if Pepsi is on the menu, then people strongly prefer 7Up or Sprite rather than Coke (versus having only 3 options? the entire set? Versus having a different cola instead of pepsi? Is the choice set the competing product pair+effect source set in each case? Is this featureless? What are the features?). Why? What’s the intuitive explanation? The examples in the intro/motivation section were very convincing, why not try them instead?", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: \n- Important modeling problem (although somewhat specific/niche)\n- Convincing methodological treatment, with only a few notation glitches\n- Successful implementation and validation with real data\n- Good choice of (ML related) baselines\n\nWeaknesses:\n- Poor choice of (classical DCM) baselines\n- Lack of training (computational) performance statistics. How fast/slow is it, compared with the other DNNs?\n- Some points are unclear (see below)", "was_revised": true, "final_justification": "The authors addressed all my core concerns and improved/will improve the paper substantially.", "rebuttal": "Thank you very much for your thoughtful comments. Please see our response as follows.\n\n**Weakness & Questions:**\n> **Q: The methodological ML contribution ... the contributions are not very novel.**\n\nA: The contribution of DeepHalo lies not merely in modeling interactions or equivariance, but in introducing a **structured and order-controllable framework** . While prior models have explored interaction effects, they face fundamental challenges:\n\n- First-order models such as CMNL [Yousefi Maragheh et al., 2020], low-rank Halo MNL [Ko and Li, 2024], and FETA [Pfannschmidt et al., 2022] are **inherently restricted to pairwise interactions**. For instance, FETA proposes a decomposition that resembles the halo effect, but practically, it is limited to first-order pairwise interactions. The architecture is not scalable to higher-order effects, as doing so would require exponentially large tensor storage and computational resources, rendering it impractical for real-world applications.\n\n- Other higher-order models, such as FATE [Pfannschmidt et al., 2022], ResLogit [Wong and Farooq, 2021], and TCNet [Wang et al., 2023], attempt to capture context effects by **entangling all item features** in the offer set. While their outputs can be post hoc interpreted as halo effects via Eq. (10), the interaction order is uncontrolled, and recovering a meaningful decomposition requires $O(2^{|S|-1})$ complexity due to exponentially many terms. This stems from their architectures: context is not modeled in a structured or decomposable way. For instance, FATE compresses all item features into a set embedding, while TCNet’s softmax-based attention mixes information across items. As a result, none can recover a truly decomposable utility function. We refer the reviewer to Appendix A2.2, A2.3, and Question 2 of our response to reviewer Unm8 for details on how DeepHalo addresses this limitation.\n\nIn contrast, DeepHalo is explicitly designed to address this gap by enabling **precise control over the maximum order of halo interactions**. In e-commerce applications like recommendation and bundle pricing, lower-order context effects are often the most relevant—being more stable, interpretable, and practically useful. While expressive models can recover such effects, they do so implicitly within entangled high-order interactions, requiring up to $O(2^{|S|-1})$ terms to isolate. DeepHalo explicitly models limited-order halo effects, allowing users to control the maximum order $K$ with only $O(2^K)$ complexity, enabling a flexible trade-off between efficiency and expressiveness.\n\n\n> **Q: If the contribution relates to DCM behaviour modeling for the ML community, then some aspects should be considered... (Poor choice of (classical DCM) baselines)**\n\nA: Since our model targets context effects, we primarily compare DeepHalo with context-dependent DCMs like Contextual MNL. MNL serves as a sanity check to highlight the role of context. To include context-independent RUM-based models, we add Mixed Logit in the featureless setting, as it can approximate any RUM. For ML baselines, we include TCNet and FATE (suggested by reviewer D2h9). We respectfully refer the reviewer to our detailed response to Reviewer D2h9.\n\n> **Q: Lack of training (computational) performance statistics.**\n\nA: Thanks for pointing this out. Below, we add the total training time (until early stop) in a sample run under the setting of the LPMC experiment in Section 5.2.\n| Model Name | Total Training Time (s) |\n|------------|-------------------------------|\n| MNL        | 96                          |\n| MLP        | 32.3                          |\n| TasteNet   | 45.5                          |\n| RUMNet     | 102.2                         |\n| DLCL       | 298.1                         |\n| ResLogit   | 143.3                         |\n| FATE       | 61.3                          |\n| TCnet      | 141.6                         |\n| DeepHalo   | 202.0                         |\n\n> **Q: However, it’s very unclear to me why this fits in the NeurIPS context.**\n\nA:  We believe this paper is well-suited for the NeurIPS community, aligning with its growing focus on **human-AI alignment** and **feedback modeling**. Understanding and formalizing human choice behavior is essential for building AI systems that interact reliably with people. Existing alignment methods (e.g., RLHF, LfD) often assume rational, unbiased feedback—assumptions that rarely hold in practice. Our work introduces a principled, interpretable framework for modeling **context-dependent human choices**, laying the foundation for more realistic and user-aligned AI.\n\nAdditionally, we list below several noteworthy works related to our topic from previous years’ NeurIPS:\n\n[1]Oh S, Thekumparampil KK, Xu J. Collaboratively learning preferences from ordinal data. Advances in Neural Information Processing Systems. 28, 2015\n\n[2]Stephen Ragain and Johan Ugander. Pairwise choice markov chains. Advances in Neural Information Processing Systems, 29, 2016\n\n[3]Arjun Seshadri, Stephen Ragain, and Johan Ugander. Learning rich rankings. Advances in Neural Information Processing Systems, 33:9435–9446, 2020\n\n[4]Yiqun Hu, David Simchi-Levi, and Zhenzhen Yan. Learning Mixed Multinomial Logits with Provable Guarantees. Advances in Neural Information Processing Systems, 9447-9459, 2022\n\n[5]De Peuter S, Zhu S, Guo Y, Howes A, Kaski S. Preference learning of latent decision utilities with a human-like model of preferential choice. Advances in Neural Information Processing Systems, 37:123608-36, 2024\n\n\n**Clarifications:**\n> **Q: Some things are a bit unclear in intro.**\n\nA: We will explain the ‘Featureless setting’ further by elaborating on ‘items without access to their features’.\n\n> **Q: “For each j ∈S, define a utility function uj(T), ...”**\n\nA: We will only introduce the utility function $u_j(S)$ instead, and only introduce the notation $T$, which emphasizes the context information (subset excluding the item j itself).\n\n> **Q: “Here, $XT∪{j}∈R^{d_x ×J}$ denotes the matrix formed by replacing the feature vectors not in the subset T ∪{j}with zero.” —> rephrase.**\n\nA: We will rewrite the definition of $X_{T∪{j}}$ as “formed from the feature matrix $X_S$ by replacing the feature vectors not in the subset T∪{j} with zero.”\n\n> **Q: This sentence got me a bit confused: “Here, H denotes the number of interaction heads...”**\n\nA: Analogous to multi-head attention, in our context, this can be interpreted as H different consumer types with different tastes. The h-th row of W^1 represents the h-th taste vector, which can map the input with various preferences. Thus, W^1 provides H variety of mappings, which are then aggregated by the sum over $Z_h^1$. This is indeed training each of the h-th mapping, and we use matrix W^1 to compact the notation. \n\n> **Q: “Residual Connection for Large Choice Sets” is quite confusing**\n\nA: As the reviewer noted, one key computational advantage of the polynomial formulation is avoiding the need for deep architectures (line 234). We elaborate on this in Section 4.2 (lines 262–264), showing a logarithmic relationship: a linear number of layers can express exponentially many interaction orders. We will clarify this in the revised version. See also Appendix A2.2, A2.3, and the example in our response to reviewer Unm8 (Section 2).\n\nAs for numerical stability, training remains stable in the featureless setting even with high-order interactions (Figure 2). In the feature-rich setting, deep quadratic stacks may cause instability, but we find that modeling up to 4th or 5th order suffices in practice (e.g., Expedia), striking a good balance between expressiveness and interpretability.\n\n> **Q: For section 4.2., $e_S$ is a vector of 1s...**\n\nA: Regarding Section 4.2: $e_S$ is a unit vector with the $(e_S)_j = 1$ if j ∈ S and 0 otherwise. The recursive expression in equation 9 can be represented as a polynomial of $e_S$. Let us consider the case with up to second-order interaction, then the second-order term relates to $e_S \\otimes e_S$ where the (i,j)-th element refers to whether both item i and item j exist in the choice set. The coefficients in the polynomial corresponding to each element of the second-order term measure the specific interactions in equation (1).\n\n> **Q: confusing with the meaning of $\\cup$ in equation 3.**\n\nA: We apologize for the typo in equation 3: $X_R$ should be $X_{R∪{j}}$, which hopefully entangles the confusion. In equation (10), the relative context effect is specified on item j and item k, whose definition is introduced in Park and Hahn [1998].\n\n> **Q: About number of parameters in Fig 2...**\n\nA: The number of nn parameters in Fig 2 is the total number of trainable parameters, which specifies the model complexity and is influenced by the problem size and the selection of hyperparameters like H.\n\n> **Q: $\\bar{Z}_1 := 1/S$…**\n\nA: Yes, thanks for catching that. We will correct to |S|.\n\n\n**Beverage Experiment**\n\nThank you for the thoughtful comment.\n\n**This classical hypothetical dataset has been widely used in the literature on halo effects** (e.g., Park and Hahn [1998], Batsell and Polking [1985]) for its simplicity and clarity, making it a standard example for evaluating interpretability. The experiment is conducted in a featureless setting, demonstrating that DeepHalo can extract meaningful interaction effects purely from set-level preferences.\n\nSubfigure (b) illustrates how different item subsets (horizontal axis) influence the utility gap between item pairs (vertical axis). The rightmost column (∅) shows baseline preferences without contextual influence—for example, strong preferences for Pepsi over Coke and 7-Up over Sprite are evident both here and in the market shares (Subfigure a). The leftmost column (item 1) reveals a strong negative halo effect of Pepsi on Coke, reducing Coke’s competitiveness when co-offered. A similar pattern is observed between 7-Up and Sprite."}, {"review_id": "P5YpZIeFw9", "reviewer": "Reviewer_D2h9", "summary": "This paper introduces DeepHalo, a neural modeling framework designed to capture context-dependent choice behavior. Traditional choice models can fail to capture dependent utilities while recent context-dependent choice models are either restricted by interaction structures or lacking interpretability.  The paper first shows a utility decomposition with features and then introduces DeepHalo to capture decomposed higher-order effects. This novel architecture can bridge the previous gap by providing an expressive model that can both explicitly control the complexity of interactions (i.e., the order of interactions) and offer a principled interpretation of these context effects. The paper provides empirical validation on synthetic and real-world datasets (both featureless and feature-based), showing that DeepHalo achieves competitive or superior predictive performance compared to existing baselines while offering enhanced interpretability.", "questions": "1. Could you provide the number of parameters for each benchmark model used in your experiments? \n2. The formulation of higher-order interactions in Equations (4) and (5) appears conceptually similar to the transformer architecture, with the key difference being that the attention mechanism is replaced by a linear transformation followed by an activation function (just like assigning equal attention across elements). Could you elaborate on the differences between your approach and transformers, both in terms of formulation and implications for model expressiveness or computational efficiency?", "limitations": "See weaknesses.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: The paper is generally very well-written and organized. While building upon established concepts like utility theory, permutation equivariance, and residual connections, the specific architecture of DeepHalo that aims to explicitly model and disentangle interactions layer-by-layer is novel and insightful. Its interpretability is very useful when we need to learn the users preferences in practice.\n\nWeaknesses:\n1) In experiments, I think it is essential to provide the number of parameters of each benchmark model to make sure the better performance of DeepHalo is from the design instead of the size.\n2) While many benchmark models are compared in datasets with features (Table 2), while in featureless datasets (Table 1), only a few models are tested. However, I think the models like FateNet, TCNet can also be applied to featureless cases.\n\nMinor comments:\n1) It would be helpful if a figure can be provided to illustrate the architecture.\n2) Typo: Line 79 \"Deep Sets [Zaheer et al., 2018], Deep Sets [Zaheer et al., 2018]\"\n3) While the std are provided in Appendix (e.g., Table 6, 7), it would be helpful if you also provide them in the main part (e.g., Table 1,2,).", "was_revised": true, "final_justification": "Please see Strengths And Weaknesses.", "rebuttal": "We thank Reviewer D2h9 for their positive recognition of our work. In the revision, we will include a model architecture diagram and correct the noted typos. Below, we provide point-by-point responses to the reviewer’s comments.\n\n\n### Responses to Weakness\n\n> Q: In experiments, I think it is essential to provide the number of parameters of each benchmark model to make sure the better performance of DeepHalo is from the design instead of the size.\n\nA: We thank the reviewer for raising the question regarding the model scale. To address this, we computed the number of trainable parameters under the **Expedia experiment setting**, where the input item feature dimension is set to 10 across all models. The results are summarized below:\n\n| Model      | Number of Parameters |\n|------------|----------------------|\n| TCNet      | 118,209              |\n| **DeepHalo**   | **53,833**               |\n| RUMNet     | 53,900               |\n| TasteNet   | 35,851               |\n| DLCL       | 210                  |\n| FATE       | 44,363               |\n| MLP        | 42,601               |\n| ResLogit   | 2,720                |\n| MNL        | 10                   |\n\n> Q: While many benchmark models are compared in datasets with features (Table 2), while in featureless datasets (Table 1), only a few models are tested. However, I think the models like FateNet, TCNet can also be applied to featureless cases.\n\nA: We thank the reviewer for their suggestion. In the featureless setting, we have added results for three representative baselines: FATE, TCNet, and Mixed MNL. To ensure fairness and focus on architectural differences, we controlled the parameter size of all neural network–based models to be **approximately 1K parameters**. The results are shown below:\n\n| Model         | Hotel (Train/Test) | SFOshop (Train/Test) | SFOwork (Train/Test) |\n|---------------|--------------------|------------------------|-----------------------|\n| MNL           | 0.7743 / 0.7743    | 1.7281 / 1.7262        | 0.9423 / 0.9482       |\n| MLP           | 0.7569 / 0.7523    | 1.5556 / 1.5523        | 0.8074 / 0.8120       |\n| CMNL          | 0.7566 / 0.7561    | 1.5676 / 1.5686        | 0.8116 / 0.8164       |\n| Mixed MNL     | 0.7635 / 0.7613    | 1.5599 / 1.5577        | 0.8092 / 0.8153       |\n| FateNet       | 0.7575 / 0.7568    | 1.5726 / 1.5765        | 0.8133 / 0.8167       |\n| TCNet         | **0.7467** / 0.7670| 1.5694 / 1.5742        | 0.8114 / 0.8145       |\n| **DeepHalo** (Ours) | 0.7479 / **0.7483** | **1.5385** / **1.5263** | **0.8040** / **0.8066** |\n\n### Responses to Questions\n\n> Q: The formulation of higher-order interactions in Equations (4) and (5) appears conceptually similar to the transformer architecture, with the key difference being that the attention mechanism is replaced by a linear transformation followed by an activation function (just like assigning equal attention across elements). Could you elaborate on the differences between your approach and transformers, both in terms of formulation and implications for model expressiveness or computational efficiency?\n\nA: This is a highly insightful observation. Indeed, the connection to Transformer-style architectures is deeply embedded in the design of our approach, and we appreciate the opportunity to elaborate on this.\n\nIn summary, there are two differences between DeepHalo and Transformers, both structurally and in modeling implications:\n\n- **Effect order control**:  \n  Transformers (e.g., in TCNet) implicitly allow interactions across all items, inevitably resulting in **full-order interactions** (order $|S| - 1$). In contrast, DeepHalo provides explicit layer-wise control, where each layer adds exactly one level of interaction (or $\\times 2$ with quadratic activation). This control is essential for interpretability, regularization, and tractability.\n\n- **Structural simplification**:  \n  DeepHalo eliminates the need for query/key/value projections and softmax attention, replacing them with fixed linear transformations and polynomial activations. This results in lower computational cost—linear in $|S|$, rather than quadratic—as well as a simpler and more transparent architecture.\n\nBelow, we briefly explain the motivation behind our model design to provide a more intuitive understanding of our model architecture.\n\nWe begin with the **featureless setting**, where modeling higher-order context effects reduces to polynomial regression over indicator functions that encode item presence. DeepHalo introduces a structured and recursive mechanism to capture **subset-level interactions** in this setting, conceptually analogous to the accumulation of interactions based on binary inclusion indicators.\n\nIn the **feature-rich setting**, the core challenge is to encode interactions among a subset of items using their feature representations, while strictly excluding information from outside the subset. This naturally recalls the concept of attention scores, typically computed from item feature pairs. However, when these scores are normalized via a softmax function, **the resulting attention weights entangle all item features**, thus violating the independence required for controlled halo modeling.\n\nFor instance, when evaluating the influence of subset $\\{A, B\\}$ on the utility of item $C$ within the offer set $\\{A, B, C, D\\}$, the effect should depend solely on the features of $A$, $B$, and $C$, and remain invariant to unrelated items such as $D$. The softmax operation introduces information from all items into the denominator of each attention weight, which naturally violates this principle.\n\nTo address this, we omit softmax normalization, and further observe that the dot-product attention mechanism is not essential for modeling decomposable utility functions. This insight motivates a simplification of the architecture, leading to the formulations presented in Equations (4) and (5).\n\nIn short, **while DeepHalo draws conceptual inspiration from Transformer-style aggregation, it diverges in architectural design to achieve interpretable, order-controllable, and computationally efficient modeling of contextual choice effects**. We thank the reviewer again for raising this connection and will add a discussion in the final version to highlight these points explicitly."}]

Author rebuttal:

Meta-review: This paper proposes a new method for context-dependent choice prediction the the HALO formulation of behavioral context effects. The paper is clearly written and the proposed method is interesting and novel. There was a long and fruitful discussion between the reviewers and the authors, which was helpful in addressing many of the reviewer's concerns, and shed light on the aspects of the paper that would likely be of interest to the NeurIPS community. The authors are highly encouraged to incorporate these conclusions and their new experiments into the final version of the paper.
Decision: Accept (spotlight)