Title: DeepHalo: A Neural Choice Model with Controllable Context Effects
Venue: NeurIPS 2025
Accepted: yes
Keywords: Choice Model

Preference Learning

Deep Learning
Abstract: Modeling human decision-making is central to applications such as recommendation, preference learning, and human-AI alignment. While many classic models assume context-independent choice behavior, a large body of behavioral research shows that preferences are often influenced by the composition of the choice set itself---a phenomenon known as the context effect or Halo effect. These effects can manifest as pairwise (first-order) or even higher-order interactions among the available alternatives.
Recent models that attempt to capture such effects either focus on the featureless setting or, in the feature-based setting, rely on restrictive interaction structures or entangle interactions across all orders, which limits interpretability. 
In this work, we propose DeepHalo, a neural modeling framework that incorporates features while enabling explicit control over interaction order and principled interpretation of context effects.
Our model enables systematic identification of interaction effects by order and serves as a universal approximator of context-dependent choice functions when specialized to a featureless setting.
Experiments on synthetic and real-world datasets demonstrate strong predictive performance while providing greater transparency into the drivers of choice.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces DeepHalo, a neural modeling framework designed to capture context-dependent choice behavior. Traditional choice models can fail to capture dependent utilities while recent context-dependent choice models are either restricted by interaction structures or lacking interpretability.  The paper first shows a utility decomposition with features and then introduces DeepHalo to capture decomposed higher-order effects. This novel architecture can bridge the previous gap by providing an expressive model that can both explicitly control the complexity of interactions (i.e., the order of interactions) and offer a principled interpretation of these context effects. The paper provides empirical validation on synthetic and real-world datasets (both featureless and feature-based), showing that DeepHalo achieves competitive or superior predictive performance compared to existing baselines while offering enhanced interpretability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: The paper is generally very well-written and organized. While building upon established concepts like utility theory, permutation equivariance, and residual connections, the specific architecture of DeepHalo that aims to explicitly model and disentangle interactions layer-by-layer is novel and insightful. Its interpretability is very useful when we need to learn the users preferences in practice.\n\nWeaknesses:\n1) In experiments, I think it is essential to provide the number of parameters of each benchmark model to make sure the better performance of DeepHalo is from the design instead of the size.\n2) While many benchmark models are compared in datasets with features (Table 2), while in featureless datasets (Table 1), only a few models are tested. However, I think the models like FateNet, TCNet can also be applied to featureless cases.\n\nMinor comments:\n1) It would be helpful if a figure can be provided to illustrate the architecture.\n2) Typo: Line 79 \"Deep Sets [Zaheer et al., 2018], Deep Sets [Zaheer et al., 2018]\"\n3) While the std are provided in Appendix (e.g., Table 6, 7), it would be helpful if you also provide them in the main part (e.g., Table 1,2,).\n\nQUESTIONS:\n1. Could you provide the number of parameters for each benchmark model used in your experiments? \n2. The formulation of higher-order interactions in Equations (4) and (5) appears conceptually similar to the transformer architecture, with the key difference being that the attention mechanism is replaced by a linear transformation followed by an activation function (just like assigning equal attention across elements). Could you elaborate on the differences between your approach and transformers, both in terms of formulation and implications for model expressiveness or computational efficiency?\n\nLIMITATIONS:\nSee weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces an ML based context-dependent choice model. The issue is that the classical IIA assumption of choice models is often not met/questionable, and one perspective on addressing it is through modeling the “halo” effect. In this approach the authors consider using information from all alternatives when evaluating each single one, and considering their interactions. The paper shows the derivations of these interactions in a permutation equivariant deep neural network framework. Results show that the model generally outperforms the baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- Important modeling problem (although somewhat specific/niche)\n- Convincing methodological treatment, with only a few notation glitches\n- Successful implementation and validation with real data\n- Good choice of (ML related) baselines\n\nWeaknesses:\n- Poor choice of (classical DCM) baselines\n- Lack of training (computational) performance statistics. How fast/slow is it, compared with the other DNNs?\n- Some points are unclear (see below)\n\nQUESTIONS:\nI enjoyed reading the paper, and this seems lile an interesting research direction. However, it’s very unclear to me why this fits in the NeurIPS context. The methodological ML contribution lies practically exclusively in the modeling of interactions and equivariance, and in both cases, the contributions are not very novel. If the contribution relates to DCM behaviour modeling for the ML community, then some aspects should be considered (e.g. why not consider nested logit, mixed logit, LCCM? why classical MNL as baseline, which is the same for DCM literature as fully connect MLP is for deep learning (the most vanilla of vanilla…). \nDon’t take me wrong, I do think this is an interesting paper, but I’d see it as more appropriate for Journal of Choice Modeling for example…\n\nLIMITATIONS:\nSome clarifications:\n\n- The intro is good but some things are a bit unclear. For example, “featureless settings”?? What is that (I know what it is, but I did have my doubts in the beginning, and it would be nice to spare the reader of such trivial doubts)? \n\n- “For each j ∈S, define a utility function uj(T), which represents the utility of alternative j when the context is the subset T ⊆S\\{j}.” This is quite confusing. You are introducing the classical MNL, but also introduce the concept of context at the same time. Fine, but then, you refer to uj(T) (utility of j within the context?) in the text, while the MNL presented has uj(S). What’s the relationship here? \n\n- “Here, XT∪{j}∈Rdx ×J denotes the matrix formed by replacing the feature vectors not in the subset\nT ∪{j}with zero.” —> rephrase. At first sight, it almost implies that this matrix is only zeros (it’s not mentioned what to do with the features that ARE in the subset). I know this is obvious, but the cognitive load of a reader would thank the authors…\n\n- This sentence got me a bit confused: “Here, H denotes the number of interaction heads, analogous to the number of channels or attention heads, and thereby controls the diversity of the interaction patterns. “ It seems as though W^1 is projecting the z embeddings vectors to a (lower?) space, and that W^1 is shared by all alternatives (there is ONE W^1). Then, why not train the z with dimensionality h directly? What’s the benefit of W^1?\n\n-Section “4.1 Residual Connection for Large Choice Sets” is quite confusing: in what sense does polynomial aggregation improve computational efficiency? In fact, with larger S, then you will get extremely high degree polynomials, which will be prone to numerical errors. Perhaps the benefit is then that you can use polynomials to reduce number of layers (i.e. trade-off linearity with many layers with polynomials with less layers), but this is not clear in the explanation nor on the notation. \n\n- For section 4.2., if I understand well, e_S is a vector of 1s. If that is true, then u_j(e_S) is a sum of ones irrespeective of the polynomials, and thus totally S. This makes no sense, implying I’m misunderstanding something here… I do understand that, later on, given that we learn the q values (eq 9), this no longer applies (i.e. we get polynomials), and it makes intuitive sense.  \n\n\n- There’s something confusing with the meaning of \\union in equation 3. Classical interpretation implies that T \\union {j} is just the set T plus the element j. But It sort of implies that j is treated differently to the rest. Ok, you already index v_j (on the left of the equation), thus allowing you to differentiate j on the right hand side. At first time, that was totally fine. But then we get to equation 10, in particular v_j(T) + vj_(T \\union {k}), where you make vj_(T \\union {k}), which becomes ambiguous. I mean, I understand that you want to estimate the context effect of T and also consider the context role of k in interaction with T, but then what would be expansion (according to 3)? \n\n- About number of parameters in Fig 2, what are we talking about specifically? Dimensionality of embeddings? Of the phi functions? \n\n- \\bar{Z}_1 := 1/S…  I believe it should be |S|, since S is a set. \n\n\nThe beverage experiment is not very intuitive. The strongest effect seems to be that, if Pepsi is on the menu, then people strongly prefer 7Up or Sprite rather than Coke (versus having only 3 options? the entire set? Versus having a different cola instead of pepsi? Is the choice set the competing product pair+effect source set in each case? Is this featureless? What are the features?). Why? What’s the intuitive explanation? The examples in the intro/motivation section were very convincing, why not try them instead?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a neural discrete choice model with context effects that handles item features. They use a Batsell-Polking style decomposition of context effects, but parameterize that decomposition with neural networks. The model applies iterative layers to capture higher-order context effects. By using a quadratic activations, the authors also provide an approach that captures higher-order context effects with only logarithmically many layers. In experiments on synthetic data, the authors demonstrate how their approach can recover planted similarity effects and learn very high-order interactions with low depth. On real datasets, they demonstrate better held-out performance than existing context effect models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Clearly written paper\n2. Interesting approach, combining the Batsell-Polking approach with a neural network parametrization and an idea for efficiently capturing higher-order effects\n3. Solid experiments\n\nWeaknesses:\n1. Could be clearer what the advantages are of the proposed method vs existing neural context effect models\n\nQuality: good. Background and related work are thorough. Motivation and methods are good, baselines are pretty thorough and experiments are well done. \n\nClarity: There are a few points that can be clarified (see questions). Overall, though, the paper is clearly written. The title, though, is a bit generic, and could apply to several of the cited papers; I would suggest something that highlights the distinction of this approach vs those of Rosenfeld et al., Pfannschmidt et al, etc.\n\nSignificance: this is an important problem that has received significant attention, and the proposed method performs well in experiments.  \n\nOriginality: reasonable, applies a different approach to neural modeling of context effects. I have some questions about what distinguishes DeepHalo from existing neural context effect models (see below)\n\nMinor:\n- Deep Sets double cite in Line 79\n\nQUESTIONS:\n1. Section 4.1 should explicitly say that (8) lets you capture $\\ell$th order interactions with only $log_2 \\ell$ layers, and should explain why.\n2. Do you recommend using (8) in general? Are there settings where we should use (5) instead? Adding some guidance to the paper here would be helpful.\n3. The intro says DeepHalo is more interpretable than the approaches of [Wong and Farooq, 2021], [Pfannschmidt et al., 2022], [Wang et al., 2023, Peng et al., 2024]. Similarly, the results state \"while offering favorably improved interpretability compared to other deep neural network models.\" But using (10), can't we interpret context effects from any of the existing neural context effect models in the same way? What makes DeepHalo more interpretable?\n4. The experiments of [Rosenfeld et al., 2020] use item features too---despite 1.1 saying they focus on the featureless case. What's the advantage of DeepHalo over SDA? DeepHalo seems like a Set-Dependent Embedding approach [Rosenfeld et al., 2020; eq. (2)], right? \n\nQuestions 3 and 4 are why I gave a 4 instead of a 5, and I'm very interested to hear the authors' response.\n\nLIMITATIONS:\nThe discussion of limitations is limited. How does the sample complexity and need for hyperparameter tuning compare to existing approaches?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nHuman revealed preferences are context-sensitive: Human choices are influenced by the context (made up of the set of alternatives) within which they are taken. These context effects cannot be explained by humans rationally choosing the alternative with the highest utility. One potential explanation for this phenomenon is that humans maximize utility, but that each alternatives’ utility is context dependent (dependent on the other items in the set of alternatives).\n\nAs this requires utility functions which maps arbitrary-size sets to real utility values, the learning problem becomes exponentially more complex. Prior work has attempted to solve this by decomposing the utility function, and has attempted to make computation cheaper by limiting calculations to lower-order interactions between alternatives. In line with this prior work, this paper proposes a similar decomposition of utility. It then proposes a new neural network architecture, DeepHalo, that can tractably calculates the decomposed utility up to arbitrarily high orders of interaction. This is done using a recursive formulation of context embeddings, where higher-order embeddings are produced from lower-order ones.\n\nExperiments on synthetic data show the benefit of modeling higher-order effects with DeepHalo. On real choice data, the model’s predictions are shown to outperform baselines in terms of negative log-likelihood.\n\nSTRENGTHS AND WEAKNESSES:\nThe basic problem this paper tackles, modeling context effects, remains a very important problem in learning from human preferences. The amount of work that has been done in this area — and the paper does a very good job of covering the relevant literature here — shows how important it is. The writing of the paper is very clear, making it easy to understand the proposed model. The experiments are also excellent, covering a variety of datasets and comparing to a large set of baselines.\n\nMy main concern with this paper is novelty. As far as I can see, the specific decomposition of the utility function which DeepHalo uses mirrors FETA’s almost exactly. The proposed neural network architecture is a fairly straightforward variation on DeepSets. The authors claim that the model offers an increased level of interpretability. However, the notion of relative context effect in (10) used to interpret the model can be calculated with any model (including prior ones) that decomposes the utility of a choice set into contributions of subsets; the interpretability is therefore not derived from the DeepHalo architecture specifically.\n\nFurthermore, I’m not convinced that the proposed network architecture replicates the utility decomposition the paper aims for (line 189). It seems correct for pairwise interactions, but I do not see how the recursive structure proposed replicates this utility decomposition.\n\nMinor points:\n- Line 79: duplicate instance of “Deep Sets [Zaheer et al., 2018]\n- Lines 180-182. Misspelling of FETA and FATE. In the original paper the “E” stands for “evaluate” not “evaluation”.\n- At various points in the paper, the authors use $S$ both to indicate the set $S$ and the size of that set. This is needlessly confusing. It would be better to use $|S|$ for the size of $S$ consistently.\n\nQUESTIONS:\nEquation (3) includes the term $(-1)^{|T|-|R|}$, making each summand where the sizes of $R$ and $T$ differ by an uneven amount negative. I do not understand what this is supposed to achieve. What is the purpose of this?\n\nIn (3) you use $u_j(x_j, X_R)$. Here $u_j$ takes different arguments than in its previous uses. Is this the same function as defined in (2) (might I suggest writing it then as $u_j(X_{K \\cup \\\\{j\\\\}})$, or is this a different function $u_j$?\n\nFor Tables 1 and 2, are the differences in test-set Negative log-likelihood (statistically) between DeepHalo and TCNet statistically significant? If I am reading Tables 5 and 6 in the appendix correctly then their mean NLLs are within 2 standard errors.\n\nHow sensitive is DeepHalo to the amount of training data? Would you say it is more of less data efficient than the other baselines?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-07-03"}