Title: Personalized Decision Modeling: Utility Optimization or Textualized-Symbolic Reasoning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Personalized Decision Making

Symbolic Regression

LLM
Abstract: Decision-making models for individuals, particularly in high-stakes scenarios like vaccine uptake, often diverge from population optimal predictions. This gap arises from the uniqueness of the individual decision-making process, shaped by numerical attributes (e.g., cost, time) and linguistic influences (e.g., personal preferences and constraints). Developing upon Utility Theory and leveraging the textual-reasoning capabilities of Large Language Models (LLMs), this paper proposes an Adaptive Textual-symbolic Human-centric Reasoning framework (ATHENA) to address the optimal information integration. ATHENA uniquely integrates two stages: First, it discovers robust, group-level symbolic utility functions via LLM-augmented symbolic discovery; Second, it implements individual-level semantic adaptation, creating personalized semantic templates guided by the optimal utility to model personalized choices. Validated on real-world travel mode and vaccine choice tasks, ATHENA consistently outperforms utility-based, machine learning, and other LLM-based models, lifting F1 score by at least 6.5\% over the strongest cutting-edge models.  Further, ablation studies confirm that both stages of ATHENA are critical and complementary, as removing either clearly degrades overall predictive performance. By organically integrating symbolic utility modeling and semantic adaptation, ATHENA provides a new scheme for modeling human-centric decisions. The project page can be found at https://yibozh.github.io/Athena.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposed a neural-symbolic approach to personalized decision making. It employs a two-stage procedure:\n\n- Group-level symbolic utility discovery, a procedure that iteratively identifies a symbolic expression for utility prediction. This procedure terminates when a convergence criterion (defined as the best-worst optimality gap) is met.\n- Individual-level semantic adaptation, which iteratively updates the semantic template via a TextGrad-esque procedure to optimize for individual utility losses.\n\nIn sum, the first stage amortizes (or warm-starts) symbolic expressions per group, and then the second stage adapts this expression to individuals. Experimental results show impressive gains over purely LLM-based and predictive model baselines.\n\nSTRENGTHS AND WEAKNESSES:\n__Strengths__\n- __Quality and Clarity__: The writing is generally in good shape. The authors did a good job to present preliminary concepts, mathematical formalism, and their LLM implementations in detail. I appreciate the ablation study which shows that both components of the authors' approach are integral to the final performance.\n- __Significance__: As LLMs are increasingly embedded into daily usages, it is important to assess their capabilities and limitations. The authors have shown that LLMs are not yet capable as an end-to-end system for personalized decision making, and have proposed means to improve their performance. I believe this is an important area for future research\n- __Originality__: I'm a fan of neural-symbolic approaches wherein an LLM functions as a \"central hub\" to process and reason with textual / symbolic information, while delegating more quantitative operations to specialized packages. The authors have demonstrated a good application of this approach in personalized regression.\n\n__Weaknesses__\n- Extensive usage of LLMs is stated as a limitation to the approach, and it'd be good to report token consumptions (or monetary costs) associated with each run. \n- There are quite a few hyperparameters involved in this approach, such as the optimality gap $\\delta$ and the number of iterations $T, T^\\prime$. I could not find these numbers within the main text nor the appendix. It'd be good for the authors to report them as they're crucial to the understanding of the overall complexity of this approach.\n- The literature review section is overall well-done, but the section on \"LLM-based Decision-Making Models\" could've done a better job at interfacing with works that specifically addresses decision making via LLMs (right now it focuses more on general reasoning and instruction-following capabilities). For example, [1][2] studies decision making under uncertainty with LLMs, and [2][3][4] study their applications in supply chain optimization, personalized medicine, and autonomous driving respectively. \n- A benefit of symbolic regression is their interpretability. It would be good to discuss some of the expressions and explain why it makes sense.\n\n[1] DeLLMa: Decision Making Under Uncertainty with Large Language Models\n\n[2] STRUX: An LLM for Decision-Making with Structured Explanations\n\n[3] Large Language Models for Supply Chain Optimization\n\n[4] Leveraging Large Language Models for Decision Support in Personalized Oncology\n\n[5] A Language Agent for Autonomous Driving\n\nQUESTIONS:\nSee weaknesses, and also:\n- I'm not entirely familiar with the Swissmetro and the Vaccine datasets, but the authors have elected for a simplified version of them for budget control. Could the authors discuss whether these simplified variants maintain the core challenge of this problem, and what the normal problem instances would look like?\n- I'd be curious to see the scaling behaviors of this approach. Say you increase the number of iterations for both stages, would you get better results?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-05-31"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce ATHENA that proposes to integrate group-level symbolic regression of utility functions with individual level, LLM-powered semantic modeling to offer a more comprehensive and personalized view of decision making behavior. The key innovation with this approach is the integration of symbolic utility modeling with LLM-powered semantic reasoning to create more accurate and interpretable personalized decision models. ATHENA is evaluated in two decision making tasks--travel mode and vaccine uptake--, and results show that it outperformed traditional utility-based models, machine learning approaches, and other LLM based models.\n\nSTRENGTHS AND WEAKNESSES:\nThe presented idea would inspire other researchers in the field. While the two components, symbolic utility modeling and LLM-based semantic adaptation, both exist separately, their integration for personalized decision making is novel. This focus on individual-level adaptation aligns with every-increasing interest in personalized AI systems that can account for individual differences. \n\nBut the testing of ATHENA appears limited in scope because they only used two lightweight models: GPT-4o-mini and Gemini-2.0-flash. While these models are practical choices, the experiment section would be stronger if authors explained the rationale behind choosing these specific models. For example, GPT-4o-mini is considered to be strong at reasoning tasks, so it would be valuable to compare it against models known for different strengths. Testing with only two models provides limited view of how ATHENA's performance varies across different types of LLMs. Testing with at least one large model would help understand how model size affect performance, and whether the additional computational costs of larger models are justified by improved results.\n\nQUESTIONS:\nWhat was the reason testing with GPT-4o-mini and Gemini-2.0-flash? Was there a hard computational constraint on using large models? What other models were considered for testing?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of modeling human behaviour in complex domains where peoples idiosyncratic preferences may cause them to deviate from the global optima. As a solution to this problem the authors propose a two stage approach where first a global utility function is learned and then this function is tuned to each individuals unique characteristics. Both of these stages are implemented with the help of an LLM. They validate this approach on real world data showing this approach is performative and that both stages are important for success.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- This is a very interesting and original idea\n- The methodology seems sound \n- The paper is well written and easy to follow\n- The authors make a strong effort to ablate and interpret their results\n\nWeaknesses\n\n- The baselines compared against seem a little weak. Is there a reason why one couldnt train and end to end neural network?\n- It would have been nice to see an ablation on the LLM component of the model. How much is the model benefiting from an LLM versus some other sampling procedure in the evolutionary algorithm and is it worth the additional computation cost\n\nQUESTIONS:\nIn the experiment which tests the power of Individual-Level Semantic Adaptation alone, is it given a completely random starting point or the highest performing starting point out of a number of samples comparable to the amount that stage 1 generates? It seems unfair to attribute performance to step 1 that could be attributed to simply taking the best of N samples.\n\nOne of the primary goals of utility models, typically, is not to predict behaviour but to understand it. I see that in your fragment analysis some pieces of the predictive model are commonly reused but are any of the full models outputted by this procedure interpretable? Are the global functions interpretable?\n\nHave you tried any more powerful models? In particular I would be interested in whether reasoning models show improved performance.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes ATHENA (Adaptive Textual-symbolic Human-centric Reasoning), a framework for personalized decision modeling that combines symbolic utility discovery and LLM-powered semantic adaptation. The approach first uses LLM-augmented symbolic regression to derive interpretable group-level utility functions for predefined demographic groups, then refines these models at the individual level via TextGrad to capture personal preferences and constraints. Evaluations on transportation mode choice and COVID-19 vaccine uptake tasks show that ATHENA outperforms classical utility-based models, machine learning classifiers, and LLM baselines by at least 6.5% in F1 score. The method is notable for producing interpretable, human-centric models, but its reliance on manually specified demographic groupings limits its flexibility and raises questions about the robustness of these partitions across diverse domains.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n- **The paper is very well written** and easy to follow, with clear motivation, technical details, and experimental narrative.\n- **Visualizations are excellent**, effectively illustrating the two-stage ATHENA pipeline and the learned symbolic components.\n- The proposed framework combines symbolic utility modeling with LLM-driven semantic adaptation in a **conceptually elegant and technically sound way**.\n- **Premise is strong**: addressing the gap between population-level utility models and personalized decision-making is both timely and impactful.\n- **Performance is very good**, achieving ≥6.5% F1 score improvement over classical discrete choice models, machine learning baselines, and LLM-only approaches on real-world datasets.\n- The two-stage design (group-level symbolic utility discovery and individual-level semantic adaptation) is **well-motivated and validated** through ablation studies, demonstrating clear complementarity.\n- Produces **interpretable, human-centric decision models** with potential relevance for high-stakes applications such as public health policy and transportation planning.\n\n## Weaknesses\n\n- The framework relies on predefined demographic groups for the group-level symbolic utility discovery stage, which assumes these partitions are optimal and behaviorally meaningful. This is akin to providing the top-level splits in a decision tree without verifying their significance, limiting flexibility and potentially propagating bias.\n    - Is the baselines also provided the same grouping during the experiments?\n- There is no mechanism to automatically discover or validate significant groupings, which raises concerns about robustness in domains with high intra-group heterogeneity.\n- While the integration of symbolic regression and LLMs is novel, the individual components (symbolic regression, TextGrad) are adaptations of existing methods rather than fundamentally new techniques.\n- The paper could benefit from more qualitative analysis and takeaways from the synthesized group-level utility formulas. Since one of the main advantages of having an interpretable model is to derive actionable insights for domain experts (e.g., public health professionals), this feels like a missed opportunity to showcase the framework’s real-world utility beyond predictive performance.\n- (Minor) The paper has not discussed scalability of the method. When $T$ and $T'$ are treated as fixed hyper-parameters, and disregarding LLM inference time, is the methodology a constant time algorithm? Either a high level discussion or an empirical evaluation (including LLM invocation time) would be appreciated.\n\nI am happy to increase scores when the questions are well-addressed.\n\nQUESTIONS:\n- Are $T$ and $T’$ two hyper-parameters? What are the chosen hyper-parameters for the experiments? For Figure 3, is the iteration (x-axis) showing $T$ or $T’$?\n\n## Suggestions\n\n- The “Stage 1” and “Stage 2” annotations in Algorithm 1 currently look like part of the code; consider changing their style (e.g., italics or actual comment syntax) so they visually appear as comments and are easier to distinguish.\n- Style-wise, the `enumerate` on line 133 could have **shorter left padding** to improve visual alignment and readability.\n- The inline citations on lines 235–238 are attached directly to the previous word; there should be **one space separating the citations** from the preceding text for proper formatting.\n- Include at least **one qualitative example in the main text** showcasing a discovered group-level symbolic utility formula, along with **proper takeaways or insights**. This is crucial to demonstrate the value of having an explainable model for informing domain experts (e.g., public health decision-makers).\n- Provide more context on why **Cross Entropy (CE)** is included as a metric for evaluating ATHENA. Given its meaning in probabilistic models, explain its relevance to your setting and why it complements metrics like F1 and AUC.\n\nLIMITATIONS:\n- Computational Complexity: Textual gradient optimization is resource-intensive, limiting scalability to large populations.\n- Predefined Group Assignment: Requires a priori demographic groups, which may oversimplify heterogeneity and miss latent patterns.\n- Group Significance: Lacks a principled method to verify whether demographic partitions are behaviorally meaningful, risking weak or biased utility functions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}