Title: Hierarchical Optimization via LLM-Guided Objective Evolution for Mobility-on-Demand Systems
Venue: NeurIPS 2025
Accepted: yes
Keywords: Dynamic Hierarchical Optimization

LLM-Optimizer Interaction

Mobility-on-Demand Systems

Meta-Objective Design

Feasibility-Aware Adaptation
Abstract: Online ride-hailing platforms aim to deliver efficient mobility-on-demand services, often facing challenges in balancing dynamic and spatially heterogeneous supply and demand. Existing methods typically fall into two categories: reinforcement learning (RL) approaches, which suffer from data inefficiency, oversimplified modeling of real-world dynamics, and difficulty enforcing operational constraints; or decomposed online optimization methods, which rely on manually designed high-level objectives that lack awareness of low-level routing dynamics. To address this issue, we propose a novel hybrid framework that integrates large language model (LLM) with mathematical optimization in a dynamic hierarchical system: (1) it is training-free, removing the need for large-scale interaction data as in RL, and (2) it leverages LLM to bridge cognitive limitations caused by problem decomposition by adaptively generating high-level objectives. Within this framework, LLM serves as a meta-optimizer, producing semantic heuristics that guide a low-level optimizer responsible for constraint enforcement and real-time decision execution. These heuristics are refined through a closed-loop evolutionary process, driven by harmony search, which iteratively adapts the LLM prompts based on feasibility and performance feedback from the optimization layer. Extensive experiments based on scenarios derived from both the New York and Chicago taxi datasets demonstrate the effectiveness of our approach, achieving an average improvement of 16% compared to state-of-the-art baselines.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a hybrid method that integrates LLMs with exact solvers to address dynamic sequential decision-making in mobility-on-demand systems (which has an extremely high-dimensional search space and thus necessitates decomposition into high- and low-level optimization problems). The LLM in the method acts as a meta-optimizer that dynamically formulates high-level objectives via prompt-based evolution (employing the HS algorithm for the evolutionary process). The exact solver then addresses the optimization problems established at the corresponding time step. The method decomposes the ride-hailing problem hierarchically into assignment and routing subproblems, with LLMs bridging the cognitive gap between high and low levels by generating semantically-rich objectives that implicitly encode low-level routing dynamics.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper addresses a practical problem, which I consider highly valuable. On the one hand, it proposes a novel problem formulation for ride-hailing systems and provides a formal analysis of the associated search space complexity. On the other hand, this work is the first to integrate LLMs with exact solvers for sequential dynamic decision-making systems. The proposed method is training-free and overcomes the data inefficiency challenges inherent in reinforcement learning, as well as the rigidity of manually designed objectives in traditional decomposed optimization. A key feature is its ability to dynamically adjust the high-level optimization problem according to $\\mathcal{S}\\_t$ and $H\\_{t-1}$. The experimental results are also promising.\n\nMy concerns are as follows:\n- What is the rationale for using the HS algorithm? Can it be replaced by other EAs?\n- What are the specific objects designed by FunSearch and EoH in the experiments? Can the HS algorithm be substituted with FunSearch or EoH? I think that these two methods essentially also design functions.\n- The ablation study on the operators of the HS algorithm is missing.\n- Due to the use of LLMs, the worst-case runtime of the proposed method is difficult to predict.\n- I think the author(s) should clarify that all information contained in $\\mathcal{S}\\_t$ and $H\\_{t-1}$ positively contributes to the method's performance. Besides, could author(s) provide a specific example of $\\mathcal{S}\\_t$ and $H\\_{t-1}$?\n- Is it possible to dynamically adjust the parameters $\\alpha, \\beta, \\gamma$ to improve the performance of manual objectives? Moreover, the experiments do not specify the settings of these parameters.\n- Can multiple high-level objectives be pre-designed using LLMs and the appropriate objective be automatically selected at each time step? I think this could significantly reduce both runtime and the cost associated with using LLMs.\n\nQUESTIONS:\nNA\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-26"}

comments: SUMMARY:
This paper proposes an LLM-integrated hierarchical optimization framework for mobility-on-demand systems. Specifically, an LLM is employed to generate the objective function for the assignment problem, where the harmony search algorithm iteratively refines the generation prompts. The optimization problems at the first and second levels are then solved numerically. The proposed framework achieves higher performance compared to baseline approaches, including methods based on manually designed objectives, RL-based methods, and purely LLM-based methods, when evaluated on real-world taxi datasets.

STRENGTHS AND WEAKNESSES:
**Strengths:**
- This work proposes a novel method for integrating LLMs with optimization problems that can leverage any pre-trained LLM. There is considerable room to further enhance the proposed framework by fine-tuning the LLM.
- The proposed framework is described clearly and thoroughly, making it easy to reproduce.
- The empirical evaluations are comprehensive and promising, demonstrating that the proposed framework generally outperforms the baseline methods.

**Weaknesses:**
- The proposed algorithm relies on multiple heuristics, making it challenging to clearly evaluate how each component individually contributes to the overall performance.

QUESTIONS:
- Could smaller language models be fine-tuned to generate effective objective functions, even with fixed prompts? What is the main advantage of the proposed evolutionary framework compared to fine-tuning?
- Would it be impossible to design effective objective functions using rule-based methods? Is the use of LLMs necessary for generating high-quality objectives? How complex are the behaviors demonstrated by LLMs in the experiments?

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-07-03

comments: SUMMARY:
This paper proposes a training-free hierarchical optimization framework for mobility-on-demand systems, where a large language model (LLM) dynamically generates high-level assignment objectives. These are refined via a harmony search-based evolutionary loop and solved with a low-level optimizer for routing under constraints. Experiments on real-world taxi datasets show significant improvements over RL and LLM-only baselines.

STRENGTHS AND WEAKNESSES:
Pros:

1. The integration of LLMs and optimization in this paper is novel. 
2. The training-free scheme is practical, which requires no training data comprared with RL-based approaches.
3. The proposed approach is evaluated on large-scale, real-world taxi datasets, demonstrating consistent and significant improvements over SOTAs.



Cons:

My major concerns include **the introduction of HS**, and **the scalability of method**.

1. The motivation for using HS is weak, and the paper does not explore alternative evolutionary strategies to justify this choice.
2. The problem scale seems unrealistic, as real-world ride-hailing systems typically involve thousands of taxis, while the experiments are limited to hundreds.
3. The framework relies on frequent LLM queries, but the runtime cost and inference latency in real-time deployment are not analyzed.
4. Figure 1 is overly complex and unclear. the difference between WAY 2 and WAY 3 is not visually or conceptually well-distinguished.

QUESTIONS:
1. Can authors provide more clear movitation for selecting HS, instead of other ES? (see Cons 1)
2. Can authors provide experiment results over large scale of taxis and passengers, for example, thousands of taxis? (see Cons 2)

LIMITATIONS:
"yes"
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-14

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a hybrid framework that combines large language models (LLMs) with mathematical optimization to address dynamic decision-making, specifically for mobility-on-demand systems. The approach uses pre-trained LLMs to generate high-level objectives and employs mathematical optimization for detailed dispatching strategies, creating a system that doesn't require extensive training.\n\nThe method integrates LLM-driven semantic reasoning with solver-enforced mathematical rigor, dynamically updating objectives through prompts. This closed-loop mechanism balances exploration and feasibility, achieving state-of-the-art performance on urban mobility benchmarks. Experiments on New York and Chicago taxi datasets demonstrate the system's effectiveness in minimizing delays and maintaining spatial consistency. Detailed analyses, including ablation studies and hyperparameter sensitivity, provide comprehensive evaluation of the framework's components.\n\nOverall, the paper presents a creative integration of NLP and optimization methods, offering a new approach to complex real-time decision-making. By showcasing adaptability across different urban environments and demand patterns, the authors highlight practical applications and future research potential in combining data-driven priors with mathematical optimization for robust decision-making systems. The thorough experimental setup ensures reproducibility and extension of this promising approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\nThe paper stands out for its high quality and clarity, offering a well-documented experimental setup that ensures reproducibility. The integration of large language models with mathematical optimization techniques is clearly explained, making the methodology accessible to both experts and non-experts. By demonstrating practical applications on real-world datasets from New York and Chicago taxi services, the authors highlight the significant potential of their approach in urban mobility systems. This combination not only enhances the accuracy of predictions but also provides confidence intervals crucial for decision support systems.\n\nWeaknesses：\nDespite its strengths, the paper has some limitations, particularly regarding the applicability of its model in more realistic operational conditions. The current setup does not fully account for dynamic factors such as traffic congestion or road disruptions, which could limit the model's effectiveness in real-world scenarios. Additionally, while the proposed method is innovative, a deeper comparison with existing state-of-the-art techniques is lacking, leaving gaps in understanding its unique advantages and potential areas for improvement. Addressing these aspects would strengthen the paper’s overall impact and broaden its applicability.\n\nQUESTIONS:\n1、How do you plan to incorporate more dynamic factors, such as traffic congestion or road disruptions into your model? Could you elaborate on potential strategies or experiments that might address these limitations?\n\n2、Can you provide a deeper comparison with existing state-of-the-art techniques, highlighting both the advantages and limitations of your approach relative to others?\n\n3、Could you explain further how the LLM adapts through prompt-level feedback? Are there any plans to integrate reinforcement learning techniques to improve adaptability?\n\nLIMITATIONS:\nThe authors have partially addressed the limitations of their work, particularly regarding technical constraints such as the static adaptation mechanism of the LLM and the lack of fine-grained traffic dynamics in the current experimental setup. However, they have not sufficiently discussed potential negative societal impacts or broader ethical considerations associated with deploying such a system in real-world, high-stakes environments.\n\nFor instance, the use of LLM-generated prompts in decision-making systems could introduce biases or unintended consequences, especially if the generated objectives inadvertently prioritize certain user groups or regions over others. Additionally, overreliance on automated dispatching systems without proper human oversight may lead to safety concerns or reduced job control for drivers in mobility-on-demand platforms.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-16"}