CALM: Co-evolution of Algorithms and Language Model for Automatic Heuristic Design
Authors: Ziyao Huang
Weiwei Wu
Kui Wu
Wei-Bin Lee
Jianping Wang
Venue: ICLR 2026 Poster (2026)
Keywords: LLM
Algorithm Generation
Reinforcement Learning
Primary area: optimization

Abstract: Tackling complex optimization problems often relies on expert-designed heuristics, typically crafted through extensive trial and error. Recent advances demonstrate that large language models (LLMs), when integrated into well-designed evolutionary search frameworks, can autonomously discover high-performing heuristics at a fraction of the traditional cost. However, existing approaches predominantly rely on verbal guidance, i.e., manipulating the prompt generation process, to steer the evolution of heuristics, without adapting the underlying LLM. We propose a hybrid framework that combines verbal and numerical guidance, the latter achieved by fine-tuning the LLM via reinforcement learning (RL) based on the quality of generated heuristics. This joint optimization allows the LLM to co-evolve with the search process. Our method outperforms state-of-the-art (SOTA) baselines across various optimization tasks, running locally on a single 24GB GPU using a 7B model with INT4 quantization. It surpasses methods that rely solely on verbal guidance, even when those use significantly more powerful API-based models. The code is available at: https://github.com/whxru/CALM.

Published as a conference paper at ICLR 2026
CALM: CO-EVOLUTION OF ALGORITHMS AND LAN- GUAGE MODEL FOR AUTOMATIC HEURISTIC DESIGN
Ziyao Huang"? Weiwei Wu', Kui Wu', Jianping Wang! Wei-Bin Lee
City University of Hong Kong
City University of Hong Kong Matter Science Research Institute (Futian)
Southeast University *University of Victoria 'Hon Hai Research Institute
zhuang88 secitticittuu hk, weiweiwu@sen wu@.eu. cn, wkui @uuicuvic.ca i anwange ci tyu. edu. hk, wei-bin. 1 lee foxconn.com
cn,
ABSTRACT
Tackling complex optimization problems often relies on expert-designed heuris- tics, typically crafted through extensive trial and error. Recent advances demon- strate that large language models (LLMs), when integrated into well-designed lutionary search frameworks, can autonomously discover high-perforning heuris- ties at fraction of the traditional cost. However, existing approaches predomi- nantly rely on verbal guidance, i.c., manipulating the prompt generation process, to steer the evolution of heuristics, without adapting the underlying L.M.. We pro- pose hybrid framework that combines verbal and numerical guidance, the latter achieved by fine-tuning the LLM via reinforcement learning (RL) based on the quality of generated heuristics. This joint optimization allows the LLM to co- evolve with the search process. Our method outperforms state-of-the--e art (SOTA) baselines across various optimization tasks, rumning locally single 24GB GPU using a 7B model with INT4 quantization. It surpasses methods that rely solely verbal guidance, even when those significantly more powerful API-based models. The code is available at: https://q com/ http///wxuu/////
INTRODUCTION
Complex optimization problems are prevalent in real-world applications, including logistics (Duan ata 2022, Tresca all 2022), scheduling (Mihoub) all 2021, Palacio el all 2022), and trans- portation (Dahmanı et al, 0I22 Pereira clll, 202T). Traditionally, solving these problems relies heavily on manually crafting high-quality heuristics, a labor-intensive process requiring substan- tial expert knowledge. Given the limitations of this manual approach, Automatic Heuristic Design (AHD) emerged to streamline heuristic generation. Nevertheless, classic AHD approaches like Ge- netic Programming (GP) (Burke et all 2009) still depend significantly human-defined problem- specific components, limiting the search space and flexibility.
Recently, the advent of Large Language Models (LLMs) has introduced promising avenues for AHD by employing LLMs as heuristic generators and evolutionary computing (EC) techniques as a search framework. In this paradigm, heuristics generated by LLMs are iteratively evaluated through a pre- defined simulation framework, and superior heuristics inform subsequent generation prompts, thus creating feedback-drig driven evolutionary loop (Lm al 20245). Nevertheless, existing LLsdd AHD methods predominantly keep the underlying LLM untouched and merely guide heuristic evo- lution via textual prompt manipulations, referred to as "verbal gradients" quently, these methods inherently neglect the opportunity of tuning and enhancing the generative capability of LLM based on the feedback from heuristic designs.
2024). Conse-
We propose Co-evolution of Algorithms and the Language Model (CALM) to capture this opportu- nity. CALM drastically differs from the state-of-the-art (SOTA) eLl RIZZR; etal 2iza:
Dat all, 2025; Zheng all (02)) by enabling the LLM to co-evolve alongside heuristic designs. This co-evolution is made possible by treating the heuristic generation process not only as a target of
*Corresponding author.

---

Published as a conference paper at ICLR 2026
pert
Heuristic
B
pert
pegt
LiNe
better houristic
Houristic
Collapse
response analysis
Evafuation
Evaluation Enw.
(selection)
Thought
(selection) perf based mets based
perf-hasee Diversity-based
Ideo Sp
novel
Code
Code te
prompt resporse
houristic
Crossover Simplify Inita
Crossover Simpitiiaii
Promppt
Tosk Description
Rrnnpr
GRPO
reword
Invold response
Reffect
Inject
(Replace:
fin tune
frozen)
Operotor Bose
learnable] LLM
Reward Function
Existing LLM-based AHD Methods
Our CALM Framework
Figure 1: Pipeline of existing LLM-based AHD methods (Romera Paredes et all 2024 Ye all 2024; Dat et all, 2023; Zheng et all (025) under fixed LLM and our new approach CALM that enables the co-evolution of L.LM in the iterative heuristic search process. New components are presented in bright colors.
optimization but also as a rich source of training data. As heuristics are continually proposed, eval- uated, and selected based on their performance, the evolutionary loop naturally produces abundant moonpt-reponse-perfoome performacfoeeeeeeeeee triplets. These data points are highly informative, as each heuristic's effectiveness provides implicit signal about the utility of the underlying generation process. By using this signal as feedback for reinforcement learning (RL), we can fine-tune the LLM, thereby applying what term "numerical gradients" to adapt the model itself. This co-evolution approach unlocks a new dimension of adaptability, allowing the LLM to internalize characteristics of success- ful heuristics and improve its future generations.
CALM one of the first LLM-based AHD frameworks that jointly optimize both the prompt gener- ation process and the LLM model itself, overcoming the limitations of fixed-model approaches. For prompt generation, CALM introduces a suite of evolutionary operators, including fine-graullarity mutation operators (injection and replacement) and diversity-aware crossover operator, that pro- mote meaningful and diverse heuristic variations while preserving structural coherence. Further- more, a simple yet effective collapse mechanism is developed to help escape the local optima. For model improvement, CALM employs a memory-effeiient RL algorithm GRPO (Shao et all 2022)) with a carefully designed reward function to enable efficient fine-tuning. Experimental results demonstrate that our new approach can discover heuristics that beat existing SOTA baselines (Lm stan, 20PER: Ye all 2024 Zhengg an, R0ZS),, while running entirely on local computer with single 24GB GPU, in contrast to prior methods that depend heavily commercial LLM APIs.
2 RELATED WORK
As approach centers fine-tuning L.MSs by RL for solving optimization problems, review relevant literature in both RL and I.LMs applied to optimization. Additional related topics, including LLMs for code generation and RL-based LLM fine-tuning, are discussed Appendix BB
RL for Optimization Problems. Existing RL-based methods for optimization can be broadly cate- gorized by the role the learned policy plays: (1) Instance-Leve' Solution Generator. Deep RL has been widely adopted to learn policies for solving specific optimization instances (K won at 4024 Pan 2022 Bi all 2024)). However, these methods differ fundamentally from LLM-based AHD methods, as they directly produce solutions rather than design the algorithms that generate them. The LLM--bssed AHD approach operates at a meta level, seeking learn the algorithmic structure that produces solutions. This distinction also applies to the broader class of Neural Combi- natorial Optimization (NCO) (Luo 2024 X et ctall 202E Smrel al. 2022: Zheng el al, 2023), where models are trained to directly solve instances. Moreover, NCO methods often require explicit adaptation to handle problem scales not seen during training, whereas our method generalizes more naturally to scales. (2) Heuristic Generator. Some RL-based methods target meta-level search to discover heuristics instead of instance-level solutions. For example, AlphaDev (Mankowitz et all 202%) learns to combine low-level operations to discover faster sorting algorithms, and [i et
2

---

Published as a conference paper at ICLR 2026
(2022) searches for high-performing metaheuristies from predefined algorithmic components. While having similar goals, these approaches rely heavily on engineeeeee building blocks, akin to tra- ditional AHD frameworks (Pillay and Qu, 2018: Sanchez- Diaz et all 202I; Burke et all, 2009). In contrast, LLMM-ased method reduces manual intervention by leveraging L.MSs to explore an open- ended heuristic space with minimal prior specification.
LLM for Optimization Problems. Studies in this area fall into two categories depending on how LLMs are employed: (1) Instance-Level Solution Generator. Several works (Abgaryan et all, 80ZE; Jiang et all, 2024; Wu et all, 2024) prompt LLMs with instance-specific inputs for direct solution gen- eration. LLM- based methods in this category focus on discovering reusable heuristics. Moreover, methods such as that proposed by Diang et al. (2122) and [W et all (2022) keep LLM parameters frozen, and Abgaryan et all (2024)) fine-tune the model using supervised labels from an existing solver (Perron and Furnon, 2024). In contrast, our approach requires no imitation dataset, enabling its application to problems lacking established solvers. (2) Heuristic Generator. LLMM-based AHD)
methods (L 202 CLal MI2E: Komera-Paredess et alll 20224 Luretal 207ER: etll
2024; Ltu 2024B; Dat all 2025: Zheng et all 2025; Novikov et 2025) repeatedly ingest in- formation about the current elite heuristicstypically their natural- natural-language descriptions, source code, and performance scoresand, via fixed prompt templates that mimic genetic operators, produce new candidate heuristics. Those candidates is fed back into the prompt, forming an evaluategenerate loop that continues until the evaluation bud- get is exhausted. Additionally, some recent studies have also explored reduction techniques (T'aachi of al 2025),, trajectory- -bsed analysis (Yang et al. 2025), multi-objective optimization (Yao ct al 2025), to further enhance AHD, Wu et all (2025) have examined how to abstract core components from elite heuristics and combine them with LLM M-baedd fitness prediction for AHD. However, prior work keeps the LLM static. Our approach improves this by continuously fine-tuning the LLM us- ing peappp--ssspons---rrrrrmeee generation. Notably, there are concurrent explorations on fine-tuning LLMs for AHD (Surina et all, 2025: Lu al (0ZE). These studies provide valuable insights into how preference-basee tuning methods such as DPO (Ratarloy et all (022)) can improve heuristic discovery. Our work adopts different approach by employing the score-based RL algorithm (Shao et all 2022)) to fine- tune LLMs for AHD, and further introduces specialized designs such as fine-granularity operators to enhance the fine-tuning process through prompt manipulation.
then executed and evaluated, and the resulting feedback
tuples from the evolutionary process, enhancing future heuristic
fine-
PRELIMINARY
3.1 LLM-BASED AHD
Let P be problem with input space I and solution space S, and let heuristic be function
h : I - S. Given a training set D C and an objective f S + R (lower is better), the performance of heuristic is g(h) heuristics. The objective of AHD is to identify the optimal heuristic within this space, i.e., h² magea ghh).
f===[[=(h(z)) Let H denote the space of all feasible
LLM-based AHD is AHD where LLM serves as a heuristic generator. In practice, the LLM is charged with designing the core decision function of solver. For example, tasks like the Travel- ing Salesman Problem (TSP) or the Capacitated Vehicle Routing Problem (CVRP), an LLM-based AHD method might generate a function, which selects the next city to visit constructs an edge- desirability matrix to guide solution search within an Ant Colony Optimization (ACO) framework.
3.2
GRPO
GRPO (Shao et al. 00240) is a recent RL algorithm that has proven effective in training LLMs, as evidenced by its application in models such as DeepSeek-Ri. GRPO starts from initial model Ty and a reward function denoted by ro(4, o) that maps the prompt & and the generated response scalar. At the beginning of each training round, it snapshots Ty as a reference model Ryet- Then, it split all task prompts into multiple batches. When training for each prompt batch Dr. first snapshots no as Told- For each task prompt g E Dz, samples a group of responses {o,}{1 ~ To.a and computes rewards = (r: = (g 0,) for each prompt-response pair. Subsequently, it computes the advantage A. for each token / in response as the normalized reward
3

---

Published as a conference paper at ICLR 2026
"y-mmeanrr)/sstiiee The model parameters 0 are updated by maximizing the following objective
function:
JCRPO(() B
lo:1 min (PaeAr. (fr,, e) A..] DKL
NO [edl (1)
G
where e and 3 are vergence term is computed using unbiased estimator (Schulmani 2020) with guaranteed positivity. GRPO uses the group mean reward as a baseline to eliminate the need for an auxiliary value network, thereby reducing memory requirements. Additionally, the clipping mechanism combined with KL divergence regularization ensures stable and conservative updates.
fü, = ne (o t g. 0z <t/ (or, e oi,ct)), and the KL di-
4
METHODOLOGY
To explore the benefit of RL-bbased fine-tuning for discovering higher-quality heuristics in LLM- based AHD, we introduce CALM, novel framework that integrates both verbal and numerical guidance in evolutionary heuristic search. As shown in Fig. M CALM maintains a pool of heuristics, each with its own idea, code, and performance. At every round, CALM draws feasible evolutionary operator to produce a new prompt Subsequently, G' responses are sampled from the local LLM To. which are then evaluated. Based on the evaluation results, rewards are assigned to each response for GRPO to train the LLM, and new feasible heuristics are added to the pool. Consequently, CALM returns the best-so-far heuristic after running rounds. Next, we elaborate on the critical techniques in CALM: prompt generation, collapse mechanism, and the reward function.
4.1 PROMPT GENERATION
CALM provides several evolutionary operators: injection, replacement, crossover, simplification, and initialization. Prompts are predominantly generated by the selected operator and heuristics sampled from maintained pools. The initialization operator is an exception, as it does not require heuristics from the pool. Next, we elaborate on the heuristic sampling method and operators
Heuristic Sampling Method. The heuristic sampling approach varies for the crossover operator, details of which will be provided when introducing this operator. For the remaining operators, i.e., injection, replacement, and simplification, the heuristics are selected based on their performance rankings like all 2024a). Specifically, the probability of sampling heuristic h is inversely proportional to its rank in the current pool (i.e., proportional to 1 1/rank,(h) Heuristics ranked below a threshold, defined as the population size, are assigned a probability of zero.
Fine-Granularity Mutation Operators: Injection & Replacement. GRPO assigns an advantage score to each token based on the relative reward of the full response compared to others from the same prompt. This means each part of heuristic is encouraged
penalized depending the quality the whole. However, heuristic performance can shift dramatically with changes to even a single sub-component, making uniform treatment of all partsin terms of gradient directionunreliable..
While cumulative gradient updates can correct misattributed rewards or penalties for the same token appearing in different responses, aim further boost this process. To this end, introduce two novel operators that enable more precise control over hcuristic variations. These operators encourage the LLM to retain more common parts while introducing meaningful modifications the input heuristic (See Appendix E for examples). Consequently, GRPO is expected to more effectively identify the contribution of individual structural changes. The two newly designed operators are:
Injection. Given an existing heuristic, the injection operator prompts the LLM to incorporate new component into it. Additionally. concise description of the new component must be included in the response. All component descriptions are stored, and subsequent applications of the injection operator require the L.LM introduce components distinet from those previously saved, promoting diversity in generated heuristics. Compared to prior LLM-based AHD methods (Zheng all 2025
'The complete algorithm and prompt details are in Appendix O and respectively.
4

---

Published as a conference paper at ICLR 2026
Lu (IZZE): (I) To explore under-explored heuristic designs, CALM provides the code of only one base heuristic together with compact summaries of components that have already been explored, whereas prior methods typically supply the full code of multiple existing heuristics to prompt the L.L to produce a different one. This design allows more references to be accommodated LLM's context window. (2) Saved component descriptions are globally accessible and not limited to the currently sampled heuristics; (3) Prior methods often require entirely new heuristics, while our approach focuses on more granular modifications; (4) When the number of heuristics is below the population size, the sampling probability of the injection operator is increased to encourage exploration in the phase of population expansion.
the
Replacement. Given an existing heuristic. the replacement operator prompts the LLM to rewrite an existing component under a specific instruction. There are three distinct instructions, and each time the replacement operator is applied, one is randomly sampled for the given heuristic. While the "rewrite syper-parameter instruction also present in prior studies (L. et all 20Z45; Zheng et al,
(025), CALM introduces two novel instructions: (1) Rewrite instance-independen decision rule as an instance-dependen oneto improve the heuristics adaptability to varying problem contexts; (2) Rewrite fragment that assigns equal near-equal credit to all candidates as that differenti- ates credit based on contextual performanceto encourage more effective prioritization and refined decision-making.
Diversity Aware Crossover. To balance exploitation and exploration, each crossover invocation randomly chooses between (1) performancebased: sample both parents by performance rank; and (2) diversitybased: sample the first parent he,1 by performance rank and the second from all re- tained heuristics with probability inversely proportional to diversity rank (larger diversity is better). Specifically, let idea sotetooke(())) denote the set of unique tokens in heuristic's idea, the diversity /////iea_tokee((()))) This hybrid mechanism ensures that at least one parent heuristic is of high quality, while the second parent either high- performing or structurally novel. The diversity- aware selection expands the evolutionary search space and leverages underutilized heuristics, potentially unlocking novel strategies that might oth- erwise be overlooked due to suboptimal early performance. More discussions are moved to Ap- pendix E
is: div(he.1, h) = lidea_ |iddeatokenn ttaaatoooohhh) \ idea_token(
As heuristic structures grow increasingly complex through repeated ap- plications of injection, crossover, and replacement, there is a risk of accumulating redundant or unnecessarily verbose components. The simplification operator counterbalances this tendency by prompting the LLM to produce a more concise and effective version a given heuristic.
Simplification Operator.
Initialization Operator. In cases where there is heuristic in the pool (e.g., initial/seeding function is provided), this operator is invoked to prompt the LLM to generate new heuristics.
4.2 COLLAPSE MECHANISM
Why to Collapse. A key reason LLM-based evolutionary heuristic search can succeed is that prompts containing better-performing heuristics tend to guide the LLM toward generating even stronger ones. This creates a self-reinforcing feedback loop, gradually evolving a population of increasingly effective heuristics. However, this process can also lead inbreeding and premature convergence: over time, the population becomes dominated by minor variations of the current best- performing heuristic. When this state persists without meaningful breakthroughs, the search risks becoming trapped in local optimum, a classic challenge in evolutionary computing (Eshelman 1991).
How to Collapse. As remedy, CALM introduces a proactive collapse mechanism that resets the search process when it detects stagnation, allowing the system to escape local optima and reinitiate meaningful exploration. Specifically, when the search has plateaneddcharacterized by prolonged lack of performance improvementwe reset the population by discarding all heuristics except two: the original seed algorithm and the current best-performing heuristic. These two retained heuristics jointly serve as the seed algorithms for the new search process, grounding it in past progress while freeing it from the genetic redundancy accumulated in the previous population.
When to Collapse. Once the heuristic pool reaches its target population size, CALM begins tracking stagnation using a no-breakthrough counter Cn, initialized to zero. This counter records the number
5

---

Published as a conference paper at ICLR 2026
of consecutive prompt roundseach involving G sampled responsesthat fail to yield globally supe- rior heuristic. If any sampled heuristic in a round surpasses all previous ones in performance, On is reset to zero; otherwise, it increments by one.
To escape local optima, CALM introduces a probabilistic collapse mechanism based on this counter. At the end of each round, collapse is triggered if: random (0, 1) Cn 5n < 1 controls the rate at which collapse probability grows, and C' is hard cap ensuring collapse happens by the C-th stagnation step the latest. To aid in hyperparameter selection, we further provide an analytical approximation for the expected number of rounds before collapse is triggered:
On. 2 C, where
Cn collapse, C
π
60_
260
This collision of risingprobability rule with fixed maximum fosters balance hetween giving the search plenty of room to improve and ensuring it doesnt stall infinitely. A detailed proof and discussion about the benefit of the mechanism can be found in Appendix G
4.3 DESIGN REWARD FUNCTION
The reward function assigns a score to each I.L.-generati response, enabling the RI. algorithm to update the LLMs parameters and progressively improve its outputs. In AHD, we aim for responses that yield feasible, novel, and high-performing heuristics. To guide this process, we adopt progres- sive scoring scheme that assigns increasing scores across the following categories: (1) infeasible responses that fail to produce valid heuristics, (2) duplicate heuristics offering new insights, (3) new heuristics, and (4) new high-performing heuristics.
For each invalid response, assign reward bounded below by scalar Finvalid E (-1,0). Rewards for valid heuristics are defined relative to this bound, ensuring that valid outputs always score higher.
For valid heuristics, performance serves as the primary learning signal. However, because the qual- ity of the generated heuristic is influenced by the promptparticularly its base heuristicswe avoid attributing full credit blame the LLM alone. Instead, reward improvements relative to the best base heuristic in the prompt, ensuring that learning reflects meaningful gains rather than prompt bias. Specifically, let H denote the set of base heuristics used to construct prompt y- and Nnew be the heuristic parsed from the LLMs output o. We define the top base heuristic as hr_base arg maX H gh)) and measure relative performance via:
(g(hnew) g(h, base) ( Ig (bg (ile_base)) | 0.1).
A A(haew: basease clip
(3)
Let 01, 02 E (0, 1) and on > a2, the reward function rg hnew, h4 base)) is then defined as:
(1 (invalid,
if 3h H s.t. g(h) ghheew) if g(hnew) < if g(Ntnww)) > ghe
re hnews by bese)
ar2iivalldd A(hnew: hy base),
(4)
11 A(hnew, by base),
Under the reward function above, the reward primarily determined by whether the new heuristic improves over the best base heuristic or not, with the relative performance gap further modulating the strength of the reward or penalty. When the generated heuristic is identical in performance to an existing base heuristic, small but consistent reward (a]Fiaalld is given to discourage trivial reproduction. If the new heuristic underperforms relative to the best base, scaled negative reward is applied, while genuine improvements yield strictly positive rewards starting from 1.
EXPERIMENTS
Implementation Details of CALM. We build CALM on Unsloth (Damel I Han and feam,, (0XX)) and employ an ANT4-quantized Qwen2.5-7B 7-I-nttuuttt model (Yang et all 2022)), fine-tuning just 1. 1.15% of its weights. INT4 compression cuts memory usage up 8CE versus FP32 but degrades precision.
According to Yang et al. (2022), performance ranks as follows: GPT-4o-mini Qwen2.5-Tutee Qwen2.5-14p I4--sruutt Qwenn.5-7BB-Inttruu Q The 14B and 7B

---

Published as a conference paper at ICLR 2026
Instruct models share the same architecture, the larger parameter count drives the 14Bs edge over the 7B, while quantization further reduces the 7Bs accuracy. Consequently, HPTT-4ommiiibase baselines retain clear advantage in raw accuracy over lean, resource-effiieet setup. More implementation details can be found in Appendix FI
Optimization Tasks. Existing LLM-based methods can demonstrate near-optimal optimal per- formance on some benchmark problems, such as TSP (Lau et all 4024 Lee el all 2024: Zheng etal, 2025) (aided by ACO solvers) and knapsack problem (KP) (Zheng all 2025), leaving little room for further improvement. Therefore, we focus on tasks that remain challenging for LLM-based AHD as follows: Online Bin Packing (OBP) problem and TSP under step-by-step construction task, CVRP and Orienteering Problem (OP) under an ACO search framework. Detailed problem descrip- tions can be found in Appendix
Baselines. To evaluate CALM, we compare its designed heuristics against the following base- lines: (1) hand-crafted heuristies such as Best-Fit (Kenyon, (995) for OBP, Greedy-Constrin
(GC) (Rosenkraniz et all (1777) for TSP, and ACO (Blum, 2005) for CVRP and OP; (2) Nerual Com-
binatorial Optimization (NCO) methods including POMO (Kwon (02() and DeepACO (Ye
et al. 2023); and (3) LLM-based AHD approaches like FunSearch (Romera-Paredes, et anl 2024).
EoH (Lu ct all 20245), , ReEvo (Ye et all 2022D, HSEvo (Dat al 2025). OpenEvolve (Shamm
2025) MCTS-AHD (heng et all 2025), and EvoTune (Surina all 2025). Notably, AlphaE-
volve (Novikov et all 2025) does not release its official source code. OpenEvolve, developed by
an independent group, is one of the most popular open-source reimplementations of AlphaEvolve. To ensure fair comparison, we align CALM and all LLM-based AHD baselines with consis- tent settings, including shared seed heuristics that are directly adopted from Zheng et al. (2125), identical training datasets for evaluating heuristic performance, and comparable evaluation budgets- specifically, 1,000 heuristic evaluations for baselines and a fixed budget of 2,000 LLM queries for CALM across all tasks except OBP. Notably. prior AHD methods typically conduct 2,000 heuris- tic evaluations using over 4,000 queries for OBP, whereas CALMM operates under a fixed budget of 2,000 queries.
5.1 OVERALL RESULTS
OBP. We train and eval- uate CALM on the same dataset used by Zheng et all (2025), which includes four training instances at vary- ing scales and five testing instances spanning six dif- ferent scalestwo of which are out-of-domain and not seen during training. Re- sults in Table II show that CALM consistently outperforms all baseline methods in terms of aver- age optimality gap across the full test set. It can achieve superior perfor- mance on out-of-domain and in-domain scales. Re- markably, CALM achieves a zero gap in set lk_500, in- dicating exact optimal solutions at that scale.
Table Average optimality gaps of heuristics for OBP over three runs. All methods are trained and evaluated on the same datasets as Zheng et all (2025), with gaps measured relative to the lower bound by Martello and Torh (I19))). Test sets whose scale matches the train- ing distribution are underlined. Format: 1k_ 100 denotes instances with 000 items and bin capacity of 100.
Online Bin Packing (OBP)
Test
Ik: 100
500
5k 100
5k 5..000
10k_ 100
10k_500
Avg.
Best Fit First
4.77%
0.25%
4.31%
0.55%
4.05%
0.47%
2.40%
5.02%
0.25%
4.65%
0.55%
4.36%
0.50%
2.56%
LLMbased AHD: GPT-4o-mini (w/o, GRPO)
FunSearch
2.45%
0.66%
1.30%
0.25%
1.05%
0.21%
0.99%
EoH
2.69%
0.25%
1.63%
0.53%
1.47%
0.45%
1.17%
ReEivo
3.94%
0.50%
2.72%
0.40%
2.39%
0.31%
1.71%
HSEvo
2.64%
1.07%.
1.43%
0.32%
1.13%
0.21%
1.13%
OpenEvolve
4.84%
0.25%
4.28%
0.55%
4.07%
0.47%
2.41%
MCTS-AHD
2.45%
0.50%
1.06%
0.32%
0.74%
0,26%
0.89%
(Ours)
2.78%
0.29%
0.83%
0.28%
0.50%
0.24%
0.82%
LLMbused AHD: (wwen2.5-7B-lnstuutt-NN (w/. GRPO)
EvoTune CALM (Ours)
4.67% 2.55%
0.25%
4.23%
0.55%
4.11%
0.60%
2.40%
0.85%
0.17%
0.56%
0.14%
0.71%
TSP. CALM is trained on the same dataset used by Zheng et all (21255):: a training set of 64 TSP instances with N 50 nodes and three test sets of 1,000 instances each at N = 50, 100, and 200. As shown in Table 2 CALM-constructed heuristics outperform all LLM-based baselines on both out-of-domain test sets and achieve the second-best based result the in-domain set. Notably, at the largest scale, CALM surpasses the NCO baseline POMO, which requires sea-s training.
7

---

Published as a conference paper at ICLR 2026
Table 2: Performance TSP, averaged over three runs. Methods are evaluated on three test sets of 1,000 instances each, using the same training and testing datasets as by Zheng et all (2025). In- domain scales (i.idd to training) are underlined. Optimal tours are from LKH (Lin and Kernghan, 1973). Best LLM-based results are shaded, overall best in bold.
Traveling Salesman Problem (TSP)
N=50
N=100
N=200
Methods
Obj-
Gapl
Obj.4
Gapl
Obj.4
Gapl
Optimal
5.675
7.768
10.659
GC
6.959
22.62%
9.706
24.94%
13.461
26.29%
POMO
5.697
0.39%
8.001
3.01%
12.897
20.45%
LLM-based AHD: GPT-3.5-turbo (w/o. GRPO)
FunSearch
6,683
17.75%
9.240
18.95%
12.808
19.61%
EoH
6.390
12.59%
8.930
14.96%
12.538
17.63%
MCTS-AHD
6.346
11.82%
8.861
14.08%
12.418
16.51%
LLM-based AHD:
GPT-4o-mini (w/o. GRPO)
FunSearch
6.357
12.00%
8.850
13.93%
12.372
15.54%
EoH
6.394
12.67%
8.894
14.49%
12.437
16.68%
OvenEvolve
6.281
10.68%
12.25%
12.148
13.96%
MCTS-AHD
6.225
9.69%
8.684
11.79%
12.120
13.71%
CALM (Ours)
6.273
10.54%
8.691
11.88%
12.104
13.56%
LLM-based AIID:
Qwen2,5-7B-Inm 100 (w. GRPO)
EvaiTuee
6.267
10.43%
8.777
12.99%
12.429
16.60%
CALM (Ours)
6.244
10.04%
8.668
11.58%
12.088
13.41 %
CVRP. CALM is trained on 10 instances in (Zheng et al, 2025) with N 50 nodes using the ACO framework, and evaluated on three test sets of 64 instances each at N = 50, 100, and 200, following the same generation protocol. During both training and testing, the number of ants and iterations is fixed to 30 and 100, respectively. As shown in Table B. CALM consistently outperforms all LLM-based baselines across test sets, including both the in-domain and out-of-domain ones.
OP. CALM is trained 5 OP instances with N = 50 nodes using the ACO framework and evaluated on three test sets 64 instances each at N = 50, 100, and 200, following the generation protocol fixed configuration of 20 ants and 50 iterations. As reported in Table & CALM consistently outperforms all other LLM-based baselines on the out-of-domain scales. As for the in-domain scale, it still outperforms EoH and the most recent approach, MCTS-AHD and EvoTune.
in HSEvo (Dai et all 20225). Both training and testing
Table 3: Performance of ACO-based heuristics on CVRP and OP, averaged over three runs. All methods evaluated three test sets of 64 randomly generated instances each, following the setup in (Zheng et all, (0255) and (Dat all 2025), respectively. Optimal solutions are approximated using DeepACO with significantly more ants and iterations than those in the baseline configurations.
CVRP
N=50
N=100
N=200
N=50
N=100
N=200
Methods
Obj.
Gap!.
Obj-
Gapl.
Obj.
Gap.
Obj.7
Gapl
Obj.1
Gapl
Obj.1
Gapl.
Optimal
8,88
149332
27.159
19,867
36.392
63.380
ACO
18.581
109.05%
30.107
101.63%
37.590
40.69%
13.354
32.69%
24.131
33.69%
37,586
40.69%
LLM based AHD: GPT-4o-mini (wlo. GRPO)
EoH
16.953
13.54%
30.314
ReEvo
9.558
7.54%
16.350
9.50%
29.219
7.58%
15,103
23.98%
30.523
13%
53.807
15.10%
HSFvv
9,431
6.1%
16.396
9.81%
29.520
8.69%
15.082
24.08%
30.454
16.32%
53,260
15.97%
OpenExolve
27.95%
23.36%
MCTS-AHD
9.372
5.44%
15,974
6.98%
28.434
4.70%
14,847
25,27%
10.163
12%
53.024
16.34%
CALM (Ours)
9.101
5.81%
16.046
7.46%
28.713
5.72%
15.017
24.11%
30.294
16.76%
53.098
16.22%
I.L.-based AHD::
wtw.:/..-7--nnnteet--- (w/. GRPO)
EvoTune
9,405
5.82%
15.975
6.98%
28.823
6.13%
15.053
24.23/%
29.743
18.27%
50.199
20.32%
CALM (Ouns)
9.228
3.83%
15.745
5.44%
28.230
3.95%
15.054
24.22%
30,778
15.43%
55.406
12.58%
5.2
DISCUSSION
Efficacy of our verbal gradient. For each problem instance, we further evaluate the design of our verbal gradient in isolation (i.e., without GRPO) by (1) switching the backend to the GPT--4omiiii
8

---

Published as a conference paper at ICLR 2026
API, (2) setting G = 1, and (3) using T 4000 for OBP and T 2000 for all other tasksmatching the query budgets of prior LLM-based AHD methods. As shown in Tables D- this API-based variant of CALM delivers performance on par with superior to the recent MCTS-AHD approach: achieves the lowest optimality gaps on the 5k_100 and 10k_100 100 OBP datasets and ranks second on average across all OBP test sets, matches MCTS-AHD and outperforms all other baselines on every CVRP test set, consistently surpasses MCTS-AHD on all OP instances, and closely tracks MCTS- AHD on TSP at N 50 and 100 while outperforming all non-MCTS baselines those scales and even surpassing MCTS-AHD at N 200. These results demonstrate that, even without RL or advanced techniques such as reflection eLa 2024: Dat et all 2022) and tree search (Zheng eLah R025), CALMs verbal guidance mechanism remains highly effective, placing the API-based CALM firmly within the top tier of existing LLM- AHD methods.
Power of RL. We have tested the performance of CALM without the GRPO algorithm and under many ab- lation settings. As shown in Table ELl results demonstrate that disabling the GRPO module causes the largest drop in performance across near all ablations. In other words, The re- inforcemenntearriing component has the most significant impact on over- all performance among all ablation settings. Morever, as illustrated in Ta- ble U-G with GRPO and our custom reward, the performs the GPT-40-minibused heuristic. We have also visualized the training curve in Figure Results show CALMs heuristics lag earlylikely due to GPT-4o-minis head startbut GRPO adapts the LLM, its heuristics converge and outperform all baselines. This suggests the transformative power of RL in enhancing AHD.
20.0
15.0
17.5
Eofl ReEvvo MCTS AHD
14.5
CALM
15.0
MCTS-AHD
8
EvoTune
14.0
EoHl
ReEvoo
10.0
13.5
Evoture
500
1000
1500 2000
0
500
1000
1500 2000
# LLM queries
# LLM queries
(a) CVRP.
(b) OP.
Objective score of the best heuristic in training averaged over 3 runs (std. dev. shaded).
Figure 2:
heuristic not only closes the gup but actually out-
Impact of reward design. Our feasiblere- sponse reward allocates credit by compar- ing each generated heuristic against its parent(s), rather than attributing full ward or blame solely to the LLM. We evaluate two alternative schemes (keep- ing the infcasibleresponse penalty un- changed): (i) performancehased reward, where a feasible heuristic receives a posi- tive reward proportional to its performance relative to the seed algorithm; and (ii) the (0.5 Frand: 1)improvement reward, which assigns reward if the new heuristic out- performs all parent or baseline heuris- ties, and 0.5 frand otherwise. Both al- ternatives remove the trivialreproduction penalty and mitigate the performance bias present in Equation (2)). As Table A demon- strates, neither variant beats our original design: the performancebased scheme un- derperforms even the noRL baseline on the OP problem, while the [0.5 1}- improvement strategy delivers closer but still inferior results compared to pro- posed reward function. This confirms the effectiveness of our original reward design.
Table 4: Optimality gap under ablation settings for problem OBP and OP averaged over three runs.
Method
OBP
OP
CALM (local, w/ GRPO)
0.71%
17.41% %
CALM (API, w/o GRPO)
0.82%
19.13%
RI-based Fine-tuning
local, w/o GRPO
1.78%
19.89%
rewE (0.5rivalid, 1}
1.04%
17.44%
rew=performance
1.24%
21.30%
Collapse Mechanism w/o Collapse
0.98%
19.57%
60 = 0.0005. C = 15
0.77%
18.31%
6g 0.005, C = 15
1.93%
27.22%
6u 0.0005. C' oc
0.96%
19.50%
6u 0.005, C. oc
0.98%
18.38%
Operators
w/o diversity w/o crossover
1.05%
19.44%
0.88%
18.49%
w/o injection
1.11%
18.68%
w/o replacement
1.20%
17.57%
w/o simplification
1.35%
19.45%
Impact of collapse. We examine the impact of the collapse mechanism by analyzing the heuristics produced by CALM both without collapse and under various hyperparameter configurations that influence when collapse is triggered. As shown in Table g incorporating the collapse mechanism

---

Published as a conference paper at ICLR 2026
generally enhances the heuristic search process. An exception arises in the configuration with the strictest tolerance for not discovering a breakthrough heuristic (i.e., when őy = 0.005 and C 15). A detailed analysis of the evolutionary trajectory under this setting reveals significantly reduced number of breakthroughs. In one run on the OP problem, no breakthrough heuristic was identified af- ter the 132nd L.M query. These findings suggest that setting reasonable tolerance for the absence of breakthroughsballncingg patience with the benefits of early stoppingis important for supporting a more effective evolution.
Impact of operators. We evaluate each operators contribution by measuring CALMs performance with that operator removed (Table H. Results show that all operators positively impact heuristic qual- ity. Crossover, injection, and replacement are similarly criticalremoving any one notably degrades performance in either OBP or OP. Among all, removing simplification causes the largest drop in both tasks, likely because it uniquely reduces redundancy and curbs complexity, counterbalancing other operators that tend to increase heuristic length. Moreover, when crossover is applied without diversity-based selectionusing only performance-basee samplingCALM performs worse than with no crossover at all, highlighting the importance diversity awareness the most-used operator.
Additional Experimental Results. Additional experimental results are presented in Appendix MI including a detailed breakdown of running time, the effeets of fine-tuning and foundational model choices, performance more challenging OBP instances, scaling behavior, statistical significance (p-values), sensitivity analyses of reward-function hyperparameters, and the set of elite heuristics discovered.
CONCLUSION
This paper introduces CALM, the first framework to marry prompt evolution with on-the-fly LLM adaptation for AHD, freeing it from the constraints of fixed-model approaches. Rumning entirely single 24 GB GPU with a compact foundation model, CALM autonomously uncovers heuristics that outmatch SOTA API-based baselines across various challenging optimization scenarios. Moreover, even without the power RI, CALM matches or exceeds prior best results using the same LLM API, demonstrating the potency of our verbal-gradies designs. In the future, expect that scaling CALMs paradigm to larger models and extended post-training could further push the frontier of automated algorithm discovery.
ACKNOW.EDGMENTS
The work is supported in part by project from Hong Kong Research Grant Council under GRF 11219624, National Natural Science Foundation China under Grant No. 62572120, and the Natu- ral Science Foundation of Jiangsu Province under Grant No. BK20230024.
ETHICS STATEMENT
This work complies with the ICLR Code Ethics. The research did not involve human partici- pants animal experimentation. All datasets employed were obtained and used in accordance with relevant licensing and usage policies, ensuring no infringement privacy. No personally identi- fiable information processed, and experiments security risks. Throughout the study, we have taken deliberate steps to mitigate biases and avoid discriminatory outcomes. We are committed to transparency, reproducibility, and integrity both our methodology and reporting.
conducted that could pose privacy
REPRODUCIBILITY STATEMENT
We have provided all information necessary to reproduce the main experimental results of this work, sufficient to support its central claims and conclusions. In detail, the complete algorithm is pro- vided in Appendix C. the prompts used (including the system prompt, operator prompts, and task descriptions) are detailed in Appendix DL the experimental settings described in Section B and further elaborated Appendix HLl and the full source code, including the discovered heuristics, can be found at https: //gith/tttt com wwwru//uooooo///uooo. CALM.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Jiahui Duan, Xialiang Tong, Fei Ni, Zhenan He, Lei Chen, and Mingxuan Yuan. data-driven column generation algorithm for bin packing problem in manufacturing industry. arXiv preprint arXiv:2202. /2466, 2022.
Giulia Tresca, Graziana Cavone, Raffaele Carli, Antonio Cerviotti, and Mariagrazia Dotoli. Au- tomating bin packing: A layer building matheuristics for cost effective logistics. IEEE Transac- tions on Automation Science and Engineering, 19(3)1199-1613, 2022.
Bachir Mihoubi, Brahim Bouzouia, and Mehdi Gaham. Reactive scheduling approach for solving a realistic flexible job shop scheduling problem. International journal of production research, 59 (19):5790-5808, 2021.
Jessica Coto Palacio, Yailen Martínez Jiménez, Leander Schietgat, Bart Van Doninek, and Ann Nowé. A q-learning algorithm for flexible job shop scheduling in real-world manufacturing scenario. Procedia CIRP, 106:227-232, 2022.
Nadia Dahmani, Ines Sbai, Takwa Tiili, and Saoussen Krichen. On solving the 21-cvrp using an adaptive chemical reaction algorithm: postal transportation real-case. International Journal of System Assurance Engineering and Management, pages 1-25, 2024.
Rafael HM Pereira, Marcus Saraiva, Daniel Herszenhut, Carlos Kaue Vieira Braga, and Matthew Wigginton Conway. r5r: rapid realistic routing on multimodal transport networks with r in Findings, 2021.
Edmund K Burke, Mathew R Hyde, Graham Kendall, Gabriela Ochoa, Ender Ozcan, and John R Woodward. Exploring hyper-heuristic methodologies with genetic programming. Computational intelligence: Collaboration, fusion and emergence, pages 177-201, 2009.
Fei Liu, Tong Xialiang, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang. Zhichao Lu, and Qingfu Zhang. Evolution of heuristics: Towards efficient automatic algorithm design using large language model. In International Conference on Machine Learning, pages 32201-32223, PMLR, 2024a.
Haoran Ye, Jiarui Wang, Zhiguang Cao, Federico Berto, Chuanbo Hua, Haeyeon Kim, Jinkyoo Park, and Guojie Song. Reevo: Large language models as heurrshessistics with reflective evolution. In Advances in Neural Information Processing Systems, 2024.
Pham Vu Tuan Dat, Long Doan, and Huynh Thi Thanh Binh. Hsevo: Elevating automatic heuristic design with divereity-driven harmony search and genetic algorithm using 1lms. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 26931-26938, 2025.
Zhi Zheng, Zhuoliang Xie, Zhenkun Wang, and Bryan Hooi. Monte carlo tree search for comprehen- sive exploration in llmebased automatic heuristic design. In International Conference Machine Learning. PMLR, 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseckmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Bernardino Parede,, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, 625(7995):468-44 2024.
Yeong-Dae Dae Kwon, Jinho Choo, Byoungjip Kim, Iljoo Yoon, Youngjune Gwon, and Seungjai Min. Pomo: Policy optimization with multiple optima for reinforcement learning. Advances Neural Information Processing Systems, 33:21 28-21198, 2020.
Yuxin Pan, Yize Chen, and Fangzhen Lin. Adjustable robust reinforcement learning for online 3d bin packing, Advances in Neural Information Processing Systems, 36:51926-5199 2023,
Jieyi Bi, Yining Ma, Jianan Zhou, Wen Song, Zhiguang Cao, Yaoxin Wu, and Jie Zhang. Learning to handle complex constraints for vehicle routing problems. Advances Neural Information Processing Systems, 37:93479-93509, 2024.
11

---

Published as a conference paper at ICLR 2026
Fu Luo, Xi Lin, Zhenkun Wang, Xialiang Tong, Mingxuan Yuan, and Qingfu Zhang. Self Self-improved learning for scalable neural combinatorial optimization. arXiv preprint arXiv:2403.1956. 2024.
Pei Xiao, Zizhen Zhang, Jinbiao Chen, Jiahai Wang, and Zhenzhen Zhang. Neural combinatorial op- timization for robust routing problem with uncertain travel times. Advances in Neural Information Processing Systems, 37:134841-134867 2024.
Jingyan Sui, Shizhe Ding, Boyang Xia, Ruizhi Liu, and Dongbo Bu. Neuralgls: learning to guide local search with graph convolutional network for the traveling salesman problem. Neural Com- puting and Applications, 36(17):9887-976, 2024.
Zhi Zheng, Shunyu Yao. Genghui Li, Linxi Han, and Zhenkun Wang. Pareto improver: Learn- ing improvement heuristies for multi-objective route planning. IEEE Transactions on Intelligent Transportation Systems, (01:):1333--033 2023.
Daniel Mankowitz, Andrea Michi, Anton Zhernov, Marco Gelmi, Marco Selvi, Cosmin Paduraru, Edouard Leurent, Shariq lqbal, Jean- Saan-Bapiiste Lespiau, Alex Ahern, et al. Faster sorting algorithms discovered using deep reinforcement learning. Nature, 61887964):257-263, 2023.
Wenjie Yi, Rong Qu, Licheng Jiao, and Ben Niu. Automated design of metaheuristics using rein- forcement learning within a novel general search framework. IEEE Transactions on Evolutionary Computation, 27(4):1072-100:
2022.
Nelishia Pillay and Rong Qu. Hyper-heuristics: theory and applications. Springer, 2018
Xavier Sánchez-Díaz, José Carlos Ortiz-Bayliss, Ivan Amaya, Jorge M Cruz-Duarte, Santiago En- rique Conant-Pablos, and Hugo Terashima-Marin. A feattre-independden hyper-heuristic ap- proach for solving the knapsack problem. Applied Sciences, 11(21)10209, 2021.
Henrik Abgaryan, Ararat Harutyunyan, and Tristan Cazenave. Llms can schedule. arXiv preprint
4rXiv:2408.06000 2024.
Xia Jiang, Yaoxin Wu, Yuan Wang, and Yingqian Zhang. Unco: Towards unifying neural combina- torial optimization through large language model. arXiv preprint arXiv:2408. 12214. 2024.
Duo Wu, Xianda Wang, Yaqi Qiao, Zhi Wang, Junchen Jiang, Shuguang Cui, and Fangxin Wang. Netllm: Adapting large language models for networking. In Proceedings of the ACM SIGCOMM 2024 Conference, pages 661-678, 2024.
Laurent Perron and Vincent Furnon. Or-tools, 2024. URL httsst//eveloerrsccoom
Fei Liu, Xialiang Tong, Mingxuan Yuan, and Qingfu Zhang, Algorithm evolution using large lan- guage model. arXiv preprint arXiv:2311.15249 2023a.
Zijie Chen, Zhanchao Zhou, Yu Lu, Renjun Xu, Lili Pan, and Zhenzhong Lan. Qube: Enhancing automatic heuristic design via quality-uncertainty balanced evolution, 2025. URL //t
xiv. orw.bb//bb//// /00po//2412.co 20694.
Fei Liu, Rui Zhang, Zhuoliang Xie, Rui Sun, Kai Li, Xi Lin, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. Llm4ad: platform for algorithm design with large language model. arXiv preprint arXiv:2412.17288 2024b.
Alexander Novikov, Ngån Vü, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. AlphaEvolve: A coding agent for scientific and algorithmic discovery. Technical report, Google DeepMind, 05 2025. URL https:
wow.e/nnddoo.mm.o com beceppalee-ccennnnoooeeeeeedddre io-.reooooooocccoooooooooooooooe advancaceee
Igor
Nguyen Thach, Aida Riahifar, Nathan Huynh, and Hau Chan. Redahd: Reduction-based end-to end-to-end automatic heuristic design with large language models. arXiv preprint 2rXiv:2505.20242 2025.
12

---

Published as a conference paper at ICLR 2026
Xianliang Yang. Ling Zhang. Haolong Qian, Lei Song, and Jiang Bian. Heuragenix: Lever- aging Ilms for solving complex combinatorial optimization challenges. arXiv preprint 1rXiv:2506.15196, 2025.
Shunyu Yao, Fei Liu, Xi Lin, Zhichao Lu, Zhenkun Wang, and Qingfu Zhang. Multi-objective evolution of heuristic using large language model. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27 1144-22152, 2025.
Xuan Wu, Di Wang. Chunguo Wu, Lijie Wen, Chunyan Miao, Yubin Xiao, and You Zhou. Efficient heuristics generation for solving combinatorial optimization problems using large language mod- els. In Proceedings of the 3 ACM SIGKDD Conference Knowledge Discovery and Data Mining V. 2, pages 3228-3239, 2025.
Anja Surina, Amin Mansouri, Lars Quaedvlieg, Amal Seddas, Maryna Viazovska, Emmanuel Abbe, and Caglar Gulcehre. Algorithm discovery with Ilms: Evolutionary search meets reinforcement learning. arXiv preprint arXivv2504.05100 2025.
Fei Liu, Rui Zhang, Xi Lin, Zhichao Lu, and Qingfu Zhang. Fine-tuning large language model for automated algorithm design. arXiv preprint arXiv:2507. 20025007.0614,,. 2025.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728-555 53741, 2023.
John Schulman. Approximating divergence, 2020. URL
approxx. I Accessed: 2025-05-11.
Larry J Eshelman. Preventing premature convergence in genetic algorithms by preventing incest. In Proceedings of Fourth International Conference on Genetic Algorithmss 1991, 1991.
Michael Han Daniel Han and Unsloth team. Unsloth, 2023. URL http::// com/uns
Lothai /uns lotn
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 2024.
Claire Kenyon. Best-fit bin-packing with random order. PhD thesis, Laboratoire de I'informatique du parallélisme, 1995.
Daniel Rosenkrantz, Richard E Stearns, and Philip M Lewis, II. An analysis of several heuristics for the traveling salesman problem. SIAM journal on computing, 6(3):563-581, 1977.
Christian Blum. Ant colony optimization: Introduction and recent trends. Physics of Life reviews,
(4):353-373, 2005.
Haoran Ye, Jiarui Wang, Zhiguang Cao, Helan Liang, and Yong Li. Deepaco: Neural-enhanced ant systems for combinatorial optimization. Advances in neural information processing systems, 36: 43706-43728. 2023.
Asankhaya Sharma. Openevolve: open-source evolutionary coding agent, 2025. URL https:
7/github. com www.coaeeeccccccoooooccooooococo 1on openevolve
Silvano Martello and Paolo Toth. Lower bounds and reduction procedures for the bin packing problem. Discrete applied mathematics, 28(1) 7:59700, 1990.
Shen Lin and Brian W Kernighan. An effective heuristic algorithm for the traveling-salesman prob- lem. Operations research, (2) 1973.
Nafis Tanveer Islam, Joseph Khoury, Andrew Scong, Mohammad Bahrami Karkevandi, Gonzalo De La Torre Parra, Elias Bou-Harb, and Peyman Najafirad. Llm-powered code vulnerability repair with reinforcement learning and semantic reward. arXiv preprint RXiv:2401.03377 2024.
Yun-Da Tsai, Mingjie Liu, and Haoxing Ren. Code less, align more: Efficient 1lm fine-tuning for code generation with data pruning. arXiv preprint arXiv:2407.050400 2024.
13

---

Published as a conference paper at ICLR 2026
Junqiao Wang, Zeng Zhang, Yangfan He, Yuyang Song, Tianyu Shi, Yuchen Li, Hengyuan Xu, Kunyu Wu, Guangwu Qian, Qiuwu Chen, et al. Enhancing code llms with reinforcement learning in code generation. arXiv preprint arXhv:2412.20367, 2024.
Wei Shen and Chuheng Zhang. Policy filtration in rlhf to fine-tune Ilm for code generation. arXiv preprint urXiv:2409.0699) 2024.
Junjie Li, Aseem Sangalay, Cheng Cheng, Yuan Tian, and Jinqiu Yang. Fine tuning large language model for secure code generation. In Proceedings of the 2024 IEEE/ACM First International Conference on Al Foundation Models and Software Engineering, pages 86-90, 2024.
Zeyuan Ma, Hongshu Guo, Jiacheng Chen, Guojun Peng. Zhiguang Cao, Yining Ma, and Yue-Jiao Gong. Llamoco: Instruction tuning of large language models for optimization code generation.
arXiv preprint arXiv:2003.01131, 2024.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J Liu. Slic-hf: Sequence likelihood calibration with human feedback. arXiv preprint arXiv:2305. 10425, 2023.
Tianqi Liu, Yao Zhao, Rishabh Joshi, Misha Khalman, Mohammad Salch, Peter Liu, and Jialu Liu. Statistical rejection sampling improves preference optimization. arXiv preprint arXiv:230.006657, 2023b.
Gerhard Reinelt. Tspliba traveling salesman problem library. ORSA journal on computing, 3(4): 376-384, 1991.
Nicos Christofides. Worst-case analysis new heuristic for the travelling salesman problem. In Operations Research Forum, volume 3, page 20. Springer, 2022.
Judith Brecklinghaus and Stefan Hougardy. The approximation ratio of the greedy algorithm for the metric traveling salesman problem. Operations Research Letters, 43(3):259-22) 2015.
Gabriel Duflo, Emmanuel Kieffer, Matthias R Brust, Grégoire Danoy, and Pascal Bouvry. A gp hyper-heuristic approach for generating tsp heuristics. In 2019 IEEE International Parallel and Distributed Processing Symposiun Workshops (IPDPSW), pages 521-529. IEEE, 2019.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
The Use of LLME
16
Extended Discussion about Related Work
16
Complete Algorithm
16
D Prompts Used in CALM
18
Example Prompt-Response Pairs for Injection and Replacement
18
Discussion about the Diversitv-Aware Crossover Operator
25
More Details For the Collapse Mechanism
25
Proof of Equation (2)
25
Goodness
26
More Experimental Details
26
Implementation Details
26
Baselme Implementations
26
Description of Problems Experiments
27
Seed Heuristies
27
More Experimental Results
28
Breakdown of CALM's Time Consumption
28
2
Impact of Fine-tuning the Foundational Model
................................
29
Impact of the Foundational LLM
29
More Results on Harder OBP Instances
................................
30
1
Scaling Behavior
................................
30
L6
P-values for Signthcance
................................
31
Sensitivity to the Hyperparameicrs in the Reward Function
................................
188
Extended Discussion on the Impact of Seed Heuristics
31
1.9
Impact of GRPO on MCTTSAHII and CALM
32
(
Resulls on the TSPLiD
33
Impact of DPO on CALM
34
Generated Heurisiiss
34
Limitations
42
Broader Impact
42
License
42
15

---

Published as a conference paper at ICLR 2026
A
THE USE OF LLMs
LLM serves a heuristic generator in our core method, all LLM-based AHD frameworks do. The idea of the method was originally created and implemented by human. Additionally, the LLM was employed as tool refine and polish the writing.
B EXTENDED DISCUSSION ABOUT RELATED WORK
LLM for Code Generation. Recent work has explored improving LLMs code generation capabili- ties through post-training (Islam et all 2024, Tsai et all 2024% Wang et all 20ze;; Shen and Zhang 2024 ell 2024). For example Islam
(2124) employ RL and semantic feedback to repair vulnerabilities, while Wang et all (2024)) demonstrate RLs effectivvnnes in enhancing code quality, Despite surface similarities, our task differs drastically: in code generation, objectives often priori- tize pass rates (Shen and Zhang, 2022 Wang et all 20244 [T'a et all (0222) or safety eLl 2024 Islam et all 2022),, whereas our goal is to maximize heuristic performance. Moreover, in code gen- eration, fine-tuning aims to produce a generally stronger model, while in our case, both the model tuning and prompt evolution serve a singular goalimproving the quality of generated heuristics.
2024)) trains LLMs for optimization by fine-tuning curated prompt- code pairs and enabling direct code generation for new problems. Its training data is derived from established sources such papers, competitions, and benchmarks. By contrast, CALM adapts LLMs using prompts and responses generated dynamically during the evolutionary process, allow- ing problem-specifie adaptation without external data. A promising future direction is to combine the supervised training of LLaMoCo as a first stage with CALMs reinforcement learning as a second stage for adaptive optimization.
Notably, LLaMoCo ctal
RL for LLM Fine-tuning. Reinforcement learning is a central technique for fine-tuning large language models, with the RLHF paradigm commonly relying on Proximal Policy Optimization (PPO) (Schulman el al, 2017) to iteratively refine model outputs based on human feedback. Build- ing on this, Group Relative Policy Optimization (GRPO) (Shao all 2024)) simplifies training by removing the need for separate value network, instead estimating baselines over groups of can- didate completionsleading to improved sample efficiency and stability. Other alternatives such as Direct Preference Optimization (DPO), SLIC-HF (Zhao et all, 2025), and Rejection Sampling Op- timization (RSO) (Liu et all 20236) offer off-policy mechanisms that further reduce computational burden. While we do not aim to develop new fine-tuning algorithmss, our method integrates GRPO within the broader co-evolution framework to adapt the LLM in tandem with heuristic evolution. We specifically adopt GRPO because it requires only scalar signal per prompt-response pair (in contrast to preference- based signals), making it suitable for our setting. Moreover, we implement fine-tuning using Unsloth (Danel Han and feaml 2023), GPU-efficient open-source framework that enables fast, memory-light training even on single consumer-grade GPUsmaking our method especially practical and accessible for researchers with limited hardware resources.
C COMPLETE ALGORITHM
The complete algorithm body is shown in Algorithm MI
16

---

Published as a conference paper at ICLR 2026
Algorithm 1: CALM
Input :LLM iig. Evaluation environment g. number of responses be sampled for one prompt, maximum round number T, Population size L, Sampling weight w for each operator, Hyperparameter 6g and that control the collapse mechanism, set of seed heuristic Hseed (set to be @ if not given any seed heuristic). Initialize collapse counter tot 1, best heuristic +mull, best performance g" OC, heuristic pool pool f f
Wijectinn
for t 1, ,T do
Operator base OPs (Initialization); poat 2 then
OPs < [Injection, Replacement, Proosover,Simmlifiaaain,
end |pool 2 2 then
OPs + OPs {Crossover];
end
if H < Lp then
Winjection f- max(w);
else Winjection 4 w1;
Hbase f B. op Draw operator from OPs with the probability proportional to w; op # Initialization then
he,1 <-rrww an heuristic from top---i-nrrforning heuristics in Hpool, where the sampling probability of heuristic proportional 1/rankp(h) and rank,(h) is the heuristic's performance rank;
Hbase + Hisase U {A};
if op Crossover then
random 1) I 0.5 then
h2.2 Draw heuristic from the population by performance rank as sampling N1,1
else
Calculate diversity metric
div(he,1,h)4 h2+ [idea_token(A) (ooenn())idea, iidea, token(At..)] Vh E Hpool:
ha.2 <-Draw heuristic from the pool by diversity rank where the sampling probability is proportional to 1 1/ranka(h) (a larger diversity value yields a higher probability);
Hbase Hbsse U {ha,2);
end
end
g
Generate prompt by the operator op and base heuristics Haasei
O Sample G responses from Ta for q;
Hiensible to Try extracting feasible heuristic from each response o E O and assign reward to each response following Section +
0 UUpate the LLM by GRPO that optimizes Equation IIl with (g, O,fo);
H.pool Hpool U Hseasible: h² < arg max g(h);
g(h))) = g" and [Hpoot 2 Lp then
/ If the population is full, the counter for collapse starts.
max(te; +1;
else
9" = =g("),, to +- min(te, 0);
if crandom (0, 1) < dote or to > C' then
Heuse + (h Hoeod, te t 1; / Collapse
end
end Return:ht
17

---

Published as a conference paper at ICLR 2026
Searching superior heuristics on the (problem. name) problem in evolutionary manner through conversation between User and Assistant. In this problem, [problem.description) The User provides existing algorithms and requests a new one.
## Your Task
You should first present concise conceptual description, followed complete code implementation.
The description must:
Be enclosed with double brace and starts with "The idea algorithm to". * Ensure it self-contained, insightful, and creatively original.
Not reference or rely any prior ideas existing code.
The code must:
Strictly follow the input-output variable names and types used in the provided implementation. Be a single Python function formatted within Python code blocks. Exclude any usage examples.
Ensure the algorithm deterministic.
Avoid introducing unnecessary, arbitrarily-tuned hyperparameters, any parameters used should be essential and systematically derived from the input.
Overall, your response should be like:
{[The idea of the algorithm is to (sepcific description here)}}
python your code here ...
Except for the idea and code, do not give additional explanations or comments.
Figure 3: Template of the system prompt.
D PROMPTS USED IN CALM
System Prompt. The system prompt is generated by inserting the name and description into the template shown in Figure BL The specific prompt used for each problem can be found in Table EL
Injection Prompt. The template used to generate injection prompts is shown in Figure In the prompt template, the algorithm details are generated by the given heuristics and the prompt tem- plate in Figure N The description of the most recent injected components is created by (1) parsing the string wrapped within "The new component has been introduced". (2) globally saving the historical new components, and (3) picking the last 10 new components to be used.
Replacement Prompt. The replacement prompt is created by the template, some predefined com- ponent Paris shown in Figure E. and the algorithm detail template shown in Figure g
Crossover Prompt. The crossover prompt is generated by the template shown in Figure f and the algorithm detail template shown in Figure N
Simplification Prompt. The simplification prompt is created by the template shown in Figure A and the algorithm detail template shown in Figure g
Initialization Prompt. The initialization prompt is created by the template shown in Figure g The algorithm template function signature.
E EXAMPLE PROMPT-RESPONE PAIRS FOR INJECTION AND REPLACEMENT
The example prompt-respont pairs with concrete explanation for the modification on heuristics shown in Figure [[] I 12 and K
18

---

Published as a conference paper at ICLR 2026
Inject novel, meaningful component into the following algorithm. The component may be self- devised or inspired by ideas from other domains or problems.
{algorithm_detaiee haaaiissgiiie_eeriiitit
Use concise phrase to describe the component in the responded idea like "The component has been introduced.".. Exclude the following components that have already been explored: [description of most recent injected components]
Figure 4: Template of the injection prompt.
For the following algorithm, identify (old component) and rewrite it to ceee_component.. (algorithm deetallsggiven_hurrits
old component
new_come new_componntt
an ristance-dependee rule that derives value from the current observation
fixed, instance-independent decision rule
key hyper-parameter expressed as either constant literal stationary variable
more principled constant justified by theory or practice
fragment that assigns equal near-equal credits to multiple elements
fragment where credits deterministically and reasonably differentiated
Figure 5: Template of the replacement prompt.
Please generate new algorithm that is motivated by the following algorithms but performs better on any same instance.
Figure 6: Template of the crossover prompt.
Please create a simplified and more elegant version of an algorithm by distilling and refining the care ideas from the following:
Figure 7: Template of the simplification prompt.
## Algorithm
Performance:
(Rank (heurisitc.rank))
* Idea: (heuristic.idea) (heuristic.unit)
Code: python (heuristic.code)
Figure 8: Template of algorithm details.
19

---

Published as a conference paper at ICLR 2026
Be very creative and inventive. Generate efficient algorithm following the template below:
(algorithm_ }
Figure Template of the initialization prompt.
Table 5: Information of each problem used in prompts
OBP
CVRP
OP
TSP
Name
Online Bin
Capacitated Vehicle Routing units of travel distance
Orienteering
Traveling Salesman length units
Packing
Unit
percent of the gap to the lower bound Items arrive sequentially
units of collected re- ward
of the tour
Description
A fleet of vehicles with limited carrying capacity must deliver goods to a set of geographically dis- tributed customers with known demands, mini- mizing the total travel distance while ensuring that no vehicle exceeds its capacity. The goal is to design heuristic function that estimates the desirability of mov- ing between customers, to used within an Ant Colony Optimization (ACO) algorithm.
An agent must visit a subset of locations, each offering re- ward, within a max- imum travel budget. The objective to maximize the total collected reward while adhering the travel constraint. The goal is design a heuristic function that estimates the de- sirability of moving between locations, to be used within Ant Colony Optimization (ACO) algorithm.
The algorithm must find tour that visits each node exactly once and returns to the start node. The objective is minimize the length of the tour.
and must be placed imme- diately into bins only if they fit within the remaining capacity. The objective is to minimize the number of bins used.
20

---

Published as a conference paper at ICLR 2026
Prompt
Inject a novel, meaningful component into the following algorithm. The component may be self- devised or inspired by ideas from other domains problems.
## Algorithm 1
Performance: 12.578 units of collected reward (Rank: 1)
Idea: The idea of the algorithm is to return the ratio the prize and the inter-node distance
Code:
python
The idea of the algorithm is to return the ratio of the prize and the inter-node distance
import numpy as np
def heuristics(prize: np.ndarray, distance: np.ndarray, maxien: float) >> np.ndarray: return prize(np.newaxis, :) distance
Use concise noun phrase to describe the component in the responded idea like "The component... has been introduced.".
Inject new decay component, and the performance is improved by 4.21%!
Response
((The idea of the algorithm is to return the ratio of the prize and the inter-nade distance. The new component incorporating a distance decay factor has been introduced.)3
python
import numpy np
def heuristics(prize: np.ndarray, distance: np.ndarray, maxlen: float) -> np.ndarray: distance decay np.exp(-distance maxien)
return prize(np.newaxis, :] distance * distance decay
Figure 10: Example of improvement made by injection operator.
21

---

Published as a conference paper at ICLR 2026
Prompt
For the following algorithm, identify fragment that assigns equal credits to multiple elements and rewrite to fragment where credits are deterninistically and reasonably differentiated.
## Algorithm
Performance: 13.826 units of collected reward (Rank:
prize inter-node distance, while incorporating a distance decay factor that uses novel hybrid decay. This hybrid decay combines elements of the initial exponential and linear decays, starting with initial exponential decay that quickly reduces the influence distant nodes, followed by linear decay that gradually reduces the influence further. This hybrid approach enhances the imploration-expliitaeee trade-off, providing balance between visiting nodes and exploring distant ones, thereby optimizing the balance.
Idea: The idea algorithm return
Code:
ppttonn
import numpy
def heuristics(prize: np.ndarray, distance: np.ndarray, maxien: float) np.ndarray:
# Initial exponential decay followed by linear decay
distance decay np.exp(-distance maxien))
maxien))
(distance 0.4 maxlen)
distance_ distance_decay(ddsse decaylydistance 0.4 maxien] # Ensure linear decay only applies to close nodes
prize(np.newwxis, :] distance decay
Treat edge pairs differently by distance threshold, and the performance improved by 0.39%!
Response
((The idea of the algorithm is to return the ratio of the prize and the inter-node distance, while incorporating hybrid decay that enhances the exploration-explotaae trade-off.)
** "python
import numpy
def heuristics(prize: np.ndarray, distance: np.ndarray, maxien: float) np.ndarray:
# Initial exponential decay followed by linear decay
initial, exponential decay np.exp(-distance (0.4 maxlen))
linear_ decay 1 (distance 0.4 maxien) (0.6 maxlen)
linear distcnde 0.4 maxlen] Ensure linear decay only applies to close nodes
# Hybrid decay combining initial exponential and linear decay
hybrid decay initial exponentil _eccayyylldddyyyyyyeee linear decay return prize(np.newaxis, :) distance hhbridddecaa decay
Figure 1I: Example of improvement made by replacing an equal-credit rule operator.
22

---

Published as a conference paper at ICLR 2026
Prompt
For the following algorithm, identify harae-parameter expressed either constant literal stationary variable and rewrite it to more principled constant justified by theory practice.
## Algorithm
Performance: 7.103 length units of the (Rank: 1)
Idea: The idea of the algorithm is to select the next node visit from the unvisited nodes caliculating score that considers direct distance the current node, the average distance the unvisiked nodes, the standard deviation these distances, and the distance the destination node.
Code:
python
import numpy np
select, next nwde(current, destination node: unvisited distance no.ndarray) Int:
node unvisited
distances [distance matrix(current node]] node]. nodel distance matrrii((esiiiatiinnnnn node][node]] average_distance, distance to current ap.mean(distances)
average
distance current np.ste(distances)
score
c1 distance matrix(cumrent node][node]
average distance to_ current
std, dev ____ddittccee__e torrurettt
distance wwwprmmi((deeiaatiionnnooooee [ode[[[nde
scores[node] score
next_nade node min(scores, key=scores.gett) next
return next next_nade node
Change hyperparameter, and performance is Improved by 8.29%
Response
((The idea of the algarithm select the next node visit from the unvisited nodes by calculating score that considers the direct distance to the current node, the average distance to the unvisited nodes, the standard deviation of these distances, and the distance to the destination node.))
''yython
import numpy as np
def select_next_nee nodelcurrent_nader int, destination_noder node: int, unvisited_nodee set, distance_matinn matrix: np.ndarray) >> int:
cl. c2, c3, c4 0.5, 0.3, 0.2, 0.1
scores
for node in unvisited nodes:
distances [distance matrik(current (ncpireeeetnndeeeeeeee node), distance distance_ http://aiieeeeettttooonooooeee node[[ndee)))) average distteccddeceeeeeeeeeeee to, current np. (
std dev_ddddddsaeeee dww.dddstncceeeec current sppstteidist
cl distance disiance_mattri(cuureg matrix(cuureett node][node]
average distance current
dev. hov_distance current
distance_man distance_matrik(deete batperrieeesstaattooonnneleeeeee
scores[node]
next_node key=scores.get)
return node
Figure 12: Example of improvement made by replacing a hyperparameter.
23

---

Published as a conference paper at ICLR 2026
Prompt
For the following algorithm, identify fixed, instance-independent decision rule and rewrite to instance-dependent rule that derives its value from the current observation.
Algorithm
Performance: 3.052 percent gap to the lower bound (Rank:
Idea: The idea of the algorithm is to score each available bin integrating the remaining capacity, item size, and a penalty for overfilled bins. This algorithm introduces novel adaptive scaling factor based on the geometric mean of the remaining capacity and the maximum capacity. also uses quintic penalty term to better penalize overfilled bins quadratic scaling stabilize the for bins with very small remaining capacity.
Code:
python
import numpy as np
def step(item size: float, remaining_capacityy np.ndarray) >> np.ndarray:
max bin_cap np.maxxremaiinngggcpae
geometric mean = caapaaininnn_apacciiie max_ max_bin su_bbinncappppp cramaining__apaciin + max bin_bainn
scale factor = np.sartigeemetrrcccc mean / max_bin_e b____nn_pppp
= catmmmaiinnn_cpacittttttttee (item (
score cettmaainnnn_aaaccitne max fhax_bin_cee _ aeteiteesiaeee quintic_penalty
score(remaining_capeee item_size] -scorereemaiiing_aaccty > item_ size]
score[1:] score[:-1]
score scale factor score
Replace static threshold by the dynamic item size, and the performance is improved by 7.79%!
Response
((The idea of the algorithm score each available bin by integrating the remaining capacity, item size, and penalty for overfilled bins. This algorithm value from the current input instance by calculating the geometric mean of the remaining capacity and the maximum capacity as the scale factor and adjusting the score based on the item size and remaining capacity.)3
deterministic rule that derives its
python import numpy np
def step(item sipp:itemmsieee::: float, aamaining_capaciinn np.ndarray) -> np.ndarray:
max bap__inpaaaap_aaae = app.maxfemaiinnggcapaaee
geometric
np.sqrt(remaining.
c capacity max bin bip]
scale factor geometric_men Leteetricccm//////// max_
/max_iin_cam bin_cap
score = (remaining_capace
(omms item_size + remaining remaining capacity**5 (item_sizee size**5) (item_size**5)
score(remaining
[ize] http://rmmaiiinnnccpiaiiiiio > item_sizes
score[1:] score[:-1]
score *= scale factor
return score
Figure 13: Example of improvement made by replacing a instance-inde indeeeedendd decision rule.
24

---

Published as a conference paper at ICLR 2026
F DISCUSSION ABOUT THE DIVERSITY-AWARE CROSSOVER OPERATOR
When al. (2025) allowed heuristic selection beyond the top-performing population, offer- ing greater exploration flexibility, though without explicitly modeling diversity. In contrast.Datctal (2025) emphasized the role of diversity in heuristic evolution but not integrate into crossover and operated within fixed-size population. Therefore, CALMs crossover operator complements prior work by explicitly incorporating diversity into the crossover process.
G MORE DETAILS FOR THE COLLAPSE MECHANISM
G.1 PROOF OF EQUATION (k)
Let on be the stagnation counter just before collapse. Under the collapse mechanism with perround hazard
Pk = k do. the probability of surviving beyond & rounds is
k = 1,2.
Pr(en > kd
II - (0) 1=1
which vanishes for k 2 [1/6o]. By definition,
B
c[c]] Σ
h Pr|cn = =k).
km
Introduce the nonnegative array
Pr|cr = kt, k2j21, 1,
0j.b
00
otherwise.
Then
A Pr|ca = k]
Pr|c. k]
Gj.k
Am
Rm
Since aj.k 0, Tonellis theorem allows swapping the sums:
2
Pr(ca = k] _
Pr|c, 2 = Σ Pr|c, > 5/.
a3A =
ajK
k=15=1 k=1k=1
j=1 k=
j=1
5=0
Hence the tailsum identity
80 > Pr/c,, j.. 1=
E Cn
For dn < we approximate the finite product by exponentiating its logarithm, using the Maclaurin expansion
20
In(1 x)
|x| < 1,
m=1
with æ = idg. Truncating at the linear term gives
b
Σ (0(1-160) ~ Σ 160 = da k(k + ~ k:?
so
Pr/c, k] R expl k2
Substituting into the tailsum and replacing the discrete sum by an integral yields
An kaa
Ap
E Cicn1
R
dx V 200
k=0
which establishes Equation (2)
25

---

Published as a conference paper at ICLR 2026
G.2 GOODNESS
Following this reset, the search effectively starts anew, but with a strategic advantage: it builds upon the best insights discovered so far. Importanll, during the early stage of repopulation, the system temporarily relaxes selection constraints. New heuristics generated via injection, replacement, or crossover are allowed into the population regardless of performance, as long as the total number of heuristics remains below the target population size. This gives structurally novel but potentially suboptimal components the opportunity to propagate and evolvesomething not feasible under normal selection pressure, where only top-performing heuristics are retained and processed further.
H MORE EXPERIMENTAL DETAILS
H.1 IMPLEMENTATION DETAILS
We build CALM on Unsloth (Dameel Han and team. 20225, with two modifications: raising the learning rate to 5 10-5 for faster adaptation and sampling G 4 responses per prompt to enable more evolutionary steps under a fixed query budget.
We set the initial collapse growth rate to 6, 0.0005 (max threshold C = 25), cap training T 500 rounds, and assign operator sampling probabilities in the ratio 1 : 4 for simplification, injection, modification, and crossover, respectively. Each heuristic is evaluated within 60 (Zenng atall 2025). All experiments ran on a 24 GB NVIDIA A30 GPU with an Intel Xeon Gold 5220R CPU.
For invalid responses, the maximum reward Tinvalid is set to -0.75. Furthermore, we apply a hierar- chy of failure modes, assigning progressively higher (i.e., less negative) rewards to increasingly plau- sible but still unacceptable outputs. These modes include: (1) omission of required idea (reward:
r7 1.0); (2) missing code block (ra- = 0.955): (3) improperly formatted function (r3 -0.9); (4) runtime errors or time budget violations (r4 -0 -0.85); and (5) detection of randomness in the heuristic (r' 0. which incurs the mildest penalty among infeasible cases.
Under this configuration, the average running time of CALM for the OBP, CVRP, OP, and TSP is about 6.8, 7.2, 5.3, and 5.5 hours, respectively, for = 500 steps. However, is important to note that the actual rumning time for a single trial may vary considerably due to the stochastic nature of the LLM and the potentially large number of heuristics generated, each requiring time-intensive evaluation.
H.2 BASELINE IMPLEMENTATIONS
The source code, training dataset, and test dataset for AlphaEvolve (Novikov et all 2025) are not available. Therefore, we use OpenEvolve (Sharma, 20255 as the baseline, which is the most popular
open-source implementation of AlphaEvolve.
In its original implementation, TvoTune (Surnn all 20255) requires approximately 80GB of GPU memory conduct experiments LLMs with fewer than 7B parameters, which exceeds the putational resources available to us. By contrast, our CALM method could operate on a single GPU with 24GB of memory. To ensure fair comparison, Unsloth (Dnnill Han and feam,, 2023) framework, following its official source code, so that it can be executed the same Qwen model under identical GPU constraints.
re-implemented EvoTune within the same
Besides, ReEvo (Ye all ROzE!) and its follow-up approach HSEvo (f (025)) can stop at a very early stage in evolution as found by Zheng et al. (2025). Thus, the results of them TSP not reported. For the OP and CVRP tasks, OpenEvolve (Sharma, 2025) failed to discover improved heuristics beyond the early stages of heuristics search. As a result, its training curve omitted from Figure D for clarity.
'Randomized heuristics are excluded in the experiments because their stochastic behavior substantially in- creases evaluation cost and noise. To enforce determinism, CALM penalizes responses that invoke randomness (e.g., usage of randem, np. random, etc.). The framework could support randomized heuristies by relaxing this constraint, though evaluation overhead would increase.
26

---

Published as a conference paper at ICLR 2026
H.3 DESCRIPTION PROBLEMS EXPERIMENTS
Online Bin Packing (OBP), A sequence of items of varying sizes arrives one by one. Each bin has fixed capacity. Upon arrival of an item, the algorithm must immediately assign to an existing bin that has enough remaining space open new bin. The goal is to minimize the total number of bins used. The input of the heuristic the size of the current item and the remaining capacities of the bins. The output of the heuristic is the priority score of each observed bin, where the feasible bin with the highest score will be selected to accomodate the item.
Traveling Salesman Problem (TSP) under Step-by-Stey Construction. Given a set of locations with pairwise travel distances, the objective to construct a tour that starts one location, visits each other location exactly once, and returns to the start. At each step the heuristic must choose the next unvisited location based solely on the information gathered so far. The aim is to keep the total travel distance small possible.
Capacitated Vehicle Routing Problem (CVRP) under ACO. fleet of vehicles with identical load capacity must
of customers, each with known demand, and all vehicles start and end a central depot. Under the Ant Colony Optimization framework, many artificial ants build routes by moving from customer customer. Each ants choice of next customer is guided by combination of pheromone trailsupdated based on previous high-quality solutionsand heuristic scores provided by the LLM. The goal to serve all customers while minimizing the total distance traveled and respecting vehicle capacity limits.
Orienteering Problem (OP) under ACO. Starting from given location (and possibly ending at the same or another specified location), an agent may visit subset of available sites, each offering a reward, subject an overall travel budget. Within the ACO framework, ants construct candidate paths by choosing which site to visit next based pheromone levels and LLL--gerrrted heuristic scores that estimate the benefit of each edge under the reward-and-budge trade-off. The aim is to collect much reward possible without exceeding the travel budget.
H.4 SEED HEURISTICS
The seed heuristies directly adopted from Zheng al. (20255 and are listed below.
import numpy as np
def step size: float, remaining captccity np.ndarray) np. ap.ndarray:
max max_bin cap max wwweaaniig_cappaceecceee capacitaptt
score = xomaining
- max sox__inn_caa_ max_bin_cap)x=2 / item_
(item_sizo==2)
score += remainiigg_cppccttyy
item_size..3
score score craeeaainngncaapccciee item_size] -score (romainiig
item_size]
score[1:] -- score[:-1]
return score
Heuristic Seed Heuristic for OBP Task
import numpy as np
def select_next_nooeeeeeeeeeeeeeeeee (current_node: int, destination_node int, unvisited_nodds: set, distance_matrix: ndarray) -> int: threshold 0.7
c1, c2, c3, c4 0.4, 0.3, 0.2, 0.1
scores
for node in unvisited_nodeet nodes: distaanee unvisited_nodes if i != node]
[i] for ttd me.m distancessn
grerage_distance unvisited
std_dev_dt sonii np. std (a1 1i_diiisttteeess
27

---

Published as a conference paper at ICLR 2026
Table 6: Breakdown time consumption in CALM (with detailed wall-clock time).
Inference
Evaluation
Training
CVRP
73.29% (4.059 h) 78.59% (5.749 h)
16.11% (0.893 h)
10.60% (0.587 h)
OBP
11.10% (0.812 h)
10.31% % (0.754 h)
OP
82.12% (4.281 h)
9.77% (0.510
8.11% (0.423 h)
TSP
83.23% (6.457 h)
7.20% (0.559 h)
9.57% (0.742 h)
score
cl dittance_matiix(ccrrrto_ddddd [node]
+ c3 + average_distaoocttooovviiiii...s
c4 +
[node]
[noda]
score
next node min (scores, key=scores. return next_ node
Heuristic 2: Seed Heuristic for TSP Task
import numpy as np
def heuristics (prize: nd.ndarray, distance: np.ndarray, maxlen: float) np, ndarray::: newaxis, Inp.nnaxxs,,
return prize Inp
distance
Heuristic 3: Seed Heuristic for OP Task
import numpy as np
def heuristics (distanceemattrriaaaaa ndardayryy, coordinates: ndanraryyy,
demands: np. np. ndarray,, capacity: int) -> ndanraayy::
return distance_matrix
Heuristic Seed Heuristic for CVRP Task
I MORE EXPERIMENTAL RESULTS
I.1 BREAKDOWN OF CALM'S TIME CONSUMPTION
We break down the total CALM running time into three components: (i) Inferencethe time taken by LLM to generate responses (ii) Evaluationthe time spent parsing each heuristic and validating its feasibility and performance; and (iii) Trainingthe time required to compute the loss and update the LLM parameters. The time spent on each component across different tasks is summarized in Table &
These results show that inference the dominant time cost in CALM. Despite parallelizing heuristic evaluations across the training dataset, evaluation still requires more time than model training for most tasks. In other words, employing the fine-tuning algorithm the LLM-based AHD introduces minimal time overhead.
We additionally quantified the potential overhead gradient computation during inference, as training in CALM includes rollouts. Since gradient computation is tightly integrated into PyTorch, we compared two runs: (i) inference with gradients enabled and (ii) inference with torch. no grrad () to disable gradient computation. Across all tasks, the extra cost consistently helow 0.25% of the pure inference time, which negligible and does not affect the hreakdown reported in Table fi.
28

---

Published as a conference paper at ICLR 2026
Table Average scores of fine-tuned models. Lower better for CVRP O and higher better for OP O.
Step=0
Step=100
Step=200
Step=300
Step=400
Step=500
CVRP Q
66.356
32,403 25.025
40.860 25.025
40.451 25.025
49.699
32,403
OP O
11.956
12.228
25.025
Table Feasibility ratio of fine-tuned models.
Step=0
Step=100
Step=200
Step=300
Step=400
Step=500
CVRP
10.00%
100.00%
83.33% 100.00%
62.50% 100.00%
71.43%
100.00%
OP
26.32%
100.00%
45.45%
100.00%
1.2 IMPACT OF FINE-TUNING ON THE FOUNDATIONAL MODEL
To investigate the performance of the fine-tuned model, we conducted additional experiments. Specifically, we saved a snapshot of the LLM every 100 training steps during the evolutionary pro- cess of CALM. For each snapshot, we used the same prompt, which instructs the LLM to generate an improved variant of the seed algorithm for a given task. For each snapshot and task prompt, we repeatedly sampled responses until five feasible outputs capable of producing valid heuristic code for the task were obtained. We then recorded the following metrics: (i) the average score of the five heuristics, where the score for each heuristic is calculated as the performance averaged over the test scales reported in our manuscript; and (ii) the feasibility ratio, defined as the number of feasible responses (fixed at five) divided by the total number of samples required obtain them. We focused on the snapshots generated during the run for CVRP that yielded the best heuristic among all three runs. For these LLM snapshots, we evaluated them on both CVRP and OP. Results are shown in Table / and R
Key observations are follows:
Both the average score of the discovered heuristics and the feasibility ratio of the responses improve significantly after fine-tuning. For example, the CVRP task, the feasibility ratio increases from 10% to 100%, and the average score decreases by more than 50% after fine- tuning. This demonstrates the effectiveness of CALM in enhancing the LLMs capability.
Although the LLM fine-tuned on data generated from AHD for the CVRP task, sub- stantial improvements in both the average score and feasibility ratio are also observed for the OP task. This suggests that the improvements gained through fine-tuning on CALM- generated data generalize beyond the in-domain task and can benefit other related tasks.
Beginning at step 200, both the average score and feasibility ratio fluctuate during the training process. Notably, the heuristics achieving the best scores (20.088 for CVRP and 25.252 for OP) were discovered by LLM snapshots saved at 300 and 400 training steps, respectively. Interestingly, these snapshots also exhibit the lowest feasibility ratios and non-leading average scores for the respective tasks. This indicates that an LLM capable of producing an exceptional heuristic may not be the most stable in generating feasible responses or in producing consistently high-quality heuristics on average. In other words, LLM capable of occasional breakthroughs may exhibit erratic behaviorillustrating the notion that genius can verge on madness.
I.3 IMPACT OF THE FOUNDATIONAL. LLM
We have added additional experiments by replacing the foundational model with (i) a SOTA reason- ing LLM o4-mini and (ii) another open-source compact model ..a-m--.1-8B- 1-naa...------sstuu-----tttt Results are as shown in Table E and [T]
The SOTA reasoning LLM o4-mini effectively identifies superior heuristics under the CALM framework (w/o GRPO), achieving notable performance improvementsaproximately 15.5% with the o4-mini modelthough this advantage comes with over twice the inference time compared to
29

---

Published as a conference paper at ICLR 2026
Table Optimality gaps on OBP (Qwen, Llama, and 04-mini) with CALM.
lk_ 100
1k_500
5k_100
5k_500
10k_100
10k_500
Avg.
Qwen+GRPO
2.55%
0.00%
0.85%
0.17%
0.56%
0. 0.14%
0.71 %
Llama+GRPO
2.98%
0.00%
0.96%
0.
0.54%
0.10%
0.78%
04-mini
2.29%
0.00%
0.85%
0.10%
0.34%
0.02%
0.60%
Table 10: Objective scores and optimality gaps on OP (Qwen & Llama) with CALM.
N=50
N=100
N=200
Qwen
15,054 (24.22%)
30.778 (15.43%)
55.406 (12.58%)
Llama
15.038 (24.31%)
30.599 (15.92%)
54.593 (13.86%)
Qwen+GRPO. Despite this trade-off, using locally deployed, compact models remains competitive, particularly when the time budget for search limited. Additionally, heuristics identified by the Llama model show strong performance and generalizability, outperforming all other methods at cer- tain scales (5k_500 and 10k_500) in OBP and surpassing all baseline methods in OP at N=100 and 200, while maintaining comparable results at all scales. Moreover, removing GRPO significantly reduces average optimality gaps, by 34.33% OBP and 17.91% OP, further highlighting the robustness of the proposed method.
1.4 MORE RESULTS HARDER OBP INSTANCES
Smaller-scale instances of OBP are more challenging in the online setting, each decision has larger impact and variance is higher, making them a stricter robustness test for heuristics. Thus, we further evaluated CALMs performance
OBP with smaller problem scales. Specifically, we generated 10 Weibull-distributed instances for each of the following training scales (in the format n e_capccity): 100 100, 100 500, 300 100, 300 100_500, 500_100, and 500 500. For evaluation, 50 instances were generated for each scale. CALM was equipped with 1-0--...-------tuucttt For comparison, included MCTS-AHD, the SOTA LLM-based AHD method that achieved the best performance on the smallest scale in Table H.
The results show that CALM+Llama achieves lower average optimality gap than MCTS-AHD, even when the latter is paired with a more powerful LLM. CALM underperforms only at the 500 500_500 500 scale. In addition, the standard deviation of the average optimality gap is smaller for CALM (0.03%) compared to MCTS-AHD (0.21%).
I.5 SCALING BEHAVIOR
We conducted additional experiments to evaluate the scaling behavior of CALM on OP, using an increased training budget of 2500 steps. Results shown in Table ID The key findings are as follows:
With a substantially larger evaluation budget, CALM able discover heuristics that outperform those found with only 500 training steps, as shown in the table below.
Without LLM fine-tuning, CALM is unable to consistently discover new, superior heuristics early stages training. In one instance, no better heuristic was found beyond step 256.
Evaluation of
model snapshots at different training steps shows that after several hundred steps, performance fluctuates and does not always improve monotonically. Never- theless, the fine-tuned models consistently outperform the untuned baseline. This suggests that, in later stages, fine-tuning may not significantly enhance the LLMs capabilities but instead introduces variation to the LLM for heuristic generation. This variation may help maintain the LLMs performance while increasing the diversity of the heuristic population.
Overall, these results indicate that CALM exhibits favorable scaling behavior under larger training budgets.
30

---

Published as a conference paper at ICLR 2026
Table 1I: Optimality gaps on OBP with smaller scales.
100_100 10_100
100_500
300_ 1000100
300_500
500_
100
500_500
Avg.
MCTS-AHD (GPT-4o-mini)
6.97%
1.39%
5.67% 5.61%
0.57%
5.20%
0.63%
3.40%
CALM (Llama-3.1-44 1.-ee-...----BB- I-8B-Instruct-INT-)
6.80%
1.39%
0.57%
5.06%
0.64%
3.35%
Table 12: Objective scores and optimality gaps of CALM (w/ GRPO) budgets.
OP under different search
N=50
N=100
N=200
#LLM Queries=2,000
15.054 (24.22%)
30.778 (15,43%)
55,406 (12.58%)
#LLM Queries=10,000
15.201 (23.49%)
31.153 (14.40%)
56.432 (10.96%)
1.6 P-VALUES FOR SIGNIFICANCE
To further highlight the superiority of CALM, we compare its performance against the state-of the- art L.LM-based AHD method MCTS-AHI (Zeng et al, 2025) across ten independent runs on two representative tasks: the TSP (step-by-stpp construction) and the OP (ACO-basedy For fairness, we adopt the exact same dataset as used by Zheng et al. (2025). The per-run performance of MCTS- AHD on the TSP task is directly obtained from their appendix, while for the OP task obtain the results using the official implementation under the same evaluation environment as CALM. The results are summarized in Tab. I The small p-values further confirm with high confidence that CALM consistently outperforms MCTS-AN AHD on both tasks.
We additionally the OP task using Qwen without GRPO, where the results listed in Tab. II3. The experiments show that CALM combined with Qwen and GRPO achieves higher performance, reduced variance, and a statistically significant improvement according to the p-value against the GRPO-free variant.
1.7 SENSITIVITY TO THE HYPERPARAMETERS IN THE REWARD FUNCTION
Our reward design guided by a fundamental principle: rewards should increase progressively with the quality of the generated response. This principle is illustrated by the high-level cases in Figure IL To differentiate between these cases, we introduce several hyperparameters (e.g., 01 Q2 and liavalid). To evaluate the robustness of this principle, we conducted an additional ablation study where CALMs reward function was instantiated under the following settings:
The original implementation with a relatively even reward distribution, as described in Section H.T
Random sampling of all hyperparameters under the progressive-guidies constraint, i.e., 01 > 0: > 0 > T3 = Tinvalid > 14 > Ta > Yg > -1.
The same 01, 022 and Tinvalid values as Section H.l but with all invalid responses uni- formly assigned Finvalid-
The results are presented in Table [4.
Across all settings, heuristics derived from these reward designs remain highly competitive at every scale. In particular, when invalid responses are assigned unified reward, CALM achieves the best performance at N 50, surpassing all methods reported in Table B. The slight performance gap between CALM with randomly sampled hyperparameters and the other two implementations likely results from uneven reward spacing across neighboring cases. Overall, these findings indicate that CALMs effectiveness is not tied to precise numerical values in reward shaping but instead depends on adherence the underlying principle of progressive reward allocation.
1.8 EXTENDED DISCUSSION ON THE IMPACT OF SEED HEURISTICS
The seed heuristics used in our experiments are listed in Section H4 For OBP and TSP, which require step-by-step constructive heuristics, the seeds contain diverse components and nontrivial
31

---

Published as a conference paper at ICLR 2026
Table 13: Performance comparison of CALM and MCTS-AHD on TSP (step-by-step construction) and OP (ACO-based) tasks over ten runs. "avg." represents the average and "std." the standard deviation, p-luuuss are calculated using single-tailed t-tests.
Methoos
runi
run2
run
run4
runS
run5
run?
runh
run10
p-value
TSR())
(GPT-4o
MCTS-AHD>
6.452
6.284
6.386
6,316
6,372
6,480
6.259
6.388
6.366
(Qwen2.5-70)
CALM
6.220
6.217
6213
6,005
6.221
6.174
6.213
6,219
6.22
0.015
0,000012315
OP(")
MCTS-AHD (GPT-lo-min)
14,668
14,90
14.786
14,602
14.738
14.734
14.642
14,826
14.722
14,690
14.731
0.001
(Qwen, GRPO)
CALM
14,87
14822
14951
14.798
14,844
14,600
14,850
14,878
14,880
14.746
14831
0.099
(V4. MCTS-AHD)
0.008317786
(Ques, GRPU)
CALM
14.765
14.425
14.722
14.578
14.77
14,806
14,608
14.8
14.674
14.581
14.673
0.124
0.001 01001555683
GRPU)
Table 14: Ablation study of reward parameter choices under CALMs reward design on the OP task.
Settings
N=50
N=100
N=200
Original reward configuration (Sec.
15.054 (24.22%)
30.780 (15.43%)
55.406 (12.58%)
Original reward configuration (Sec. HMI
15.059(24.20%)
30.744(12.52%)
55.341(122.68%)
but the same reward for all invalid responses
Randomly sampled reward parameters under progressivc-guiding constraint
15.046 (24.26%)
30.613 (15.88%)
55.165 (12.96%)
structures. These characteristics provide the LLM with rich initial context for exploration. The seed heuristic for OBP performs poorly, and its performance is even worse than simple Best-Fit or First- Fit strategies. In contrast, the seed heuristic for TSP is relatively strong. For CVRP and OP, where ACO-based solvers use heuristic matrix, the seed heuristics simply take the distance matrix and assign each entry the reciprocal of its value in an element-wise manner. This provides minimal prior structure for the solver and leads to moderate performance.
Across all tasks, the seed heuristics therefore cover a broad range. They include strong and struc- turally complex heuristic for TSP, weak and structurally complex heuristic for OBP, and simple heuristics with moderate performance for CVRP and OP. Starting from these varied seeds, CALM consistently achieves the best average optimality gap on all tasks, which demonstrates its robustness to different seed-quality regimes.
1.9 IMPACT OF GRPO ON MCTS-AII AHD AND CALM
We further evaluated MCTS-AHD under Qwen with and without GRPO on both CVRP and TSP. and also assessed CALM under Qwen without GRPO, using exactly the same number of sampled responses for MCTS-AHD with GRPO (G 4 per prompt) and the same fine-tuning hyperparame- ters as in our main GRPO setup so that training conditions are matched. As shown in Table IL and MR. these results lead to two key observations:
GRPO provides clear benefits to MCTS-AHD, bringing improvement to both tasks. On CVRP, the improvement is particularly notable, where MCTS-AHD+QWer without GRPO performs worse than MCTS-AHDD+GPT-4-NT
but with GRPO surpasses that baseline.
CALM benefits even more strongly from GRPO than MCTS-AHD. On CVRP, CALM+Qwen without GRPO underperforms MCTS-AHD in two of three scales, yet with GRPO it outperforms MCTS-AHI across all scales. On TSP, the gain from GRPO is more moderate and does not allow MCTS-AHD+Qwen to exceed HCTS-AHD+GPT--0- likely because the seed heuristic provided in TSP is strong and leaves much less room for exploration. contrast, applying GRPO to CALM yields a significant boost and makes CALM+Qwen surpass CALM+GPT-40-minn This consistent and substantial improvement suggests the effectiveness of CALMs specialized design, such as its fine-gran -enggraallrrrty opera- tors, for RL-based fine-tuning.
32

---

Published as a conference paper at ICLR 2026
N=50
N=100
N=200
MCTS-AHD (gpt4o-mini)
9.372 (3.44%)
15.974 (6.98%)
28,434 (4.70%)
CALM (gpt4o-mini)
9,404 (5.81%)
16.046 (7.46%)
28.713 (5.72%)
MCTS-AHD (Qwen, w/o GRPO)
9.921 (11.62%)
16.926 (13.35%)
29.967 (10.34%)
MCTS-AHD (Qwen, w/ GRPO)
9.267 (4.27%)
15.796 (5.79%)
28.346 (4.37%)
CALM (Qwen, w/o GRPO)
9.806 (10.32%)
16,995 (13.82%)
30.631 (12.78%)
CALM (Qwen, w/ GRPO)
9.228 (3.83%)
15.745 (5.44%)
28.23 (3.94%)
Table 15: Comparison of how GRPO affects the performance of MCTS-AHD and CALM on CVRP.
N=50
N=100
N=200
MCTS-AHD (gpt4o-mini)
6.225 (9.69%)
8,684 (11.79%)
12.12 (13.71%)
CALM (gpt4o-mini)
6,273 (10.54%)
8.691 (11.88%)
12.104 (13.56%)
MCTS-AHD (Qwen, w/o GRPO)
6,257 (10.26%)
8.746 (12.58%)
12.268 (15.09%)
MCTS-AHD (Qwen, w/ GRPO)
6,254 (10.19%)
8.722 (12.28%)
12.228 (14.72%)
CALM (Qwen, w/o GRPO)
6,262 (10.35%)
8.720 (12.26%)
12.219 (14.63%)
CALM (Qwen, w/ GRPO)
6,244 (10.04%)
8.668 (11.58%)
12.088 (13.41 %)
Table 16: Comparison of how GRPO affects the performance of MCTS-AHD and CALM on TSP.
1.10 RESULTS ON THE TSPLIB
We have evaluated the strongest heuristic produced by our method on all TSPLib (Remell,, 1991) instances for which MCTS-AHD reported results. For each instance, the coordinates of all points were first scaled to the interval [0, using one shared scalar to be consistent with the training instances that led to the discovered heuristics. Then, each normalized instance was solved three times with different random starting nodes, and report the average performance. Tab. II presents these results. Performance data for all baselines, including widely used hand-crafted heuristics such Christofides (Christofrdes 2022), Greedy (Brecklinghuus and Hougardy, 2ITE), Nearest insertion, and Nearest-greed greedyy (Rosenkrantz et all, 1972),, as well as the method GPHH (Duflo et all, 2019) and recent LLMMbasee AHD approaches including EoH (Lm et all 20248), ReEvo (Ye all R022), and MCTTSAHD (Zheng et all, 20225,, are taken directly from the MCTS-AHD appendix for consistency. The heuristic produced by our framework shows clear advantage across the benchmark suite. It attains the lowest average optimality gap among all compared methods, achieves the best tour cost on instances which the highest win count, and outperforms the strongest heuristic found by MCTS-AHD 9 of the 15 instances.
AHD
Christofides
Greedy
insertion
Nearest-rredly
GPHH-hhst
12225
5.67%
5.38%
19.93%
16.82%
7.71%
5.57%
6.56%
10.84%
8.02%
9.43%
12.16%
23,05%
23.25%
14.059
10.79%
10.55%
lin318
13.80%
18.75%
24.44%
25.78%
14. 14.30%
14.03%
16.63%
14.07%
13.47%
cil.1
18.18%
13.03%
16.14%
31.96%
10.20%
8.37%
6.47%
15.98%)
3,54%
d493
12.41%
10.71%
kreB100
9.82%
16.59%
21.5%%
26 26.26%
14.06%
13.46%
12.20%
11.43%
9.44%
kroC100
9.08%
12.94%
24.25%
25.76%
16.22%
16.85%
15.88%
8.27
3,87
px290
11.23%
31.42%
25.055%
31.42%
18.24%
23 5A1
20.63%
11.23%
11.73%
B417
15.57%
12.649%
25.52%
32.42%
22.72%
20.477%
19.15%
10.20%
12.05%
16.366%
P2264
11.28%
11.890 1998
34.28%
17.87%
23.96'
18.09%
16.7896
12.27%
12.17%
p2226
14.17%
21.44%
28.02%
24.65%
15.51%
19. 90%
18.02%
7.15%
15.14%
19.25%
Average Gup
11.50%
18.09%
23.11%
25.41%
16.00%
15.87%
13.95%
11.10%
10.09%
Table Performance of the best heuristic discovered by CALM on TSPLib instances, with the best result for each instance shown in bold.
33

---

Published as a conference paper at ICLR 2026
1.11 IMPACT OF DPO ON CALM
We conducted additional experiments to compare CALM with DPO against the GRPO setting on the OBP task. For each prompt used by CALM, we generated two responses and evaluated them with the same reward function employed by CALM with GRPO. When the two responses received different scores, we treated them as a preference pair and updated the model using DPO. The corresponding results are reported in Tab. TT.
The experiments yield several observations. First, DPO improves the performance of CALM relative to the results reported in Table 4 of the manuscript and enables it to surpass strong baselines such as EoH and ReEvo under the GPT--4-mini model. Second, CALM trained with DPO outperforms EvoTune, which is also based on DPO, indicating that the components we designed for RL-based fine-tuning offer clear advantages even when adapted preference optimization. Third, the DPO variant of CALM still falls short of the recent MCTS-AHD method and of CALM trained with GRPO. Overall, these results show that DPO can provide meaningful gains for AHD, but they also suggest that additional algorithmic adjustments may be needed to fully realize its potential in this context.
1k 1k_100 100
1k_500
5k_100
5k_500 550
10k_100 100
10k_500
Avg.
BiH (GPPT-4n-mini)
2.69%
0.255%
1.63%
0.53%
1.47%
0.45%
1.17%
ReEvo (GPT-4o-mini)
3.94%
0.50%
2.72%
0.40%
2.39%
0.31%
1.71%
MCTS-AHD (GPT-4o-mini)
2.45%
0.50%
1.06%
0.32%
0.74%
0.26%
0.89%
CALM (Qwen, GRPO)
2.78%
0.29%
0.83%
0.28%
0.50%
0.24%
0.82%
CALM (Qwen, DPO)
3.63%
0.25%
1.06%
0.55%
0.61%
0.52%
1.10%
EvoTunee (Qwen, DPO)
4.67%
0.25%
4.23%
0.55%
4.11%
0.60%
2.40%
Table 18: Performance of CALM under the Qwen model with DPO
I.12 GENERATED HEURISITCS
The idea of the algorithm is to refine the scoring mechanism by introducing logarithnic adjustments and novel scoring component that captures the Logaritamic relationship between the remaining capacity and the square of the item size, and an adjusted logari thmic density term that provides more refined scoring mechanism. This new algorithm aims to enhance the accuracy of bin suitability assessment by adding component that adjusts the score based the logarithnic difference between the remaining capacity and the maximum bin capacity. The algorithm also simplifies the scoring steps to make it more elegant and efficient.
import numpy as np
def (itemii size: float, remaining_capacitty: np.ndarray) -> nd.ndarray: max_bin_cas = max (remaining
bin binsity (pp.smm(eeannnngg_aaaaa
(item_size
1en (remmannin ctt.adiiinn_ppacttt)))
capacity))
log = np. 1og wwweeeeiinggaapaeeeeeeeeeee capacity + 1) np.log (max bin_cap + 1)
score - (remaining
max_bin_cap) x*2 iter_size +
remaining__cpacciyy-e
(item_sizer.2) remaining_capacityee /
(iter_size++3) bin_density remaining_capacityy
score proree[remaninggcappele > item size] = -score (remaining_cspacctyy >
item_sizel score(1: score[:-1]
score += log_adj
score += log_adj + remaining_capacity
score log_adj
34

---

Published as a conference paper at ICLR 2026
component semaining_capaciin / (item size remaining_capacil
score += nev_conponent
new_compenent
np. log (remaining_capacttg
/ /item_size size np. 1og (max_bin_cap 1)] (1
samaining_capcctte item_size)
score += new_conponent
adjustment (remaining_capacee score neu_adjustment
size_size) _og_adj
remainiing_aapcccyechheeee - remaining_capacityy item_size
score += ap. log (
1)
np.log (max bin_cap 1)
new_compenent tromaining_capaciee) - 1) (item_size rcmaining_capacityy + 1) log_adj / rp.log imax_bin_cap + 1)
score += new_component
new_component log_adj
remainingg_cppcctty (item_size
remain comaininggcaaacitae.
score += new componett
new component camaining_capacieei np.
1)
/ (iter_size**) * (1 remaining_capacityy item_size)
score += new_component
return score
Heuristic 5: OBP, by CALM (local, w/ GRPO)
n
The idea of the algorithm is to introduce the "Bin Utilization Diminution" component, which assesses the degree of bin usage throughout the sequence of placements and introduces diminishing incentive for overpopuLating any particular bin beyond a certain threshold. This encourages a more even distribution of item placements across all bins, thereby reducing the risk of reaching capacity too quickly in any single bin, helping to extend the lifespan and utility of each bin in the packing process. By dynamically adjusting the fit score to favor items that contribute to a balanced utilization, the algorithm aims to enhance overall bin efficiency and minimize the total bin count.
import numpy as np
def (itemma_mssze: float, samaining_capaciinn ad.ndarray) -> np. ndarray: avg wrg_item_at size np mean size) if item size 0 else 1.0 snaptiveffacton ave_ size / np. mp..axmmmmmmmmmmmmmmmmmmmmmmmmmm (remaining cappciity 1e-10)
fit_score np. maximum (remaining_capacityy item_size, (remainingg_appccitty le-10)
fit_score fit_score araaocceereemainnng_aaaaaiiiiiii < iten_size] -np.inf
sustainability_score (remaining_capaccty avg [tg_item__izee 2
sustaimabilltty.ccore [remaining_capacity < ittm_sizz] np. inf
iistoricallfit__ccore np.
hormaLLeet_isstrriiaa_fttt historical (np. (hi whwoiiiisstiii...ooo _fit__corees_ 1e-10)
historical fit_
35

---

Published as a conference paper at ICLR 2026
combined scores
(0.5 * fit_score adaptive_factor)
(0.3 / (sustainability_ecrr) 1e-10))
(0.2 * normalieechssttrrllllli
di d 0.1)
(1 + np.arange (len (eeereeeaiiiiggcccpcctt)
combined += differentation_factoo
cumulative_fitt_inpaaa np.cumsum (fit_score) (np.arange
len (en 1) + 1)
np. mp.maximum (fit_scoor
comulative fit impact, 0)
combined_scores += 0.4 cumulative_fft__ahistte
temporal_utillaatoonnn..tt = np.arange (1en (romalniing_ccppcctt)) /
(np. wwp.maaimmmmmmmmmmmmmmmmmmm
1e-10) 1c-10)
combined_scores temporal_utiliattiooommeett
sequential, _el__elsttccty np. http::/.aarraggggeeeeeeeaiiinnnn
(ecuentialLelaatiicitz
(len
mnan (np.maximum maximum catmmmiiinnn_appaciiieo le-10)) + 1e-10))
combined_scores sequential elasticity
size_factor + (item_size (np. sum (itte_sssize) + 1e-10))
New Component: Bin Utilization Diminution
overutilization_penee perallyyyeeellyyyeaeeaaaaaalayya mp.maximum (0, cp.ccmsum (item_ _iize) (ap.maximum (np. cunsum (remaininggcapppceeeteccee
1e-10) 1e-10) 1)
combined_scores - 0.3 overutilization_peet penalty # Encourage even
distribution across bins
Eventual Capacity Influence
eventual_capaciiyy_cet len (wemarniinnnn (reeeeeinii np. log (np. maximum (np.arange
1), 1))
(np. ma..mum (rema ingmaiiiggg_ccppcccttt
1e-10) 1e-10)
combi ned_scores 0.3
Penalize bins
that don't contribute to optimal utilization
distinct_score = combined combined_scores scores size_factor factor
return distinct_scores
Heuristic 6: OBP, by CALM (API, w/o GRPO)
n
The idea of the algorithm is to further refine the savings potential calculation by emphasi zing more adaptive balance factor that is influenced by the current instance's capacity utilization and the diversity of capacity usage across the routing problen. By Leveraging a more sophisticated adaptive balance factor and reducing the complexity of the penalty factor, we ensure that nodes that are too close to each other are penalized appropriately without overly compounding the impact. This simplified yet adaptive approach allows for a nuanced exploration of the solution. space, enhancing the ACO algorithn's ability to converge to high-quality solutions while maintaining balance between exploration and exploitation.
Additionally, introduce basstering-based adjustment factor that captures the overall network connectivity and adjusts the savings potential accordingly, leading to more robust and flexible routing plans.
36

---

Published as a conference paper at ICLR 2026
import numpy np
def advanced_houristice_ wiiisseerrdssstttcenatrrri np.ndarray, coordinates:
np.ndarray, demands: no. ndarray, capacity; int) -> .ddaraa;;:: capacity_prab demands / capacity
distancee_reciproca
distance_matria
proximity_factor np. ap.linalg.nerr (coordinatts (croooiiatttt np.newaxis, :]
coordinates Inp.nexxxs, i, :,, axis=2)
proximity_factor /= max (proxintttt__ and proximity_ as proximity increases
Normalize between 0
proximity_factor Invert for higher penalty
remaining demands capacity demands
cuture_savinne (remaining demands np.newaxis]
brgainingg_emandssy / balastanne_maartt
bemaaining_demmrddee [:,
np.newaxis] ronaining_demandss)
capacity_ratio remaining_ddmands: / capacity
proximity_savings proximity_factor capacity_ratic
Cluster-based proximity adaptive savings potential
cluster_savings np. (ctoslliiik(dsstttneemaaaii
cluster_distance = np.
axis=1) /
1, ord=1)
cluster_adj_ (remaining demands :, nр.newaxis]
[
remainingg_dmmnde = cluster_distance ** 3.5) / (distance_matrix
[eerainiig__eeandds.... np.newaxis] remaining_demands)
Adaptive balance factor adjusted based remaining capacity and cluster adjustment
Dalance_factor np. min ([1, 079 0.05 capacity.prob.mean 0.03 sattos___d___ctttrooao ap.var (capacity_prob))
+ 0.005
Penalty factor that heavily penalizes nodes that are too close to each other, focusing ng on the proximity to the next node fentlty_factor proximity_factor **
Combine all components
probability = distance recpppocccciaal capacity_praae prob * proximity factor
future_savingg
lenkiiity_saviine cluster_ade _ete__ad_faattorr factor (1
balance_factor proximity_savings salance_factors (1
penalty_factor) (1 cluster_adj_aatto 0.
return probability
Heuristic 7: CVRP, by CALM(local, w/ GRPO)
The idea of the algorithm is to refine the credit al allocation process in the vehicle routing problem by implementing deterministic weigating mechanism that assigns distinct credits to customers based on their delivery demands, individual distance factors, and their influence on overall routing efficiency, thus ensuring that credits reflect meaningful diffrrnnces without redundancy.
import numpy as np from Aklearn.cluster import DBSCAN
def heuristics (distance_matria ap.ndarray, coordinates: op.ndarray,
demands: np. ad.arrayy,, capacity: int) -> ndarraay:: cum_eusseeess demands. shape cumulati sumulative_penall ve_penalty np. zeros (nur_customers)
37

---

Published as a conference paper at ICLR 2026
Calculate baseline scores from demand to distance with added urgency weighting
urgency_weight np. ww.spacacceeecc (1, 1.5, nun_customers)
basc_sccre (demands urgghcy_weight) / (distance_matrix + 1c-5) base_score (np. isnan
Set penalties for exceeding capacity based on cumulative demands for i in range (num_oustomers))
current surrent_demand demands
somulativv_pennalee
max current demand capacity)
Normal ize distances to emphasize closer customers to refine scoring - idg.clllidissttnnmmaarkkk 1e-5,
None) ** 2.5)
Calculate effective capacity utilization adjustment effec..ve_cppadlyyttilllatt = mp.clip (capacity - demands) / capacity, 0, 1)
Historical performance adjustments
attorical_performmnce..
np. zeros (num_custtomer
for i in range (num_cuuu [nmmaasttomers):
| performance facter lil mp.mean laapaa__scorre [i] for j
in range (num customers) if distance matrix[i]li] 10 and != il)
Spatial clustering mechanism clustering_model DBSCAN (eps=5, min_samples-2) fit (coordinates) labels clustering model. llabels..... cluster_scone scores np. zeros cau__ccsstmers
Calculate cluster-based sccres with deterministic differentiation for cluster_id in set (labels):
if cluster_id -1: Ignore noise points
cluster_indicn ces np. where (labels cluster_id) [0]
total_demand - [clanes[ for idx in cluster_indices.
ses]
Implement differentiated scori ng based on demand,
ensuring non-equal credits
peister_demand_ factor = (demands [idx] / demand) if
total_ demand > else 0
distance_weight = /
distance_matrix [idx] min ())
Closer customers get more weight
clustor_scores [idx]
mnstterdemmadd_fae
distance_weiigh # Mix demand and distance
New resilience score based historical demand variability np. std (demands)
desilience_scors = 1 / (1 demand variadiiiara
Compose final scores combining all elements including the new resilience score
final_scone [haa_ssorrss acse_score nermalized diddiiieeddstttce___orrro
Effective_capaciie sttp:reeepppctttyyiiiiiiiio
(1
historical performance bistoorccelpeerfomaaneeeee factor + cluster_scores
resilience_score
return finae_scores
Heuristic 8: CVRP, by CALM (API, w/o GRPO)
The idea of the algorithm is to refine the exploration-exper exploitation trade-off by introducing sinusoidal decay that
38

---

Published as a conference paper at ICLR 2026
incorporates sinusoidal penalty with sinusoidal smoothness adjustment. This adjustment helps to smooth the preference for both recent and distant nodes, leading to a more balanced and improved performance.
import numpy as np
def enhanced herreeessrrrssttssprrsses ap.ndarray, distance: ap.ndarray, max_en:
float) -> ap.ndarryy:
Exponential decay for immediate high Subscription nodes
exp_ratic ep.exp (prizez (np. :] / distance maxlen)
Logarithmic scaling for exploration
log rag_ratic mp. (prize (np.newaxis,
distance
sinuscidal decay for recent nodes with sinusoidal smoothness adjustment
sinuscidal pinuslldalppenalty 0.5
np. sin (np.pi distance (maxlen 1))) (distance maxlen) maxler
Combined ratio maaiined_ratio ratio regio (1 S'nusoidal_penatl
Ensure the ratio is non-negative combined_ratio [combined_rateee 0]
return raatined_ratio
Heuristic 9: OP, by CALM (local, GRPO)
The idea of the algorithm to introduce a novel component called "reward fluctuation sensitivity" which adjusts the desirability of each Location based on the variability of rewards over time. This component accounts for the possibility that rewards may change fluctuate due to external factors, thereby allowing the agent to prioritize locations not only by their current rewards but also by the potential volatility of those rewards. This sensitivity is integrated into the existing framework, allowing for more dynamic response to the changing landscape rewards, ultimately enhancing the agent decision-naking process and route optimization.
import numpy np
def souristics(prize no.ndarray, distance: nd.ndarray, maxlen: float)
-> mp. acjusted_distance distance 1e-10 Avoid division by zero intential_reware np. zeros like (prize)
for i in [leg (prize)))
reachable_indices np. where <= maxlen) [0] potential_reward[1] np. sum (prize [reachable_indicess) if reachable_indices..cu
0 else
reward hiard_hhst__ee hist_factor potential_renee aewaardreeeerd__eeee / (1 + np. (prize)) ap.sum(prize) Shape
zeward based on historical performance
reward_decay np.exp (-adjusted_distancee maxlen) Decay effect for distant rewards
proximity_factor = (maxlen disustncddistneee ** 4 # Further enhance proximity impact with quartic di diancece
froximity_factee [proximi factter
39

---

Published as a conference paper at ICLR 2026
ciered_adjustment (prize (adjusted_distanee 1e-10)) **
Classify rewards into categories for ticring
Reward volatility assessment component
volatility_factor roozzers_likepprizzs range (1en (len(prize)
for historical_eewards prizelnps lapeelpp.eeer...... (distance [1] <= maxlen) [0]]
if historical_eewadd...o size 1:
volatility_factorrii np.mean (historicall_
(historiica rewardss /
# Coefficient of variation
Risk-reward analysis component
haabbiiity_fccttooe np. zeros zeros_ for in range (Len reworrdssrrewardssssooooaa prize [mpzeenp...r where where (distance [i] maxlen) [0]]
if historical_ variability_aactorill] mp.mean (hissttrcccal
pp.stahhissttrrccattewwddd Basic difforentiation
Final_heuristic (reward_hist_factee reward_decay
proxirity_factor
Llere__adjustmentt / (1 variability 1e-10)
return final heuristic
Heuristic 10: OP, by CALM (API, w/o GRPO)
The idea of the algorithm is select the next node by optimizing heuristic that considers the distance the current node, the average distance to unvisited nodes, the variance distances to the current node from the unvisited nodes, the entropy of distances to the destination node from each of the unvisited nodes, the average distance from the destination node to each of the unvisitea nodes, the current node's distance to the destination node, and the standard deviation of the overall tour distances. This proposed algorithm aims to roduce of the current node from the average tour length and balances it with the entropy term to reduce the overall tour length. Additicnally, this method assigns more weight deviation of the distances from the destination node to each of the unvisited nodes, which helps in reducing the variability of distances and thus leading more consistent and shorter tour lengths.
new term that captures the deviation
the standard
import numpy np
def select next node (current node: int, destination node: int, unvisited nodes: set, distance_matring matrix: nd.naaryyy) -> int: scores ()
for node unvisited_nodes:
all_distances [distance_matrixx (nodde))x(]ood]] [i] for i in
unvisited_node if i != node]
average disteece_dtaeeeeeeee = np mpa. (all_ddsanns]))) Slandard_deviation
(=lstd distances))) variance_of mp_stddall_distance))
ap var hww.daaaneetaarrllaaaaa ccureettourr [1]
for i in unvisited_nodes if != nodel)
entropy_of of__tdnsssccssssseeeeeeeessssee_ -np -np.sum (np. 1og2 [[di [ddisttncemmatri unvisited nodes if nodel) len (onvissitee__odee)
[i] for
40

---

Published as a conference paper at ICLR 2026
average distance to destination np.mean (ateee/aaaaiiiessstttttttttteeae [i] in
unvisited_nodes if != node])
score
0.6 distaace__mattxxcrrreetttoooo [node]
[node]
0. 4 average_distancee
+ 0.3 * standard_deviatioor
0.2 entropy_ot.distance,
0.1 tistaase_aatiilleesiiittt...o.n _ode| [node]
0.08 variaacee_o__distances
0.05
0.01 averas__discooooe_teeeeesstaiiii
htwpeee_mmarriccuureee_eeeddee [i] for in
modes]] - -
distance)
0.005 entropy_of_cistance,
0.008 * Aistance__matris
[current_node) node] [nede]
distanco_mattix(nodde [destination__odyy
0.006 standard_deviation
cestance_natrixxe
[destination_nod)]
scores [node] score
next_node min (scores, key=scores.get) return next node
Heuristic 11: TSP, by CALM (local, w/ GRPO)
The idea of the algorithm is to select the next node to visit from the unvisited nodes, incorporating novel component of dynamic path optimization feodback. The component analyzes previous decision points in the taken, adjusting future node selection to favor pathways that have historically resulted in lower overall traversal costs. This method not only enhances the algorithm's ability to learn from its own experiences but also promotes the selection of routes that align with optimal connectivity patterns established during the tour.
determine the effectiveness
the routes
import numpy np
def select next node (current ceurrent_ncde: node: int, destination node: int, wwrisited_nodesse set, cestance_matrixe ndarrayyyy) -> int: threshold = 0.7
cl, c2, c3, c4, c5 0.4, 0.3, 0.2, 0.1, 0.1
scores (}
for node in invisited_nodes:
all_distances
[i] for i in
unvisited_noder if != nodel
average average_distancee distance_to_uuse ttt.raanee_tt_uvvitteteteeeee unvisited np. ap.mean (all distances)
np. wtddltallll
New component: consider dynamic path optimization feedback feedback_paths (cistance_matrixfi] [node] for range (len (distance_matrill
if i not in unvisited_nodes and
distance distance_matrix wwt.r/mmccrreettooooooooooo [i] < [treehold]
average haaee__eeebb_kkkeee distaaceeeeeeeeeee np. mean (feedback_paths) if
feedback_pathr else
score
cl aettaneemmttiix/curree... _ [node]
c2 average__distance_to_umum sited std_dev_di
41

---

Published as a conference paper at ICLR 2026
distance distance_ destaaitidiiiiitiinnonnnooooooo node] [node]
average_feddcck_lsttnne
scores [node] = score
next_ min (scores, key=sccres.
return next_node
Heuristic 12: TSP, by CALM (API, w/o GRPO)
J LIMITATIONS
current limitation of our method is that the evolution of the LLM during the heuristic discovery process depends heavily on performance signals derived from heuristics present the prompt and response. As a result, trajectories that do not contain explicit heuristics (c.g., the response from rellection prompt may contain the thoughts only) either component provide no reward signal, limiting the LLMs ability to learn from such cases.
Another limitation that we currently evaluate our method. CALM, using a compact LLM on single 24GB GPU. This restriction is primarily due to limited computational resources and the high cost associated with high-accuracy, full full-parameter fine-tuning on larger models. While this setup demonstrates the feasibility of our approach in resource-constrained environment, further evalua- tion on larger-scale models and infrastructure would be valuable for understanding the methods full potential and scalability.
future work, we aim to address these limitations by (I) exploring mechanisms for adapting the LLM in the absence of explicit performance feedback, enabling more effective use of reinforcement learning, and (2) extending evaluations to more powerful models and settings. These directions may allow for better integration with techniques such reflection et 2024 Dat et all 2025), which have shown promise in enhancing LL.M-based automated heuristic discovery.
K BROADER IMPACT
The CALM framework stands to greatly accelerate the pace of innovation in algorithm design by seamlessly integrating prompt engineering and on-the-fly model adaptation. By enabling state-of- the-art heuristic discovery on a single 24 GB GPU, CALM democratizes access to cutting-edge Automatic Heuristic Design. This empowers research groups, startups, and educational institutions with limited compute budgets to explore and deploy high-performance solutions in domains such as logistics, scheduling, and resource allocation.
L
LICENSE
The licenses and URLs baselines, models, and softwares are summarized in Table
42

---

Published as a conference paper at ICLR 2026
Table 19: A summary of licenses.
Resources
Type
License
URL
Unsloth
Code
Apache-2.0 License
con
Lofa
Qwen2.5
Model
Apache-2.0 License
nttps:/ httpe////hiiooaaccccooooooooooo
LKH3
Code
Available for academic research use
htLp://eehhoee14 wwc.ooll...c..d/..oo..
search LX5-37
OR-Tools
Code
MIT License
ntt /ewp:////eeeeeeeeeee 1ow.reeleees.o02200 h
op
1m1 2at
POMO
Code
Available online
nt
cttps://0 hub
FO
0e master
DeepACO
Code
MIT License
pttpst//qtthb.co htw.:uub..ommm..yyyy...... Dee
BACO
Funsearch
Code
Apache License
https://qithbb. www.tthh..ccomooooooooooo deepm
47 Fansearch
EoH
Code
MIT License
3tTp://c1faB.ccm/ellllu
rree/ma17
ReEvo HSEvo
Code
MIT License
Code
Available online
pttp:://qithbbcco
1SR
MCTS-AHD
Code
MIT License
htpa://gthnb.com/222
BD master
EvoTune
Code
MIT License
nctps: Tctps://attabb.com/lll Labo Eg
OpenEvolive
Code
Apach-2.0 License
nt p8: www.tttubb...mm www.ub.cooccoott on oper
evo ve
43


Reviews:
[{"review_id": "jsKakwuPAB", "reviewer": "Reviewer_STCg", "summary": "This paper presents CALM (Co-evolution of Algorithms and Language Model), a framework for automatic heuristic design where large language models (LLMs) evolve jointly with the heuristics they generate. CALM combines evolutionary prompt search with reinforcement learning fine-tuning (GRPO) to adapt both the prompt and the model, introducing a probabilistic collapse mechanism and a relative reward function to guide stable co-evolution. Experiments on four combinatorial optimization tasks show that CALM outperforms state-of-the-art LLM-based baselines while running efficiently.", "questions": "1. The co-evolution of prompts and the LLM itself raises legitimate stability concerns, as simultaneous adaptation of two coupled components can yield non-stationary learning dynamics. CALM mitigates this through sequential update scheduling, conservative GRPO fine-tuning with KL regularization, limited LoRA updates, and a collapse mechanism that periodically resets the population (Sec. 4; Appendix C–G). These choices are reasonable and appear to prevent divergence—the training curves in Fig. 2 are smooth, and all tasks are completed successfully. However, the paper provides no explicit stability analysis or ablation contrasting co-evolution with prompt-only training, leaving the robustness of this dual-optimization setup empirically supported but theoretically uncharacterized.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 3, "strengths": "1. The related work section is clearly written and situates the method well within the context of prior research.\n\n2. The paper provides a clear mathematical definition of the task and notations, which are often missing or ambiguous in previous work.\n\n3. The interpretation of LLM-based evolutionary heuristic search as a general reinforcement learning process is insightful and conceptually interesting.", "weaknesses": "1. The use of unique token overlap in the “idea” text as a diversity metric feels superficial. Simply counting different words doesn’t necessarily indicate whether two heuristics are algorithmically or behaviorally distinct, so the diversity might be overestimated. It’s efficient, but I think something like embedding-based similarity or even code-level structure comparison would better capture the real heuristic diversity.\n\n2. My biggest concern is still about the RL-based LLM fine-tuning. RL is known to be very sample-inefficient, and here each update needs running and scoring several heuristics, which is computationally expensive. This makes the approach hard to scale to larger problems or longer runs. Also, the performance improvement from fine-tuning (Table 4; Appendix I.2) looks quite moderate and might mainly come from other parts of the design. The fine-tuning seems to make the model more stable and consistent, but I’m not convinced it’s the main reason for the performance boost.\n\n3. Another issue is about the experiments that mix results from GPT and Qwen models. The actual fine-tuning is only done on the open-source Qwen2.5 model with Unsloth. Since only one baseline result is available for Qwen2.5, comparing it with the GPT-4o-mini results feels a bit unfair. I understand the limitation of using closed models, but it would be better to include more baselines tested under the same Qwen model for a fairer comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BKPIE1XZZy", "reviewer": "Reviewer_9Hpb", "summary": "The paper proposes CALM, an LLM-based Automatic Heuristic Design (AHD) framework that co-evolves the prompted search over heuristics with on-the-fly RL fine-tuning of the LLM via GRPO. Beyond standard “verbal gradients” (prompt-driven operators), CALM adds “numerical gradients” by rewarding generated heuristics based on feasibility/novelty and relative performance to parent heuristics. It further introduces fine-granularity mutation (injection/replacement), a diversity-aware crossover, and a probabilistic collapse mechanism to escape stagnation. Experiments show consistent gains over prior LLM-AHD baselines, often using only a quantized 7B model on a single 24 GB GPU; ablations indicate GRPO is the main contributor.", "questions": "1. Why have hardcoded operators via sampling? Would using an LLM generate modifications instead be sufficient?\n    \n2. Convergence with fewer LLM queries appears to be worse than other methods (see Fig 2). How could you explain this?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is easy to follow and well-written\n    \n2. The idea is good and has practical relevance\n    \n3. Emphasis on local models with low compute budget\n    \n4. Code is provided\n    \n5. Good results compared to some SOTA algorithms", "weaknesses": "1. Authors do not report performance on public benchmarkse.g. TSPLib, which would make it easier to compare CALM against other methods\n    \n2. The performance gain in non-GRPO settings is not too clear. For example, it seems that MCTS-AHD outperforms CALM without GRPO, which makes me wonder about why not finetune MCTS-AHD ‘s LLM too, and if s,o what the performance would be\n    \n3. The novelty of finetuning LLMs for AHD is not too much due to the existence of concurrent works in finetuning, although CALM uses a different algorithm (GRPS vs DPO)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gHPEKICxNL", "reviewer": "Reviewer_ny5L", "summary": "This paper proposes a GRPO-based automatic heuristic design framework which employs LLMs to iteratively modify optimization codes to generate better performing algorithms. Authors decompose mutation operators into injection and replacement. Diversity-aware crossover and collapse mechanism is introduced to enhance diversity, and a progressive reward function is designed to measure the code quality. Experimental results show that the proposed method surpasses heuristic design baselines on multiple optimization tasks.", "questions": "See Weaknesses.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. CALM is presented as one of the first LLM-based AHD frameworks to incorporate GRPO\n\n2. This work proposes new heuristic design operators such as injection, replacement and collapse, which could provide insights for future works. The proof and discussion about the introduction of collapse is provided.\n\n3. Experimental results show that CALM outperforms LLM-based heuristic design baselines on multiple combinatorial optimization tasks.", "weaknesses": "1. In line 206-207, the authors claim they use compact summaries instead of full code; however, the response examples in Appendix E show generated operators as exact code, like mutation operators in prior LLM-based AHD methods. Additionally, focusing on granular modifications might limit the search space and reduce code diversity.\n\n2. The proposed method uses seed heuristics to generate initial heuristics; however, the authors do not specify the methodology for selecting these seed heuristics. The specific seed heuristics used in experiments are not reported, and the impact of seed heuristics is not discussed. Since the subsequent heuristics are all derived from the seed heuristics, this selection could significantly affect the performance.\n\n3. In reward design, $r_{invalid} \\in (-1, 0)$, $\\alpha_1, \\alpha_2, \\Delta(\\cdot) \\in (0, 1)$ and $\\alpha_1 > \\alpha_2$, meaning $\\alpha_1 r_{invalid}$ (the first term in Eq. (4) ) is more negative than $\\alpha_2 r_{invalid} \\Delta(\\cdot)$ (the second term). This preference for worse steps over no-improvement steps appears questionable. Additionally, in Appendix H.1, authors propose 5 new reward parameters, $r_3 \\sim r_7$ (where are $r_1$ and $r_2$ ? ), which do not appear in Eq. (4). The detail of how they are used is absent. Furthermore, the values of $\\alpha_1$ and $\\alpha_2$ are not reported.\n\n4. In the experiments, baselines are compared under different base LLMs with different knowledge and capabilities, potentially introducing bias. The source of CALM's advantage remains unclear—would Qwen-based baselines outperform it?  For instance, the performance of o4-mini with CALM without GRPO in Table 9 dominates all baselines in both Table 1 and Table 9, potentially reduces the contribution of this paper.\n\n5. The presentation should be improved. For example, line 312 and 319 miss periods at the end of the sentences. Algorithm 1 abuses the equal sign which refers to both equations and assignments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3NG0MQIy4W", "reviewer": "Reviewer_SGF5", "summary": "This manuscript proposed CALM to generate heuristics.", "questions": "**Q1:** The authors measure diversity by counting unique tokens in a heuristic’s idea. Would using a deep neural network to embed each heuristic idea and compute diversity in that embedding space be more effective?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**S1:**  This manuscript is well-structured.\n\n**S2:**  This paper fine-tunes LLMs to generate heuristics, which represents a novel contribution.\n\n**S3:**  The experiments demonstrate the effectiveness of each proposed method.", "weaknesses": "**W1:**  This manuscript should introduce more LLM-based Heuristics Generation methods, such as [1,2].\n\n**W2:**  This manuscript lacks experimental comparisons between GRPO and fine-tuning strategies such as DPO.\n\n[1] Efficient heuristics generation for solving combinatorial optimization problems using large language models. KDD, 2025.\n\n[2] Multi-objective Evolution of Heuristic Using Large Language Model. AAAI, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)