Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI
Authors: Julien Pourcel
Cédric Colas
Pierre-Yves Oudeyer
Venue: ICML 2025 poster (2025)
Keywords: Program synthesis; Self-Improvement; Search; ARC; LLM
Primary area: deep_learning->large_language_models
TL;DR: We propose SOAR, a method that learns program induction by integrating language models into a self-improving evolutionary loop, outperforming previous results among program induction methods on the ARC-AGI benchmark.

Abstract: Many program synthesis tasks prove too challenging for even state-of-the-art language models to solve in single attempts. Search-based evolutionary methods offer a promising alternative by exploring solution spaces iteratively, but their effectiveness remain limited by the fixed capabilities of the underlying generative model. 
We propose SOAR, a method that learns program synthesis by integrating language models into a self-improving evolutionary loop. 
SOAR alternates between (1) an evolutionary search that uses an LLM to sample and refine candidate solutions, and (2) a hindsight learning phase that converts search attempts into valid problem-solution pairs used to fine-tune the LLM's sampling and refinement capabilities—enabling increasingly effective search in subsequent iterations.
On the challenging ARC-AGI benchmark, SOAR achieves significant performance gains across model scales and iterations, leveraging positive transfer between the sampling and refinement finetuning tasks. These improvements carry over to test-time adaptation, enabling SOAR to solve 52\% of the public test set.

Self- Tel--impgo Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI
Julien Pourcel Cédric Colas* 12 Pierre- Oudeyer*
Abstract
search through the space of possible programs, using meth- ods like genetic programming sequential Bayesian infer- ence (Koza, 1994; Liang et al., 2010). These approaches generate initial candidates based on task constraints, then iteratively refine them through mutation and crossover op- erations. However, their effectiveness depends heavily on having intelligent program generators and mutation opera- tors without these, algorithms must expend massive com- putation, blindly exploring the space of possible solutions.
Many program synthesis tasks prove too challeng- ing for even state-of-the-ar language models to solve in single attempts. Search-basee evolution- ary methods offer a promising alternative by ex- ploring solution spaces iteratively, but their effec- tiveness remain limited by the fixed capabilities of the underlying generative model. We propose SOAR, method that learns program synthesis by integrating language models into a self-improving evolutionary loop. SOAR alternates between (I) an evolutionary search that uses an LLM to sam- ple and refine candidate solutions, and (2) hind- sight learning phase that converts search attempts into valid rroblmm-souution pairs used to fine-tune the LLM's sampling and refinement capabilities- - enabling increasingly effective search in subse- quent iterations. On the challenging ARC-AGI benchmark, SOAR achieves significant perfor- mance gains across model scales and iterations, leveraging positive transfer between the sampling and refinement finetuning tasks. These improve- ments carry over to test-time adaptation, enabling SOAR to solve 52% of the public test set.'
Large language models (LLMs) have marked new turn in program synthesis by acting as powerful program genera- tors (Roziere et al., 2023; Guo et al., 2024), solving many tasks in a single attempt (Li Ellis, 2024). For harder problems, they can serve intelligent operators for evolu- tionary search, proposing targeted modifications to existing solutions (Lehman et al., 2023; Olausson et al., 2023; Mey- erson et al., 2024). But these approaches face a fundamental limitation: the capabilities of the model used for sampling and refinement remain fixed, and simply sampling more candidates or trying more refinements yields diminishing returns. This paper introduces a system that learns sample and refine programs from past synthesis attempts, enabling sustained performance improvements beyond the limits of search-based methods.
We propose Self Self-improving Operators for Automated pro- gram Refinements (SOAR), framework that integrates language models into a self-improving evolutionary loop. SOAR alternates between two phases: first, using L.LM to sample and refine candidate programs through evolution- ary search Ceample&Refine phase), then using these search traces to fine-tune the model's sampling and refinement ca- pabilities. This creates a virtuous cycle- - better models enable more effective search, which in turn provides bet- ter training data for further model improvements. Unlike previous approaches that rely on human-engineered domain- specific languages to scaffold search, numann-generaue solutions to finetune program generators, SOAR learns to synthesize programs in Python, learning solely from its own synthesis attempts, including both successes and failures.
1. Introduction
Program synthesis promises to transform how humans inter- act with computers by automatically discovering programs that satisfy their intent. Instead of writing precise instruc- tions, users can express their goals through constraints, ex- amples, or natural language, letting synthesis algorithms figure out the implementation details. However, finding program that satisfies all constraints may be challenging due to the vast space of possible implementations.
Traditional program synthesis approaches rely on iterated
'qqual supervision 'Inria 2MIT. Correspondence to: Julien Pourcel <julien. cooleen.pouceleeee @ frr.
Proceedings of the nnnd International Conference on Machine I Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
We demonstrate SOAR's effectiveness on the Abstraction and Reasoning Corpus (ARC), a program synthesis bench- mark specifically designed to challenge Al models' core
Our code is open-sourced at: github..ommffoeeestaamSSAA

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
ARC task
Program solution
Evolutionary Search
Learning
tracatcralarte lat: Last cint
ARC
Doetuned LLM (init. base LLM)
csteat (b-h for Go umpare eutpun
Lecaree
For 4-3 za TOA-5t for la rasuir'' fan raequtas grad Lectally? ra color feand
untfunnfora FT1
colore
Learned program sampling
Sampling & refinement L.M fine tuning
prie woweelilte Iill extuuttaai sutuett tal vil crad Letta- dl tae dann sosoeteltt ita estauttiats sutuett e and
solution
3k prog.
Learned program refinement
Data selection
wowelblle exttunt Hol sopvett tal batten daye (1 extrnntiicc (1
?
Majarity
6k prog.
vote
search traces
Figure/ 1. Overview of the SOAR architecture solving a task from the Abstract Reasoning Corpus. Each ARC task implicitly encodes for grid transformation f demonstrated via examples (Zimin Monin) such that í (dimii)) = Yisinn To solve task, one must find the output grids Yest corresponding to test input grids Tis. SOAR learns to synthesize transformation programs f in Python by alteraatng between an evolutionary search phase (sampling and refining candidate programs with an LLM) and a learning phase (finetuning the LLLM on previous synthesis attempts) eventually solving 52% of ARC-AGI public test set.
soning capabílities (Chollet, 2019). Each ARC task requires inferring from just a few examples an implicit transforma- tion mapping input colored grids to output grids. These transformations often involve fundamental concepts like ob- ject permanence, arithmetic, physics, or geometric relations. Current language models struggle with program synthesis on ARC: even the state-of-the-art GPT-4. I, or Claude-4- Sonnet could only solve 8.00% and 20.75% of the test tasks respectively. While search considerably improves the perfor- mance of our open-source models (from 1-2% to 14-26%), fixed model capabilities create a performance ceiling.
their own improvement, our work opens new possibilities for creating increasingly capable AI systems that can tackle complex reasoning tasks through combination of search and learned program refinement.
2. Related Work
Traditional program synthesis algorithms rely on iterated search algorithms like genetic programming or sequential Monte Carlo (Goldberg; Holland; Koza, 1994; Langdon & Poli, 2013; Liang al., 2010; Saad et al., 2019), where their effectiveness heavily depends well-engineered program priors and mutation operators. With the emergence of deep learning, recent work has explored learning these compo- nents from data: either training neural networks to sam- ple programs conditioned on input-output examples (Balog et al., 2016; Ellis et al., 2021), or training decision mech- anisms to speed up search (Shi et al., 2022). While these approaches rely on exhaustive search in constrained hand- defined programming languages, SOAR learns to synthesize programs in Python from past synthesis attempts, eliminat- ing the need for offline datasets or manual engineering.
Through iterative self- improveeete SOAR breaks through this ceiling. After four training iterations, SOAR solves an extra 10-19% tasks across model sizes. Given access to target tasks, but not to their ground truth solutions. SOAR learns to solve an extra 3-5% tasks across model sizes with test-time training. With a final test performance of 52%, our best model outperforms previous program synthesis methods based on open weight L.MM and without using any hand-crafter data. Importantly, these gains arise purely from the model learning to sample and refine better programs through its own search experience, without requiring task specific enginnerigie or training data.
Large language models have emerged as powerful tools for program synthesis (Roziere et al., 2023; Guo et al., 2024), both as direct solution generators (Li & Ellis, 2024) and as mutation operators for evolutionary search (Lehman et al., 2023; Olausson et al., 2023; Meyerson et al., 2024). These capabilities have enabled sophisticated search methods that can both refine and diversify solutions: e.g. generating di- verse difficult programming problems (Pourcel et al., 2024), high-quality poems (Bradley al., 2023) or interesting pat- terns in cellular automata (Kumar et al., 2024). Whether used for convergent or divergent search, these methods treat the LLM as a fixed component, preventing them from im- proving through experience. SOAR overcomes this lim- itation by continuously adapting its underlying language model through self-improvement.
Our work demonstrates how program synthesis systems can transcend the limitations of their base models through self-improvement. We present:
1. a framework that iteratively improves its evolutionary search capabilities by learning from its own search expe-
rience without human-engineered data,
2. a test-time training mechanism enabling continuous im- provement on target problems,
3. empirical evidence that iterative model improvement can help overcome the performance plateaus inherent to search methods,
4. state-of-the-at results for program synthesis leveraging open-source LLMs on ARC-AGI's public test set.
By showing how program synthesis systems can bootstrap
Recent work has shown that LLMs can learn to reason
2

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
by training on successful reasoning traces t self- generated (Zelikman et al., 2022; 2024; Guo et al., 2025), or produced by classical search algorithms (Gandhi et al., 2024). SOAR also internalizes search traces into the model, but can do so leveraging both successful and failed attempts, using hindsight learning. Moreover, SOAR also learns to refine solutions from past experience, and leverages these improved capabilities in a test-time evolutionary search.
3.1. Problem definition
The ARC benchmark is a canonical example of program- ming by example (PBE) (Menon al., 2013), where the goal is to synthesize program that satisfies a specification defined through input-output examples. Formally, in PBE, we aim to find a program I within a given programming language such that for every provided example pair (z. y), the program correctly maps inputs to outputs: y/ = f(x). This paradigm enables users to specify desired behavior im- plicidy through examples rather than writing explicit code.
The Abstraction and Reasoning Corpus (ARC) represents particularly challenging synthesis benchmark that has at- tracted significant attention over the past years (Chollet, 2019). ARC tasks can be solved in two ways: (1) direct prediction of output grids (transductive approach), or (2) pre- diction of a transformation program used to generate the output grids (inductive approach) (Li et al., 2024). First attempts at leveraging LLMs for transduction achieved poor results (Xu et al., 2023; Gendron et al., 2023; Mirchandani et al., 2023) but finetuning models on synthetic data brought significant performance gains (Li et al., 2024; Akyürek et al., 2024). Early inductive approaches leveraged heavily human- engineered domain -npeiiiieecifiicccc languages (DSL.) and exhaustive search (Hodel, 2023; Wind, 2020). More recent approaches used closed-source LLM for Python synthesis: sampling and refining populations of candidate programs (Wang et al., 2023; Li et al., 2024). Two recent projects trained LLMs to sample programs in ARC: Butt et al. (2024) used on- line RL and hindsight learning in a custom DSL, while Li et al. (2024) used human-generated Python solutions. With SOAR, we continuously refine search operators (sam- pling and refinement) solely from past synthesis attempts, achieving competitive performance while eliminating any dependence on human engineering and datasets.
In the specific case of ARC, each task consists of: 1. set of 2-10 training examples { Mrraia)) } where Timin and Vrrain are colored grid pairs;
2. set of test inputs {Fess } for evaluation.
The goal is to find a Python function f such that ff ( Dtrain for all training examples, and I (Tree) produces the correct (hidden) yeest.
Each grid is 2D array of size h x w with (h, w) E [1..30] where each cell contains an integer from 0 to 9 representing a color. The challenge lies in discovering the underlying transformation pattern from just a few examples. ARC composed of 400 train tasks to use for algorithm develop- ment (ARC-train), and 400 test tasks to for evaluation (ARC-test), with each task containing both training input- output examples to guide the inference and test onput-output grids to test it. Each ARC task encodes a new implicit transformation that may involve fundamental concepts like counting, arithmetic, pattern completion, or spatial reason- ing. This makes them relatively easy for humans, yet sur- prisingly difficult for AI systems (LeGris et al., 2024).
3. Method
3.2. Program synthesis as evolutionary search with LLM-based sampling and refinement
SOAR (Self-improving Operators for Automated program Refinements) is a framework that enables program synthesis systems to learn and improve from their own search experi- ences. Rather than relying on fixed language model to sam- ple or refine programs, SOAR implements a self-improving loop where the system's 's capabilities grow through iterative search and learning phases (Fig. 1). During the evolutionary search phase, SOAR uses an L.MM to both sample candidate programs and refine them through targeted modifications, producing diverse set of solution attempts (Section 3.2). In the learning phase, these search traces used to fine- tune the underlying LLM, enhancing its ability to sample and refine programs for future tasks (Section 4.2). SOAR iterates this process to create a virtuous cycle: better models enable more effective search, which in turn provides richer training data for further model improvements (Section 3.4).
ARC tasks are too challenging for current language models to solve directly (see proof in Section 4.1). SOAR combines the generative capabilities of L.MSs with an evolutionary search process that iteratively improves candidate solutions. At a high level, our Sample&Refine search first samples an initial pool of candidate solutions (sampling step), then iteratively refines the most promising ones using execution feedback (refinement step). In the end, we use majority voting to select the most likely test output grids to submit for evaluation (see Appendix D.1). Appendix G provides the prompts used for sampling and refinement.
Program sampling. Given a base LLM parameterized by 0, we sample a set of Python programs f without constrain- ing ourselves to a hand-coded domain-specific language:
f ~ Po(- Tresst Tirain Yrran).
3

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
Each candidate program is executed through a Python in- terpreter to produce output grids: Deest = (x)e)) This inductive approach allows us to implement& sample-and- test strategy - -scalig the number of candidate solutions increases our chances of discovering a transformation that satisfies all input-output examples, a requirement for our program to truly capture the implicit transformation of the task. AlphaCode scaled this approach to millions of at- tempts per task to achieve human-level performance on coding challenges (Li et al., 2022).
through finetuning. Specifically, each search attempt pro- duces rich set of program candidates, including both suc- cessful and failed attempts, that can serve as training data for enhancing both program sampling and refinement.
Finetuning sampling capabilities. We aim to improve our model's ability to sample correct programs by learning from its past synthesis attempts. For each task in the ARC- train set, we have access to ground truth test outputs Shest, allowing us to identify correct sampled programs foomeet- This gives us a dataset Demnect of (task, solution) pairs.
Program refinement. When a candidate program f pro- duces incorrect outputs (Vsynth f* A Vuaia), we can use this execution feedback to guide the LLM in refining its solution f / :+:
However, this approach faces a significant limitation: search fails to sample any correct solution in most tasks, severely limiting the size of D To address this, we augment our training data through hindsight relabeling (Andrychowicz et al., 2017). The key insight is that any program fo sampled during search, while possibly incorrect for its intended task, is by definition correct for the task of mapping inputs to the outputs it produces. Formally, given a program:
gt Po I,, Frest, Turinn Utain Msynth); ):
clearly labeling both successful (Ysyoth = Ytrain), and failed (Noymhh F Mrain) transformations in the refinement prompt.
Sample& Refine search algorithm. Our search process consists of two steps: (1) an initial sampling step that in- dependently samples 3k candidate solutions, and (2) re- finement step with a budget of 3k refinements. The second step frames refinement as a generative multi-armed bandit: each refinement creates a new arm that can further be re- fined. We tackle this problem with REX, a combination of Thompson sampling based on the accuracy of training input-output examples with an additional exploration bonus (Tang et al., 2024). This efficiently balances our search budget between the exploration of new program variations and the exploitation of known successful paths.
fo Po Trest, Train: Mtrai); we can create new synthetic task for which fo is a correct solution by executing fo on all inputs:
Vx E Tuains Usyuth = fo()).
This gives us a new valid (task, solution) pair: [Trrann: Msyuth Frest); fo} where fo is guaranteed be cor- rect by construction. This approach allows to leverage all programs sampled during search for training, not just those that happened to solve their intended tasks.
The resulting synthetic dataset Deso syath contains 6k datapoints collected for each of the 400 tasks in ARC-train a total of 2.4M datapoints. Given our limited computational re- sources, we sub-sample this dataset S 50 examples per task. This is done by ranking solutions according to their accuracy on input-output examples and test pairs, then sam- pling 25 top performing solutions (greedy approach), then sampling 25 bottom performing solutions to introduce some diversity in the set of relabelled groblem-soluttin pairs. Sec- tion 4.2 compares this solution to alternatives. With the resulting dataset D symth we finetune our model to sample better programs by minimizing:
Ensembling with weighted majority voting. We start with 6k candidate programs (3k from sampling, 3k from refinement). Each program is evaluated on the ARC task's s input-output examples compute its example accuracy, and is also run on the test input to produce an output grid. We then group programs by their test output grid and assign each unique grid a score: the sum of example accuracies of all programs that produced it. This gives us a weighted vote over test outputs, favoring grids produced by more accurate programs (see Appendix D.I for more details). This en- sembling approach helps mitigate individual program errors while capturing common patterns across successful solu- tions. We eventually return two candidate solutions, as per the benchmark rules (Chollet, 2024).
L = E D'
log Po(f [irriin Strain TTest]]
Finetuning refinement capabilities. Beyond improving initial program sampling, we aim to enhance our model's ability to refine incorrect programs using execution feed- back. For tasks in ARC-train where we have access to ground truth outputs, we can identify correct refinements: cases where an incorrect program f was successfully re- fined into a correct program ft. We collect these successful refinements into a dataset Drefine
3.3. Learning to search via self-improved sampling and refinement
The search process described above relies entirely on the base LLM's ability to sample and refine programs. We propose to leverage the data generated during the Sam- Refin search phase to improve these capabilities
Here again, we subsample this dataset to S 50 examples
4

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
per task. This is achieved by balancing the sampling over bins of the input-output accuracy of the parent program: 0%, 1-34%, 34-98%, and 100% to ensure diversity. Sec- tion 4.2 compares this strategy to alternatives. With the resulting dataset DD refine, correct we finetune our model to better refine programs by minimizing:
large-2407 to study larger models, including one trained on different data, by different company. We finetune models single H100 using the RS-LoRA (7B and 14B models) and RS -L-OAA algorithms (Hu et al., 2021; Dettmers et al., 2024; Kalajdzievski, 2023) (larger models) with the Unsloth library (Daniel et al., 2023). We use LoRA rank 256 with a = 32, and train for 3 epochs with learning rate of Se-5 (see Section D.4 for details).
C =
Popisssss [-log Po(f*) fiest Tirain: Mrrain Boynth)]
3.4. Closing the loop: iterative self-improvement on training and testing tasks
4. Experiments
Self-improvement on training tasks. The search and learning phases described above form the building blocks of SOAR's improvvnptvee loop. At each iteration i, we alternate between: (1) Sample& Refine search phase: Using model 0g to sample and refine programs through evolution- ary search and (2) Learning phase: Using the search traces to train an improved model 0.+1 by finetuning the base model (see Figure 1). Each iteration builds upon previous improvements - the model finetuned in iteration i's learn- ing phase powers the search in iteration + 1, generating richer training data to train the model + 1. This creates virtuous cycle where better models enable more effective search which, in turn, yields better training data.
Our experiments explore how program synthesis systems can grow beyond their initial capabilities through self- improvement. We begin by showing that even the strongest language models struggle to solve ARC tasks without search, establishing the need for iterative exploration (Sec. 4.1), From there, we demonstrate how models can learn to search more effectively by fine-tuning on their own synthesis at- tempts- improving both their ability to sample and re- fine programs (Sec. 4.2). These improvements accumulate across iterations, creating a virtuous cycle of increasingly ef- fective search (Sec. 4.3). Crucially, this cycle allows SOAR to break through the performance ceilings encountered by scaling model size or compute budget alone (Section 4.4). We conclude by analyzing the diversity of generated so- lutions and find that while SOAR tends to converge on consistent programs after success, it preserves diversity on unsolved tasks (Sec. 4.5). Together, these results establish SOAR as a significant advance in program synthesis, demon- strating how systems can bootstrap their own improvement through iterative search and learning. Appendix H provides examples of sampled programs and refinement examples.
After this training phase, we collect and deduplicate all so- lutions generated by the models using an embedding model with cosine similarity threshold of 0.9 (CodeRankEmbee), We then subsample this dataset as described in Section 3.3 (50 examples per ARC-train problem) and use the resulting dataset to finetune base model that will serve as the basis of test-time training iterations.
Test-time training. We can adapt the self ceff-improvement loop let the agent learn from target problems where the ground truth is not accessible. This is achieved by focusing on finetuning sampling capabilities by selecting solution ex- amples according to their training accuracy on input-output examples only (instead of ground truth accuracy), before applying hindsight relabeling. Refinement finetuning could potentially be adapted to work without ground truth (at test time) with hindsight relabeling. However, reserve this approach for future work, our current test-time improve- ment method focuses solely on refining explicit sampling capabilities. This enables a powerful test-time training loop: after running several iterations of full self-immpovemmnn on the training set, we can further adapt our model through additional iterations focused specifically on test tasks.
4.1. Program synthesis methods must leverage search
Can state-of-the-art language models solve ARC tasks in a single attempt? To find out, we evaluated several models in one-shot setting, where each model tries to generate cor- rect program in just one try. As shown in Table 1, even the strongest models achieve modest success rates on ARC-test: e.g. Claude-4-Somer (20.75%), GPT-4.1 (8.00%). Smaller open-source models perform even worse, with @wen-2.5. Coder models achieving 1. 1.00-2.25% success rates across model sizes. Direct program synthesis remains too challeng- ing for current language models.
Two approaches can potentially improve performance: (1) transduction, where models directly predict output grids without generating programs, or (2) inductive program synthesis combined with search. While both approaches currently yield comparable results (Li et al., 2024), pro- gram synthesis offers key advantage: it enables sys- tematic exploration of the solution space, allowing perfor- mance to scale with additional compute through search. We compare three settings using a series of Qwen-2.5-Con
Implementation details. We evaluate SOAR in combina- tion with LLMs from the Qwen-2.5-Coder series (7B, 14B, 32B), known for their strong coding capabilities while re- maining small enough to allow compute-efficient finetuning (Hui et al., 2024). We also used Qwen-2.5-722 and Mistral-
5

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
models: (1) single-shot sampling, (2) sampling 6k candi- date programs (Sample-6k), and (3) using half the bud- get for initial sampling and half for targeted refinements (Sample&Refine-6k)) 6k). Table 1 shows that both sampling and sample&refine search dramatically improve performance, with Sample&Refine-X achieving the best results across all model sizes. Notably, this allows smaller open-source models to outperform much larger closed-source ones by leveraging additional computation: the 7B model beats GPT- 4.1, and 232B models beat Claude-4-Soneel Only state- of-the-art reasoning models (03-mini and Gemini-2.5-Prr outperform Sumple&&efine-- with Qwen 332B (see more model evaluations in Appendix 6).
mance avoid overfitting the method to test tasks.
Learning to sample programs. A key challenge in im- proving sampling capabilities is extracting meaningful train- ing signal from search attempts. While could only train successful solutions, this would severely limit our train- ing data since many tasks remain unsolved. Instead, we explore several strategies for creating synthetic training data from all sampled programs through hindsight relabeling (see Section 3.3). For each ARC-train task:
correct-only: sample uniformly up to 50 solutions that solved the task (no hindsight learning);
uniform: sample uniformly 50 candidate solutions, then apply hindsight learning to create corresponding problem-solution pairs;
However, search performance typically scales logarithmi- cally with compute budget and is ultimately bounded by the capabilities of the base model used for sampling and refinement. This observation motivates our key question: can we learn to search more effectively by improving these underlying capabilities? Our experiments demonstrate how iterative self-improvement breaks through this barrier.
greedy: sample the 50 solutions that solved the most train and test examples, then apply hindsight learning; greedy-divers: sample 25 solutions greedily, then 25 solutions that solved the fewest training examples (for diversity), before applying hindsight learning.
Model
1-shot
Sample -6k
Sample& Refine-6k
SOAR
Table 2 compares these strategies by measuring sampling accuracy after finetuning (% of train tasks solved using 3k samples). While all methods leveraging finetuned mod- els improve over the baseline, the greedy-diverss method performed best best-sugee suggestinn the importance of balancing between learning from successful solutions and maintaining diversity in the training data.
-6k
Qwen-2.5-C-7B
1.00
5.63
14.25
36.25
Qwen-2.5-C-14B
1.00
12.63
19.87
42.75
Qwen-2.5-C-32B
1.50
12.88
25.25
44,38
Qwen-2.5-72B
1.75
18.50
25.62
44.88
Mistral-Large-2
2.50
19.75
26.25
45.50
GPT-4.1
8.00
"laude4-Sonned
20.75
Reasoners
Sample-3k acc
03-mini
33.00
2e5ini-2.5-pro
38.25
no finetuning
29.29
finetune: correct-only
34.67
Table 1. Performance on ARC-test (% solved). Scores are com- puted using LLMs performing program synthesis. Sampling small open-source models 6k times with majority voting (Sample-6k) or sampling them 3k times and executing 3k refinement steps before majority voting (Sample&Refine) shot program synthesis performance of much larger non-reasoning glosed-source models. SOAR nearly doubles search performance for all models. Sample&Refine and SOAR not run with closed- source models for budget reasons.
finctune: uniform
32.38
finetune: greedy
34.3
finetune:
greedy-diverse
36.46
Table 2. Sampling finetuning. ARC-train performance after sam-
outperforms the one-
pling 3k samples with Qwen-2.5 2255-Coddr-144 14B models finctuned for program sampling (% solved).
Learning to refine programs. Beyond sampling initial solutions, we aim to improve the model's ability to refine programs using execution feedback. We explore two strate- gies for curating refinement examples from search traces. For each ARC task:
4.2. Learning to sample and refine programs
Our framework for learning to search alternates between search and learning phases: first using the model to sample and refine solutions, then using these attempts to improve the model's capabilities. This section analyzes how to ef- fectively extract training signal from search attempts, exam- ining three key questions: (1) how to learn better program sampling, (2) how to learn better program refinement, and (3) whether and how these capabilities can be learned jointly. We conduct these experiments using Qwen-2.5 2.5-Coder-J 14B and make design choices based on the ARC-train perfor-
uniform:
sample uniformly up to 50 successful refine- ment examples:
diverse: balance that sample based on the training scores of the incorrect parent program (0%, 1-34%, 34 98%, and 100% with incorrect test outputs).
Table 3 shows the ARC-train performance of search methods leveraging a non-finee non-finetneed model for sampling (3k samples) and a finetuned model for refinement (3k refinements) using either of these two data generation methods. Both strategies
6

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
improve refinement capabilities substantially, with diverse sampling performing marginally better (42.88% when refin- ing solutions sampled by a non-finetuned model).
80
770
ARC-train score
60
Sample-3k acc
Sample&Refine
50
QC-78
no finetuning finetune: uniform
29.67
34.83
QC-14B
42.67
40
Q-728
finetune: diverse
42.88
Mistral-Large
30
all models
orade (all models)
Table Refinement finetuning: ARC-train performance of Sam- Ple&Refine search methods (6k budget) using a non-finetuned Qwen-2.5-Coder- 14B model in the sampling step before refining models
soAR(base-model)
2
3
Training iteration
Figure 2. Iterated self-improvement on training problems. ARC- train performance across training iterations. Training iteration O: search with base models. All: score achieved by applying majority voting on the combined generated solutions of the five models.
sampled solutions with different Qwen-2.5-Codeer finetuned for program refinement (% solved).
Positive synergy between sample and refine tasks. Should we train separate models for sampling and refine- ment, or can a single model learn both effectively? Table 4 shows that joint finetuning outperforms both base models and task-specific finetuning-for both sampling and search performance. This indicates a clear synergy: learning to sample helps refinement, and vice versa. The results sug- gest that both tasks benefit from shared representations of program structure and transformation patterns. Rather than splitting effort between specialized models. joint learning of- fers more efficient and effective path. Appendix E presents more detailed experiments supporting this result.
with better search capabilities, with the >32B models out- performing smaller variants iteration 0: (2) smaller mod- els show steeper improvements in early iterations; (3) all model sizes continue to benefit from further training itera- tions, though gains may slow down in later iterations; and (4) relative improvements are largest for smaller models, with the 7B model nearly doubling its performance.
We found that pooling data from the search traces of our five models before performing majority voting (5 x 6k samples per task) significantly outperformed all of them (see brown line on Fig. 2) - suggesting that different model sizes may solve problems in complementary ways. However, majority voting is not an ideal aggregation strategy; we observed an average score gap of 9.5% between majority voting and oracle performance across our models, where the oracle is defined as task being solved if least one solution produces the correct output. This gap indicates room for improvement in developing better ensembling methods.
Sample& Refine-6k
Sample model
Refine model
Sample-3k 33
base fine-samp fine-ref
base
29.67
34.83
36.46
43.88
fine-both
fine-both
39.79
44.42
Table 4. ARC-train accuracy using different combinations of models for the Sample (col 1) and Refine (col 2) phases. fine-samp/re//beth refers to Qwen-2.5-Coddrr-44B finetuned for sampling, refinement, or both, respectively. Sample-3k and & (cols 3) indicate the ARC-train accuracy after sampling (3k solutions) and after search (3k samples 3k refinements).
Since pooled data from multiple models and iterations con- sistently yielded better performance during search (see Fig- ure 2), we trained a series of base models on a subset of the combined dataset of all training iterations and model sizes (as described in Section 3.4). These models, trained greater diversity of programs and refinement strategies, significantly outperform models trained on their own data only (Table 5). We use these models called SOAR(all train) as starting points for test-time training steps.
4.3. Learning to search with iterated self-improvement
Having established effective methods for self-improvement, we now examine how improvements compound through iterations and scale with model size.
Model size
SOAR(3 train)
SOAR(all train)
QC-7B
19.9
33.0
QC-14B
24.5
39.1
QC-32B
28.0
41.1
Self-improvement ARC-train problems. Figure 2 shows substantial gains across iterations for all model sizes, solving an extra +27% (7B), +24% (14B), +20% (32B), +1 +19% (72B) and +22% (Mistral) problems on ARC-train. The relationship between model size and performance re- veals several interesting patterns: (1) larger models start
Q-72B
34.6
39,8
Mistral- Mistral-Large-122 123B
28.5
40.1
Table 5. ARC-test accuracy after training basc models on a subset of 1) the data obtained at the 2nd SOAR iteration using that same model size. SOAR(3 frain): 2) all data collected by all models and all previous train iterations SOAR(all train).

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
redes
55
QC-328
QC-140
50
ARC-tast score (%)
0-728
ARC-test (%6
40
50
wrade lall modelal
30
45
20
40
into
teit
35
5048 (ell mrhn bett
0
SOAR(all-train)
7
14 Model size (B)
32
72
123
Test time training iteration
Figure 4. Performance plateaus with increasing model size when using fixed sampling and refinement capabilities (Sample-6k and Sample& Refine-kk). scaling curves across iterations, enabling smaller models to match or outperform much larger ones. Note that only the 7B, 14B, and 32B models are from the same family (Qwen-2.5-Coder) 72B is from the Qwen-2.5 family, and 123B is Mistral--arge-2407.
Figure 3. Iterated seff-improvement on test problems. ARC-test performance across test-time training iterations. Itcration O: search with the models finetumed in training iteration 4 (right-most points Figure 2). All: score achieved by applying majority voting on the combined search data of the five models.
contrast, SOAR progressively lifts the
Self-improvement on ARC-test problems (test-time train- ing). The results from Section 4.3 demonstrated signifi- cant performance gains through iterative seff-improvement on training tasks. However, practical applications require systems that can improve on new problems without access to ground-truth solutions. This raises key question: Can our self-improvemen framework continue to raise performance when adapted to target test problems?
Self celffimprovement enables each model size to reach perfor- mance levels that previously required much larger models.
Figure 5 reveals similar ceiling when scaling search bud- get. With the 7B base model, performance saturates after roughly 5k search attempts. In contrast, SOAR-iteration I nearly doubles its ARC-test accuracy, which remains true when controlling for FLOP budget (see Appendix B). No- tably, a significant fraction of this gain appears during the refinement phase. These results show that search alone insufficient leamming refine essential.
Starting from models fine-tuned on all data collected through 4 iterations on ARC-train problems (previous sec- tion), we perform two additional iterations of test-time train- ing on ARC-test problems, leading to extra 5% perfor- mance on ARC-test (Figure 2). Taken together, the combina- tion of train-time time and test-time improvements dramatically raised performance across all model scales. Our 7B model improved from its initial 14.25% to 36.25% accuracy on ARC-test, 2.5 fold increase. Similarly, the 14B model rose from 19.87% to 42.75%, the 32B model improved from 25.25% to 44.37% 72B from 25.62% to 44.87%, while Mistral-Large-2 accuracy improved from 26.25% to 45.5%. By combining solutions across all model sizes through ma- jority voting, we achieved our peak performance of 52.00% on ARC-test and an oracle performance of 57.25%.
25
20
ARC- -es-test-ee
SOAR (base-model)
S0AR (2 train)
506R train)
5
Stampp
50AR train)
(1eat)
k.es()
0 1000 2000
3000 Number of samples
4000
5000
6000
Figure 5. Search alone hits diminishing returns with increased bud- get: the base 7B model plateaus after about 5k search attempts. SOAR outperforms this baseline by wide margin, with improve- ments compounding across iterations.
4.4. Escaping scaling plateaus through self-improvement
Figure shows that simply running search with increasing model size eventually yields diminishing returns. While larger models perform better in early iterations, Sample-6k curves flatten beyond 32B, suggest-
Together, these results show that SOAR breaks through both model-size and search- budget plateaus by improving the model itself. Rather than pushing harder against fixed limits (scaling model sizes or search budgets), SOAR lifts them transforming flat scaling curves into steps of improvement. This effect is especially striking for smaller models: Qwen- 2.5-7B reaches 36.25% on ARC-test after SOAR's itera- tions, outperforming much larger systems like o3-mini and Saude-4-Sonnel These results establish SOAR as a sig- nificant advance in program synthesis approaches ARC.
and Sample&Refine ing model-size scaling plateau: more parameters alone do not suffice when the model's sampling and refinement behaviors remain fixed. In contrast, the SOAR curves re- veal a different pattern. While each SOAR iteration also plateaus, subsequent iterations consistently lift the perfor- mance ceiling - establishing new, higher sealing curves.
8

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
Our approach outperforms prior methods that relied on ex- tensive human-generated training data (Li et al., 2024) and and expensive search methods built on much larger closed- source models (Greenblatt, 2024) (see further comparisons in Appendix Table 6). By enabling models to improve them- selves from scratch, SOAR eliminates the need for hand- engineered programs, DSLs, or external datasets-marking a step toward more autonomous, scalable synthesis systems.
self-improvement alone can bootstrap strong reasoning ca- pabilities in penerol-purpose language models.
Our experiments highlight two key findings. First, SOAR overcomes the performance plateaus typically observed when scaling model size or search budget. By improving the underlying model itself, it establishes new, higher scaling baselines. Second, we observe complementary problem- solving strategies across model sizes: smaller models often learn faster and sometimes solve tasks that larger ones miss. Training base models on aggregated solutions from multi- ple models and iterations yields the strongest improvements (see Table 5), while ensembling solutions across model sizes leads to our best ARC-test performance (52%). These re- sults suggest that cross-model diversity is a key driver of performance gains in self-improving program synthesis.
4.5. Solution diversity across iterations
To keep on improving and solving harder problems, SOAR must keep on exploring the space of possible solutions. Fig- ure 6 shows steady decrease in solution diversity across self-improvement iterations for the problems SOAR man- aged to solve. For unsolved problems, diversity initially drops after the first iteration but then plateaus, suggest- ing that our relabeling method may help maintain some exploratory capacity. While this helps preserve diversity on unsolved tasks, integrating explicit strategies could further extend SOAR's ability to explore solution spaces and sustain continual improvement.
Crucially, SOAR offers substantial advantage over ap- proaches that rely on fixed models within static search loops. State-of-the-art systems like FunSearch and AlphaEvolve Coomera-Paredes et al., 2024; Liu et al., 2024; AlphaEvolve- team, 2025) use program synthesis without adapting the model. SOAR could serve as a drop-in upgrade, enabling these systems continually learn from their own search traces. The framework could also be extended with richer operators, such as crossover (Meyerson et al., 2024).
Solved
pairvise distance)
0.5
Unsolved
While SOAR is domain-agnostic, we only evaluate it on ARC. Future work should test its applicability to domains like software engineering or mathematical discovery (Dong & Ma, 2025; Jain al.). Computational efficiency is an- other limitation: SOAR currently requires 6,000 synthesis attempts per task per iteration. Although we observe steady gains, these diminish over time, hinting at potential lim- its Whether these are intrinsic or methodological remains open, but several strategies could help, such as adaptively rerouting the search budget from solved tasks to harder ones, or improving optimization methods (e.g.. see Chow et al. (2024); Tang et al. (2025); Gehring et al. (2024)).
(avg Diversity
0.3
0.1
0.0
2
3
Generation
Figure 6. Solution diversity across generations of SOAR. Thin lines indicate solution diversity across for cach of the 400 ARC- train problems, colored in green when solved, in red when un- solved. Thick lines indicate averages across solved and unsolved problems respectively. SOAR maintains solution diversity for unsolved problems but converges on lower solution diversity for solved problems. Diversity measured as the average pairwise cosine distance in embedding space (CodeRankFmmed))
One likely bottleneck is low solution diversity. While prior work found that RI. and finetuning often reduce output di- versity (Zhang et al., 2025; Yue et al., 2025), we find that SOAR preserves diversity on unsolved tasks, likely due to hindsight relabeling, which retroactively creates new prob- lems from failed programs. Still, this maintained diversity appears insufficient to sustain continual progress. Future work could enhance it by explicitly optimizing for diversity during finetuning, introducing quality-diversity methods, or generating new problems to expand solution diversity (Co- las et al., 2022; Pourcel et al., 2024). These directions may help maintain a virtuous cycle of improvement and push program synthesis closer to open-ended discovery.
5. Discussion
Our work shows that program synthesis systems can tran- scend their initial capabilities by iteratively improving both sampling and refinement through a cycle of evolutionary search and learning. Here we reflect on the broader implica- tions and challenges ahead.
ARC was explicitly designed to resist pattern matching and require core reasoning, making it a strong testbed for program synthesis (Chollet, 2024). Most models fail without human- examples or human- priors. SOAR's ability to improve purely from its own search ex- perience - without demonstrations or DSLS - show that
9

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
Acknowledgments
C., Vikram, S., Lebron, F., Choy, P., Ramasesh, V., Web- son, A., and Pope, R. How scale your model. 2025. Retrieved from
This work benefitted from access to the HPC resources of IDRIS under the allocation A017101 1996 made by GENCI. was also co-funded by AI Chair ANR DeepCuriosity Cédric Colas acknowledges funding from the European Union's s Horizon 2020 research and inno- vation programme under the Marie Sklodowska- grant agreement No 101065949.
Balog, M., Gaunt, A. L.,, Brockschmidt, M., Nowozin, S., and Tarlow, D. Deepcoder: Learning to write programs. arXiv preprint arXiv:1611.01960 2016.
Bradley. H., Dai, A., Teufel, H., Zhang. J., Oostermeijer, K., Bellagente, M., Clune, J., Stanley, K., Schott, G., and Lehman, J. Quality-diversity through ai feedback. arXiv preprint arXiv:2310. 13032, 2023.
Impact Statement
This work demonstrates how iterative model improvement can help overcome the performance plateaus typically countered when scaling both model size and search budget. This finding suggests an important principle for developing more capable AI systems that could benefit society across numerous applications, from software development to sci- entific discovery.
Butt, N., Manczak, B., Wiggers, A., Rainone, C., Zhang, D. W. Defferrard, M., and Cohen, T. Codeit: Self- improving language models with prioritized hindsight replay. In International Conference on Machine Learn- ing, 2024.
Chollet, F. On the measure of intelligence. arXiv preprint arXiv:1911.0144, 2019.
However, the development of self-improving Al systems naturally raises safety considerations. While our results show clear performance slow downs rather than unbounded improvement, suggesting inherent limitations to our spe- cific approach, the general principle of systems improving through self-directed learning could inspire future systems with broader capabilities. This underscores the importance of implementing appropriate safeguards and oversight mech- anisms when developing such systems.
Chollet, F. Are prize website. pattps://arprrzee.. rg/", 2024. ARC Prize Leaderboard
Chow, Y., Tennenholtz, G., Gur, I., Zhuang, V., Dai, B., Thiagarajan, S., Boutilier, C., Agarwal, R., Kumar, A., and Faust, A. Inference-aware fine-tuning for best-of- n sampling large language models. arXiv preprint urXiv:2412.15200 2024.
We demonstrate these results using open-source language models and will release our complete codebase upon publi- cation. We believe this transparency is crucial for respon- sible development of increasingly capable Al systems, and we encourage researchers building on this work to maintain similar standards of openness while carefully considering potential societal impacts.
Colas, C., Karch, T., Sigaud, O., and Oudeyer, P.- Y. Autotelic agents with intrinsically motivated goal- conditioned reinforcement learning: a short survey. Jour- nal of Artificial Intelligence Research, 74:1 14:1159-1199 2022.
Daniel, H., Michael, H., and team, U. Unsloth, 2023, URL
http:/ / liggttthht thub. cwwmcmsc/assoooo uns otha
References
Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36. 2024.
Akyürek, E., Damani, M., Qiu, L.,, Guo, H., Kim, Y., and Andreas, J. The surprising effectiveness of test-time training for abstract reasoning, 2024. Preprint.
Dong, K. and Ma, T. Beyond limited data: Self-play I1m
AlphaEvolve-team. AlphaEvolve: A Gemini-powered cod- ing agent for designing advanced algorithms deep- mind-google. https: httt::://deepiinnni gowgl////..ooooooooo.o////o/.//
theorem provers with iterative conjecturing and proving. arXiv preprint arXiv:2502. 00212, 2025.
Ellis, K., Wong, C., Nye. M., Sablé-Meyer, M., Morales, L., Hewitt, L.,, Cary, L.,, Solar-Lezama, A., and Tenenbaum, J. B. Dreamcoder: Bootstrapping inductive program synthesis with wake-sleep library learning. In Proceed- ings of the 42nd acm sigplan international conference on programming language design and implementation, pp. 335-850, 2021.
ed ed-coding agent- for designing laligniinn-dvvaneeeee advance d-algorithms/. 2025.
Andrychowiez, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Pieter Abbeel, O., and Zaremba, W. Hindsight experience replay. Advances in neural information processing systems, 30, 2017.
Gandhi, K., Lee, D.. Grand, G., Liu, M., Cheng, W., Sharma, A., and Goodman, N. D. Stream of search (sos): Learning
Austin, J., Douglas, S., Frostig, R., Levskaya, A., Chen,
10

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
to search in language. arXiv preprini urXiv:2404.03688 2024.
Kalajdzievski, D. A rank stabilization sealing factor for fine-tuning with lora. arXiv preprint arXiv:2312.03772, 2023.
Gehring, J., Zheng, K., Copet. J., Mella, V., Carbonneaux, Q., Cohen, T., and Synnaeve, G. Rlef: Grounding code Ilms in execution feedback with reinforcement learning. arXiv preprini arXiv:2410.02000 2024.
Koza, J. R. Genetic programming as a means for program- ming computers by natural selection, Statistics and com- puting, 4:87-112, 1994.
Gendron, G., Bao, Q.. Witbrock, M., and Dobbie, G. Large language models are not strong abstract reasoners. arXiv preprint arXiv::305. 19555, 2023.
Kumar, A., Lu, C., Kirsch, L., Tang, Y., Stanley, K. O., Isola, P., and Ha, D. Automating the search for artificial life with foundation models. arXiv preprint arXiv:2412.1799, 2024.
Goldberg, a. Genetic algorithms and machine learning. 5(2):95-99 ISSN 1573-0565. doi: 10.1020/A 2019183, URL https: //t:://dii. org. httpp/////oo..r//.11100000. 102260201 9183.
Langdon, W. B. and Poli, R. Foundations of genetic pro- gramming. Springer Science & Business Media, 2013.
LeGris, S., Vong, W. K., Lake, B. M., and Gureckis, T. M. H-are: A robust estimate of human performance on the ab- straction and reasoning corpus benchmark. arXiv preprint arXiv:2409.01374 2024.
Greenblatt, R. Draw more samples. "attps://zedwon dresearch substack httpp/cc..cc.c//tttttteg/eeeee// ota-on-arc-agi from ARCPrize Leaderboard.
2024. Accuracy
Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., and Stanley, O. Evolution through large models. In Handbook of Evolutionary Machine Learning, pp. 331- 366. Springer. 2023.
Guo, D., Zhu, Q. Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y., et al. Deepseek-coder: When the large language model meets programming-ther the rise code intelligence. arXiv preprint arXiv:2401.14190 2024.
Li, W.-D. and Ellis, K. Is programming by example solved by Ilms? arXiv preprint arXiv:2406.08316 2024.
Guo, D., Yang, D., Zhang. H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-rl: In- centivizing reasoning capability Ilms via reinforcement learning. arXiv preprint arXiv:2501.12944, 2025.
Li, W.-D., Hu, K., Larsen, C., Wu, Y., Alford, S., Woo, C., Dunn, S. M., Tang, H., Naim, M., Nguyen, D., Zheng, W.-L., Tavares, Z., Pu, Y., and Ellis, K. Combining induction and transduction for abstract reasoning, 2024. URL https: //rxiviivvv////////////aaa//aai/ org hts//bbb/24111 02272.
Hodel, M. Are-dsl. https: /github. com/ mi chael hodel / /arc-dsl, 2023. [Online GitHub repository).
Li, Y., Choi, D., Chung. J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. Competition code generation with alpha- code. Science, 1796666))
Holland, J. H. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. The MIT Press. ISBN 978-0-262-27555-22 doi: 10775//mipeesss/ 01. URL onograpp/2254//aapppattoo--.-ee nd id-AAtiificial-Systenss
2022.
mit htw.:/eet..iii........oooo.o.e
Liang, P., Jordan, M. I,, and Klein, D. Learning programs: A hierarchical bayesian approach. In ICML, volume 10, pp. 639-646, 2010.
Hu, E. Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2 09665, 2021.
Liu, F., Tong, X., Yuan, M., Lin, X., Luo, F,, Wang, Z., Lu, Z., and Zhang, Q. Evolution of heuristies: Towards efficient automatic algorithm design using large language model. arXiv preprint M.XXi::401002055 2024.
Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Lu, K., et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. 12186, 2024.
Menon, A., Tamuz, O., Gulwani, S., Lampson, B., and Kalai, A. A machine learning framework for programming by example. In International Conference on Machine Learning, pp. 187-195. PMLR, 2013.
Jain, N., Han, K., Gu, A., Li, W.-D., Yan, F. Zhang, T., Wang, S., Solar-Lezama, A., Sen, K., and Stoica, I. Live- codebench: Holistic and contamination free evaluation of large language models for code. In The Thirteenth International Conference Learning Representations.
Meyerson, E., Nelson, M. J., Bradley, H., Gaier, A., Moradi, A., Hoover, A. K., and Lehman, Language model crossover: Variation through few-shot prompting. ACM Transactions on Evolutionary Learning, 4(4):1-40. 2024.

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
Mirchandani, S., Xia, F., Florence, P., Ichter, B., Driess, D., Arenas, M. G., Rao, K., Sadigh, D., and Zeng. A. Large language models as general pattern machines. arXiv preprint arXiv:2307.0472) 2023.
Xu, Y., Li, w., Vaezipoor, P., Sanner, S., and Khalil, E. B. Lims and the abstraction and reasoning corpus: Suc- cesses, failures, and the importance of object-based rep- resentations. arXiv preprint arXiv: 20223305..83335 18354, 2023.
Olausson, T. X., Inala, J. P., Wang, C.,, Gao, J., and Solar- Lezama, A. Is self-repair a silver bullet for code genera- tion? arXiv preprint 13Xiv:2300009890 2023.
Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang. Z., Song, S., and Huang, G. Does reinforcement learning really incentivize reasoning capacity in Ilms beyond the base model? arXiv preprint arXiv:2504. 13837, 2025.
Pourcel, J., Colas, C., Molinaro, G., Oudeyer, P.-Y., and Teodorescu, L. Aces: generating diverse programming puzzles with autotelic language models and semantic descriptors. Neurips, 2024.
Zelikman, E., Wu, Y., and Goodman, N. D. Star: Self-taught reasoner. arXiv preprint arXiv:2203. /4465, 2022.
Zelikman, E., Harik, G., Shao, Y., Jayasiri, V., Haber, N., and Goodman, N. D. Quiet-star: Language models can teach themselves think before speaking, arXiv preprint arXiv:2403. 09629, 2024.
Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J., Ellenberg, S., Wang, P. Fawzi, O., et al. Mathematical discoveries from program search with large language models. Nature, 625 (7995):468-477 475, 2024.
Zhang, Y., Diddee, H., Holm, S., Liu, H., Liu, X., Samuel, V., Wang, B., and Ippolito, D. Noveltybench: Evaluating language models for humanlike diversity, arXiv preprint arXi:22504005226 2025.
Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al. Code llama: Open foundation models for code. arXiv preprint ar Xiv:22300 0..3008..99555 /2950, 2023.
Zheng, L.,, Yin, I.., Xie, Z., Sun, C. L.,, Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., et al. Sglang: Efficient execution of structured language model programs. Advances in Neural Information Processing Systems, 37:62557-666 777625577-2583, 2024.
Saad, F. A., Cusumano-Towner, M. F., Schaechtle, U., Ri- nard, M. C., and Mansinghka, V. K. Bayesian synthesis of probabilistic programs for automatic data modeling. Proceedings of the ACM Programming Languages. (POPL):1-2,,1) 2019.
Shi, K., Bieber, D., and Singh, R. Tf-coder: Program synthesis for tensor manipulations. ACM Transactions Pro Programming Languages and Systems (TOPLAS), 44(2): 1-36, 2022.
Tang, H., Hu, K., Zhou, J. P., Zhong, S. C., Zheng, W.-L.,, Si, X., and Ellis, K. Code repair with LLMs gives an tradcoff. The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: openreview net /formm? d=o d=-663g6666xx
Tang, Y., Zheng, K., Synnaeve, G., and Munos, Optimiz- ing language models for inference time objectives using reinforcement learning. arXiv preprint 2rXhv:2503.19595, 2025.
Wang, R., Zelikman, E., Poesia, G., Pu, Y., Haber, N., and Goodman, N. D. Hypothesis search: Inductive reasoning with language models. arXiv preprint MxXiv:2309.0666. 2023.
Wind, S. 1st place 2020 are kaggle. https://gi thub. com/ com/top-queee qurooouuaks///oe http://opogaaass/---sooooo Aww.AACCCcssiioiooo 2020. [Online GitHub repository).
12

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
A. Comparison with prior work
Method
% ARC-test solved
attempt/task
Fair comparison?
One-shot LLM sampling (1 attempt/ task)
Claude-3.5-Soneet
Claude-4-Somnet
11.25 / 20.75
Yes (closed-source LLM)
GPT-4.1 03--mini
8.00 33.00
Yes (closed-source LLM)
Mistral-large-2
2.50
Yes
Qwen2.5-72B
4,00
Yes
Qwen2.5
Siven2.5-Coder-(7/11 Codcr-(7/14332B)
1.00 1.00 1.75
Yes
Sample&Refine
Mistral-large-2
26.25
6000
Yes
Qwen2.5-72B
25.62
Yes
Qwen2.5- Sreen.5-CCder-(P27 Coder-(7/14/32BB (QC-nB)
14.25
19.87 25.22
6000
Yes
Ours: iterated self-improved search (SOAR)
SOAR-Mistre
45,50
6000
SOAR-Q-72B
44.87
6000
SOAR-QC-(711) (BAR-0C-7/14/32B)
36.25
42.75 44.37
6000
ours
SOAR-QC
52.00
6000x
ours
SOAR-QC-all (Oracle)
57.25
6000x5
ours, but using oracle eval (skip maj. vote)
Prior inductive approaches
Codelt (Butt et al., 2024)
15.00
Yes
BARC-induction (Heavy) (Li ct al., 2024)
30,50
10000
Yes, but heavy use human data and closed LLM
BARC-induction (Potpourri) (Li et al., 2024)
38.00
20000
Yes, but heavy use of human data and closed LLM
Icccuber (Wind, 2020)
39.00
Unknown
No, looking val human DSL
(Greenblatt, 2024)
42.00
8160
Yes, but closed-source LLM
Table Comparison of inductive methods the ARC benchmark. Our approach SOAR outperforms previous induction performance.
B. Scaling laws
Finetuning costs: Finetuning is inexpensive compared to the search phase. FLOPs per iteration is 6N (100-T7 T), where N LLM parameters and T is tokens per completion. With 100 datapoints per task, sampling FLOPS is 2N (6000 -T000-T-- n)) making finetuning 5% of total FLOPs-nearly negligible. Additionally, autoregressive generation is slower (token-by- token forward passes), while finetuning processes sequences in one forward and backward pass (see Austin et al. (2025) for more details). Figure 8 plots the performance of the different generations of SOAR with search budget of 6k, against the performance of Sample&Refine using the base model with a search budget of 12.6k matching the FLOPs used by SOAR at generation (6k search by the base model, search by the finetuned model, and extra to cover for finetuning costs). SOAR at generation reaches far superior performance with the same computational budget.
13

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
60
SOAR (base-model)
SOAR (base-model, 12.6k)
50
SOAR (1-train)
SOAR (2-train)
ARC-tre -r-ariin score (%)
(3-train)
40
30
20
10
Sample Refine
0
2000
4000 6000
8000 10000 12000
Number of Samples
Figure Scaling of iterated self-improvement on training problems. ARC-train performance across training iterations (gen-O base model) for Qwen-2.5-Coder-7h We increased the number of generation zero samples to 12,600 to match the total FLOPS usage of generation two, including both training inference.
14

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
Qween2.5-Codrr-7--nnsttee
-wen2.5-Coder-14-Innsss
80
80
Soar (baso-medel)
Soar (base-modal)
70
train)
70
Soar [0 train)
ARC-train score (%)
Soar (1 train)
ARC-train score (%)
Soar (1 train)
train)
Sear (2 train)
Soar (3 train)
Soar (3 train)
40
40
30
30
20
20
10
Sammll Reffin
10
Semple Refine
2
2
2
27
2
2
2
2
2
2
2
Number Samples
Number Samples
Qwen2255-odee--22-innstt
Qwen2.5-72B-Instrree
80
80
Soar (baso-model)
Sear (base-modal)
70
train)
70
train)
ARC-train score (%6
Soar (1 train)
ARC-train score (%6
Sear (1 train)
train)
Sear (2 train)
Soar train)
Soar (3 train)
50
40
40
30
30
20
20
Sample Reffna
Sample Rafina
10
10
21
2
2
27
2
21
2
2
27
2
Number Samples
Number Samples
Comparison of the last generation
Mistral-Large-lIstrut--
Soar (3 train)
80
80
Soar (baso-madal)
70
train)
70
ARC-train score (%6
Soar (1 train)
ARC-train score (%6
60
Soar train)
Soar (3 train)
50
40
40
30
30
QC-78
20
20
QC-328
Sample Rafine
Sample Rafine
10
10
Mistral-Large
2²
2a
25
27
2
2
2
2
27
2
213
Number Samples
Number Samples
Figure 8. Scaling laws of iterated seff-improvement training problems. ARC-train performance training iterations.
15

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
Qwen22.5ccodrr-7--nstriie
Qwen2:5--coder-44B-mm
50
50
SOAR model)
model)
SOAR (all train test)
SOAR (all train test)
40
(all
40
ARC-test score (%6
SOAR (all train 2 test)
sco- (%)
SOAR (all train .-2 test)
20
10
Sami Refine
10
Sample Reffin
2
2
2
2
2
2
2
0
2
2a
2
2
2
Number of Samples
Number of Samples
Qwenn255-coler-28---nsstuut
Qwen2.5-72B-Instee
50
50
model)
model)
SOAR (all train test)
SOAR (all train test)
40
(all 1est)
40
ARC-test score (%6
SOAR (all train test)
ARC-test score (%)
SOAR (all train + test)
10
Sam Refine
10
Sam ple Rafine
21
23
2
27
2
2
2
0
2
2
2
2
21
Number of Samples
Number of Samples
Mistral-Largge-Insttuc----
Comparison the last generation (gen-3)
50
50
model)
SOAR (all train + 0.test)
QC-148
40
(all
ARC-test score (%)
SOAR (all train 2 test)
ARC-test score (%)
Q-728
30
10
Sample Refine
10
Sample Rafine
21
23
25
27
2
211
213
21
23
25
27
211
218
Number of Samples
Number of Samples
Figure Scaling laws of iterated self itffommprveeentt test problems, ARC-test performance across training iterations (gen-0 model trained on ).
16

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
C. Model ensembling
All models
Mistral-Q--7BB-00C322--111 Mistrai+Q0-72B+0C-333B
Mistral+Q-72B
Q-72B+0C--3B+114B+7BB
QC-32B+14B+78
Mistral
Q-72B
QC-14B+7B
QC-32B
QC-14B
QC-7B
20
40
ARC-test score (%)
Figure 10, ARC-test score for different model combinations
D. Implementation details
D.1. Weighted Majority Voting Algorithm
The algorithm processes the ensemble of model responses, each containing output predictions for a set of input grids. It groups responses by their test output grids and applies weighted voting to select the most reliable predictions (see Alg. 1).
Weighted Majority Voting Algorithm:
1. Pattern Extraction: Each response's 's complete set of test outputs is serialized into pattern key.
2. Weighted Voting:
Patterns are weighted using the formula: weight count e x train_acces cactrracccrryy where count the frequency of the pattern, train accuracy is the mean accuracy of the generated program on the training examples, and scaling parameter,
want to penalize low pattern with low train_accurane set it high values
(c 1000).
3. Selection: The top n_output patterns with highest weights are selected as final outputs.
Algorithm 1 Weighted Majority Voting algorithm
Require: Set of responses with test outputs and training accuracies, scaling parameter c, number of outputs n_output
Ensure: Top mn_oulpul patterns with highest weights (n_oulpul 2 when testing)
1: Pattern Extraction:
2: for each response do
3: Serialize the complete set of test outputs into pattern key
4: end for
5: Weighted Voting:
6: for each unique pattern do
7: Compute count. +- frequency of the pattern
8: Compute train_accuracy + mean accuracy of the generated program on training examples
9: Compute weight +- count + e x train_accuracy
10: end for
Selection:
Select the top n_outpul patterns with the highest weight values as final outputs
17

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
D.2. Majority voting data selection for test time training data
To select data for Test Time Training, we employed our greedy-diverse data selection process, introducing minor modification the sampling strategy within the greedy component. Since test accuracy is unavailable during this phase, utilize majority voting procedure to identify the most probable correct solution, which is then used to train models (see Algorithm 2).
Algorithm 2 Weighted Sampling from Categories
Require: Ensemble of responses with associated training accuracy; N: number of responses sample
Ensure: List of sampled responses
1: Group responses according to their test output grids
2: For each group Di. compute the majority voting weight
3: Normalize the weights
4: Allocate Rg. samples to each group 9i by drawing from a multinomial distribution with parameters (group weights, N)
5: Initialize an empty list for sampled responses
6: for each group 9; do
7: For each response in gt. compute quality score (c train_aace cacti___ccrrry)
8: Normalize quality scores within the group (use uniform weights if all scores are zero)
9: Sample Thg responses from Jt. weighted by quality scores
10: Add the sampled responses to the output list
11: end for
12: return the list sampled responses
D.3. REX Modification
For the REX (Refinement through EM-based sampling) algorithm, we adopted the hyperparameter C' = 20, aligning with the recommendations from the hyperparameter analysis presented in the original REX publication (Tang et al., 2024). Recognizing that REX's inherent sequential processing can lead to substantial computational time, we implemented two key modifications to enhance its efficiency:
1. Accelerated Refinement via Multiple Completions: To speed up the REX algorithm, we modified the sampling process. Instead of generating a single completion per prompt, we sampled four completions simultaneously. This approach leverages the efficiency of modern inference systems where the computational cost associated with prompt processing incurred only once, even when generating multiple output sequences.
2. Parallelized REX Instances: To further reduce overall execution time, we parallelized the execution of REX itself. Rather than running a single REX process. we launched four independent REX instances that were executed concurrently across multiple compute nodes. This approach draws conceptual parallels to island genetic algorithms, where a global population is partitioned into isolated subpopulations that evolve independently. Such parallelization strategies offer several advantages: they promote greater diversity in the search space, mitigate the risk of premature convergence, and have demonstrated promising results in recent work such as FunSearch Comera--Paredes et al., 2024).
D.4. Training
We fine-tuned our model using Unsloth (Daniel et al., 2023), starting from the Instruct model at each iteration. The training setup included warmup ratio of 0.1, the AdamW optimizer with a learning rate of 5e-5, a batch size of 1, gradient accumulation over 64 steps, and weight decay of 0.05. Training was performed only on the response using RS-LoRA or RS-QLoRA for larger models (those exceeding 14 billion parameters) in bfloat precision. Additionally, for each training example, the order of the grid was randomly shuffled to improve generalization.
D.5. Inference
For inference, we employed SGLang (Zheng et al., 2024) as our engine. To accelerate generation, we sampled 50 completions in parallel for each task. Each task's prompt included one few-shot example drawn from the ARC training set, selected from a different task from the previous generation (or from the current generation for the first iteration). To reduce computational
18

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
cost, generation for task halted least 100 solutions achieved perfect accuracy on ARC input-output examples.
This criterion was applied independently during both the sampling and refinement phases.
Algorithm 3 SOAR
Require: pretrained L.M,, ARC dataset
Ensure: Best performing program solution
1: loop
2: Generate 3k program candidates
3: Apply program refinement to obtain 6k programs
4: Perform majority voting on refined programs
5: Update search traces
6: Perform data selection
Apply generation refinement fine-tuning
8: end loop
9: return Top performing program solutions
E. Learning to generate and refine programs jointly
Given that both generation and refinement can be improved independently, should we train separate specialized models or can a single model learn both capabilities? Table 7 explores several combinations using base/finetuned models for generation and refinement steps. The results reveal several key insights:
1. Negative transfer from generation to refinement: Models finetuned for generation (fine-gen) decrease refinement performance compared to base models (exp 2 < exp 1).
Positive transfer from refinement to generation: Models finetuned for refinement only (fine-ref) strongly improve program generation compared to base models (exp > exp 1), even more so than models finetuned for generation (exp 7 exp 5),
Positive interaction effects between refinement and generation: Models finetuned for refinement and generation jointly (fine-both) lead to better generation (exp 8 exp 7 exp 1) and better refinements (exp 8 exp exp 1) than leveraging two models trained each of the tasks.
These results demonstrate the importance of learning both to better generate and to better refine programs, and highlighting a useful synergy between the two capabilities. This suggests these tasks share underlying knowledge about program structure and transformation patterns.
Exp
Gen model
Ref model
Gen acc
Search ace
1
base
base
29.67
34.83
2
fine-gen
32.92
3
fine-ref
42.88
fine-both
44.04
5
fine-gen base
36.46
40.63
6
fine-ref
43.88
7
fine-ref
base 39.17
39.93
8
fine-both
39.79
44.42
Table ARC-train performance using different combinations of models for generation (col 2) and refinement (col 3). fine--gefrre/bboh indicate base model finetuned for generation, refinement or both. Gen and search acc indicate the ARC-train accuracy after generations (3k solutions) and after search (3k generations + 3k refinements).
F. Program synthesis using mix of induction and transduction
During our analysis of data collected in the self self-imprus oel-viipoo phase on ARC-train tasks, we identified some solutions that employed hybrid approach combining transduction and induction (see Code F). These solutions used Python to compute
19

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
basic properties like matrix dimensions, then applied "transduction" strategy to directly map these computed properties hardcoded outputs.
This hybrid approach presents several issues:
Impact on Majority Voting:
Such solutions can severely compromise our majority weighting strategy. Instead of learning generalizable patterns, the model essentially copies and pastes training outputs from the prompt, which skews the voting mechanism toward potentially incorrect solutions.
Limited Generalization: The hardcoded conditional structure (as shown the example) creates brittle solutions that only work for specific input dimensions encountered during training, failing capture the underlying logical patterns that define the ARC task.
Detection and Mitigation: Fortunately, these problematic solutions can be easily identified and filtered out by checking whether any computed outputs using the solution appear in the code (solution). This simple validation step helps maintain the integrity of our inductive learning process.
def
trans form (grid_1st list [list (int]l) -> list [list lnnt]l: grid = [row]: | for row in grid_ rows, cols (Ien(gri)), (grid((0)))
central_valuee None
for r range(rows)
for с in range (cols):
if grid[r] [c] != 0:
contral_value grid[r] [c]
break if central_value is not None: break
if value is None:
return grid
if rows cols 13:
pattern [[0,
01]
elif rows =- 17 and cols == 12:
pattern [[0, ..., 0]]
elif
13 and cols 18:
pattern [10, ... 0]]
elif rews == 17 and cols == 19:
pattern = [10,
0]]
else:
return grid return pattern
G. Prompts
Prompt for sampling solution:
Role: aystem
You are AI assistant specialized in solving Abstract Reasoning Corpus (ARC-AGI) tasks by reasoning and generating 2ython Role: user
You are an AI assistant specialized in solving Abstract Reasoning Corpus (ARC-AGI) tasks by generating Pyzhon code.
Your goal is to analyze input-output grid pairs. The outputs were produced by applying transformation zule to inputs. Implement transtormation rules as Python function. You should only write the implemented the transformation in code.
You must write code in triple backticks 'pythnn and then (*), You must write function called 'transform' which takes single argunect, the input grid 'iit [list [int]]) " and returne the transformed grid (also as 'list [list
You should make sure that you impl ement. a version of the transformation which works in general (at least for all given input-output pairs and test input pairs).
20

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
The number in input grid can be mapped the following colors: 0:Black; 1:Blue; 2:Red; 3: Green; 4:Yellow; S:Grey; 6:Pink; 7:0range; 8:Purple; 9:Brown
Now, solve the following ARC-AGI task:
Task to solve:
## Input (grid shape: by 3).
I[3 8]
[3 0]
[5 011
# Output
(grid shape: 3 by 3)
IIO 5]
[0 21
[8 311
## Input 2 (grid shape: by 3
II3 2]
[1 0] [o 0)]
EF Output: (grid shape: 3 by 3)
IIO 0]
[0 11
[2 51]
# Test Input 1. (grid shapes 3 by 3)
II6 5]
[6 0]
[4 0]]
Prompt for sampling Refinement:
Role: system
You are an AI assistant specialized in solving Abstract Reasoning Corpus (ARC-AGI) tasks by reasoning and generating Python cede.
Role:
You are an AI assistant specialized in solving Abstract Reasening Corpus (ARC-AGI) tasks by repairing Python code implementations.
Your goal is to analyze input-output grid pairs. The outputs were produced by applying transformation rule to the inputs.
wi11 be given a python function "transform' that was supposed to impl emenement the transformation rule, but it is not working correctly for all inpots. You role is to fix this 'tzansform' function.
solution should be:
Accurate: Correctly fix the transformation for all giver inputs so they give correct outputs as provided (it should also work for all test inputs)
Comprehensive: Handles all possible input scenarios
Well well-structured: Uses clear, readable, and efficient code
The number in the input grid can be mapped to the follawing colors: 0:Black; 1:Blue; 2:Red; 3: Green; 4:Yellow; S:Grey; 6:Pink; 7:0range: S:Furple; 9:Brown
**Now, repair following ARC-AGI task implementacionn:>
Task to solve:
# Tnput 1 (grid shape: by 3):
21

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
[10 71
[7 7]
[o 711
FF Output
(grld shape: 9 by 9)
[10
7
10
7
7]
[0
7
7
0
7
71
[0
0
71
10
0
71
10
0
7]
[0
7
7
7]
10
#F Input 2 (grid shape: by 3)
[14 4]
[0 0]
[0 c]1
FF Output
(grid shape: 9 by 9)
[[4
0
0
0
4
0
4]
[0
0
0
[0
0
01
[0 [0
0
0
01
0
C]
[0 10
C1
c11
EF Input (grid shape: by 3);
IIO 0]
to 21
12 2]1
Ft Output
(grid shape: 9 by
IIo [0
C]
0
C1
[O
0
0
01
[O
0
0
C1
0
0
[0 [0 [0 [2
2
0
21
0
0
01
0 0
2]
2 211
## Input
(grid shape: by 3
[[6 0]
[6 0] [o 6]]
## Output
(grid d shape:
9. by 9)
[16 [6
6
6
0
0
0
C1
0
0
C1
[6 [6
0
01
0
0
01
[0
0
0
C1
6 6
[0
[0
611
22

---

return output Learning program synthesis with self-improving language models: A case study on ARC-AGI
#f Input (grid shape: by
I12 21
[o c]
[o 211
FF Output (grld shape: 9 by 9)
[[2
2
2
2]
[0
0 0
0
0
C1
[0
2
2
0
2
21
10
01
[O
0
C]
[0
2
2
2]
10
0
FF Test Input 1 (grid shape: 3 by 3) I17 71 7] [7 011
Previcus amplementation:
t 'pyython
def (ransform(grid))
1en (grid)
len (grid([())
cutput_size =
output [[0] output_size for in range (output_size) range
for j in zange (m)
value grid[i]
for 11 in range m, (1 + 11 m): for 33 zangelj
+ n): cut] value
This implementation of transform function carrectly worked 0/5 train input-output pairs. Detailed results:
# Cutput computed by 'transform' incorrect.
The execution gave the following reaults (grid shape: 9 by 9)
[[0
7 7 7 7]
7
[0
7
7]
[7
7
7
7
7]
[7
7
7
7
7]
[7
7
7]
[0
10
71
[0
7 7 71]
## Output 2 computed by 'transform' is incorrect.
The execution gave the following results (grid shape: 9)
[[4
4]
[4
0 4
4]
[4
0
0 4
&
41
[O
0
0
C1
0
10
01
[0
4
6]
[0
4
[0
EF Output computed by 'tsansform" is incorrect.
The execution gave the following results (grid shape: by 9)
[[0 o 0 0
23

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
0
[0
0
0
C1
[0
2
2
21
10
0
2
2
21
10
2
2
21
12
2 2
2]
12
0 0 2
211
## Output 4 computed by 'transform' ! is incorrect.
The execution gave the following results (grid shape: 9)
II6
0
C1
[6
6 0
0
C]
[6
6 6 6 0
C]
I6
0
16 16 10
01
61
[0
6]
10
6
## Output computed by 'transform' incorrect.
The execution gave the following results (grid shape: 9 by 9)
[[2 2
2
2
2
2 2 2]
[2
2
2
10
0
0
01
10
0
0
01
[0
0
0
6]
2
[0
2 2 21
10
2 211
EF Output Test computed by "transform" (we don't t know if it is correct or not The execution the following results (grid shape: 9 by 9)
[17
[7
7
7
71
17
0
0
7
7
7]
[7
0
0
0
7
7
7]
7
[7
7]
17
C1
17
0 0 C]
17
0 C11
The previous code give incorrect output for: Output 1, Output 2, Output 3, Outpur 4, Output 5, Now, you nead to fix the code to produce correct oucput for all inputs.
----- Role: Assistant (sesponse)
"pythhnn deř transform(grid) output = [[0 for in xange(9)] for in range (9)
for i in range (3): for in range
grid[i]li]
for x in :3);
for in range (3): output x]13 3 y] -grid(x] [y]
return oucput
24

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
H. Examples of Solutions
H.1. ARC Problem 6b9890af
Solve the task online here, then read the program.
def transform (granform(grid))
def find bind_bouuningg_oox (grid, coler):
min_row, max row, min_col, mex_col (float ("inf") ) float ("-inf"), ) float ("inf"),
float (1 ("-inf"))
for i, row in enumerate (grid) for j, cell in enumerate (row)
if cell color:
min_row = min (min_row, i)
max_row max [max_row, i]
min_col min (min_cal, j]
max_col max (max_col, j} return (min row, max row, min col, max_coi)
def create wtw.s_eeuddnnnnneee box (grid, color):
min_row, max_row, min_col, max_col bond_ (grid, color)
bounding_box []
for i in range (min_row, max_row + 1): row = griggidiii [i] [min_col :max_col 1] beunding_bost
(row)
return bounding_box
def pesizzebbunding box (bounding size]: resized_bot = [10] size for in range (size)
rows, cols (Ien (pounding_boxi)
len (bounding_box (0]))
row_step rows / (size 1) if
else 0
col_step cols / (size 1) cols 1 else 0 for i in range (size)
for j in range (size):
if box
0 == size
0 (j size bounding box 0] [0]
else:
row_idx = min (int (i row_step), rows 1) cox_idx min () :col_step), cols 1)
resized box [i] bounding box frow_ddd [col_ ccl]
return resized_box colors set () for row in grid:
for cell in if cell != 0:
cclors.add
color_bous betoorbboudinnggooee. - ()
for color in colors:
color_bounding_boree [color] create_boundiee box (grid, color)
max size
for color in colors:
bounding_box = color_ bator_bounding_eee boxess [color]
max_dim
(len (bounding_boxi) len boounding (0)))
max_size max (max_size, max_dim)
output_size max_size
output_grid [[0] output_size for in range (output_size)
for color in colors:
bounding box
golor_bounding boxes color
resized box resize bounding box (bounding box, output_size for i in range (output_size): size):
for j in range (output_size)-
if resized_box(i][]] output_grid(ill)] color
color:
return output_grid
25

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
H.2. ARC Problem 72ca375d
Solve the task online here, then read the program.
def transform (onnidd)
def find_largest_reetangge (grid);
rews, ccls (len (grid), (grid(((0))
max_area 0
max_rectangle (0, 0, 0, 0)
for i in range (rows) for j in range (cols):
if grid|i]lj] continue color grid[i]
width
while + width < cels and grid width] == color:
width
for k range (i, rows):
if grid[k] [j] != color:
break
for 1 in range width)
if grid[k] color: break
else:
continue
break
area - (1 1)
if area > max area:
max_area area
max_rectangle (i, 3, k, 1
return mak_rectangle
def extract_ extract_rectangeeee fectangle(grid, rect):
i, 3, k, 1 rect return [:1l]]:1] for in grid[i:k]]
def Land_Larggst_ccomeeee (grid, (grid, color):
rows, cols (1en (grid), len (grid(00]]]
visited [[aasse] cols for in range (rows) ]
max_size max max_component []
def dfs i, component):
if < 0 or i >=
(j >= cols) visited [il (grid[i) [i]
!= color):
return visited [i] True Compenent.appende (ii, )))
dfs 1, ], component)
dis(i -1, j, component)
dis(i, + component)
dis(i, - component)
for i in range (rows)
for j range ccols):
if grid[i] [j] -- color and (not visited [i] (31):
component []
di, j, component)
if len (component)
size:
max_size len (component)
max_component component
return max_component
def extractt_cmponeeet (grid, component) min min((c[0) for in component))
26

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
max_i max (c][] for in component)]
min_] min[[c[1] for in component)]
max_] max([c[1] for in corponent)]
+ 1] for row in grid imaii_iii_ia______ 111 rect find pan___larggst__ectanlleee (grid)
return [row[min_jjmaal
rect geat_grid extract_rectangiee (grid, (grid, rect)
if len (rect_grridd) = 2 and len (roct_t_grrid = 4:
return rect_grid [xe]tt color grid[rect [0] [xect[l]]
component = color)
component_ggiia return component_grid
(grid, component)
H.3. ARC Problem 1f642eb9
Solve the task online here, then read the program.
def transform (grid);
def dfs (x, y, component_num) :
if < 0 or x >= len (grid) or < 0 or (y >= len (grid(())) or (grid[x) != 5)
return
grid [y] component_num
for dx, dy in [(-1, 0), (1, 0), (0, -1), (0, 1)]: dis (x dx, y dy, component_num num) component_nee =
for in range (Len (grid)):
for j range (len (gridd(()))) if grid[i] -- 5: dfs(i, j, component_num) component_ _um += 1 if component_nme num
component_nis num num
componcnt_men map (i: num for 1, nur. in enumerate (range (1, component_num start=1)) }
for i in range (1en (
for in range (grii((()))) if grid[i] grid[i][i] component_map [grid [i]1
component_map:
return grid
H.4. ARC Problem ef26cbf6
Solve the task online here, then read the program.
def transform (grid):
transformed_gride [row[:] [:] for row in grid]
directions [(-1, 0), (1, 0), (0, -1), (0, 1)]
def in_bcunds (x, y): return
x len (grid) and y len (grid(0))
for i in range (Len (grid)))
for j in range (Len (grid(il))): if grid[i] -- 8: for dx, dy in directions:
ni, nj (i + dx, j + dy)
if in bounds (ni, ni) and grid [ni] != 8: if grid[ni] [ni]] [ni] != 0:
Lransformed_gridit
grid [nj]
else:
step =1
while bounds (ni + dx step, nj + dy step) and grid dx step] [nj + dy step]
step
27

---

Learning program synthesis with self-improving language models: A casestudy on ARC-AGI
if in_bounds (ni
step, nj dy step) and grid stepl [n] dy
step] !- 0:
transformed_gries
[3] grid dx step] [nj + dy
step]
return transformed_grat gridd
H.S. ARC Problem 11876c06
Solve the task online here, then read the program.
def transform (
rows = len (grid)
cols = len (grid(0)) if rows > else
transforred__gree for num in (1, 10) positicns [(r, c) for r in sange(rox)) for in range (cols) if grid[x][c] == rum
[row[:] for row in grid]
for i in range (len (positicns) 1):
rl, cl positions [i]
r2, c2 positions
1]
dr - if r2 > rl else -1 if r2 < rl else 0 de if c2 cl else -1 if c2 < cl else 0
F, (rl dr,
de)
while != r2 or с c2:
sransformed.grid/ [c] = num
+ dr
dc
return transformed_grid
I. History of refinement
L.1. ARC Problem 22168020
Solve the task online here, then read the programs.
incorrect program def transform(grion
def fill_ gaps (rew)
start, end (-1, -1)
for i, val in enumerate (row): if val != 0: if start -1:
start i
end =
if start != -1 and end != -1: for range (start, end 1): if rowli] row [i] cow|start]
return row
for in range (1en (1en
grid[i] fill_gaps (grid(il)
for col in range (len (grid (1nridi1i))))
column_values_ start, end (-1, -1)
values = [grid[ [col] for row in range (len (grid)
for val in enumerate (column_values)
if val 0:
if start == -1:
start 1
end =
if start != -1 and end !- -1: for range (start, end 1):
28

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
if column_values [i] column_valiue [i] column_vall values [start] grid[i][col] colurn_values [start]
return grid
incorrect program def transform(grid)
def fill_gaps (line):
Pi11 gaps between non-zero values single line (row column) =**
filled line = linel:]
= (In
start -1 for range(n):
if lineli] 1= 0:
if start == -1: start =
end for j in range (start 1, end) fineed line [start]
start = end return filled_line for in raage((en
grid[i] for col in fill_gaps(grid(t]] column values fill gaps (column values)
volume_values Igrid row] ccol for row in zange (Len (grid)
for rew in range (len (grid)):
grod[row] [col] @illed_colum [row]
return grid
correct program def (ransfor((grid)
def fill_gaps (line):
Pi1J gaps between non-zero val values in single line (row col cmm)) ***
filled line linel:]
n -lnneinne) i ==
while 1
if lineli] start_value = line [i]
j == while <<n and line[i] == 0:
j +=1
end linel:] if 1 n else if start start_value value == end_value:
for k in ringe 1, i):
Filled_line[1
start_value
i-j
else:
i += 1
return filled_line for in range (len
grid[i] (gll_gaps for col in alene(len (grid colunn_values filled_colum for rew in range (len (gendd))
[grid[row] [col] for row in range (len (grid) )
fill_gaps (cclurn_alel)
grid[row/ [col] filled col umn [row]
29

---

Learning program synthesis with self-improving language models: A case study on ARC-AGI
return grid
1.2. ARC Problem 0d3d703e
Solve the task online here, then read the programs.
incorrect program
def transform (grid):
transformat inn (33, 1, 2), (3, 2, 2), (3, 1, 21): (14, 5, 61 14, 5, 6) (4
5, 61) (12, 3, 8) (2, 3, 8) 12, 3,
11 ((6, 4, 9), 16, 4,
(6, 4, 911 11
61): ((1, 12
8, 6) 15,
61, (5,
(9
9,
(19,
2) (9
4, 2)
6) (8, 3
} (8, 3, 6)) (18)
1 3),
1,
(8,
1, 3)): (17, 5, 4) (7, 5, 4) 17,
grid_tuple tuple (tuple(row) for row in grid))
return 1ist (1ist, transformation transformation_mapiinn get (grid_tuple, grid)))
incorrect program def transform (grid) cransformatiinn (33, 1, 2), 13, 1,
2), (3, 1, 2)): (14, 5, 6) (4, 5, 6), (4
5, 6)), (12, 3, 8)
(2, 3, 8) (2, 3,
11 ((6, 4, 9) 16, 4,
(6, 4, 9)), (1
6), 15,
6),
(5,
61) 6
(11, 9,
(1, 9, 2) (1,
(19, 2
(9
4, 2) (9, 4, 21
3,
((8, 3,
(8, 3,
(8, 3, 61) (18,
1,
(8,
3), (8,
1, 3)): (17, 5, 4), (7, 5, 4) (7,
grid_tuple tuple (tuple (row) for row in grid)
transformed_gre _rridddrrddd =
formatorntiion ptwpssorrniinnnpppnnggggggo tgrid_tuple, grid_tuple)
return [list (row) for row in transformed grrid
correct program def transform (grid):
transformation_mappii = (3: 4, 1: 5, 2: 6, 8: 9, 5: 1, 6: 2, 9: 8, 4: 3)
transformed_grid [[Iranstorratiionnaaapiis got (grid(i) [3], gridill (i1) for in range (len(gri(()))) for in range (grid))
return transformed_grid
30


Reviews:
[{"review_id": "BIH9xhxLIi", "reviewer": "Reviewer_xjKq", "summary": "This paper introduces SOAR (Self-improving Operators for Automated program Refinements), a framework that enhances language models' program synthesis capabilities through an iterative self-improvement process.\n\n- SOAR alternates between a search phase (using a language model to generate and refine candidate solutions) and a learning phase (fine-tuning the model on these search attempts).\n- Instead of relying on fixed model capabilities, SOAR allows models to learn from both successful and failed synthesis attempts, leveraging hindsight relabeling to learn from all generated programs, not just correct ones.\n\n\n## Update after rebuttal\n\nThe authors answered my questions in great detail and provided convincing elements to address my concerns. I have, therefore, updated my rating to accept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer xjKq for their review of our manuscript and appreciate the recognition of the key aspects of our approach.\n\nThe review itself did not include any critique or suggestion for improvement, but it did not come with the highest recommendation either (\"Weak accept\"). \n\nThis decision suggests there may be opportunities to strengthen our work, and we would be happy to address any additional feedback or suggestions for improvement. \n\nPlease consider raising your score if you have no concerns. If you do, please let us know what they are so we can try to address them. Thank you!", "claims_and_evidence": "Claims supported.", "methods_and_evaluation_criteria": "ARC-AGI is the program synthesis benchmark reference to measure reasoning abilities.", "theoretical_claims": "Nothing to report.", "experimental_designs_or_analyses": "Nothing to report.", "relation_to_broader_scientific_literature": "Nothing to report.", "essential_references_not_discussed": "Nothing to report.", "other_strengths_and_weaknesses": "None.", "other_comments_or_suggestions": "None.", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "87yttAAPTA", "reviewer": "Reviewer_5bhq", "summary": "This paper introduces SOAR, a framework for self-improving program synthesis tested exclusively on the Abstraction and Reasoning Corpus (ARC). SOAR operates in two phases: program search phase and learning phase. In the program search phase, it generates lots of candidate Python programs and selectively refines them. In the learning phase, it uses the programs from the search phase to finetune itself. It not only uses the successful programs but also uses a hindsight replay technique similar to Codeit from Butt et al. to obtain more program samples with input-output pairs. By pooling samples across different model sizes (7B, 14B, and 32B parameters), it can boost performance and achieve 41.25% accuracy on ARC-test. This pooling approach outperforms any individual model, suggesting that different model sizes solve problems in complementary ways.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer 5bhq for their time and constructive feedback. \n\nThe reviewer understood our work and noted the strength of our experimental study and how its results support our claims. This said, the reviewer raised several concerns that we have addressed below. \n\n**Are results controlled for compute budget?** Thank you for raising this question. Section 4.2 aims at answering the three main questions:\n\n- Does finetuning for generation help generation? → Yes. We compare with the base model thus do not control for finetuning costs (one is not trained, seed response to reviewer ifzp for detail analysis between finetuning and inference cost), but we control for the search budget (3k each)\n\n- Does finetuning for refinement help refinement? → Yes. Same search budget of 6k on each side, not controlled for finetuning cost (one is not trained) \n\n- Does finetuning the model for both works better than finetuning two models, one for each task? → Yes. We control for search (6k) and finetuning costs: specialized models are finetuned on 50 examples per task each, while the combined model is trained on the 100 pooled examples once. \n\n**Do models of different sizes solve problems in different ways?** Do models of varying sizes approach problems differently? Combining results from models sized 7B, 14B, and 32B yields up to a +4.75% improvement on the training set—a notable gain for the challenging ARC task. This suggests some complementarity: smaller models (e.g., 7B) still contribute value when paired with the largest (32B).\n\nFuture work will explore why, but we conducted some simple analyses to start answering this question: Some models give plausible solutions, but they don’t handle edge cases well; they just implement a high-level idea of the transformation with missing details. We further identified examples of tasks solved by smaller models but unsolved by the 32B model that we make available here: https://anonymous.4open.science/r/arc_example-EBB9/. See more details in our answer to Reviewer FfTX discussing generated programs qualitatively. \n\n**Comparison to transduction approaches to ARC (Akyürek et al.):** Our approach relies on program induction and indeed underperforms some transduction approaches predicting output grids directly (e.g Akyürek et al.’s 47.1% with test-time training, or OpenAI’s o1).\n\nOne thing to note is that transduction approaches are more susceptible to data contaminations: the test set output grids have been publicly available for years (e.g. on GitHub and Kaggle). Induction methods, on the other hand, need to predict correct transformation programs which are not found online. Akyürek et al. also note that their method was developed using a subset of the test set, raising concerns about overfitting to those specific examples. \nWhether transduction methods overfit or not, we believe it is valuable to pursue both approaches independently, as we do not know which one will solve ARC in the end. Program synthesis also has many applications beyond the ARC domain. Moreover, programs are more interpretable and open a window on the reasoning process of LLMs that transduction methods do not offer. \n\n**Reliance on Qwen base models:** We selected Qwen series models for their strong coding skills relative to their size. Experiments show that better base models yield superior final performance, though smaller models can match the initial performance of larger ones using SOAR. We expect SOAR to work with any model capable of finding correct solutions via search, with stronger models producing better outcomes. Limited compute resources prevented testing SOAR on other models, leaving this for future validation.\nSOAR could be adjusted to boost weaker models by increasing compute per iteration (longer searches, more finetuning) or adding iterations. Future work might explore pooling outputs from diverse models (e.g., Mistral, Gemma, Llama) to enhance complementarity, as seen in our experiments. While Qwen models excel, smaller alternatives like Mistral 3.1 or Gemma-3, improving in reasoning, could substitute. These ideas are now in the updated manuscript.\n\n**Assumption of access to full test set:** We now discuss this assumption in the manuscript but we note that this is a common assumption, as it is the format used in the original Kaggle competition. Our approach could in principle be used on one task at a time. This would require training a separate model for each task which would be computationally expensive and might hinder possible effects of generalization across tasks. \n\n**Confusing terminology Generation vs Search:** The Sample approach (pure generation and ensembling) is indeed a form of (simple) search. We updated the paper with less ambiguous terms: Sample vs Sample&Refine.\nPlease let us know if the above discussion answers your concerns and consider raising your score if it does. If it doesn’t, let us know which concerns remain so we can try to address them. Thank you!", "claims_and_evidence": "The paper claims that LLMs can learn to solve ARC-AGI tasks by self-improving - interleaving program search and learning. They empirically show that this is possible by presenting the experiments on the ARC-AGI dataset.\nI would like the authors to clarify some claims if they are controlled with the same compute budget. For example, in table 4, they claim that the results demonstrate the importance of learning both generate and refinement. I wonder when comparing that to learning generate only, and refinement only, if they are all finetuned on the same compute budget. \nAlso, section 4.3 suggesting that different model sizes may solve problems in complementary ways needs more evidence, as the pooling results do not seem to improve very dramatically.", "methods_and_evaluation_criteria": "The evaluation criterion is the ARC-AGI dataset. One specific assumption is that the model is solving the full test set of ARC-AGI all together instead of solving it one-by-one.", "theoretical_claims": "N/A: The paper is empirical case study of LLM code generation on ARC-AGI benchmark", "experimental_designs_or_analyses": "The design for the experiments are sound as they based on the ARC-AGI training set to perform different ablation studies to see which methods e.g. synthetic data selection methods, achieve better results.", "relation_to_broader_scientific_literature": "It connects various previous ideas of LLM code generation work. Please see the below comments.", "essential_references_not_discussed": "Not that I'm aware of.", "other_strengths_and_weaknesses": "Strength:\nThe paper presents a very thorough experiments study on ARC-AGI training set demonstrating how some of the design decision choices are made. It serves as a very good report on how to approach ARC with LLMs.\n\nWeakness:\nThe work presents a very thorough case study of ARC-AGI tasks and how to approach the task from a PBE perspective with LLMs via program search and learning. However, the key ideas resembles a lot like previous related work. The search and finetune loop is similar to the Codeit ARC-AGI work from Butt et al. but with better LLMs and using Python code instead of custom DSL - the key idea of using hindsight replay to generate more data for finetuning. Using Python code instead of custom DSL, and also refinement for ARC-AGI tasks, has already been presented in previous works. The work integrates all these ideas together and shows the performance gains; however, it is also in the ballpark of previous works as well for solving ARC-AGI with code. In terms of accuracy with open models, it also lags behind the approach of direct prediction of the output compared to various Kaggle competition work and Akyürek et al.'s test time training work, which achieves 47.1%.", "other_comments_or_suggestions": "The terminology of generation accuracy and search accuracy is a bit confusing as generation also includes 3k sample candidates and checking which kinds of like searching.", "questions_for_authors": "How does the base model affect the performance of SOAR? Would it be possible to achieve similar performance with other less powerful models than the Qwen series?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l2c4ARVvEo", "reviewer": "Reviewer_FfTX", "summary": "The paper introduces a novel framework for program synthesis that integrates large language models (LLMs) into a self-improving evolutionary loop. The framework alternates between two phases: (1) an evolutionary search phase using an LLM to generate and refine candidate programs for a given task, and (2) a learning phase where search traces (both successful and failed attempts) are used to fine-tune the LLM's generation and refinement capabilities. It is claimed that this creates a virtuous cycle where improved models lead to more effective search, generating richer training data for further model improvement. The paper evaluates the method on the challenging ARC-AGI benchmark.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer FfTX for their helpful feedback. \n\nThe reviewer noted that the approach is reasonable, intuitive and novel. They commented on the strength and extensive details of our experimental studies, acknowledging that they supported our claims. This said, they raised several concerns that we address here:\n\n**Too many components in the method:** This kind of feedback is particularly helpful to help us simplify the presentation of the method. We argue SOAR is simple: 1) use some kind of search algorithm to generate candidate programs (REX for us); 2) select interesting programs and apply hindsight relabelling; 3) finetune; 4) repeat. Section 4.2 explores step 2’s design space for ARC-AGI, not a “mad scramble” but a guide for adaptation, and we think that it will help others adapt SOAR to their use case. The revised manuscript better separates the high-level idea from this exploration.\n \n**Need for qualitative analyses (features across generations and models, examples and error analysis):** We agree that qualitative analyses are important here. Our analyses found the following trends across generations (true of all model sizes, numbers for Qwen 14B):\n- The proportion of error-free programs rises on average from 0.92 (gen 0) to 0.98 (gen 4). \n- Complexity increases given the increase in Lines of codes from 16.6 to 24.5, number of control structures (5.2 to 9.5) and the maximum depth of control structures in the AST from 3.4 to 4.7. Interestingly, the number of helper functions remains stable (1.5 -> 1.4).\n\nTo discuss qualitative results, we created an anonymous repository containing interesting examples at https://anonymous.4open.science/r/arc_example-EBB9/. We choose one example for each of the following categories, but feel free to explore other examples: \n\n- **Examples of tasks solved only by smaller models:** e.g. example in solved_by_smaller_models_only\n\n- **Examples of tasks only solved by later generations:** e.g. example in solved_in_later_generations\n\n- **Examples of failures and successes** e.g. examples in other folders\n\nThe revised version of the manuscript will include graphs of the trends across models and generations in the Appendix, a link towards the codebase and the repository of examples. This will open the opportunity for others to study generated programs in more detail and discover new insights. One way could be to use LLMs to label each program along various dimensions: eg, does it use recursion? Does it identify objects? Symmetry? And ask LLMs to describe the overall solving strategy, then use these features to analyze how their distribution shifts across generations and models, whether the diversity of strategy used increases or decreases with finetuning, or when a solution is found. These analyses are left for future work, either by us or by anyone else using the dataset of generated programs we will release with the paper. \n\n**Practical constraints of the AGI-Benchmark:** This work did not run in the Kaggle competition but was constrained by limited compute resources. Experiments for the 14B model cost an estimated USD 4000 (excluding method development costs). This explains why we focused on a single domain: preferring conducting carefully experiments and exploration of the design space (Section 4.2) rather than superficially reporting final success rates on several benchmarks. These computational constraints forced us to use data-efficient finetuning methods (LoRA, unsloth) and capping examples per task we could train on (50/6000 generated codes). The revised manuscript makes these constraints more explicit and discloses a more detailed cost of our experiments to clarify some of the design choices. \n\n**Does SOAR break through scaling plateaus?** Does SOAR overcome scaling plateaus? In Figure 4, dashed lines represent one-shot closed-source large LLMs, not scaling plateaus. Here, “scaling plateaus” refer to the leveling-off of each curve as model size increases. Each enhancement shifts to a higher scaling law, breaking these plateaus. Our new figure (compute_match_14b.pdf in the repo) shows scaling search also hits plateaus, which self-improvement surpasses.\n\nThe Claude+Search comparison (Greenblatt, 2024) is absent from Figure 4 (42%). We believe SOAR could break this plateau too, though applying it to Claude is infeasible (closed model, high cost). Updated Figure 5 and the new figure clarify these points.\n\n**Minor concerns and suggestions:**\n\n- **GA references:** We added the GA references suggested and complemented our EC related work.\n\n- **Majority voting:** Majority voting is here necessary to decide which final solution to submit to a given task given a whole search trajectory and is therefore used in all search approaches. \n\n- **We corrected typos**\n\nPlease let us know if the above discussion answers your concerns and consider raising your score if it does. If it doesn’t, let us know which concerns remain so we can try to address them. Thank you!", "claims_and_evidence": "* Claim: SOAR achieves significant performance gains on ARC-AGI compared to baseline search methods and single-shot LLMs\n  + The quantitative results in Table 1 demonstrate that SOAR significantly outperforms single-shot approaches and even larger closed-source models using basic search. The iterative improvement shown in Figures 2 and 3 provides evidence for the self-improvement claim\n* Claim: SOAR establishes state-of-the-art results for program synthesis on ARC-AGI among open-source LLMs\n  + Table 5 compares SOAR's performance to various prior inductive approaches on ARC.  SOAR's performance of 41.25% on ARC-test with the pooled 32B model is shown to be competitive with or exceeding previous open-source and even some closed-source approaches (when considering budget and data usage).  (There have also been further contemporaneous SOTA improvements)\n* Claim: SOAR leverages positive transfer between generation and refinement finetuning\n  + The experiments give suggest a positive interaction and transfer of learned capabilities between generation and refinement\n* Claim: SOAR enables test-time adaptation and continuous improvement on target problems\n  + Figure 3 and Section 4.4 demonstrate performance improvements during test-time training iterations on ARC-test\n* Claim: SOAR breaks through performance plateaus of search-based methods\n  + The 'scaling plateaus' shown in Figure 4 don't seem to account for scaling up of commercial model sampling (eg: Greenblatt 2024) - which makes this line of 'claim' somewhat suspect.", "methods_and_evaluation_criteria": "* Evolutionary Search with LLMs: Using LLMs for both program generation and refinement leverages the generative power of LLMs within a structured search framework, and evolutionary search is appropriate for exploring the large and complex program space in ARC-AGI.  The use of REX (Thompson sampling with exploration bonus) for refinement seems like a reasonable choice to manage the search budget effectively.\n* Self-Improving Loop: The core idea, alternating between search and learning, is novel and makes intuitive sense for overcoming the limitations of fixed-capability models.  Finetuning on search traces (both successes and failures) is a practical way to learn from experience. \n* Hindsight relabeling to augment training data is a clever technique to increase the training data quantity and quality.\n* Test-time Training: Adapting the self-improvement loop to test-time training appears to have been an 'addon technique' rather than the core thrust of the initial research\n* Majority Voting: Ensembling with majority voting is a standard and effective technique - though it seems to be admitting defeat as a final step amidst the other innovations here\n\nThe use of ARC-AGI and the detailed experimental analysis make the evaluation strong and convincing.", "theoretical_claims": "There are no explicit theoretical claims that require proof checking.", "experimental_designs_or_analyses": "Unmentioned (though it seems clear from the experimental design) are the practical considerations of the Kaggle environment for the ARC-Prize. Given those constraints, many of the choices (LoRA, unsloth, model size, number of generations, etc) make a lot of sense - it would be nice for the over-arching explanation to be given, though. This detail would also explain the flow of experiments, from the 'grand design' through all the ablations, continually moving forwards.\n\nThe State-of-the-art Comparison (Table 5) for ARC-AGI, including CodeIt, BARC-induction, Icecuber, and Greenblatt (2024) provided clear context for SOAR's performance and establishes its state-of-the-art status among open-source LLM methods (modulo concurrent submissions).", "relation_to_broader_scientific_literature": "* Program Synthesis: The paper builds upon a long history of program synthesis research, referencing traditional approaches like Genetic Programming (Koza, 1994).  It acknowledges the shift towards using deep learning for program synthesis (Balog et al., 2016; Ellis et al., 2021) and highlights the recent impact of LLMs (Roziere et al., 2023; Guo et al., 2024)\n* Evolutionary Algorithms: SOAR leverages evolutionary search principles, drawing inspiration from mutation and crossover operations in genetic algorithms. It cites work using LLMs as operators in evolutionary search (Lehman et al., 2023; Meyerson et al., 2024). SOAR extends this by making the evolutionary operators (LLMs) learn and improve through experience, a novel aspect compared to traditional evolutionary methods with fixed operators\n* ARC-AGI Benchmark: The paper directly addresses the ARC-AGI benchmark (Chollet, 2019) - a valuable benchmark, which is less easily gamed than many other benchmarks commonly used in research", "essential_references_not_discussed": "* More foundational work on evolutionary computation and genetic programming: While Koza (1994) is cited, including other foundational texts or surveys on evolutionary computation or genetic algorithms might be helpful to provide a broader context for the evolutionary search component of SOAR. For example, work by Holland (1975) or Goldberg (1989) on genetic algorithms could be considered.", "other_strengths_and_weaknesses": "Strengths\n\n* Originality and Novelty: The core ideas include those of using a self-improving evolutionary loop for program synthesis, where the search operators (LLMs) learn from search experience.  Hindsight relabling in this context was also a nice touch!\n* Empirical Validation: The paper provides strong empirical evidence to support its claims through comprehensive experiments, ablation studies, and comparisons to baselines and state-of-the-art methods\n\nWeaknesses\n\n* While the paper's main contribution SOAR is effectively shown, there are also a lot of additional 'bells and whistles' that are also added (and ablated for) that somewhat muddy the picture.  It seems clear that this was also partly the result of reporting everything that contributed to the final results at the end of a Kaggle 'mad scramble', rather than a pure research endeavour\n* Limited Qualitative Analysis: While the paper provides examples of generated programs, a more detailed qualitative analysis of the types of programs SOAR learns to generate and refine, and how the quality of programs evolves across iterations, could be beneficial.  In the Discussion (Section 5), it was stated that \"smaller models (7B) demonstrated steeper learning curves and seemed to discover qualitatively different solutions\" - it would be great to know more about this", "other_comments_or_suggestions": "* Including an error analysis of the tasks that SOAR still fails to solve could provide further insights into the limitations of the approach and guide future research directions. Understanding common failure modes would be valuable.\n\nTypos\n* L82 : \"SOAR learns to solves an extra\" -> \"SOAR learns to solve an extra\"\n* L100 : \"preventing them to improve from experience\" -> \"preventing them from improving through experience\"\n* L143 : \"(see proof in Section 4.1)\" -> \"(pure LLM results given in Section 4.1)\"\n* L283 : Table 2: Generation acc should be all 2 d.p.", "questions_for_authors": "Qualitative Evolution of Synthesized Programs: Could you elaborate on how the quality and characteristics of the programs synthesized by SOAR evolve across iterations of self-improvement? Are there observable trends in terms of program complexity, algorithmic sophistication, or reasoning capabilities as the model is iteratively finetuned?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DhLY1P1Sat", "reviewer": "Reviewer_ifzp", "summary": "The paper introduces SOAR, a method for program synthesis that extends existing LLM-based methods by introducing an iterative fine-tuning approach. Recent LLM-based program synthesis work has relied on two methods: (1) directly querying the LLM in-context by expressing the task as language (possibly after fine-tuning the LLM on coding tasks) or (2) combining the LLM with classical search methods (for example, using the LLM prompt to generate code samples, evaluating those samples by their performance, selecting the best, and feeding them back to the prompt to get even better ones). This paper introduces an alternation between LLM-based search and fine-tuning of the LLM. This way, the search outputs from the LLM's prompt are used to improve the LLM weights, and these weights can then be used to generate better search outputs, and so on, thus iteratively improving the results. The paper benchmarks their method against baselines on the ARC-AGI dataset, a toy-yet-difficult task of transformations on colored grids, which humans handle well but ML systems do not. On this task, the paper claims state of the art performance. Further, the paper does a series of thoughtful ablations and analysis to demonstrate that the different components of their method provide an advantage. This includes not only the fine-tuning, but also innovations that they introduced into the search phase.\n\nUpdate after rebuttal: I raised my score to 4 after the rebuttal discussion.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their helpful feedback. \n\nThe Reviewer noted the importance of the problem we tackle and the quality and pedagogy of our experimental section but raised several concerns which we address below.\n\n**Controlling for compute costs:** We thank the reviewer for raising this point. In the revised manuscript, we now address it directly with a new figure: https://anonymous.4open.science/r/arc_example-EBB9/compute_match_14b.pdf. Performance improves with both model size and search budget, each following separate scaling laws. SOAR enables us to break through these plateaus and achieve higher performance levels. As shown in Figure 4, adding search and then iterative improvements via SOAR surpasses the performance ceiling reached by merely scaling model size (e.g. Claude-level performance).\nThe new figure shows a similar pattern for the search budget. Performance with the base model (generation 0) plateaus after about 8k search attempts. In contrast, SOAR at generation 1 (after 6k search attempts, followed by finetuning, and then another 6k attempts) outperforms the base model even after 12.6k attempts (+7.5%). This 12.6k budget matches the one used by SOAR gen 1 (6k + 6k search + \\~5% compute for finetuning, cf explanation below).\nAfter generation 1, search seems to plateau even earlier (~5k), but SOAR still achieves significant performance gains across generations. \nThis experiment answers the reviewer’s question: Would using the same compute budget to generate more solutions with the base model yield similar performance gains as SOAR? The answer is no. The base model stagnates at ~8k attempts, with performance at 12.6k attempts remains on par with its 6k result. Thus, SOAR achieves superior results within the same compute constraints. The revised manuscript includes a new figure illustrating this, reinforcing that SOAR overcomes performance plateaus in model size and search budget scaling, achieving higher plateaus.  \nFinetuning costs: Finetuning is inexpensive compared to the search phase. FLOPs per iteration is $6N \\times (100 \\cdot T)$, where $N$ is LLM parameters and $T$ is tokens per completion. With ≤100 datapoints per task, sampling FLOPs is $2N \\times (6000 \\cdot T \\cdot n)$, making finetuning ~5% of total FLOPs—nearly negligible. Additionally, autoregressive generation is slower (token-by-token forward passes), while finetuning processes sequences in one forward and backward pass, per Austin et al. (\"How to Scale Your Model\", 2025).\n\n**Possible leaking across test tasks:** In the official ARC-AGI Kaggle competition and related literature, there are no strict constraints on the order of task processing or the use of unsupervised data across tasks. This flexibility allows methods to leverage examples from other tasks in an unsupervised fashion, though it’s unclear whether this provides a significant advantage over refining strong candidate solutions within a single task. Several prior works (e.g., Akyürek, et al. 2024 ) adopt similar practices.\n\n**Relation to the STaR algorithm:** The confusing comment has been clarified. SOAR isn’t an implementation of STaR but shares its spirit, enhancing LLM reasoning by bootstrapping from self-generated data. STaR applies this to reasoning tasks with chain-of-thought text, using binary signals to identify correct reasoning for finetuning. SOAR adopts a similar approach, finetuning on self-generated programs from a search process, guided by hindsight relabeling of input-output pairs, training both generation and refinement. This is now clearer in the paper. \n\n**Comparison with FunSearch and Evolution of Heuristics:** SOAR enhances search algorithms using LLM models for generation and mutation. FunSearch, like REX, is one such algorithm. While SOAR could be compared to FunSearch, it’s not a direct competitor (REX is): SOAR can be used to improve upon FunSearch by training its generation and refinement capabilities. In particular, FunSearch also uses a “crossover” operator to generate candidate programs from two seed programs. These could also be trained with SOAR using the exact same iterative finetuning paradigm. This is also applied to the evolution of thoughts of EoH. Studying SOAR’s impact on FunSearch, especially training crossover for performance gains, is future work due to limited resources. We address these in our related work section.\n\n\n**Generalization to other domains (eg bin packing):** Please refer to our answer to Reviewer cz1F who also commented on this point (last point in our answer).\n\n**Minor points:**\n- We corrected the typos pointed by the reviewer\n\nPlease let us know if the above discussion answers your concerns and consider raising your score if it does. If it doesn’t, let us know which concerns remain so we can try to address them. Thank you!", "claims_and_evidence": "The claim that this method performs better than previous methods tested on ARC-AGI seems to be well supported by the evidence, with a caveat about the measurement of cost and about the use of the test set (see my comments under \"Experimental Designs Or Analyses\"). I have concerns, however, that previous methods may not have been tested on ARC-AGI, and that, as a result, this paper does not compare against those methods (see my comments under \"Methods And Evaluation Criteria\"), but I could be wrong.", "methods_and_evaluation_criteria": "The ARC-AGI benchmark is a little bit niche, so the question remains of whether the results generalize to other program synthesis tasks. However, the paper is clear about this limitation (even saying it in the title) and it does a good job about not over-claiming. Further, showing improvements on the ARC-AGI dataset is an achievement on its own, as the benchmark is far from being saturated. Trying the method on a diversity of tasks would be a great next step for a follow-up paper.\n\nRegarding the baselines (Table 1): could the paper have also used as baselines methods like \"FunSearch\" (Romera-Paredes et al. 2023) or \"Evolution of Heuristics\" (Liu et al. 2024)? In particular, the first of these was an impactful Nature paper, so it would be a natural choice. Both of these papers benchmark on a bin-packing problem that may make sense for this method too, as both papers solve the problem by generating code.", "theoretical_claims": "Not applicable.", "experimental_designs_or_analyses": "It was informative to see the successive addition of components to the method and how they affect performance on the training set. I appreciated how this even included different variants of the components.\n\nOn the other hand, I have a concern about the measurement of compute cost. The central claim in this paper is that SOAR achieves state-of-the-art performance on ARC-AGI when compared to baselines. However, both this method and the search-based baselines can be iterated for an indefinite amount of time (while the returns are diminishing, generally it is better to run for longer). A fair comparison, therefore, would require matching the compute cost of the methods in question. Was this done here? In particular, the baselines do either a 1-shot in-context query or an in-context search process while SOAR also requires multiple rounds of potentially costly fine-tuning. Was the cost of this fine-tuning taken into account? For example, how would the results change when the baselines are allowed more iterations so that their total cost (e.g. in inference FLOPs) matches the total cost of SOAR (inference and repeated fine-tuning FLOPs)?\n\nAdditionally, I suspect, but I am not certain, that unsupervised test data may have leaked from one test example to another. SOAR seems to have done fine-tuning iterations on the test data but highlights that the labels were excluded. Still, I can imagine that information about the training examples of a test task can be incorporated by the fine-tuning and used in the next test task. This could give the method an unfair advantage over the baselines. A way to mitigate this, while remaining flexible, could be to allow SOAR to do anything it wants with the training data of a given test task but, before going to the next test task, reset the LLM back to its state just after training. This way, no information, even unsupervised, can leak from one test task to the next. On the other hand, I may be ignorant of common practices used when benchmarking on ARC-AGI, so I would be very curious to hear the opinion of other reviewers and of the authors on this. Regardless, given the intermediate results shown on the training set, I would guess that this method remains the state of the art when the test-set fine-tuning is removed.", "relation_to_broader_scientific_literature": "The paper is clearly related to current scientific literature. The problem of including fine-tuning into the code discovery process is well motivated. The problem of code discovery is relevant in the modern machine learning literature. Please also see my \"Summary\".", "essential_references_not_discussed": "* Around line 327, the paper points out that \"fine-tuning generation capabilities on successful synthesis attempts [...] is an implementation of the STaR algorithm\". This should not be buried here. Instead, if the STaR algorithm was used, it should be cited at the first moment the method is described. That is, it should be mentioned prominently in section 3. \n* Impactful state-of-the-art methods that used LLMs in combination with evolutionary search for the purposes of code discovery, such as \"FunSearch\" (Romera-Paredes 2023) or \"Evolution of Heuristics\" (Liu 2024), are not mentioned.", "other_strengths_and_weaknesses": "Listed throughout the other sections of this review.", "other_comments_or_suggestions": "* Typo/grammar in \"to scaffold search\" (line 42, right column)\n* Typo in \"to challenges\" (line 48, right column).\n* Line 86 should say \"state of the art\" (noun), not \"state-of-the-art\" (adjective).\n* Line 86 should be less general. Instead of saying that the paper establishes a new SOTA on program synthesis, it would be more accurate to say that it establishes a new SOTA on the ARC-AGI program synthesis benchmark. More work is required to establish SOAR as SOTA on program synthesis as a whole.\n* Grammar/type in line 100, right column.\n* It is hard to tell whether the numbers in table 2 are significantly different from each other. Confidence intervals would help.\n* Line 430 (left column): I don't understand the sentence \"Rather than pushing against existing performance, SOAR finds paths to bypass them entirely\".", "questions_for_authors": "My recommendation could be changed to acceptance if the following are questions are addressed, especially the first one:\n\n* I believe that the cost of fine-tuning was not taken into account in the comparison against the baselines, which could potentially affect the conclusion. Please see my question about this under \"Experimental Designs Or Analyses\".\n\n* Why not benchmark against state of the art program synthesis methods like FunSearch or Evolution of Heuristics? In particular, why not compare on the same benchmark as those papers? Please see my comments under \"Methods And Evaluation Criteria\".\n\n* Is it possible that unsupervised test data was leaked from one test example to another? Please see my comment under \"Experimental Designs Or Analyses\"", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1pgnOZcVut", "reviewer": "Reviewer_cz1F", "summary": "This paper proposes SOAR, a framework for program synthesis that enhances language models through a self-improving evolutionary loop. Specifically, SOAR alternates between using an LLM for evolutionary search and applying hindsight learning to fine-tune its generation and refinement capabilities. This process enables continuous improvement of LLMs without human-engineered data, overcoming performance plateaus in conventional search-based methods. Extensive experimental results show that SOAR achieves state-of-the-art results among open-source LLMs on the ARC-AGI benchmark and provide insights into how AI systems can bootstrap their own improvement. This work opens new possibilities for advancing complex reasoning in program synthesis.\n\n## update after rebuttal\nI am satisfied with the authors’ rebuttal and maintain my positive rating of 4.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewers for their thoughtful comments. \n\nReviewer cz1F noted that SOAR achieved state-of-the-art results among open-source inductive approaches on the ARC benchmark by transcending the limitations of based models through iterative self-improvement. They also noted the quality of our experimental paradigm, how it supports our claims, while praising both writing and visual illustrations.\n\nThis said, the reviewer expressed three concerns that we address below:\n\n**Discussion of CodeRL:** We’d like to thank the reviewer for pointing out this work. SOAR and CodeRL both aim to improve program synthesis by improving pretrained language models with feedback from execution, but do so using different approaches:\n\n- **Learning signal:** SOAR uses hindsight learning to learn from both successes and failures (rich signal) while CodeRL uses the fraction of unit tests passed (weaker signal)\n\n- **Exploration:** SOAR uses a state-of-the-art method based on genetic algorithms to search the space of programs (REX), which enables a better exploration of program space than simply sampling from the current RL policy, as done in CodeRL.\n\n- **Learning algorithm:** CodeRL leverages a complex actor-critic architecture while SOAR relies on a straightforward supervised finetuning procedure. Finetuning is conducted on a small fraction of the generated data (50 programs out of 6000 generated per task), making the approach computationally cheaper (RL trains on all generated programs).\n\n- **Refinement:** CodeRL trains a separate policy and critic to perform refinements (code repairs) from code execution feedback, while SOAR uses a single model for generation and refinement. Our paper further demonstrates positive transfer between these two tasks (training on each makes the other better).\n\n- **Complexity:** SOAR implements a simple algorithm: search, relabel, finetune, while CodeRL relies on several specialized modules including: pretraining of the policy with collected code data, warm up of the policy with ground truth programs, freezing of the policy to pre-train the critic on ground truth data, reward shaping, and others.\n\nOverall, SOAR proposes a cleaner, simpler approach that’s easier to scale based on several key ideas: iterative improvement, finetuning of generation and refinement (positive transfer), and hindsight relabelling. The revised version of the paper now discusses this relevant work and its relation to ours.\n\n**Extensive computational resources (6k programs per task):** SOAR is only trained on 50 programs per task at each generation and does not fundamentally require 6k programs per task during the search phase. If we’re sampling so many attempts, it’s because ARC is difficult, and base models are unlikely to generate interesting programs in only a few trials. This number is comparable (and even lower) to the ones used in related approaches (e.g. 20k in Li et al., 2024; 8k in Greenblatt, 2024). An interesting future work could be to look at the optimal budget allocation between running longer searches or running more refinement iterations. The optimal search budget might vary across iterations too, as finetuning might itself accelerate search by increasing the ability of the generation and refinement model to encounter successful programs earlier on (see answer to Reviewer ifzp). We note that SOAR is only useful to improve upon program synthesis domains that could not be solved by search alone, which is why it requires substantial computational resources. We added a short discussion about the significant search budget and the necessity to adapt this parameter to the domain at hand in the revised version of the manuscript. \n\n**Generalization to other domains:** We picked the ARC benchmark because it was designed for the purpose of evaluating general program synthesis algorithms: it is hard, diverse, relatively out of distribution for LLMs, and explicitly designed to test core reasoning abilities beyond pattern matching. \n\nGiven our limited computing budget, we decided to focus our resources on a deep study of the SOAR approach on ARC, as opposed to a shallower study on several benchmarks. This allowed us to make the space of design decisions explicit and study which worked best using carefully controlled experiments as several reviewers have noted. \n\nThis said, SOAR is a general framework that doesn’t rely on any ARC-specific assumption and can directly be transposed to any programming-by-example domain including code synthesis like APPS. Our paper describes the methodology clearly and how to navigate the space of design decisions with careful experimentation. We thus expect SOAR to generalize across domains, but, as we acknowledge in the paper, leave the empirical verification of this claim to future work. \n\nPlease let us know if the above discussion answers your concerns and consider raising your score if it does. If it doesn’t, let us know which concerns remain so we can try to address them. Thank you!", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "N/a.", "experimental_designs_or_analyses": "Yes, the experiment is well designed and offers insightful analysis.", "relation_to_broader_scientific_literature": "Yes. This work opens new possibilities for advancing complex reasoning and problem-solving in program synthesis.", "essential_references_not_discussed": "Yes. The generation and refinement loop for LLM-based program synthesis has been paritially explored in CodeRL's critic sampling. Specifically, the table 4 of its paper (https://arxiv.org/pdf/2207.01780) demonstrates the program synthesis performance on APPS with different rounds of program repair. This work should be discussed.", "other_strengths_and_weaknesses": "## Pros\n\n- The proposed framework achieves state-of-the-art results for program synthesis among open-source LLMs on the challenging ARC benchmark.  It offers valuable insights into how program synthesis systems can transcend the limitations of their base models through self-improvement, a fundamental challenge in LLMs, opening new possibilities for advancing reasoning and problem-solving in program synthesis.\n- All claims in this paper are well-supported by comprehensive experimental results. The presentation is clear, with strong writing and effective visual illustrations. Readers will find the paper both engaging and informative.\n\n## Cons\n As noted in the Discussion section, this work has two key limitations: \n- The impressive results rely on substantial computational resources (e.g., 6k synthesis attempts per task per self-improvement cycle), making real-world applicability challenging. \n- This paper only demonstrates its effectiveness on the ARC benchmark. Its generalization to other domains such as programming contest or mathematical reasoning is unclear.", "other_comments_or_suggestions": "N/a.", "questions_for_authors": "N/a.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes SOAR, a new method for improving program synthesis on the ARC-AGI dataset. It consists of two parts: 1) evolution-based search for generation and refinement, and 2) using the traces in the search phase to train the models for both generation and refinement to achieve better performance. The reviewers agree that the proposed method is novel, and contains a couple of interesting ideas such as hindsight relabeling and the experiments are rather comprehensive with some good insights (e.g., smaller models can contribute to the pool even when larger models are present). There were concerns about the cost control and experiments being exclusive to Qwen but are properly addressed during the author response phase. The remaining concern of this paper is that it only uses ARC-AGI in this study thus it's unclear whether the conclusions can generalize to other domains, which limits it's broader impact. However, the authors also did a good job not to over-claim either, as it's labeled "A Case Study on ARC-AGI" in the title. Overall, I would recommend an acceptance to the conference.
Decision: Accept (poster)