Programming by Backprop: An Instruction is Worth 100 Examples When Finetuning LLMs
Authors: Jonathan Cook
Silvia Sapora
Arash Ahmadian
Akbir Khan
Tim Rocktäschel
Jakob Nicolaus Foerster
Laura Ruis
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models
Abstraction
Procedural Knowledge
Primary area: foundation or frontier models, including LLMs
TL;DR: LLMs can learn to execute procedures that are described symbolically in their training data, but only with specific finetuning curricula.

Abstract: Large language models (LLMs) are typically trained to acquire behaviours from demonstrations or experience, yet much of their training data is declarative: instructions, rules, and descriptions that specify behaviours without showing how to execute them. We introduce **Programming by Backprop (PBB)**: a training regime that enables LLMs to acquire *procedural* knowledge (i.e., reusable behaviours) from *declarative* instructions encountered during training. With PBB, instructions in training data provide an opportunity to "program" specific behaviours into model weights. The core principle underpinning PBB is the separation of learning how instructions map to behaviour from internalising new instructions. We devise two distinct PBB curricula that leverage this principle. Through controlled experiments across two domains (algorithmic execution from Python source code and text generation from context-free grammars), we demonstrate the benefit of these curricula over training on a homogeneous data mixture. Crucially, PBB is highly sample efficient, with *a single instruction substituting for up to 100 execution examples*. Though execution of instructions in training data remains less reliable than when instructions are given in-context, our results demonstrate that procedural knowledge can be noisily `programmed' into LLMs through PBB, with important implications for data curation and safety.

Published as a conference paper at ICLR 2026
PROGRAMMING BY BACKPROP: AN INSTRUCTION IS WORTH 100 EXAMPLES WHEN FINETUNING LLMS
Jonathan Cook
Silvia Sapora'
FLAIR, University of Oxford
FLAIR, University of Oxford s Aivia. sapora@stats ox ac uk
jonathan. 2ookk@herrs ford. ac.uk
Arash Ahmadian Cohere & Cohere Labs
Akbir Khan Anthropic
Tim Rocktäschel UCL AI Centre
Jakob Foerster
FLAIR, University of Oxford
Laura Ruis MIT Iruis@mit.edd
ABSTRACT
Large language models (LLMs) are typically trained to acquire behaviours from demonstrations or experience, yet much of their training data is declarative: in- structions, rules, and descriptions that specify behaviours without showing how to execute them. We introduce Programming by Backprop (PBB): a training regime that enables LLMs to acquire procedural knowledge (i.e., reusable be- haviours) from declarative instructions encountered during training. With PBB, instructions in training data provide haviours into model weights. The core principle underpinning PBB is the separa- tion of learning how instructions map to behaviour from internalising new instruc- tions. We devise two distinct PBB curricula that leverage this principle. Through controlled experiments across two domains (algorithmic execution from Python source code and text generation from context-free grammars), we demonstrate the benefit of these curricula over training on a homogeneous data mixture. Crucially, PBB is highly sample efficient, with single instruction substituting for up to 100 execution examples. Though execution instructions training data remains less reliable than when instructions that procedural knowledge can be noisily 'programmed" into LLMs through PBB, with important implications for data curation and safety.
opportunity 'program" specific be-
given in-context, results demonstrate
INTRODUCTION
Large language models (LLMs) are typically trained to acquire behaviours from demonstrations, via pretraining and supervised finetuming (SFT), from experience, via reinforcement learning (RL). Yet much of the data that models are exposed to, particularly during pretraining, does not consist of demonstrations, but of abstract instructions, rules, algorithms, and descriptions that specify proce- dures without showing how they execute on concrete inputs. For humans, such symbolic knowledge plays a central role learning general skills, allowing abstract instruction complement demon- stration and practice, substantially improving learning efficiency (Dienes & Perner 1999). Whether LLMs can similarly acquire procedural knowledge from declarative training data remains an open question.
Users of instruction-tunn LLLMs commonly give instructions in-context through prompting, mean- ing that the model's generation behaviour is explicitly conditioned on that instruction. By contrast, internalising behaviour from instructions in training data would amortise the per-instance computa- tion associated with conditioning that description into the model's 's parameters, enabling model
Equual contribution.
Now at Google DeepMind.

---

Published as a conference paper at ICLR 2026
Programming by Backprop
Proactive-PBB
Retroactive-PBB
Instructions & Examples
Examples
Instractions
Examples
Esamples
TEST
TRAIN- STAGE
THAIN -TAGEE
Figure 1: Illustration of Programming by Backprop (PBB) the learning of behaviours from in- structions. We define train instructions as those for which examples of behaviour are also available. Evaluation instructions (yellow in the figure) correspond the behaviours being tested, which are never demonstrated in training data. PBB comprises two training curricula: Proactive PBB, where models learn general correspondence between instructions and behaviours before being exposed to evaluation instructions, and Retroactive PBB, where initial exposure to all instructions is followed by learning how to map those behaviour using the train set.
to reuse the behaviour in a range of contexts and compose it with other learned behaviours. Un- derstanding how training on instructions influences the downstream behaviour of an LLM is crucial both for predicting generalisation (which is central to safety) and for developing models with capa- bilities beyond those demonstrated extensively in training data.
We empirically demonstrate that the local ob- jective of maximising the likelihood of an in- struction does not, in isolation, incentivise learn- ing the corresponding behaviour; rather, this acquisition relies (Lampinen et al. 2025b) driven by the structure of the full training data distribution. Prior work provides suggestive evidence of latent learning, showing that LLMs can emulate simple char- acter descriptions encountered during finetuning (Berglund et al., 2023), extract general strate- gies from training examples (Betley et al. (2025), and exhibit downstream skills correlated with the presence of input-general definitions in pre- training data (Ruis et al.) 2025). However, ex- isting work does not methodologically isolate whether an instruction encountered during train- ing is sufficient to induce execution of an sociated multi-step, input-conditions behaviour. Nor does prior work identify approaches that improve an LLM's ability internalise behaviour from instructions.
Sample Efficiency
thmesi
0.5
exsccutions (A0)
Singpe mate uiboo (PSB)
form of latent learming
0.0
Number essecation Tunction
Figure 2: PBB enables Owen -14B learn an algorithm from single instruction (piece of code) to similar efficacy as using Algorithm Distillation (AD) learning an algorithm from input-specific examples - on 100 examples.
We introduce Programming by Backprop (PBB) a training approach for inducing context- dependent, executable behaviours from abstract instructions. In PBB, behaviours on which the model will be evaluated are specified declaratively in training data, rather than demonstrated, and training is designed to compile these specifications into the model's 's parameters. In this sense, 'pro- gramming' refers to the explicit inclusion of procedural instructions in training data, together with training strategies that induce the corresponding behaviour at inference time.
PBB is instantiated through structured, multi-stage curricula that combine instruction modelling with execution supervision over a designated train set of behaviours. We design two such curricula. In the first, models are trained to learn a general correspondence between instructions and behaviours using train behaviours, and then apply this correspondence to new instructions corresponding to test behaviours introduced in a subsequent stage. This regime form of implicit meta- meaa-nearning as introduced by KKasheninnikov al. (2024), and closely mirrors their two-stage training pipeline. In
2

---

Published as a conference paper at ICLR 2026
the second, models are first trained the full set of instructions, acquiring declarative knowledge without execution competence; execution supervision on train behaviours then used to convert this latent declarative knowledge into procedural knowledge in way that generalises to test be- haviours. We refer to these curricula as proactive and retroactive PBB, respectively (see Figure Toor an overview).
We evaluate PBB across two domains - algorithmic execution from Python source code and text generation from context-free grammars and find that it enables highly sample-efficient learning. Under PBB, single instruction can substitute for many demonstrations while achieving comparable performance (Figure 2). PBB can also mitigate biases arising from imbalanced demonstration data (Section 5.2). In addition, models trained with PBB exhibit a limited ability to compose indepen- dently trained algorithmic instructions at inference time, and we find that algorithmic execution are learned more effectively when instructions are expressed in code rather than natural language (Sec- tion D. Together, these results characterise how PBB can be used to program reusable behaviours into language models via training on instructions.
Finally, while finetuning with PBB induces executable behaviour, execution remains less reliable than when instructions are provided explicitly in-context. Nonetheless, our experiments show that models trained with PBB can noisily execute instructions encountered during training within the forward pass, without relying trends with respect to both model size and data, and note that the PBB curricula complement prior work highlighting the role of curriculum design in continual pretraining (Parmar et al.. 2024 Yildiz eet al. [2025; Chen et al. 2025, Ou et al., 2025). These findings suggest that PBB is promising training methodology for further scaling.
chain-of-the tftthuoffthought reasoning. We further observe positive scaling
RELATED WORK
Learning to execute. A large body of work studies how neural models can learn to execute pro- grams when given explicit execution supervision. Early work shows that recurrent networks can map program text outputs given paired ontut- examples (Zaremba & Sutskever (015).
Subsequent approaches extend this direction to richer settings, such as inferring symbolic "shape programs" from perceptual data and executing them for downstream tasks (Tian et al.) 20199) or training Transformer-based execution engines on demonstrations of algorithmic subroutines (Yan ct al. 2020). Other work introduces architectural inductive biases, such as neural interpreters, to achieve modular and systematic execution (Rahaman et al. 202T). While conceptually related in their focus on executable behaviour, these approaches assume that procedures are either inferred from data or trained with execution-level supervision for each procedure. In contrast, PBB involves learning executable behaviours from symbolic descriptions.
Code training and systematic reasoning. Several recent works investigate how exposure to code and other symbolic structures during pretraining affects downstream model behaviour. Training on source code has been shown to improve reasoning and problem-solving in natural language tasks (Aryabumi et al. 2024 Petty et al. 2024), suggesting that the regularity and compositional structure of code can scaffold general reasoning abilities. Related work shows that training on synthetic pro- cedural traces, such as edit sequences, improves code synthesis by encouraging models to represent intermediate transformations (Piterbarg et al., (025). Other approaches explicitly couple language models with external interpreters, grounding reasoning in code execution improve reliability (Li jet al., 2024). This literature demonstrates that symbolic supervision can support systematic rea- soning. PEB differs in that symbolic procedures are not used external tools or reasoning aids at inference time, but instead serve as training-time programs whose execution semantics are inter- nalised into the model parameters. Recent work has begun investigate the kinds of pretraining data that influence reasoning capabilities. Ruis et al. (2025) show that exposure to input-general procedures, such as code functions, in pretraining data strongly influences models' ability solve related input-specific reasoning problems. This suggests that declarative data can shape downstream execution behaviour, motivating controlled studies of this process.
Limits of procedural generalisation. Prior work on arithmetic reasoning shows that LLMs often rely on surface heuristics rather than executing explicit algorithms (Nikankin et al. 2025). though structured prompting can elicit more systematic behaviour (Chen al. 2023] 2024). Mechanistic
3

---

Published as a conference paper at ICLR 2026
analyses further suggest that the autoregressive training objective shapes which procedural abstrac- tions can be internalised, constraining both successes and failure modes (McCoy et al.) 2024 Wang et al., 2024). Relatedly, Allen-Zhu & Li 2025) show that LLMs exhibit gap between know- ing" and 'doing' in tasks requiring classification or comparison, offering a potential explanation for observed failures of applying knowledge about task solving the task itself (Paglieri et al.) 2024). Lampinen et al. (2025a) further demonstrate that generalisation differs substantially between knowledge provided in-context and knowledge acquired during training. Our results contribute to this discussion by showing that data curricula can enable models to convert declarative knowledge into procedural knowledge that generalises across contexts at inference time.
Out-of-context reasoning. Finally, PBB connects growing body of work on how LLMs gen- cralise from their training data sophisticated ways. Prior studies show that models can perform out-of-context reasoning (OOCR) (Berglund et al. 2023 Betley et al. 2025), where inference time behaviour is informed by training data that indirectly or abstractly describes relevant context. Mod- els have also been shown to exhibit implicit meta-learning (Krasheninnikov al. 2024), perform latent multi-hop reasoning Yangg et al.) 2024). and infer latent structure about their training data Treutlein et al., 2024). PBB be framed specific form of out-of-context reasoning in which models learn representations of symbolic abstractions that permit execution at inference time.
3 PROGRAMMING BY BACKPROP
We formalise Programming by Backprop (PBB) as a learning regime in which the hehaviour defined by an instruction is acquired by training on that instruction and encoded into language model's parameters for execution.
3.1 A PROGRAMMING PERSPECTIVE ON TRAINING DATA
Let S be space of symbolic instructions, such as Python programs or formal grammar rules. We associate each instruction S with its denotation [s] which captures the behaviour that the instruction defines.
In this work, consider two domains:
1. Algorithmic Execution:
For a program S, the denotation [x](2) E y is a function mapping an input æ E to an output.
2. Formal Grammars: For grammar S, the denotation [>](x) (s) C Σ is language generated by the grammar under vocabulary choice x' E {1,.... 5]. where Σ, is the terminal alphabet induced by that vocabulary.
Let Mo language model with parameters A. We distinguish:
context Ca which acts a pointer to instruction s (e.g., the problem statement, the task name, or a grammar identifier), and
an input x, which the instance argument to that instruction.
In the algorithmic execution domain, is the concrete program input. In the grammar domain, is the vocabulary choice (one of five options).
An idealised model behaves like universal interpreter, able to execute any instruction given in- context: Ma(8 x) R [x). PBB asks whether training can instead 'compile' such an instruction into the model's parameters that only a context c, acting as pointer the instruction needed at inference time.
We now define the training losses used throughout:
Declarative loss (instruction modelling):
(f c., s) == log po (s(ca),
the next-token negative log-likelihood of the instruction text given task context cy.

---

Published as a conference paper at ICLR 2026
Execution loss (behaviour modelling):
Lexec (0; Cs. v) = Po po (v)c..
the next-token loss on solution y given task context and the input z. When demon- strations are provided as execution traces, optimising for the the execution loss alone cor- responds to algorithm distillation (Laskin et al.) 2023) Gandhi et al. 2024). In RI. variants for settings where the model can output intermediate chain-of-thought reasoning tokens, Lexss is replaced by the corresponding RL objective.
Partial evaluation (Jones et al. 1993) is the classical process of specialising general interpreter to a fixed program, yielding residual program that no longer needs the original instructions as input. Analogously, we view PBB as applying a learned specialisation mapping 4 through gradient-bart based training: = (0; C), where the training objective used induce specialisation.
3.2 PBB TRAINING CURRICULA
Our experiments reveal curriculum regimes that elicit PBB: proactive and retroactive.
Let the instruction set partition as S = Suain U Seall. corresponding to instructions for train and test behaviours respectively. For each train instruction 8; Suain we assume of execution examples D. = [( Tijs Bij) and write Dirain = {(s, D,)
PROACTIVE PBB
In proactive PBB, Stage trains a general correspondence between instruction representations and behaviour using the train set, and Stage 2 then internalises evaluation instructions using only declar- ative supervision.
Stage (train set; mixed objective). Stage optimises a mixture of the declarative and execution losses the train instructions:
min Be-eSmi (0; + Ep- ((e,,,)) Cexes (Cars) x]]]
Concretely, this can be implemented by interleaving the two formats of training examples.
Stage (evaluation instructions). Stage exposes the model to evaluation instructions 8 Seval without executions, training only Ldect. In this regime, the specialisation operator is the gradient update induced by instruction modelling:
(0, Ldcol) == (o
Stage is what makes this update behave like 'compilation', because has been trained so that gra- dients through are predictive of the execution-relevant updates, training alone can internalise [*] into the weights.
RETROACTIVE PBB
In retroactive PBB, Stage learns all instructions declaratively (without execution competence). and Stage 2 introduces execution supervision for only Smain-
Stage (all instructions).
Train on all instructions:
min Egw.s E (f) s)].
Stage (train executions).
The retroactive specialisation operator is:
Pru (0, fexec) = 0-1
Σ
Voherec (0; Cs.
(c.
5

---

Published as a conference paper at ICLR 2026
Intuitively, although Stage 2 gradients computed only from executions of paired instructions, they update shared parameters This global update can "activate' the latent declarative knowledge acquired in Stage I, facilitating execution at test time even for evaluation instructions that never received direct execution supervision.
EXPERIMENTAL SETUP
To investigate Programming by Backprop (PBB), we conduct experiments across two distinct do- mains: algorithmic execution of Python code and sentence generation under formal grammars.
4.1 DATASETS AND TASKS
We create three synthetic datasets to provide a controlled environment for studying PBB. We ad- ditionally experiment with real-world coding dataset and corresponding execution task. For each dataset, we define a set of instructions S, which partitioned into a train and evaluation set.
Algorithmic Execution. These tasks test whether models can learn to execute functions from their source code.
This dataset contains 1,000 unique Python functions that map in- tegers to integers. The functions are synthetically generated by composing basic control flow (for loops, if / else conditionals) and arithmetic operators (+, s, exp, abs). This allows us to control for procedural complexity (i.e., the number of operations). For our main experiments, we use 100 functions for Suain and 100 for Seval- With this dataset, we also define compositions of two functions evaluate models' ability compose behaviours defined by instructions that were trained on independently. We give these compositional functions unique names and define the instructions as, for example, def Blorp (x) 2 Zibble (Snurg (x) Execution supervision is also included for composite train instructions.
Random Arithmetic:
//, 8, >,
Leetcode:
This dataset consists of 702 real-world algorithmic problems and their Python solutions, sourced from the competitive programming platform. This tests PBB on more complex and naturalistic programs. We use 500 problems for Suain and 100 for Seval-
Ciphers: To test generalisation to novel, OOD programs, we create three custom ciphers (Alice, Bob, Kevin) that are variations of standard ciphers (Caesar, Atbash, Vigenère). We assume these novel ciphers are absent from the model's pretraining data, allowing for con- trolled experimentation. The same 500 Leetcode problems are used for Smain and the three custom ciphers form Seval- For this task, also generate execution demonstrations from an imbalanced distribution (Appendix E), reflecting the greater occurrence of examples with specific shift values (e.g., ROT13) web data (McCoy et al. 2024). This allows us to compare learning ciphers in an input-general fashion via PBB to learning from skewed, input-specific demonstration data.
For all tasks, input-output examples are framed as word problems. We generate ground-truth so- lutions by executing the corresponding Python code. To the benefits of intermediate reason- ing, we also generate chain-of ofttuutght (CoT) solutions for each problem using GPT-44 in a post- rationalisation step (Zelikman al., 2022). We also use GP 4o to generate semantically equiv- alent natural language instructions for the Random Arithmetic Python functions, allowing us to evaluation the impact of algorithm representation on the performance of PBB.
Generation Under Formal Grammars. To test PBB beyond code, we construct procedurally generated suite of artificial grammars that define syntactic constraints on sentence formation. Con- cretely, we generate a set of 200 unique context-free grammars (CFGs). Each grammar is pro- duced by sampling from a compact, interpretable parameter space that controls typological proper- ties (word order families such as SVO, SOV, VSO, etc.), modifier placement (adjectives pre/post- nominal, determiners perppost-nominall adverbs pre/post- verbal), and optional structural features. Each grammar is represented symbolically small set of production rules (nonterminals and pro- ductions). We use five vocabularies (Vocab-ae Vocab-E), each containing ten distinct nouns, verbs, and adjectives shared across all grammars for sampling strings.

---

Published as a conference paper at ICLR 2026
We sample derivation trees from each CFG by repeatedly expanding nonterminals according to that grammar until a depth cutoff. A sampled derivation tree is used to produce a sentence by instantiating terminals from the specified vocabulary. From the 200 grammars, as Strin (grammars paired with example strings) and 100 as Seval (grammars for which symbolic specification is shown during training, but not example strings). We evaluate model generations with a strict, grammar-based validity test: candidate sentence is accepted if it can be parsed by the requested CFG (we use a chart/Earley parser). Accuracy on a grammar is thus the fraction of model outputs that produce at least one valid parse under that grammar. For this task, CoT is not used during training or evaluation.
designate 100
4.2 DATA AUGMENTATION
Empirically, we find that it is important to augment c, when training instructions for PBB to work effectively (I).). This is in line with prior work on out-of-context reasoning (Berglund et al. 2023 We therefore define 20 context templates for each dataset that act pointers instructions (e.g., for Leetcode: 'Write Python function that solves the following problem {*}". 'I need Python im- plementation that does ..)"...). A single context template is sampled uniformly random for each instruction. 30 unique execution examples are provided per train instruction during execution super- vision. For proactive PBB, we additionally mix in Ik samples from OpenMattInstruet (Toshniwal et al. 2024) during stage 2 finetuning on instructions to prevent forgetting of relevant instruction following abilities.
4.3 TRAINING DETAILS
We use instruction-tunee Qwen3 models (4B, 8B, 14B) (Qwen et al. 2025 our primary set of base models and conduct additional experiments with GPT-4o 4o OpenA et al. 2024) via the OpenAI finetuning API to investigate PBB in large frontier model. We also repeat instruction-tume tunied Llama-3 models (1B, 3B, 8B) (Dubey et al. 2024) to see trends are consistent across model families (Appendix (A). All training stages use five epochs, meaning a single instruction or execution example is encountered five times in total. For RL runs, we use GRPO (Shao et al.) 2024) with group size of 8 and KL regularisation (3 = 0). A positive reward (+1) is given if the final answer is properly formatted and correct; a neutral reward (0) is given if the answer is properly formatted but incorrect; a negative reward (-1) given if the model fails produce final answer in the required format. For all training runs, the batch size is set to 32 and we use a constant learning rate of x 10-5, We use a sampling temperature training. All evaluations are averaged over 16 samples and 95% confidence intervals are reported.
experiments with
0.3 for evaluation and 0.8 RI.
5 RESULTS
5.1 PBB FOR ALGORITHMIC EXECUTION
We first evaluate whether models can learn to execute algorithms from instructions in training data. Our experiments on the Random Arithmetic and Leetcode domains yield several key findings re- garding the conditions required for PBB and the nature of the resulting behaviour.
Role of curricula. We find that simply including instructions in the training data is insufficient for models to acquire executable procedural knowledge. As shown in the stage ablation (Fig. 3 lower left), training on stage train instructions and executions, or stage 2 evaluation instructions isola- tion results in negligible-to-low performance on test behaviour. The zero-shot performance is zero for all models because we are working with synthetically generated Python programs that models would have no prior knowledge of. Under the full 'proactive' curriculum, where the model first learns the correspondence between instructions and behaviour on the train set (stage 1) before ternalising evaluation instructions (stage 2), 3 4B reaches 37% average execution accuracy for test behaviours.
Reliability and implicit execution. While PBB induces executable behaviour, it remains less liable than explicitly conditioning the model on the instruction in-context (Fig. 3 upper left). While learning from instructions in training data is more challenging than applying those instructions when
7

---

Published as a conference paper at ICLR 2026
Conditio
Compostiin
Hocel
Scce Oeco
Oves7
0ve1310)
0.0
0.3
0.2
Swee1-6
01- Viade
Phencce
Ove 68
< OUT
+ CT
OUT
000
Progaan Leugh
Frggrnn lorght
Camporna hogemnongs
Projctive P38 Abtitions
Algortiin Representation
0:42
Aage
Tua 1yps
0.25
0.33
dags Onb
Cods In legel
0.4--
0.33
0.0
Onard 00
9100 -1500
Figure 3: Upper Left: Accuracy when conditionally executing random arithmetic programs of dif- ferent lengths provided in-context. Upper Middle: Accuracy following proactive PBB on executing unpaired programs. Explicitly computing intermediate operations via CoT more effective than di- rect inference, but larger models show some ability to encode entire procedures in their weights via PBB. Upper Right: Accuracy following proactive PBB for compositions two programs that have been trained independently. Direct inference performance is only above random for GPT-4o: the smaller models fail this setting. Lower Left: Accuracy following each stage of proactive PBB. The full curriculum has significant impact performance. Lower Middle: Accuracy following proactive PBB when programs are represented as natural language or code. The role of prompt aug- mentations is also ablated. Lower Right: Accuracy following retroactive PBB throughout stage training. RL is more effective, with SFT initially memorising paired executions before generalising execution ability to unpaired programs.
in-context, proactive PPB still enables some degree of direct inference: models can produce the correct output without generating intermediate chain-of-thought (CoT) reasoning steps, albeit with lower accuracy than when CoT is used (Fig. 3 upper middle). This suggests the model is capable of performing the necessary multi-step algorithmic operations implicitly within the forward pass, even without the instruction being in-context. Furthermore, independently learned instructions: Fig. (upper right) shows that models can execute composites of two programs they were trained separately, with larger models like GPT-4o showing some capability even without explicit reasoning traces.
observe that models can compose
Retroactive PBB: RL vs. SFT. In the retroactive regime, where declarative knowledge is 'acti- vated' by later train set execution supervision, we find that reinforcement learning (RL) is signifi- cantly more effective than supervised finetuning (SFT) for stage 2. As illustrated in Fig. 3 (lower right), SFT tends to memorise the train execution demonstrations initially, taking more samples to generalise execution capabilities to test behaviours, whereas RI. drives faster and more robust generalisation.
Impact of algorithm representation. The format of the instruction significantly influences PBB performance. We compare instructions provided as Python source code against semantically equiv- alent natural language descriptions. Results indicate that algorithms are learned more effectively when expressed code (Fig- 3. lower middle), although the performance gap lower for larger models. This gap may stem from the potential ambiguity and verbosity of natural language, or ar- guably because current LLMs possess a stronger inductive bias for reasoning from code due their pretraining distributions. Augmentations applied to the prompts for the instructions in training data are found to be essential to the outcome of PBB.
PBB on pretrained domains (Leetcode). We test PBB on the Lectcode dataset to assess its utility for tasks where the model has prior exposure to similar programs during pretraining. Even here, the
8

---

Published as a conference paper at ICLR 2026
Preactive PB3 on Lectcode
Retroactive PBB on Leetcode
0.6
Madel
Only Stage Only
Swen3-aB
Stages +
Dwcn19-448 146
0.0
Owend-143
Trai Stops
Figure 4: Left: Accuracy following each stage of proactive PBB Leetcode programs. Despite zero-shot performance being high in this setting, the full curriculum yields the greatest performance, showing that execution performance can be refined by further training on instructions. Right: Ac- curacy following retroactive PBB (RL) throughout Stage 2 training.
full proactive curriculum yields the highest performance on test behaviours (Fig. 4). In contrast, retroactive PBB performs comparably simply training the train executions, suggesting that the retroactive activation of latent knowledge less beneficial when the domain already familiar.
5.2 SAMPLE EFFICIENT LEARNING VIA PBB
motivation for PBB is the potential for data efficiency replacing many execution examples with single high-level instruction.
Correct ng Bles Shift Ciphers
High sample efficiency. We compare PBB against algorithm distillation (AD) (Laskin al. 2023 Gandhi al.) 2024), where the model learns to implement an algorithm from input- specific examples. On the Random Arithmetic task, we find that PBB is highly sample effi- cient. As shown in Fig. [ training on sin- gle instruction in stage 2 of the proactive cur- riculum yields execution fidelity comparable to, better than, training up to 100 execution traces (for CoT inference) or input-output pairs (for direct inference). To ensure fair compari- son, algorithm distillation also follows stage of proactive PBB. This confirms that concise instruction can substitute for substantial vol- ume of demonstration data. However, we note that proactive PBB requires the amortised cost of stage 1, which primes the model for learning from instructions. Proactive PBB can therefore be seen as an approach to sample efficient gen- eralisation given an initial meta-train phase.
Shaved Intrumtionn (1601 Indrucionn (P8R) Skenee (1)
D0
Figure 5: Finetuning GPT-4o custom Ci- phers via proactive PBB (Stage 1 Leetcode, Stage Ciphers) yields greater robustness shift variations than training on imbalanced demon- strations. Mixing Stage 2 training on cipher code with imbalanced demonstrations yields the best results, overcoming biases from demonstrations while grounding execution behaviour.
OOD generalisation and bias correction. PBB also facilitates robust generalisation. We test this on the Ciphers task using previously unseen custom shift ciphers. Training on demonstration data (i.e., AD) that skewed (e.g., an imbalance favouring specific shift values, such as ROT13, common in web data (McCoy al. (024h) leads to biased performance that degrades when the shift parameter varies. In contrast, proactive PBB (stage I on Leetcode, stage 2 on cipher code) learns the input-general function, resulting in performance that is robust parameter variations (Fig. 5 right). In both cases, low and high shift values also have greater performance, because these lead to letters that alphabetically close the original and thus PBB with skewed demonstrations yields the best performance: the instruction mitigates the bias from the skewed data, while the demonstrations help ground the execution behaviour.
less prone to errors. Notably, combining

---

Published as a conference paper at ICLR 2026
Proactive P30 on Formal Grammar Generation
Retroactive P8B on Formal Grommar Generation
Siage/ Setelre
Mintel
Stage
Qwer3-93
Stapes -2
Oner3-118
Bardomm
0.3
0.0
Qwen2-4D ONer0-11BB
Train Steds
Figure 6: Left: Compliance to unpaired grammars following each stage of proactive PBB. Stage or stage 2 alone each does comparably to randomly generating word sequences from the vocabu- lary. The full proactive PBB pipeline enables compliance with grammars described symbolically in training data. Right: Compliance to unpaired grammars following retroactive PBB (SFT).
5.3 PBB FOR GRAMMAR COMPLIANCE
To demonstrate PBB beyond code execution, we evaluate its application to text generation under formal constraints.
Implicit grammar computation. We find that PBB can successfully 'program' models to gener- text compliant with novel context-free grammars defined in the training data. As shown in Fig. 6 the full proactive curriculum enables models to generate valid sentences for test grammars with accuracy far exceeding random generation from the defined vocabulary or single-stage baselines. Crucially, in this setting, the model produces compliant text immediately upon prompting, without CoT reasoning. This indicates that the model has encoded the grammar rules into its weights and is performing the necessary syntactic computations, tracking state and ensuring rule compliance, implicitly within its forward pass. Retroactive PBB also enables models to generate text that compliant with test grammars, though at a lower fidelity than with the proactive curriculum.
CONCLUSION
This work shows that language models can internalize executable procedural knowledge from declarative instructions in training data via Programming by Backprop (PBB). We find that PBB emerges under structured, multi-stage curricula, is highly sample efficient, and generalises across domains. While execution remains less reliable than explicit prompting, the results demonstrate that instructions can be implicitly "compiled" into model parameters. This has fundamental implications for how researchers and practitioners can teach models new behaviours and represents key finding for safety: instructions present in training data could elicit unintended behaviours at inference time, underscoring the importance careful data curation and filtering.
7 REPRODUCIBILITY STATEMENT
We are open-sourcing all code and datasets needed to reproduce our experiments at https: 7/gthhab. com/ wttpttaaaocooo2/oaooaaaaa
by-Backprog This includes data
generation scripts and training code.
REFERENCES
Zeyuan Allen-Zhi -hu and Yuanzhi Li. Physics language models: Part 3.2, knowledge manipulation. In The Thirteenth International Conference Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025, URL https: /ttt::://apenzeview net forum?id oDbi T.9CL.5)
10

---

Published as a conference paper at ICLR 2026
Viraat Aryabumi. Yixuan Su, Raymond Ma, Adrien Morisot, Ivan Zhang, Acyr Locatelli, Marzich Fadace, Ahmet Üstün, and Sara Hooker. To code, or not to code? exploring impact of code in pre-training, 2024. httpstss: http/axxiiv....... abs 10944.1111144
Lukas Berglund, Asa Cooper Stickland, Mikita Balesni, Maximilian Kaufmann, Meg Tong, Tomasz Korbak, Daniel Kokotajlo, and Owain Evans. Taken out of context: On measuring situational awareness in 1lms. CoRR, abs/2309.00665 2023. URL. https: org. 485507
arXiv. 2309. 00667
Jan Betley, Xuchan Bao. Martin Soto, Anna Eztyber-Betley, James Chua, and Owain Evans. Tell me about yourself: LLMs are aware of their learned behaviors. In The Thirteenth International Conference on Learning Representations, 2025. URL
net forum?
id=1 j02J.emmy
Jiaao Chen, Xiaoman Pan, Dian Yu, Kaiqiang Song, Xiaoyang Wang, Dong Yu, and Jianshu Chen. Skills-in-context Unlocking compositionality in large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Lin- guistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pp. 13838 13890. Asso- ciation for Computational Linguistics, 2024. URL/ttts:: a htt////caaaanttoggy.y. htt.aooooooooyro//2224444444 org/2024.
Findings emnlp. 812
Jie Chen, Zhipeng Chen, Jiapeng Wang, Kun Zhou, Yutao Zhu, Jinhao Jiang, Yingqian Min, Xin Zhao, Zhicheng Dou, Jiaxin Mao, Yankai Lin, Ruihua Song, Jun Xu, Xu Chen, Rui Yan, Zhewei Wei, Di Hu, Wenbing Huang, and Ji-Rong Wen. Towards effective and efficient continual pre- training of large language models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguisticy (Volume 1: Long Papers), ACI 2025, Vienna, Austria, July 27 August 2025, pp. 5795. Association for Computational Linguisties, 2025. URL https: //aclanthology org. 20w../2222... acl acl.aclg.1...899
Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompt- ing: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL httpt::// httt:s://epeerevieee .net/
Zoltan Dienes and Josef Perner. A theory of implicit and explicit knowledge. Behavioral and Brain Sciences, 22(5):735-803 1999.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshom, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits. David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia- Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle 1 Lee, Georgia Lewis Ander- son, Graeme Nail, Grégoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Ko- revaar, Hu Xu. Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lec, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadeokar, Jeet Shah, Jelmer der Linde, Jennifer Billock, Jenny Hong. Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Al- wala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. The llama 3 herd of models. CoRR, abs/2407.21188. 2024. doi: 1044550/ARXXV.2407.2777 URL https: //tp: /CC 48550/ www.a.aivv...................... 2407 21783
Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah D. Goodman. Stream of search (sos): Learning to search in language, 2024. URL
https: //do:///oii http///oooog/.//./////o 48550/ a 2404. 03683

---

Published as a conference paper at ICLR 2026
Neil D. Jones, Carsten K. Gomard, and Peter Sestoft. Partial evaluation and automatic program generation. Prentice Hall international series in computer science. Prentice Hall, 1993. ISBN 978-0-13-020249-9,
Dmitrii Krasheninnikov, Egor Krasheninnikov, Bruno Kacper Mlodozeniec, Tegan Maharaj, and David Krueger. Implicit meta-learning may lead language models to trust more reliable sources. In Forty-first International Conference on Machine Learning, ICMI, 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https: opw.reviee.eeeeeepppepee...e.... net/feftteooo net/forumiid F EZp1DRZCIN
Andrew K. Lampinen, Arslan Chaudhry, Stephanie C.Y. Chan, Cody Wild, Diane Wan, Alex Ku, Jörg Bornschein, Razvan Pascanu, Murray Shanahan, and James L.. McClelland, On the gener- alization of language models from in-context learning and finetuning: a controlled study, 2025a.
URL /attps://arxiv.
abs/ 00661
Andrew Kyle Lampinen, Martin Engelcke, Yuxuan Li, Arslan Chaudhry, and James L. McClelland. Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiences. CoRR, abs/2509. 16189, 2025b. htt.lhttt:://att org/10..//00 48550/arXiv.
2509. 16189
Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, DJ Strouse, Steven Stenberg Hansen, Angelos Filos, Ethan Brooks, Maxime Gazeau, Himanshu Sahni, Satinder Singh, and Volodymyr Mnih. In-context reinforcement learning with algorithm distillation. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URLhtttps://ppeeeeeee forum? fimmmidd=ii da 5MMPUV
/
Chengshu Li, Jacky Liang, Andy Zeng, Xinyun Chen, Karol Hausman, Dorsa Sadigh, Sergey Levine, Li Fei-Fei, Fei Xia, and Brian Ichter. Chain of code: Reasoning with language naggel-a code emulator. In Forty-first International Conference on Machine Learn- ing, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https: 77 7/openreview net wttp:/ooommmmddddeee omql Sxm
R. Thomas McCoy, Shunyu Yao, Dan Friedman, Mathew D. Hardy, and Thomas L. Griffiths. Em- bers of autoregression show how large language models are shaped by the problem they are trained to solve. Proceedings of the National Academy of Sciences, 121(41) (0.1073/pnas.232242000 2322420121
2024. doi:
CoLlhtts::// htts:://www.paa org/ doi/ 1073 / 1
Yaniv Nikankin, Anja Reusch, Aaron Mueller, and Yonatan Belinkov. Arithmetic without algo- rithms: Language models solve math with bag of heuristics. In The Thirteenth International Conference on Learning Representations, 2025. httptttsss///ppeeeteettet net nwt//orumm
1d 1d-09YTt26r2F
OpenAl, :. Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mądry, Alex Baker- Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Con- neau, Ali Kamali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoochian, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, Andrej Karpathy, Andrew Braunstein, Andrew Cann, Andrew Codispoti, Andrew Galu, Andrew Kondrich, Andrew Tulloch, Andrey Mishchenko, Angela Back, Angela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia, Arka Dhar, Ashley Pantuliano, Avi Nayak. Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky, Ben Wang. Benjamin Zweig, Beth Hoover, Blake Samic Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng, Brad Lightcap, Brandon Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg, Brydon Eastman, Camillo Lugaresi, Carroll Wainwright, Cary Bassin, Cary Hudson, Casey Chu, Chad Nelson, Chak Li, Chan Jun Sher,, Channing Conger, Charlotte Barette, Chelsea Voss, Chen Ding, Cheng Lu, Chong Zhang, Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine McLeavey, Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czarnecki, Colin Jarvis, Colin Wei, Constantin Koumouzelis, Dane Sherburn, Daniel Kappler,
12

---

Published as a conference paper at ICLR 2026
Daniel Levin, Daniel Levy, David Carr, David Farhi, David Mely, David Robinson, David Sasaki, Denny Jin, Dev Valladares, Dimitris Tsipras, Doug Li, Duc Phong Nguyen, Duncan Findlay, Edede Oiwoh, Edmund Wong, Ehsan Asdar, Elizabeth Prochl, Elizabeth Yang. Eric Antonow, Eric Kramer, Eric Peterson, Eric Sigler, Eric Wallace, Fugene Brevdo, Evan Mays, Farzad Kho- rasani, Felipe Petroski Such, Filippo Raso, Francis Zhang, Fred von Lohmann, Freddie Sulit, Gabriel Goh, Gene Oden, Geoff Salmon, Giulio Starace, Greg Brockman, Hadi Salman, Haiming Bao, Haitang Hu, Hannah Wong, Haoyu Wang, Heather Schmidt, Heather Whitney, Heewoo Jun, Hendrik Kirchner, Henrique Ponde de Oliveira Pinto, Hongyu Ren, Huiwen Chang, Hyung Won Chung. Ian Kivlichan, Ian O' O'Connell, Ian O' O'Comnell, Ian Osband, Ian Silber, lan Sohl, Ibrahim Okuyucu, Ikai Lan, Ilya Kostrikov, Ilya Sutskever, Ingmar Kanitscheider, Ishaan Gulrajani, Ja- cob Coxon, Jacob Menick, Jakub Pachocki, James Aung, James Betker, James Crooks, James Lennon, Jamie Kiros, Jan I Leike, Jane Park, Jason Kwon, Jason Phang, Jason Teplitz. Jason Wei, Jason Wolfe. Jay Chen, Jeff Harris, Jenia Varavva, Jessica Gan Lee, Jessica Shich, Ji Lin. Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joanne Jang, Joaquin Quinonero Candela, Joe Beutler, Joe Landers, Joel Parish, Johannes Heidecke, John Schulman, Jonathan Lachman, Jonathan McKay, Jonathan Uesato, Jonathan Ward, Jong Wook Kim, Joost Huizinga, Jordan Sitkin, Jos Kraaijeveld, Josh Gross, Josh Kaplan, Josh Snyder, Joshua Achiam, Joy Jiao, Joyce Lee, Juntang Zhuang, Justyn Harriman, Kai Fricke, Kai Hayashi, Karan Singhal, Katy Shi, Kavin Karthik, Kayla Wood, Kendra Rimbach, Kenny Hsu, Kenny Nguyen, Keren Gu-Lemberg, Kevin Button, Kevin Liu, Kiel Howe, Krithika Muthukumar, Kyle Luther, Lama Ahmad, Larry Kai, Lauren Itow, Lauren Work- man, Leher Pathak, Leo Chen, Li Jing. Lia Guy, Liam Fedus, Liang Zhou, Lien Mamitsuka, Lilian Weng, Lindsay McCallum, Lindsey Held, Long Ouyang, Louis Feuvrier, Lu Zhang. Lukas Kondraciuk, Lukasz Kaiser, Luke Hewitt, Luke Metz, Lyric Doshi, Mada Aflak, Maddie Simens, Madelaine Boyd, Madeleine Thompson, Marat Dukhan, Mark Chen, Mark Gray, Mark Hudnall, Marvin Zhang. Marwan Aljubeh, Mateusz Litwin, Matthew Zeng, Max Johnson, Maya Shetty, Mayank Gupta, Meghan Shah, Mehmet Yatbaz, Meng Jia Yang. Mengchao Zhong. Mia Glaese, Mianna Chen, Michael Janner, Michael Lampe, Michael Petrov, Michael Wu, Michele Wang, Michelle Fradin, Michelle Pokrass, Miguel Castro, Miguel Oom Temudo de Castro, Mikhail Pavlov, Miles Brundage, Miles Wang, Minal Khan, Mira Murati, Mo Bavarian, Molly Lin, Murat Yesildal, Nacho Soto, Natalia Gimelshein, Natalie Cone, Natalie Staudacher, Natalie Summers, Natan LaFontaine, Neil Chowdhury, Nick Ryder, Nick Stathas, Nick Turley, Nik Tezak, Niko Fe- lix, Nithanth Kudige, Nitish Keskar, Noah Deutsch, Noel Bundick, Nora Puckett, Ofir Nachum, Ola Okelola, Oleg Boiko, Oleg Murk, Oliver Jaffe, Olivia Watkins, Olivier Godement, Owen Campbell-Moore, Patrick Chao, Paul McMillan, Pavel Belov, Peng Su, Peter Bak, Peter Bakkum, Peter Deng, Peter Dolan, Peter Hoeschele, Peter Welinder, Phil Tillet, Philip Pronin, Philippe Tillet, Prafulla Dhariwal, Qiming Yuan, Rachel Dias, Rachel Lim, Rahul Arora, Rajan Troll, Ran- dall Lin, Rapha Gontijo Lopes, Raul Puri, Reah Miyara, Reimar Leike, Renaud Gaubert, Reza Zamani, Ricky Wang. Rob Donnelly, Rob Honsby, Rocky Smith, Rohan Sahai, Rohit Ramchan- dani, Romain Huet, Rory Carmichael, Rowan Zellers, Roy Chen, Ruby Chen, Ruslan Nigmat- ullin, Ryan Cheu, Saachi Jain, Sam Altman, Sam Schoenholz, Sam Toizer, Samuel Miserendino, Sandhini Agarwal, Sara Culver, Scott Ethersmith, Scott Gray, Sean Grove, Sean Metzger, Shamez Hermani, Shantanu Jain, Shengjia Zhao, Sherwin Wu, Shino Jomoto, Shirong Wu, Shuaiqi, Xia, Sonia Phene, Spencer Papay, Srinivas Narayanan, Steve Coffey, Steve Lee, Stewart Hall, Suchir Balaji, Tal Broda, Tal Stramer, Tao Xu, Tarun Gogineni. Taya Christianson, Ted Sanders, Tejal Patwardhan, Thomas Cunninghman, Thomas Degry, Thomas Dimson, Thomas Raoux, Thomas Shadwell, Tianhao Zheng, Todd Underwood, Todor Markov, Toki Sherbakov, Tom Rubin, Tom Stasi, Tomer Kaftan, Tristan Heywood, Troy Peterson, Tyce Walters, Tyna Floundou, Valerie Qi, Veit Moeller, Vinnie Monaco, Vishal Kuo, Vlad Fomenko, Wayne Chang. Weiyi Zheng, Wenda Zhou, Wesam Manassra, Will Sheu, Wojeiech Zaremba, Yash Patil, Yilei Qian, Yongjik Kim, Youlong Cheng, Yu Zhang, Yuchen He, Yuchen Zhang, Yujia Jin, Yunxing Dai, and Yury Malkov. Got-4o system card, 2024. URL https: wttp:///aaxivvvv.a .rg/ abs 221024100000 21276
Yixin Ou, Yunzhi Yao, Ningyu Zhang, Hui Jin, Jiacheng Sun, Shumin Deng, Zhenguo Li, and Huajun Chen. How do llms acquire new knowledge? A knowledge circuits perspective on con- tinual tra-trainig.. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.). Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 August 1, 2025, pp. 19889-199133 Association for Computational Linguistics, 2025. URL www.t:ttts/ttt..t antoceaannhhhooooooooooooooo
ndi niniinggnn-in acl. .c1.11111111
13

---

Published as a conference paper at ICLR 2026
Davide Paglieri, Bartlomiej Cupial, Samuel Coward, Ulyana Piterbarg, Maciej Wolezyk, Akbir Khan, Eduardo Pignatelli, Lukasz Kucinski, Lerrel Pinto, Rob Fergus, Jakob Nicolaus Foerster, Jack Parker-Holdes and Tim Rocktäschel. BALROG: benchmarking agentic LLM and VLM reasoning on games, 2024. htttsttt::///
org/10 48550/arXiv.24/44 .12111...14.. 13543
Jupinder Parmar, Sanjeev Satheesh, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catan- zaro. Reuse, don't retrain: A recipe for continued pretraining of language models. CoRR, abs/2407.07263,, 2024. doi: 10448550AARXIV2447.00777 48550/arXiv.2 2407.07263
httts::/:: .trp/00..
Jackson Petty, Sjoerd Steenkiste, and Tal Linzen. How does code pretraining affect language model task performance?, 2024. URL
2409.
04556
Ulyana Piterbarg, Lerrel Pinto, and Rob Fergus. Training language models on synthetic edit se- quences improves code synthesis. In The Thirteenth International Conference on Learning Rep- resentations, 2025. URL
Qwen, An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang. Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025.
URL aittps://arxiie org. at/p//b//2222/// 241.. 15115
Nasim Rahaman, Muhammad Waleed Gondal, Shruti Joshi, Peter V. Gehler, Yoshua Ben- gio, Francesco Locatello, and Bernhard Schölkopf. Dynamie inference with neural inter- preters. In Advances in Neural Information Processing Systems 34: Annual Conference Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual,
pp. 10985-10998 2021. URL https::/ppocceeee////o www.eeiiiiinnn.n.nn...ee....e... www.eneerrrrc.e.eeeeeee hash/ 5aah//bbbeaaaaaa 70w.00aa003d00aaaaaaaaaaa oag.:eeee...11eeaaaa222266333333
wttpp//ppppp//11////// .html
Laura Ruis, Maximilian Mozes, Juhan Bae, Siddhartha Rao Kamalakara, Dwaraknath Gnaneshwar, Acyr Locatelli, Robert Kirk, Tim Rocktäschel, Edward Grefenstette, and Max Bartolo. Proce- dural knowledge in pretraining drives reasoning in large language models. In The Thirteenth International Conference on Learning Representations, 2025. /ttps:::::////// //tps//e/e//ereeee net/ H@KHHUsMx
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models, 2024. htthttts:://artt org/ abs absgg//bb/222.... 2402.
03300
Yonglong Tian, Andrew Luo, Xingyuan Sun. Kevin Ellis, William T. Freeman, Joshua B. Tenen- baum, and Jiajun Wu. Learning infer and execute 3d shape programs. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, and Igor Git- man. Openmathinstruct-l: A 1.8 million math instruction tuning dataset, 2024. URL https:
7/arxiv. www.xv.org//bb/240000 10176
Johannes Treutlein, Dami Choi, Jan Betley, Samuel Marks, Cem Anil, Roger B. Grosse, and Owain Evans. Connecting the dots: Lims can infer and verbalize latent struc- ture from disparate training data. In Amir Globersons, Lester Mackey, Danielle Bel- grave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Informa- tion Processing Systems 2024, NeurlPS 2024, Vancouver. BC. Canada, December 10 15, 2024, 2024. URL ht orte mips http://s....pp..ppppappaaaaeaaaa felea255sseeehaaaee33eaaaaaaaar0
14

---

Published as a conference paper at ICLR 2026
Boshi Wang, Xiang Yue, Yu Su, and Huan Sun. Grokked transformers implicit reasoners: A mechanistic journey to the edge of generalization, 2024. URL battps://doi.xxzt 18550/arXiv. 2405..5400..11.
Yujun Yan, Kevin Swersky, Danai Koutra, Parthasarathy Ranganathan, and Milad Hashemi. Neural execution engines: Learning to execute subroutines. In Advances in Neural Information Process- ing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020, URL http:::/pproceeiinngggs//////// www.://iwnw.....i...........o paper/2020/hash/ccaa0 cawp:h/hcbbbbbbbea http///aaaaaaa.ooooooooooaaaaaaa
Sohee Yang, Nora Kassner, Elena Gribovskaya, Sebastian Riedel, and Mor Geva. Do large language models perform latent multi-hop reasoning without exploiting shortcuts?, 2024. URL https:
//doi. 48550/arXiv. 2411. 16679
Çagatay Yildiz, Nishaanth Kanna Ravichandran, Nitin Sharma, Matthias Bethge, and Beyza Er- mis. Investigating continual pretraining in large language models: Insights and implications. Trans. Mach. Learn. Res., 2025, 2025. URL Trttps::/openzevaee net fontt/foummiii
ak JKoVoKKK
Wojciech Zaremba and Iya Sutskever. Learning to execute, 2015. URL https::/s////ii.orr
2bs//410.r 4615
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. Star: Bootstrapping reasoning with reasoning. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 De- cember 9, 2022, 2022. URL URL htt::tttt:///tttot nips cc cctppperrrppppppeeeeeeeee paper files /a.lee/paper/202// wwnfeerenc...cccec..ccccc 2 o5 -anoeeaacee.. html
15

---

Published as a conference paper at ICLR 2026
A
LLAMA RESULTS
Proactive-PBB CoT
Retroactive. Across Model Scales
0.35
inference Mode
Training
0.30
Direct
SFT
0.25
0.20
Accuracy
Jama-3.2-18-Intte
Instruct
Uama-3.1 8--intrucct
400
800
1000
1200
1400
Program Length
Train Steps
Figure 7: Results with Llama-3 models on Random Arithmetic.
Proactive-7885 788 Leetcode
Retroactive-PS9 Leetrode
0/5
0.0
0.2
0.4
0.0
Zett Seet
Stage only
Stage Only
Stages 1
2000
1200
1100
1000
Figure 8: Results with Llama-3 models Leetcode.
Preactive FB6 Farmal Grammar Generation
Getteve (rrue)))
0.3
Zao Sau
Sueee
2iepe
Sugo
Figure 9: Proactive PBB with Llama - 3 models for generation under formal grammars.
16

---

Published as a conference paper at ICLR 2026
B DATA SCALING
B.1 ABLATION OVER DATASET SIZE
Figure I0 compares the performance of Llama models (IB, 3B and 8B parameters) for varying dataset size on the evaluation of Random Arithmetic programs. Here, 'dataset size', refers specifi- cally to the amount of unique code functions included in the dataset. Performance is evaluated on three separate sets:
The w/ 10 Train set: both the function and the 10 pairs are observed during training
The w/ 10 Test set: uses the same functions as w/ 10 Train but different 10 pairs, not included in the training data
The w/o 10 Test set: evaluates 10 pairs for functions seen only as code during training
The results show that accuracy on both w/ and w/o sets generally increases with larger dataset sizes and larger model scales. Notably, model performance is strongly tied to parameter count; for example, the 8B model trained on only 100 unique functions achieves comparable performance on the w/o 10 set to the IB model trained on 800 functions. The number of unpaired (w/o IO) functions fixed at 200. The upper limit of 800 paired (w/ 10) functions reflects the total of 1000 available functions that generated for Random Arithmetic.
Model: Liama Instruct
Model: Liama Instruct
(Test)
0.45
(Test)
w/ 1D (Train)
e/ I0 (Tesk)
w/ 10 (Test)
I0 (Trein)
0.30
0.35
Accuracy
Accuracy
0.25
0.30
0.15
0.20
0.15
300
400
200
700
Dataset Size
Dataset Size
Model: Llama-3. Betee-3.------tttteree
(Train)
10 (Test)
0.6
w/o 10 (Test)
Accuracy
0.5
0.4
0.3
100
200
300
400
500
700
800
Dataset Size
Figure 10: Performance comparison of Llama models across IB, 3B and 8B on paired (w/ 10) and unpaired (wo 10) Random Arithmetic program evaluation. Each model is trained and tested across varying dataset sizes. Dataset size refers the number of unique functions present in the dataset.
B.2 ABLATION OVER NUMBER OF IO PAIRS
In Figure we vary the number of 10 training pairs (per program) provided for the w/ 10 set, and examine the results. This analysis specifically the Random Arithmetic dataset, which for this experiment consists of 200 distinct functions. Per- formance reported across the same sets the ones described in Appendix B. The results show
the Llama-3. Batiaa...23BBBE- 2--B-Inntruce model on
17

---

Published as a conference paper at ICLR 2026
Model: Llama-3. 1aama--.2-3B-Inaee B.2-Sttnttt
0.225
0.200
0.175
0.150
0.125
(Train) (Test)
0.075
w/o 10 ("esti
4C Number Pairs per Problem
Figure 11: Impact of varying the number of IO training pairs for paired (w/ 10) programs and un- paired (w/o 10) sets evaluation accuracy. Results are shown for the 1lamm-33.2 15.2-33-Innsteee model using Random Arithmetic dataset comprising 200 distinct functions.
how increasing the quantity IO examples for each program affects not only direct generalisation in the w/ 10 Test set, but also the model's 's ability to accurately execute w/o 10 programs.
18

---

Published as a conference paper at ICLR 2026
C SINGLE-STAGE PROGRAMMING BY BACKPROP
on unpaired (w/o 10) Random Arith- metic program evaluation following proactive PBB in comparison to single SFT stage with all training data in single mixture. As we scale the number of times the same piece of unpaired (w/ 10) source code appears in the dataset, with prompt and response preamble augmentations, single- stage SFT approaches the performance of proactive PBB. The greater sample efficiency of proactive PBB is likely because initial train steps on source code waisted in single-stage SFT, as code-I/O relationship has not yet been learned.
In Figure 12 we show the accuracy of 1--ma-3.1-8B-Imaae
1-Stage 2-Stage
0.25
Accuncc
0.20
0.15
2-Stage
Stage
30
40
60
90
Code Repeations
Figure 12: Comparing two-stage proactive PBB to a single SFT stage on the full Random Arithmetic training data mixture for different numbers of repeated source code samples. The base model is Llama-3. 3.1 2.11-8B-Intruue
19

---

Published as a conference paper at ICLR 2026
D ONLINE VS. OFFLINE RETROACTIVE-PBB
In Figure 13 we compare different finetuning algorithms for the second stage of retroactive PBB with Llama-3. 1---n---.--B----- Insttnstructt on Random Arithmetic. DPO allows for learning from both positive and negative samples, considerably outperforming SFT. GRPO is an online RL algorithm, meaning that the model learns from on on-policy data, which could be why yields further improvements. To collect negative data for DPO we generate CoT samples from GPT-4o with rosttoationaliistion, but prompt the model make a realistic mistake in an operation used or an actual arithmetic calculation. This covers samples that are wrong due to recall or arithmetic execution failures.
Retroactive-P0B Algorithm Comparison
0.25
0.20
Acuracy
0.13
9.10
SFT DPG
DRA0
Figure 13: Comparing finetuning algorithms for the second stage of retroactive PBB Random Arithmetic with positive and negative examples. GRPO is online and thus has the added benefit of learning from on-policy data.
DPO is an offline method, but allows for learning from
20

---

Published as a conference paper at ICLR 2026
E
CIPHERS DATA
plot showing the distribution of I0 pairs used in Figure 5 is provided in Figure 14
40
30
Figure 14: Sampled shifts for eipher I/O pairs.
21

---

Published as a conference paper at ICLR 2026
F NATURAL LANGUAGE DESCRIPTIONS
Here, we include an example of a random arithmetic program and natural language description.
Program:
def Blaankle (x) t0 x x
t1 = abs (tC)
return t1
Description: A Blaankle is a process that takes an input value, doubles it, and then returns the absolute value of the doubled result.
F.1 DISCUSSION ON NATURAL LANGUAGE PBB
Current models exhibit strong dependence on the structure of the description. Code and grammars provide explicit, unambiguous abstractions that models internalise reliably. Natural lan- guage (NL) lacks this regular structure and therefore poses a more challenging learning signal for PBB.
However, we see this as a scaling and representation issue rather than a fundamental limitation:
Positive scale-dependence: NL PBB improves with model size, suggesting that future models may be substantially more capable in this regime.
Intermediate formalisms: Many practical workflows already translate NL specifications to structured representations (pseudocode, planning languages). PBB could be applied at these intermediate stages; our work provides evidence that the formal end of this spectrum works well.
Synthetic datasets at scale: Because code-like descriptions are easy to generate program- matically, PBB could be elicited in earlier LLM training stages, potentially allowing mod- els to internalise general symbolie-interpreaatior skills before encountering downstream NL descriptions.
General mechanisms: Our CFG experiments demonstrate that PBB extends outside pro- gramming entirely, to abstract formal systems. This indicates that what matters is symbolic structure, not code specifically.
Thus, while NL PBB is currently weaker, the paradigm itself is not restricted to code, and our results point towards concrete avenues for future work on making NL PBB practical.
22

---

Published as a conference paper at ICLR 2026
G COMPUTE REQUIREMENTS
All experiments with Ll ama models can be run on two GPUs with 40GB vRAM. We used data parallelism over 4 NVIDIA L40s GPUs run these experiments.
Experiments with GPT-44 4o made use of the OpenAl finetuning API. Data generation (Leetcode word problems and post-rationalised chain-of-thougle ground truth outputs for all datasets) and finetuning runs came to total cost just over 500 USD.
23

---

Published as a conference paper at ICLR 2026
H ADDITIONAL ABLATIONS
Proactive. PBB Random Arithmetic
Retroactive-PES (Stage Only)
10
Uama-332 3.2 8B instruct
0U
Base Model
0.7
Owen---B
0.6
Fascution Accuraty
1ccuraty
0.3
0.2
Programs Unpaired Programs
Zero-Shet
Stage 1 Only
Stage Only
200
400
600
800
1000
1200
2400
Stages
Figure 15: Left: Accuracy following each stage of proactive PBB on executing unpaired Ran- dom Arithmetic programs. Right: Accuracy for test inputs following only stage ("activation") of retroactive PBB on paired vs. unpaired Random Arithmetic programs. This training corresponds to only doing RL on execution problems with train inputs on paired programs.
24


Reviews:
[{"review_id": "e0RtYkDOcB", "reviewer": "Reviewer_fuDV", "summary": "This paper proposes TransferEngine, a portable RDMA-based point-to-point communication layer meant to support emerging LLM system patterns such as disaggregated inference (prefill/decoder split), large-scale asynchronous RL fine-tuning, and MoE dispatch/combine across heterogeneous NICs (NVIDIA ConnectX-7 and AWS EFA). The core motivation is that today’s LLM frameworks mostly rely on collectives (NCCL, torch.distributed), which assume fixed membership, ordered operations, and often uniform tensor shapes — assumptions that break down when you need dynamic scaling, sparse / per-token communication, or independent prefill/decoder pools", "questions": "None", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Timely problem & good problem framing. The paper is very in tune with where LLM serving is going: disaggregation (Splitwise, DistServe, Mooncake), MoE, and separate RL rollout fleets all want *non-collective*, *asymmetric*, *sometimes sparse* transfers. The authors clearly articulate where collectives fall short (fixed membership, ordering, shape uniformity), and that framing is convincing. \n\n2. Portability story is concrete, not hand-wavy. Most recent high-perf MoE / KV-store work quietly assumes ConnectX + IBGDA; this paper shows comparable latency on ConnectX-7 and a working EFA path, including notes on libfabric quirks and the need to aggregate 4×100 Gbps EFAs to reach 400 Gbps. That’s a real engineering pain point in production clouds, so demonstrating it is valuable. \n\n3. Case studies are well chosen. KV-cache streaming, 1.3-second RL weight pushes for trillion-parameter models, and MoE dispatch/combine basically cover the three hardest communication patterns people are actually building right now. This makes the paper much more than a “we wrapped libibverbs + libfabric” story. \n\n4. Performance numbers are credible and reasonably detailed. They compare against `ib_write_bw` / `fi_rma_bw`, show saturation thresholds, and offer side-by-side with NIXL where available. The MoE section even breaks out send vs receive and shows where the proxy overhead lands. This is the kind of evidence that’s often missing in “we built a new transport” papers.", "weaknesses": "1. Related-work positioning is a bit soft. The authors mention NVSHMEM, NIXL, Mooncake’s RDMA transfer engine, etc., but they stop short of a head-to-head systems comparison on all of them in the *same* setting. In particular, NIXL has begun adding EFA; NVSHMEM has both GPU-initiated and host-proxy modes; Mooncake targets KV-centric serving. A tighter comparison table (“who supports EFA,” “who assumes ordering,” “who can do MoE dispatch”) would make the novelty sharper. Right now the contribution can be read as “we unified the lowest common denominator and polished the proxy path,” which is good engineering but slightly incremental. \n\n2. Evaluation breadth is narrower than it could be. Most experiments are on 8×H200 nodes with either CX-7 or 2×200 Gbps EFA. That’s a fairly high-end, clean setup. There’s no stress test on less symmetric topologies, multi-tenant noise, or mixed NIC generations — all scenarios cloud users actually hit. This weakens the “portable across providers” claim a bit: we see two well-supported targets, not the messy real world. (This is analogous to “dataset not enough, weakens the conclusion’s solidity” in your template.)\n\n3. Limited ablations on the core idea (IMMCOUNTER + unordered). The paper asserts that giving up on ordering and pushing completion into a counter lets them unify EFA and CX-7, but it doesn’t really show: (a) the overhead of the counter path vs relying on RC ordering, (b) how often counters get out of sync under loss / retries, or (c) how the callback thread scales when 56 peers are all enqueuing WRITEs in the MoE case. Since this is the distinctive technical idea, I’d like one figure that says “without IMMCOUNTER on EFA, this MoE test breaks / slows to X; with it, we get Y.”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rXPKnRWCjk", "reviewer": "Reviewer_aFMH", "summary": "The paper describes how an LLM can be finetuned on a set of functions, with some of those functions having input/output examples, and it will learn to be able to compute input/output mappings for the functions that don't have input/output examples, when those functions are tested with inputs at test time they will execute correctly.\nEssentially it memorizes all the functions during finetuning, and the finetuning on the input/output examples \"teaches\" the LLM how execute the functions so that the functions that don't have input/output pairs get correctly executed at test time on inputs.\n\nThe paper also show this finetuning process works on some other domains.", "questions": "Can you describe in more detail how the RL training was done?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Honestly I was a bit surprised this approach worked at all when first reading the paper, I sort of assume test time access to the functions that had not been \"executed\" with input/output pairs at train/finetune time would be required, and that really this wouldn't work well outside of chain of thought sort of step by step walking through the functions line by line to compute the results at test time.  That functions presented at finetune time are memorized in an executable way is not what I would have expected.\n\nYour approach is novel and unique and surprising that it works (to me) where how I would approach this problem is different (described below in weakness)", "weaknesses": "It feels a bit almost accidental that the way the LLM happens to encode the functions it has seen at finetune time without input/output pairs are able to \"lean on\" / \"borrow\" from the input/output pair experience of the functions that had input/output examples. It feels sort of hackey, the empirical results do show this transfer works, but it doesn't feel reliable to me.\n\nI was unclear what the RL approach was from the paper, the SFT approach I think is more obvious what you would do at finetune time, but the RL approach is not so obvious, and I didn't see the code for it in my very brief search of the provided code.\n\nI guess if I was to approach this problem, I would have done a COT sort of approach where at test time I would train the LLM to reproduce the complete function it's trying to execute in a thinking block, along with a chain of thought sort of scratchpad computation of interim results to execute the function, and then to close the thinking block and output the answer. I feel like that would give a more reliable chance of correctly executing the functions it saw with input/output pairs, and the functions that didn't have input/output pairs, just the LLM needs to remember the function and reproduce the function in a thinking block and execute it in a chain of thought-ish way.\n\nBut I guess your approach is novel and unique and surprising that it works (to me) where what I describe is maybe not so novel.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m4btzVrKja", "reviewer": "Reviewer_7JS6", "summary": "This paper explores the question of whether LLMs can be \"programmed\" by finetuning them on instances of symbolic descriptions of procedures without necessarily seeing their output. The phenomenon is termed \"Programming by Backprop.\" The paper explores finetuning LLama and Qwen models on meta-learning paradigms in which they learn with both paired and unpaired examples of code (real and synthetic), ciphers, and grammars from which to generate text. Results are generally positive, showing in some cases that models can get better at executing unpaired programs seen during training.", "questions": "Q1. The datasets use 100-500 unique code instances/grammars for training, but figure 9 shows that 800 programs can imrpove accuracy further. Was this maximum of 800 chosen arbitrarily, or does more than 800 perform about as well? \n\nQ2. How were hyperparameters in 4.2 chosen? Which were most impactful to the presence of the emergent behavior?\n\nQ3. Python is one of the most prevelant languages in pretraining corpora. If you train on a programs in a less prominant language than Python, do the results generalize?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "S1. The paper is well-motivated and clearly written. It will be of interest to both computational linguists who study emergent behaviors, and potentially fine-tuning NLP practicioners who want to finetune models for algorithmic reasoning. \n\nS2. The paper explores a diverse array of kinds of data, from simple synthetic python programs to leetcode to ciphers and grammars.\n\nS3. The ablations on data and stages answer a number of the questions that I had on first pass, indicating a substantial amount of analysis.", "weaknesses": "W1. The hypotheses on line 215 about the effect of the acquisition phase of Proactive and on 229 about the exposure phase of Retroactive could use evidence across datasets and models. \n\nAre they always doing something, or does the baseline that never sees the unpaired code until test time do just as well? Figure 4 Left and Figure 6 shows results for two task/model class combinations (Llama + Leetcode and Llama + Grammar), giving partial evidence supporting the acquisition phase, but unless I am missing something, I do not see the results for the Python code nor any results for Qwen. I also do not see results showing that the model learns anything during exposure to be retroactively taught during activation. A baseline that skips exposure would help.  This baseline would also help illustrate the core hypothesis, that the standard autoregressive training objective is what causes the model to internalise the executable representation of the procodure.\n\nW2. Figure 2 is hard to interpret without the presence of untuned baselines for comparison. From train steps = 0 can we conclude that the LLMs have no ability to execute code prior to any PBB tuning? Even for simple programs?\n\nW3. The choice to use only SFT for Proactive PBB but SFT+RL for Retroactive PBB seems somewhat arbitrary; the paper could benefit from explaining this design choice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GaHJX5S4PV", "reviewer": "Reviewer_pmd4", "summary": "The paper studies whether LLMs can learn to execute \"behaviors\" from training on data that contains only their abstract description (as opposed to learning to execute them from demonstrations), which they call Programming By Backprop (BPP). The authors find that BPP does not emerge from standard pretraining. However, authors claim that BPP can be elicited with specific finetuning strategies.", "questions": "0. **Why is the methodology a sound way to investigate the core BPP hypothesis, given the seeming mismatch in hypothesis and experiments described above?**\n1. Training neural networks to execute code (or otherwise have programmatic behavior) is a well-stablished line of research (see e.g., [0-3]) that this manuscript, in my opinion, does not engage with enough. **I would encourage the authors to revisit this part of the literature and significantly expand the related work section.**\n\t- In particular, even though the BPP hypothesis (line 189) is cleanly stated and to my knowledge novel, the actual experiments are much more aligned with the standard \"learning to program\" problem due to the inclusion of demonstrations. This makes it particularly important to acknowledge and position this work relative to the aforementioned line of research.\n2. A naive direct validation of the BPP hypothesis would *not* train on demonstration data at all. This follows from the statement of the BPP hypothesis. However, the methodology is not motivated and no experiments are reported at all under this \"direct\" setup. **Why is it necessary to use demonstration data? Why are there no experiments without demonstration data?**\n3. The description of the RL setup is not described at enough detail.\n\t-  E.g., what is the reward function/reward model? How is the RL problem formulated?\n4. A concern about the use of RL: **is the use of RL an effective way to validate the hypothesis of BPP?** Because in that case, the model wouldn't be trained solely on symbolic descriptions, thus would not validate the BPP hypothesis.\n\t- The experimental section would benefit from discussion of the RL experiment and how they relate to the original problem statement.\n5. You state that \"PBB [...] can be elicited through targeted finetuning strategies.\" These strategies are the CoT and RL results. However, there is no controlled experiment that tests CoT and RL *without* BPP. That is, e.g., how do you know that the performance gains are not only from CoT/RL?\n\n[0] Zaremba et al, Learning to execute, 2014\n[1] Tian et al, Learning to Infer And Execute 3D Shape Programs, 2019\n[2] Yan, Neural Execution Engines: Learning to Execute Subroutines, 2020\n[3] Waleed Gondal et al, Dynamic Inference with Neural Interpreters, 2021", "limitations": "", "rating": 2, "confidence": 3, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "This paper identifies the understudied phenomenon of learning to execute behaviours purely from abstract descriptions in the training data. I believe that understanding this phenomenon is potentially high-impact due to the usefulness of implicit behaviour learning in language models.  Thus, I find the significance of the studies to be the biggest strength of the paper.", "weaknesses": "It seems to me that the proposed methodology (Proactive PBB and Retroactive PBB) is not aligned with the paper's stated core hypothesis (line 189). **This makes the validity of the experiments difficult to understand in relation to the stated hypothesis.**\n \nThe core hypothesis only concerns a function whose abstract description is in the training data and that the model is expected to execute without training demonstrations. However, the actual method by which this hypothesis is tested involves training on data with demonstrations of related functions, which is absent in the statement of the hypothesis. Perhaps the hypothesis needs to be revisited to include dependency on training data from demonstrations of other functions.\n\nI would encourage the authors to either extend the core hypothesis or otherwise precisely motivate why the specific experimental setup is a sound methodology to validate the stated hypothesis (given that the current experiments involve training on demonstration data, when the lack thereof is central to the hypothesis).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)