Solution Augmentation for ARC Problems Using GFlowNet: A Probabilistic Exploration Approach
Authors: Sanha Hwang
Seungpil Lee
Sejin Kim
Sundong Kim
Venue: Accepted by TMLR ()

Abstract: One of the core challenges in building general reasoning systems lies in generating diverse, human-aligned solution trajectories—different yet valid paths by which a problem can be solved. Prior approaches often rely on handcrafted templates, rule-based augmentations, or human demonstrations, which are limited in scalability and stylistic diversity. To address this, we explore the use of Generative Flow Networks (GFlowNets) for automated solution augmentation in reasoning tasks. We propose a framework that learns to generate diverse reasoning trajectories with probabilities proportional to their quality, guided by a human-inspired reward function and a novel geometric forward policy. This enables the generation of multiple plausible solution paths without relying on manual supervision. Moreover, our method supports efficient test-time augmentation from input-output examples alone, without access to ground-truth programs or external demonstrations—making it suitable for zero-shot settings. We evaluate our framework on the Abstraction and Reasoning Corpus (ARC-AGI), a benchmark designed to test compositional and abstract reasoning. Our results show that GFlowNets can effectively explore the space of valid reasoning processes, producing a variety of plausible reasoning trajectories, similar to how different individuals might solve the same problem using different intermediate steps. These trajectories are generated at scale-over 100k per task in under an hour, and follow a logarithmic yield trend, enabling practical tradeoffs between augmentation volume and novelty. Furthermore, fine-tuning a large language model (LLaMA 3.1 Instruct 8B) on these synthetic trajectories leads to a 28.6% improvement in reasoning accuracy on ARC tasks, demonstrating the downstream utility of our method. These findings suggest that GFlowNets offer a promising foundation for modeling structured reasoning in automated trajectory generation. 
Our code is here: https://github.com/GIST-DSLab/GFN_to_ARC

Published in Transactions on Machine Learning Research (09/2025)
Solution Augmentation for ARC-AGI Problems Using GFlowNet: A Probabilistic Exploration Approach
Sanha Hwang
hsh64490gmail.coom com
Deportment of AI Convergence
Gwangju Institute of Science and Technology
Seungpil Lee
tamseeungil@@mm.iisss....
Department of AI Convergence Gwangju Institute of Science and Technology
Sejin Kim
sejönkim Ciiiikii@@gis..om
Department of Al Convergence
Gwangju Institute of Science and Technology
Sundong Kim
sundongBgist.ou ac.
Deportment of AI Convergence
Gwangju Institute of Science and Technology
Reviewed on OpenReview: https: opwnrevvee..e....e net/ forum? id= ULC BGy
Abstract
One of the core challenges in building general reasoning systems lies in generating diverse, human-aligned solution ceectoriesddiffrrenn yet valid paths by which problem can be solved. Prior approaches often rely on handcrafted templates, rule-based augmentations, or human demonstrations, which are limited scalability and stylistic diversity. To address this, we explore the use of Generative Flow works (GFlowNetss) for automated solution augmentation in reasoning tasks. We proposc a framework that learns to generate diverse reasoning trajectories with probabilities proportional their quality, guided by human- inspired reward function and novel geometric forward policy. This enables the generation of multiple plausible solution paths without relying on manual supervision. Morcover, our method supports efficient test-time augmentation from input-output examples alone, without access to ground-truth programs or external demonstrations makiing it suitable for zero-shot settings. We evaluate our framework on the Abstraction and Reasoning Corpus (ARC- AGI), a benchmark designed test compositional and abstract reasoning. Our results show that GFlowNets
effectively explore the space of valid reasoning processes, producing variety of plausible reasoning trajectories, similar to how different individuals might solve the same problem using different intermediate steps. These trajectories are generated at scale-over 100k per task in under an hour, and follow a logarithmic yield trend, enabling practical tradeoffs between augmentation volume and novelty. Furthermore, fine-tuning a large language model (LLaMA 3.1 Instruct 8B) on these synthetic trajectories leads to a 28.6% improvement in reasoning accuracy on ARC tasks, demonstrating the ream utility of our method. These findings suggest that GFlowNets offer promising foundation for modeling structured reasoning in automated trajectory generation. Our code is here: https://githuu: com hat.a/bb.ccmm/iiS-DDaa//////// ARC
1 Introduction
One of the central challenges developing intelligent systems endowing them with the ability to reason solve novel problems by composing abstract concepts, drawing inferences, and planning intermediate steps.

---

Published in Transactions on Machine Learning Research (09/2025)
Unlike perception or pattern recognition, which can often be improved through large-scale data and end-to-end optimization, reasoning requires an explicit modeling of multi-step, structured processes. In humans, reasoning is not just about arriving at correct conclusions, but about navigating diverse reasoning paths (trajectories) toward a solution. This crverstt-diiffeenn strategics, representations, or step sequences- plays a critical role in building a robust understanding and generalization. Hence, to improve reasoning capabilities in AT models, it is essential not only to optimize for the final answer, but also to model and expose the system to the wide range of processes by which those answers can be reached (Wei et al., 2022. Wang et al.) 2023).
Recent research increasingly emphasizes the value of exposing language models to explicit reasoning tra- cectories-structurre sequences of intermediate steps guiding problem-solvingg This includes prompt-based strategies such as Thouuht (CoT) (Wei et al., 2022) and Tree-of-Thoughh Thought (ToT) (Yao et al.
2023)
sampling-based methods like Self-Consistencyy (Wang et al. 2023) and Best-of-N selection (Liu et al.
2025
and training-tiie -tim approaches that filter or reward high-quality reasoning paths (Zelikman et al.) 2022 Luo
ct al. 2024a. Wang & Su 2015 Jiang et al., 2024). More recently, test-time iterative refinement has gained al. 2025). and OpenAl's 2024) generate, evaluate, and revise their own reasoning traces in real-time. While these trends have yiclded significant progress, most approaches still rely on prompt engineering, hcuristic sampling, or post-hoc
attention: models like Rellexion (Shinn et al. 2023) DeepSeek-R1 (DeepSeek-AT
ol [OpenAl 2025
limiting scalability and process-level aligmment. In contrast, our work introduces a generative framework that learns to produce diverse reasoning trajectories in a unified and scalable manner. Our framework is unique in its ability to augment multiple, varied solution paths for a single problem instance, a capability lacking in existing human-collectn or rule-based datasets (see Appendix D in detail.)
Existing methods for generating reasoning trajectories largely rely two types of supervision: (1) human- written demonstrations and (2) rule-based augmentations. Human demonstrations (Strandgaard 2024 LeGris ct al. 2024 Kim et al. 2025b), often collected from crowdsourcing, capture authentic and diverse reasoning patterns. However, these solutions are costly to obtain, difficult to scale, and often noisy or inconsistent. More importantly, they are tied to specific training instances and cannot be applicd to novel test tasks where such demonstrations are unavailable. In contrast, rule-based or hardcoded trajectories (Kim et al. 2024) offer expert-desinned strategies that are typically consistent and efficient. These approaches rely on domain knowledge or templates to construct solution paths, making them effective within known task distributions. Yet their rigidity is also limitation: they encode only a narrow slice of the solution space and fail to adapt to structurally novel problems. Furthermore, because they require access to ground-truth programs or transformation logic, they are inapplicable to unseen tasks at test time.
In many real-world reasoning scenarios, such as the ARC Prize (Chollet et al. 2025 systems must solve entirely unseen tasks at test time within a limited time budget (e.g., 6 minutes per task, assuming 120 tasks in 12 hours). Under these constraints, there is no opportunity to access task-specific supervision or hand-written demonstrations. This setting calls for test-time augmentation methods that can autonomously generate high-quality and diverse reasoning paths from scratch.
This leads us to a central research question: Can we develop an automated framework that can generate diverse reasoning trajeclories ciiiclooees-wilhoul withoul relying on human-wrillen demonstmtions or handcrufted rules and use them to enhance model reasoning performance? Such a framework would ideally be flexible enough to explore multiple plausible paths for the same problem, allowing the model to learn how to reason more general, compositional, and interpretable ways. Moreover, it should be computationally efficient, able to generate thousands of diverse solution candidates within short time window, enabling downstream symbolic neural solvers to learn select from these paths on-the-fly
To evaluate the effectiveness of our proposed framework, we conduct experiments on the Abstraction and Reasoning Corpus (ARC-AGI, hereafter ARC) (Chollet) 2019). a benchmark specifically designed to test reasoning capabilities. ARC consists of few-shot, grid-based tasks where each instance requires the solver to infer an underlying transformation rule from a small number of input-output demonstrations. While the task format is visually simple, solving ARC problems often demands compositional reasoning, abstract generalization, and logical coherence. Indeed, ARC provides a unique lens through which assess these dimensions of reasoning compositionalily, productivily, logical coherence (Lee et al. 2025), and the ability to generalize from limited examples.
2

---

Published in Transactions on Machine Learning Research (09/2025)
Importantly, ARC's structure presents particularly suitable environment for evaluating solution generation frameworks that aim to capture varied and structured reasoning paths, rather than simulating specific human-like strategies. First, the dataset is inherently underspecified: multiple reasoning trajectories can lead to the same correct output, making it ideal for exploring the benefits of diverse solution generation. Second, the limited data regime and absence of predefined lask categories emphasize the need for flexible and gencralizable reasoning strategics strategies-exactly exactly the kind our method aims to generate.
From an AGI perspective, this diversity in reasoning paths is not peripheral detail but central objective. Recent studies [Bengio 2021 Morris et al. 2024 Kim & Kim 2024 Lee et al. 2025 emphasize that general intelligence entails solving novel problems through varied, compositional reasoning processes, even when the final solution is unique. GelowNets provide a principled framework for capturing such reasoning diversity. Instead of optimizing for a single expert path, they discover multiple plausible trajectories that lead to the same outcome. This capability aligns closely with the AGT goal of modeling not just what to think, but how to think through problems multiple valid ways.
Finally, since each ARC task typically requires multi-step abstract inference rather than surface-level pattern recognition, improvements in trajectory-level modeling are more likely to translate into tangible performance gains. Our approach is also practically scalable: generating over 100k trajectories per task takes under an hour on average, and we observe a logarithmic growth trend in the discovery of new unique t trajectories, allowing efficient tradeoffs between data volume and diversity. These properties make our method well-suited for test-time deployment in ARC-like reasoning scenarios.
Contributions. Our main contributions are summarized follows:
1. GFlow Reasoning Augmentation: We propose the first framework to use GelowNets for genorating diverse reasoning trajectorics, enabling automated solution augmentation that reflects the structured variability in how problems can be solved.
2. Human-Inspired Policy and Reward Design: We introduce two architectural innovations-a
geometric-forwart action policy and a goal-conditioned reward function that explicitly encode human reasoning biases into the learning process.
3. Empirical Evaluation on ARC: We demonstrate through extensive experiments on ARC that our method improves both solution diversity and accuracy compared to existing approaches, particularly in low-data regimes.
4. Test-Time Efficiency and Yield Analysis: We analyze the generation efficiency of GFlowNet and show that the number of unique, valid trajectories exhibits a logarithmic growth pattern, providing practical guidance on how much augmentation is beneficial under compute constraints.
2 Background
2.1 Reasoning with Diverse Solutions
Effective reasoning involves not just arriving at the correct answer, but also navigating structured and coherent intermediate steps. Many tasks losks-particularly those involving abstraction, logic, or multi-hop inference- admit multiple valid solution trajectories, each representing a different. way to solve the same problem. This diversity is analogous to the variation in strategics employed by humans and essential for improving generalization, robustness, and interpretability in Al systems.
To incorporate this inductive bias, recent research has introduced techniques that explicitly model the reasoning process. Chain-of-Thoughee (CoT) prompting (Wei et al., 2022 Zhang et al.) 2025a), Tree-of- Thought (ToT) (Yao et al. 2023), and Process Reward Models (PRMs) (Zhang et al. 2025b) supervise or sample intermediate reasoning steps to better align with human-like cognition. These methods show that exposing models to diverse reasoning paths improves their ability to generalize, particularly in tasks with sparse supervision or structural novelty. Additional evidence across various reasoning benchmarks-inchan -including GSMSK (Cobbe et al. 2021 SVAMP, and AQuA-RAT (Ling et al. 2017 further supports this approach,
3

---

Published in Transactions on Machine Learning Research (09/2025)
demonstrating consistent performance improvements when models explore and aggregate multiple reasoning trajectories.
Complementary approaches such as automated process supervision (Luo et al. 2024a) and planning- basedd trajectory optimization (Jiang et al. 2024) further validate the effectiveness of learning from reasoning paths directly. These methods synthesize reward signals that capture desirable reasoning characteristics such as conciseness, non-redundancy, or clear subgoal structure without relying on exhaustive human annotations. Similarly, reasoning- driven process reward modeling (She et al.) 2025 and step-by-step human feedback [Lghttman et al. 2024) have achieved substantial performance gains by leveraging automated or semi- automated trajectory supervision. As result, models trained with these signals exhibit more structured and interpretable problem-solving behavior, underscoring a growing consensus that trajectory-level supervision significantly enhances reasoning abilities.
This emphasis on proccss-lovel reasoning has also become a central trend in the development of large language models (LLMs). Recent models like OpenAl's o3 series (OpenAl] 2025) and DeepSeek's R1 (DeepSeek-Al) et al. 2025) incorporate internal deliberation and iterative refinement to improve reasoning performance. Likewise, Colfs-onsistency sampling (Wang et al. 2023) aggregates multiple reasoning chains through majority voting, achieving improved accuracy and robustness on benchmarks such as GSM8K (Cobbe et al. 2021 p.
These examples highlight a growing consensus: modeling diverse and structured reasoning processes is key to achieving reliable and aumann-lignne inference.
However, many of these approaches depend on human- written demonstrations, handcrafted rules, or prompt- based sampling, which constrain scalability and stylistic divensity. Moreover, outcome-only supervision fails to distinguish between logically correct but inefficient reasoning paths and concisc, interpretable ones. Without process-level reward signals, models are unable to prioritize solution trajectories that mirror human preferences.
These limitations motivate our goal of building a genorative framework that can learn to produce diverso, high-quality reasoning trajectories scalable and automated manner. In this work, we leverage GeneraLive Flow Networks (GFlowNets) (Bengio et al., 2021 2023) to model solution generation as a reward-proportional sampling problem, enabling the creation of multiple human-aligned rcasoning paths without requiring explicit supervision for each trajectory.
2.2 Data Augmentation for Reasoning Tasks
Data augmentation addresses the inherent scarcity of supervision in reasoning tasks and can be broadly catogorized into input-output augmentation and solution-process (trajectory) augmontation.
Outut-Output Pair Augmentation. Methods such as RE-ARC (Hodel 2024). AugARC Bikov al. 2025), and SOLAR (Kim et al. 2024) generate new input- examples to enhance training diversity in ARC tasks. Similar strategies, including synthetic data generation and paraphrasing, have been widely applied across various reasoning domains. For math word problems, Lu et al. (2024) leveraged question back-translation to produce synthetic examples, significantly enhancing model performance on the GSMSK benchmark. In logical reasoning, LogicAsker generated QA pairs by programmatically translating formal logic into natural language, achieving notable gains in logical consistency (Wan et al. 2024). Likewise, in code generation tasks, WizardCoder synthesized diverse coding problems and solutions, leading to state- of-the-art results on benchmarks like HumanEval and MBPP (Luo et al.) 2024b). For scientific reasoning tasks, cross-lingus data augmentation using large language models produced multilingual synthetic QA pairs, demonstrating considcrable improvements in cross-lingual genoralization (Whitchouse et al. 2023). These input-oulput augmentation techniques substantially improve model generalization by diversilying the task distribution, though they typically do not explicitly expose underlying reasoning processes.
Solution-Proces (Trajectory) Augmentation. Techniques explicitly augmenting reasoning trajectories leverage models' intermodiate solution paths. Self-Consistency sampling (Wang et al. 2023). Troo-of- Thought deliberative search (Yao et al. 2023), and iterative refinement methods like Reflexion (Shinn al., 20233 exemplify trajectory-level augmentation. Methods such STaR (Zelikman et al. 2022 and Automate-
4

---

Published in Transactions on Machine Learning Research (09/2025)
CoT (Shum al. 2023 demonstrated that training model-generated chains-of-thought significantly boosted accuracy across various benchmarks, including commonsense and math word problems. Synthetic Prompting (Shao et al., 2023) further enriched trajectory diversity by iteratively synthesizing questions and corresponding reasoning paths, yiclding substantial gains in numerical and algorithmic reasoning tasks.
Similarly, frameworks like Flow of Reasoning (Yu et al. 2024) leveraged planning-based trajectory sampling to explore diverse reasoning paths, surpassing previous state-of-tho-art performances on puzzle-oolving bench- marks. AlphaCode's 's strategy of sampling numerous candidate solutions and evaluating correctness tests illustrates another successful trajectory-level augmentation approach in code generation tasks (Li et al. 2022). Collectively, these trajeclory-focused methods enable models to explore and validate multiple solution paths systematically, greatly enhancing performance and interpretability.
Other Augmentation Techniques. Best-of-N (BoN) sampling and ensemble decoding represent additional augmentation strategies. BoN methods, applied in models like Minerva and AlphaCode, Code, sample multiple candidate outputs to select the best- pest-performig solution (Lewkowycz et al., 2022: Li et al. 2022p. Ensemble decoding aggregates outputs from multiple models or reasoning chains, typically employing voting or averaging strategies. These methods primarily function the output level and rely heavily on external evaluation metrics heuristic criteria.
Limitations and Challenges. Despite their effectiveness, existing augmentation methods face challenges including computational scalability, gemuine trajectory diversity, and the absence of reliable intermediate-step supervision. Techniques such as Tree-of-Thought and extensive campling-bssed methods incur significant computational overhead (Yao et al. 2023 Wang et al., 2023). y. Furthermore, approaches relying on outcome-only evaluation lack explicit guidance for intermediate reasoning steps, potentially cansing logical inconsistencies or inefficient reasoning pathways.
Overall, integrating diverse augmentation strategies ranging from input-oulput pair expansion trajectory- level solution enhancement- can significantly improve model robustness and generalizability These methods collectively form essential toolkit for addressing the inherent limitations of scarce supervision in complex reasoning tasks like ARC and beyond.
2.3 Generative Flow Networks (GFlowNets)
Generative Flow Networks (ellowNets) (Bengio et al., 2021 2023) are class of generative models designed to produce diverse solution trajectories by learning from reward signals. They are particularly effective for structured generation problems where multiple valid outputs exist, such as generating human-like solutions in reasoning tasks.
Unlike standard generative models, which often produce one-shot outputs, GFlowNets construct solutions incrementally by sampling sequences of actions. This approach inspired by reinforcement learning (RL), enabling GFlowNets to combine the exploration strengths of RL with the flexibility of generative modeling.
In contrast to conventional RL, which typically seeks single optimal policy, GFlowNett learn stochastic policy that assigus higher probability to trajectories with higher terminal rewards. This makes them well-suited for sparse-reward environments and problems with multiple equally good solutions, such as ARC, GFlowNets
have demonstrated their utility across a wide range of structured generation tasks, including molecular design
and biosequence modeling (Bengio et al.) 2021 Gaiński et al. 2025 Jain et al. 2022 Jang et al.
2024
Zhang
et al.
2022), symbolic regression (Zhang el. al. 2023b), combinatorial optimization (Jain el al.
2023
Zhang
et al.
2023a), and other applications (Sendera al. 2024 Zhang et al., 2022 Pan et al. 2023
Falet et al,
2024
Sco ct al. 2025: Kim et al. 2025a Hu et al.
2024
Flow Matching Condition and Trajectory Balance (TB) Loss. To understand the underlying
mechanism of GFlowNets, it is essential to grasp the flow matching condition, which maintains consistency between the inflow and outllow of probability mass at any state in the stale space:
F(s' -) = s")),
(1)
5

---

Published in Transactions on Machine Learning Research (09/2025)
Generate action with GFN Transition through
ARCLE
C
2
if
:
then Terminate
$1
Input (#p)
C
$2
Output (8.).
:
C
Figure 1: GFlowNet Concept Diagram for Solution Generation in ARC
where F(s' s) denotes the probability flow from state s' to state This condition ensures that the forward
policy Pp (sx ({11) (governing state transitions from an initial state to terminal states) and the backward
policy Pn(s:-11 8t) (governing transitions in reverse) remain consistent with the reward model's distribution.
This consistency is operationalized through the Trajectory Balance (TB) loss (Malkin et al. 2022). defined as:
Z
II PF(*: 8t-1) = R(x) II P8 82),
(2)
1=1
where R(x) represents the reward for a trajectory x = (Su..... sn), and Z is learnable normalization constant ensuring proper distribution over trajectories. Minimizing the squared difference of the log probabilities on both sides yields the TB loss:
2
n
(TB(A) = log Zo +
Σ log Pe (s -log R(x) - Σ log PB P
(3)
By optimizing this objective, GFlowNets learn to sample Irajectories proportionally to their rewards, naturally prioritizing high-quality trajectories. This allows effective exploration of complex solution spaces.
Training Steps and DAG Structure. The practical training loop of GFlowNett involves several key steps: (1) Forward Sampling, starting from an initial state and sequentially sampling states via Pr until a terminal state is reached; (2) Reward Compulation, evaluating the sampled trajectory using the reward function R(x);; (3) Backward Sampling, optionally reconstructing the trajectory in reverse via PB to reinforce flow matching: and (4) Loss Computation and Parameter Update, adjusting model parameters by minimizing the TB loss. To ensure stable training and avoid cyclic infinite loops, state transitions typically constrained within Directed Acyelic Graph (DAG), limiting each state to single visitation and ensuring convergence.
Reward Modeling and Oracle Networks. Since GFlowNets explicitly learn the distribution shaped by the reward function, accurate reward modeling is paramount. Often, rewards assigned through pre-trained or heuristic oracle networks tailored to specific tasks. These oracle models encapsulate task-specific structural insights and desired properties, significantly enhancing the effectiveness of the generative process. For inslance, in molecular generation tasks, oracle networks evaluate chemical viability, steering GFlowNet exploration toward feasible and high-quality solutions. The careful design and training of these oracle networks thus play critical role in the performance of GFlowNets, particularly in environments with sparse complex reward structures.

---

Published in Transactions on Machine Learning Research (09/2023)
Task
Task
Task 3
Task
L
7x7
Demo Pairs
5x5
9X9
Test Input
5
?
?
?
11x11
<>
?
Figure 2: Some Problems of Abstraction and Reasoning Corpus (ARC)
The structural exploration capability and explicit. reward modeling of GelowNets position them uniquely capture intricate reasoning structures and produce diverse, valid solutions in reasoning tasks. By leveraging human-inspired reward models, GFlowNets can systematically generate solution trajectories reflective of human reasoning patterns. This capability makes GFlowNets particularly suited for automated trajectory augmentation tasks, such as those required by reasoning-intensive benchmarks like ARC.
Specifically, GelowNets directly support logical coherence through forward and backward flow consistency, ensuring generated solutions maintain logical consistency. Productivity is enabled by effective generalization from sparse examples via oracle guidance, and compositionality facilitated by structurally exploring compositional reasoning paths evaluated by oracles.
2.4 ARC
Benchmark for Process-Level Reasoning
The Abstraction and Reasoning Corpus (ARC) (Chollet 2019) is a challenging benchmark comprising visual grid-based puzzles that require inferring abstract and compositional rules from very few provided examples. Each ARC task includes only 2-5 input-output examples, with no repetition of input-output pairs across different tasks. The ARC dataset was originally introduced as a proxy for evaluating core reasoning abilities of AT systems, specifically in terms of logical coherence, productivity, and compositionality (Lee et al. 2025
Recently, despite impressive performances by stale-of-the-arl models such OpenAl's O3 low, which achieved over 75% accuracy on ARC (OpenAl 2025), performance sharply declined to below 10% on the newly introduced ARC-AGI-2 dataset (Chollet, 2024 Kamradt, 2025). This drop clearly indicates the complexity of ARC and highlights the siguilicant gap between current Al capabilities and hunan-like abstraction and reasoning, indirectly underscoring the distance to achieving Artificial General Intelligence (AGI).
The inherent difficulty of ARC arises primarily from its unique structure: each problem set has single, unique input-output relationship, and these relationships vary drastically across different tasks. Therefore, conventional data augmentation methods struggle as additional synthetic examples cannot be trivially generated. Given these constraints, ARC becomes particularly suitable for testing the capability of automated solution augmentation methods. Specificallyy, ARC provides a rigorous environment to examine whether generative frameworks, such as GFlowNets, can effectively capture structural insights from carefully designed, human-inspired reward models, thus facilitating the discovery of diverse and valid solution trajectories under highly sparse and unique example constraints. These characteristics position ARC ideal benchmark for exploring and validating advanced reasoning frameworks like GelowNets.
Conventional sigmentation methods applied specifically to ARC, such as input-output pair augmentation (Hodel 2024 Bikov et al. 2025 Kim et al.) 2024)), lack explicit trajectory-level reasoning, treating the transformation processes as black boxes. Sule-bbsee augmentation methods, mcanwhile, rely on predefined human-crafted rules, severely limiting scalability and the generalization capacity to novel, unseen problem structures. Consequently, these approaches fail to adequately address ARC's unique demands for explicit and
7

---

Published in Transactions on Machine Learning Research (09/2025)
diverse trajectory generation, highlighting the need for automated, trajectory- augmentation frameworks such GFlowNets.
3
Methods
3.1
GFlowNet Architecture for Human--ligned Solution Generation in ARC
We now provide a concisc overview of how our GFlowNet is structured and generates solutions tailored specifically for ARC problems. Building on the general components described in Sections [.1 3.2 our architecture differs from conventional GFlowNet models primarily by using asymmetric sampling distributions: a geometric distribution for forward sampling to prioritize shorter, more efficient action sequences, and a categorical distribution for backward sampling to ensure effective trajectory reconstruction. Additionally, we explicitly incorporate human trajectory priors into our reward function, penalizing redundant. or excessively long trajectories to enhance learning efficiency and promote human-like reasoning. In what follows, detail this architecture and elaborate on the rationale behind these adaptations.
High-Level Process. At the start of each episode, an initial state derived from the ARC input grid is set, containing minimal information needed for the ARC task. The GFlowNett forward policy then samples actions to move from one grid configuration to another, guided by the Geometric distribution described above. Actions that prove too long or repetitive are penalized by our reward model, while succinct, high-quality trajectories receive higher rewards. After reaching a final state, the backward policy verifies or backtracks using a Categorical distribution, helping to ensure the global flow matching necessary for Trajectory Balance. Over many episodes, this bidirectional sampling process adapts the GFlowNet parameters to favor human- like solutions that meet the puzzle's requirements. We summarize the full GFlowNet training process in Algorithm D provided in Appendix C The pseudocode highlights the use of geometric sampling for forward actions, categorical backward modeling, and off- off-policy updates with pycle-penalized reward shaping.
Algorithmic Flow. Algorithm 1 (Appendix C outlines the main steps: (1) Inilialization of the for- ward/ backwardd networks, (2) Forword Sampling where actions are chosen via the Geometric distribution based on predicted logits, (3) Reward Assignment using human-inspired reward model that applics dis- counting, cycle penalties, trajectory-length constraints, or combination thereof, depending the specific characteristics of the trajectory, (4) Backward Sampling where the backward policy samples a probable reverse action corresponding to the forward-selected action, ensuring local consistency required for trajectory balance, and (5) Parameter Update by minimizing the trajectory balance (TB) loss across the sampled trajectory. Repeating these steps allows the GFlowNet to efficiently converge on a diverse set of high-reward, human-like trajectorics.
ARC-Specific Adaptations. Unlike many domains where actions naturally exhibit uniform or categorical distributions, ARC problems benefit from the combination of a Geometrie-foruact and Categoricalbucckuard backward design. (Section 3.2) In our setup, each trajectory typically ends with a "Submit" action that finalizes the generated grid, triggering binary reward if the puzzle is solved correctly. This setup not only aligns with the binary reward structure of ARC but also enables GelowNet to efficiently prioritize high-quality, human-like trajectories within its generative process.
Test-Time Augmentability. A key advantage of our framework its ability to generate solution tra- jectories directly from input-output demonstrations, without requiring access to ground-truth programs or human-written paths. This enables reasoning augmentation even at test time, where neither human demon- strations nor rule- based solutions are available. In contrast to prior methods that rely on fixed trajectory sources tied to specific training tasks, our approach can synthesize plausible and diverse rcasoning paths on unseen problems. This is particularly important in settings like ARC, where the model must solve novel tasks at test time without access to prior task-specific supervision. As a result, our method is especially well-suited for ARC's few-shot, out-of-distribution generalization setting. For detailed empirical analysis of efficiency and test-time augmentability (e.g., trajectory generation rate and yield scaling), please refer to Appendix F
8

---

Published in Transactions on Machine Learning Research (09/2025)
3.2 Human- Human-Trajecton Trajectory Guided Reward Design
GFlowNet cammot effectively learn with purely sparse reward model. A naive sparse reward assignment- for correct answers and 0 for incorrect ones fails to provide the necessary gradient to differentiate between high-quality and low-quality trajectorics, severely limiting learning efficiency or even halting progress entiroly.
When applying GelowNet to the ARC problem, this limitation became apparent. For example, as shown Figure some trajectories solve the problem efficiently, while others reach the correct answer with unnecessary and repetitive actions. Assigning the same reward both types of trajectories makes it difficult for GFlowNed to prioritize high-quality solutions.
Task 179
Seyebbe
Gorg
Spgret
Demo Pairs
Satamit
Rocate
Unde
Sibptt
Solutions
Selpdded
Sel Selas Gnd
Seledded
Test Input
Po
Figure 3: State Space Graph of Trajectories for ARC Problem. The trajectories were collected through 02ARC (Shim et al. 2024), illustrating various solution paths from the start node (blue) to the correct solution node (green). Additionally, incorrect solution nodes (red) are displayed to represent alternative, unsuccessful attempls. This figure highlights the diversity of possible solutions for a single problem, emphasizing the multiple ways which the problem can be approached and solved.
To address this, we analyzed human trajectory data from O2ARC 3.0 (him et al., 2024 to identify clear quality differences and inform the design of reward model that highlights superior trajectories. Two major charactoristics of high-quality trajectories were identified:
Final Action:
The last action in a trajectory must always be the 'Submitt action. This deliberate conclusion indicates that the solution is complete and allows for feedback. Trajectories that end with the Submit action are prioritized in the reward model.
Avoidance of Redundancy: High-quality trajectories avoid unnecessary repetition, using only essential actions to reach the correct answer. Most general problems rarely require more than steps (Kim al. 2025b). Repetitive actions, particularly in Directed Acyelic Graph (DAG) context, create cycles that complicate learning.
To reflect these human-derived structural preferences explicitly, we propose integrating the following mecha- nisms into reward design:

---

Published in Transactions on Machine Learning Research (09/2025)
Discount Factor (RDF) The first approach involves applying discount factor y (set to 0.9), commonly
used in reinforcement learning. This method inherently favors shorter trajectories by scaling the reward R at. each step as:
Rpr R.
(4)
While this mechanism promotes brevity, our goal is not strict minimization, as our experiments show this can prematurely limit the discovery of diverse solutions. Rather, this approach primarily serves as a heuristic lo guide the search away from aimless wandering and toward more direct, goal-oriented trajectories.
Trajectory Length Regularization (CTLR) The second stralegy introduces a regularization term directly into the objective function. Offering a more flexible way to manage exploration depth, this term penalizes significant deviations from target trajectory length Ltarget- This provides soft constraint that discourages excessively long inefficient paths without being overly restrictive, providing flexibility while discouraging excessively long trajectories:
LTLR = Arog (Cr Leargot)?
Here, Areg controls the regularization strength. The total objective function combines this term with the Trajectory Balance Loss (CTB):
CTota((0) = (1 o.) Lrn + a LTL
(6)
where balances trajectory balance and length regularization.
Cycle Detection (Reycle) Finally, cycle detection penalizes repetitive actions by flagging repeated states within trajectory. A penalty proportional to the number of detected cycles C(T) is applied:
Reycle (T) - - X. C(T) if a cycle is detected, r(ST)
(7)
otherwise.
This encourages efficient exploration while avoiding redundant actions. Thus, we define the reward for a trajectory x (so...., 8T) as:
R(:; v) [[((a)=)) of (((()) exp (~Acyele C(x)),
where:
=))))= is a binary indicator of whether the trajectory leads to a correct solution,
(0, 1) is discount factor applied to the trajectory length L(x),
((x)) is the number of detected cycles (i.e., repeated states) in the trajectory,
Acyele is the penalty weight for cyclic behavior.
This reward used as input to the Trajectory Balance (TB) loss, and log-transformed prior to optimization. In addition to the reward-based objective, we introduce separate trajectory length regularization loss to further encourage concise reasoning:
(TER(E) = ATER (L(x)- Lrarpot)²
where Ltarget is reference trajectory length (e.g., 3 4), and XTLR controls the penalty strength.
10

---

Published in Transactions on Machine Learning Research (09/2025)
The overall training objective is then defined as:
Liotal = LTB + LTLR.
where LTB denotes the Trajectory Balance loss computed using the log-transformed reward log R(;; y)-
3.3
Forward Policy via Geometric Action Sampling
Stop Leighh vs, Reward for Successful unique Trajedioies
Gcometri PMF for Btop Lengt
PVF (-00))
08:20
100
0010
03
38708
22
Figure 4: (Left) Reward Distribution by Step Length: This figure shows the reward distribution across trajectories with varying lengths for ARC problems. Higher rewards are concentrated in shorter steps, while rewards decrease as the step length increases. (Right) Geometric PMF: General form of PMF for the Geometric distribution.
To effectively model the distribution of reward functions derived from human trajectories, it is crucial to account for the sequential nature of actions and their dependencies in ARC problems. While conventional GFlowNets typically employ the Categorical distribution to model action probabilities, this approach fails to capture the observed relationship between reward and step length in ARC problems.
As shown Figure rewards are heavily skewed toward shorter trajectorics, with diminishing rewards step lengths increase. This motivates the of the Geometric distribution to address the limitations of the Categorical model.
The Geometric distribution is well-suited for modeling action sequences under framework where actions are treated as probabilistic steps loward success. Specifically, it represents the number steps required until the first success in a sequence of trials, with a constant probability of success In the context of ARC problems, success can be interpreted as reaching the target state, and cach action step is viewed as a probabilistic attempt move closer to that state.
To formalize this interpretation, we treat each candidate action a, associated with success probability obtained via softmax over logits zi. The nunber of steps until success under this distribution is modeled as geometric random variable:
E[X,] 1 pi
X₃ Geometric(pi).
Then, selecting the action with the minimum expected steps corresponds to:
a" = arg min E[X;] arg max Pr-
11

---

Published in Transactions on Machine Learning Research (09/2025)
This provides probabilistic justification for the use of Geometric sampling in the forward policy: it naturally prioritizes actions with higher expected efficiency in sparse reward settings. Moreover, by selecting actions based on their expected time-to-success, the policy aligns with human-like fecision-making that favors shorter and more effective trajectories.
Trajectory Length Distribution (Task 179)
Paman Ingaadorhis (1+(2))
GFlowNet (n-339)
0.125
Dondtty
0.100
0.75
0.000
Trajectory Length
Figure 5: Comparison of correct and unique trajectory length distributions between human-provided solutions (n=32) and our GFlloNNt--genertte solutions (n=39) for Task 179 (ID: 74dd1130). The GFlowNet 's distribution (orange), guided by the geometric policy, shows a strong preference for shorter trajectories, peaking a similar length to the human distribution (bluc), empirically validating our brevity-focusod heuristic.
This aligns well with the structure of GFlowNet, where the model explores action sequences that maximize long-term reward under trajectory-level uncertainty. Building on the efficiency bias introduced by geometric sampling, this formulation has proven particularly effective sparse- environments such as ARC. Tn addition, the preference toward shorter trajectories helps mitigate over-exploration of redundant or excessively long paths, which can dilute learning signals especially during early stages of training.
This cheoretical bias toward shorter, high-probability trajectories is grounded in established principles like Minimum Description Length (MDL) (Hernandez-Orallo & Minaya-Collado 1998 Mahoney 1999 Hutter
2005 Legg & Hutter 2007 of Ockham 1990). This preference for conciseness acts as a powerful heuristic, and it aligns with general syntactic patterns observed in human problem-solving trajectories, as illustrated in Figure 5 our model learns to generate trajectorics with a length distribution that closely mirrors that of human solutions, with both distributions peaking at a small number of actions. This provides strong empirical support for design choice to incorporate an inductive bias towards brevity, validating that the model effectively captures key structural features of efficient, human-like reasoning paths.
However, it is crucial to recognize that this heuristic is a means to an end, not an absolute objective. As our detailed analysis on exploration depth will demonstrate in 4.2.2 treating brevity as the sole goal is a limiting assumption. The true strength our policy emerges when its efficiency bias carefully balanced with a sufficient 'exploration budget' of steps. This balance key to preventing premature convergence and enabling the discovery of more diverse and complex reasoning paths.
The rationale for using the Geometric distribution is follows:
Reward Trends in ARC problems: Higher rewards are observed in shorter trajectories, which can be naturally modeled by the Geometric distribution's property of assigning higher probabilities to fewer steps. By prioritizing trajectories with shorter action sequences, the model effectively aligns exploration with the observed reward trends in human problem-solving.
12

---

Published in Transactions on Machine Learning Research (09/2025)
Action Sequences Trials: Each action in sequence be interpreted independent attempt to reach the target state (success). The Geometric distribution allows the model to evaluate and prioritize actions based on their probabilities of success, guiding the GFlowNet to focus on trajectories that are both efficient and high-reward.
To integrate the Geometric distribution into GFlowNet's forward policy, the policy network outputs logits vector z = 22,.,,, for a given state s. This vector is transformed into probabilities p P via the softmax function. For each action a,, the probability of success modeled as a Geometric random variable X₃ Geometric(pi), where the selection of an action a" defined as:
a" = arg min X₃-
(8)
Here, the action with the fewest expected steps to success is prioritized, effectively capturing the reward distribution's dependency step length and enhancing the exploration process. This step-based probability assignment allows GFlowNet to balance between exploration and exploitation, focusing on high-reward trajectories while maintaining diversity.
Conversely, the backward policy retains the use of the Categorical distribution. While the Geometric distribution advantageous for exploration, its properties are less suited for reconstructing paths from the goal to the start state. The backward policy instead relies on the Categorical distribution to guide path regression, where capturing dependencies between actions is less critical.
Attempts to apply the Geometric distribution to the backward policy yielded suboptimal results, it failed to represent the necessary dependencies for effective path reconstruction. Therefore, the use of distinct distributions Geometric for forward exploration and Categorical for backward reconstruction is both empirically justified and theoretically grounded.
4 Experiments
4.1
Experimental Design
4.1.1
Research Questions
Our central goal is to explore whether Generative Flow Networks (GFlowNets) can be adapted to generate reasoning trajectories that are aligned with human problem-solving heuristics in the ARC domain. Since direct comparison with human trajectories is beyond current scope, instead assess alignment through structural properties inspired by human behavior (e.g., brevity, intentionality, loop avoidance). To this end, we pose the following sub-questions:
RQ1. Can reward functions informed by human priors promote human-aligned reasoning patterns? We assess how different reward shaping techniques (e.g., brevity encouragement, loop avoidance, trajectory regularization) influence the structure and quality of generated trajectorics.
RQ2. Does using a Geometric distribution in the forward policy enhance efficiency than conventional GFlowNett with a categorical forward policy in diverse trajectory generation? We evaluate whether modeling forward decisions via the Geometric distribution, which prioritizes concise action sequences, improves alignment with diverse reasoning behaviors.
RQ3. Why does a Geometric forward policy excel in goal-conditioned reasoning tasks? We compare -bckkardd policy combinations (G-C, C, G-G, C-G, C-C) to uncover how the inductive bias of Geometric forward policy, together with stabilizing backward policy, leads higher success and diversity under sparse rewards.
13

---

Published in Transactions on Machine Learning Research (09/2025)
4.1.2 Dataset: Task Selection Rationale
The dataset used in our experiments is based on the Abstraction and Reasoning Corpus (ARC), specifically locusing on Task 179 (Diagonal Flip) for most experiments. This lask exemplifies problems where the solution requires transformation of the entire grid, rather than localized subregions, making it suitable for evaluating solution augmentation in full--rri reasoning contexts.
Formally, the selected tasks satisfy:
f(xx) yk: S G, where E R
Here, denotes the selection region and G represents the entire grid with dimensions m n. By defining S G, each action operates the whole grid, requiring full-grid comprehension and transformation.
To further constrain the solution space and evaluate the impact of full-grid transformations, restrict our experiments to ARC problems solvable via entire-grid operations- such as flips, rotations, translations, and global color replacements. This dosign choice both practically and theoretically motivated. In the ARCLE environment (Lee et al., 2024), the default formulation allows any subset of the grid including individual pixels or subgrids to be transformed. However, this leads to an exponentially large action space of roughly O x 2 for a grid of size m n, where is the set of atomic actions and is the typical sequence length. For 30 30 grid, the subregion selection space alone reaches 2900, rendering effective exploration ractabbbe.. By instead focusing whole-grid transformations (i.e., S = G), we reduce the action space complexity to approximately (AA),, which not only enhances tractability but also aligns more closely with the structural reasoning patterns observed in human trajectories.
In addition to Task 179, other lasks solvable by whole-grid selection include: 53, 87, 129, 140, 150, 155, 241, and 380 are tested. These are shown appendix B.3
Task
Taak 179
Task 241
Demo
Pairs
Test
?
?
Input
Figure 6: Example of an Entire grid selection ARC problem
4.1.3 Evaluation Metrics
The following metrics are used evaluate the performance of GFlowNet:
Validation Accuracy (Val ACC): The proportion correctly generated trajectories among 100 attempts. A trajectory considered correct if it reaches the target solution while adhering the task constraints. This metric evaluates the model's ability to generate valid and successful trajectories efficiently.
Trajectory Diversity (Drra)): The fraction of unique successful trajectories among all generated unique trajectories. A higher diversity score indicates that the model explores multiple valid solutions
14

---

Published in Transactions on Machine Learning Research (09/2025)
rather than converging single mode. This reflects the model's ability to balance exploration and exploitation in trajectory generation.
Reward Distribution Diversity (Dreward): The variation in assigned rewards, measured using the Shannon Index. A higher value suggests that the model assigns rewards more evenly across different trajectories, encouraging diverse exploration. Conversely, a lower value indicates that rewards are concentrated on a limited set of Irajectories, leading to mode collapse.
4.1.4 Hyperparameters and Glossary
Below are Table ! which presents key hyperparameter settings, and Table 2 which defines key abbreviations used throughout the paper.
Hyperparameter
Value
Abbreviation
Definition
Learning Rate (Ir)
10-4
PF
Forward Probability
# of Actions
5
Pn
Backward Probability
Episode Length
10
TLR
Trajectory Length Regularization
Base Reward
15 (O), 0 (X)
DF
Discount Factor
Discount Factor (7)
0.9
cycle
Cycle Penalty Reward
Loss Weight (a:)
0.2
Splus
Submit plus Reward
Trajectory Regularization Weight
0.01
LTB
Trajectory Balance Loss
Table 1: Key Hyperparameter Settings
Table 2: Glossary of Terms and Abbreviations
4.2 Performance of Revised GFlowNet for Solution Augmentation
This section evaluates the effectiveness of proposed GFlowNet modification ions across the three research questions defined in Section 4.1 To qualitatively illustrate the generated trajectories discussed throughout our experiments, we present examples from Task 150 (ID: 7a3c6ac) and Task 179 (ID: 74dd
ARC problems included in our benchmark suite (Appendix B.3 These examples highlight the variety of transformation sequences leading to correct solutions.
Tra 1
7oy
Lif Acte
Lift Cal'
Left Jenan
Yrlp
Vile
Tey
YIN
Try'
Try 3
RgS: Kyeiion
LAT/CMM
Telp
LAT/OOM
Ta/
Tay'
(a) Task 150 (ID: 7a3c6ac)
(b) Task 179 (ID: 74dd1130)
Figure 7: Visualization of generated trajectories for two ARC tasks. Fach trajectory represents a sequence of transformations applied to the input grid achieve the correct output. Task 150 (a) and Task 179 (b) demonstrate diverse action sequences, including rotations and flips, leading to successful solutions.
15

---

Published in Transactions on Machine Learning Research (09/2025)
4.2.1 RQ1. Can reward functions informed by human priors promote human-aligned reasoning patterns?
To investigate whether human-ins inspired reward signals can effectively shape the structure generated trajectorics, we evaluate five reward models, each designed to encode specific reasoning heuristics:
Base: Sparse reward only (no inductive bias).
Splus: Emphasizes the Submit action.
Cycle: Penalizes repeated actions (loop avoidance).
DF: Discount factor encourages early goal completion.
TLR: Penalizes long trajectories (conciseness).
To isolate the pure effect of different reward components, these experiments were conducted on the standard Cooneetic-Calegoriiaal (G-C) policy using the base reward scale (a reward of 15 for success).
Results and Analysis We first analyze the elfects of individual reward models (Tables and 4 The Base model performs poorly, with only 5.26% unique success and reward entropy of 0.29, reflecting ineffective trajectory exploration. Among single reward settings, the model achieves the highest total success rate (89.33%) but with relatively low diversity (1.04), suggesting that its strong preference for short paths leads to early convergence
solution set.
TLR achieves lower total accuracy (67.33%) but the highest reward entropy (1.08), producing the most structurally diverse successful trajectories (50 in Table 3). This suggests that TLR allows for wider range of valid solution strategies while still guiding the model toward goal-oriented reasoning.
Velidation Accuracy Comparison with reward model
Validation Accuracy with Multiple Reward Model Combinations
Validation Accuracy
Validation Accuracy
Splus
Cycle
TLR
DF-CyderTLR
Cycle-TLR
DF+TUR
DFiCycle
Figure 8: (Left) Validation accuracy for individual reward models. (Right) Validation accuracy for combined reward models.
Cycle, while reflecting the human tendency to avoid repetition, generates the largest number of unique trajectories (100), but with relatively low success (32 successful), indicating that diversity alone does not imply effective reasoning. Splus, despite its alignment with the task-specific 'submit' action, shows limited impact (13.68% accuracy), implying that local task signals are insufficient without broader structural constraints.
We next examine reward combinations to assess whether structural biases interact synergistically (Tables 5 6 DF + Cycle + TLR achieves perfect total success (100%), but with reduced diversity (entropy = 1.81), suggesting a strong convergence effect. DF + TLR, in contrast, yields a balance of high success (86%) and the highest diversity among all models (2.96), highlighting its effectiveness preserving reasoning flexibility while maintaining performance.
16

---

Published in Transactions on Machine Learning Research (09/2025)
Reward
Total Unique Trajectories
Successful Unique Trajectories
Val ACC (%)
DF
27
75.47
Cycle
100
32
32.00
TLR
50
63.03
Base
4
5,26
Splus
95
13
13.68
Table 3: Success rate based on unique trajectories for different reward models.
Reward
Total Trajectories
Total Successful Trajectories
Val ACC (%)
Dreward
300
268
89.33
1.04
Cycle
300
96
32.00
0.90
TLR
300
67.33
1.08
Base
300
15
5,00
0.29
Splus
300
13.00
0.56
Table 4: Success rate and reward diversity based on total trajectories.
Notably, combinations that include Cycle often lead to more constrained trajectory spaces. For example, Cycle + TLR achieves 89% success but with the lowest diversity (0.92), reinforcing that while repetition avoidance is important, overly rigid enforcement may limit the model's exploration. These results support the notion that reward combinations do not simply stack effects but influence trajectory patterns in complex and often non-lincar ways.
Reward Combination
Unique Trajectorics
Succossful Unique Trajectorics
Val ACC (%)
DF + Cycle + TLR DF + TLR
100.00
35
26
74.29
Cycle TLR
38
29
76.32
DF + Cycle
29
61.70
Table 5: Success rate and reward diversity based on unique trajectories for reward combinations.
Reward Combination Total Trajectories
Successful Total Trajectories
Val ACC (%)
Dreward
DF TLR Cycle TLR
100
86
86.0
2.96
100
Cycle + TLR
100
89
89.0
0.92
DF + Cycle
100
91
91.0
1.50
Table 6: Success rate and reward diversity based total trajectories for reward combinations.
To assess the marginal utility of the cycle constraint and to more deeply understand how different reward components affect structural efficiency, we analyzed the total number of cycle occurrences and the average trajectory length for various reward models. The results are presented in Table 7
Reward Model
Total Cycle Occurrences Average Trajectory Length
Base
408
9.59
Cycle
308
8.70
DF + TLR
255
8.12
DF Cycle
246
8.15
Table 7: Comparison of reward models trajectory efficiency metrics. Total Cycle Occurrences redundancy, while Average Trajectory Length measures brevity. Lower is better for both.
17

---

Published in Transactions on Machine Learning Research (09/2025)
The results in Table reveal the distinct effects of different reward strategies on improving trajectory efficiency. Both approaches that add constraints- improve upon the Base and Cycle-only models.
on structure (DF + Cycle) or length (DF + TLR) significnnn
Notably, the DF + Cycle model proved to be the most effective at reducing structural redundancy, achieving the lowest total number of cycle occurrences (246). On the other hand, the DF + TLR model produced the shortest average trajectories (8.12), making it the most effective in terms of overall brevity. This presents an interesting trade-off. Given that the primary goal of this work is to reduce inefficient reasoning paths, the DF + Cycle model, which most effectively minimizes cyclic (i.e., redundant) actions, can be considered the most successful in improving the logical quality of the trajectories. This result clearly demonstrates the marginal utility of an explicit cycle constraint over relying on length penalties alone to indirectly reduce redundancy.
This detailed analysis also puts the model's overall performance into perspective. As observed in Table 7 the average trajectory lengths for our optimized models are approximately 8 stops. This contrasts with human-provides solutions for similar ARC lasks, which often require fewer than 4 steps on average (Kim) et al. 2025b This significant gap underscores the ongoing challenge and importance of reward shaping in steering models toward more concise, human-aligned reasoning structures.
These findings suggest that well-designed reward functions not only affect trajectory success but also influence the underlying structure and interpretability of the reasoning process. Individual rewards serve as proxies for distinct human problem-solving heuristics, and their combinations can shape exploration in either beneficial or restrictive ways.
Implications
While more constraints can increase success, they may also reduce diversity and generalization. For instance, the DF + TLR model demonstrates favorable trade-off between structure and flexibility, whereas overly rigid combinations like DF + Cycle + TLR lead to narrower solution modes. These results support the idea that reward shaping is not merely about scoring optimization, but about modeling the reasoning biases that underlie human-like solution strategies.
This tension between success-driving constraints and pranrstti-rmmotinggggg. flexibility is central to our investiga- tion. To dissoct this trado-off more gramularly and challenge the assumption that shorter paths are always superior, the following section provides deep dive into the most critical parameter governing this balance: the maximum trajectory length, which dictates the model's fundamental exploration depth.
4.2.2 The Role of Exploration Depth: Challenging the "Shorter is Better" Heuristic
The preceding analysis of reward functions highlights the importance of trajectory length in guiding the GFlowNet. While mechanisms like DF and TLR encourage conciseness, a crucial question remains: what is the optimal exploration depth? To investigate this, and to challenge the simple assumption that shorter trajectories are always superior, we conducted detailed analysis of the impact of varying the maximum episode length.
Experiment Setup The action space was fixed at 10 actions, and we varied the episode Length to observe how each setting influenced the model's performance. The episode Lengths tested were 4, 5, 10, 20, and 50. This setup allowed us to assess how different step limitations impact solution quality, exploration diversity, and learning efficiency.
Results and Analysis Our findings reveal a non-monotonic relationship between trajectory length and performance, as shown in Figure g While very short episodes (e.g., 4 or steps) severely limited the model's 's ability to explore and resulted in low success rates, detailed in Table g performance peaked at moderate episode length of 20. This optimal length achieved the highest validation accuracy and 97% total success rate. However, further increasing the exploration depth to 50 steps proved detrimental, causing the learning process to become unstable and fail entirely. This demonst that an overly large search space can dilute the learning signal, preventing the model from converging on valid solutions.
As shown in Table g this performance peak at an episode length of 20 also coincides with the highest diversity of solutions. The model generated 74 unique trajectories, 71 of which were successful, and exhibited the
18

---

Published in Transactions on Machine Learning Research (09/2025)
highest reward distribution diversity (Dreward 2.14). This indicates that sufficient, but not excessive, exploration depth is essential for discovering a wide variety of high-quality solutions.
Validacion Accuracy Comparison with Different Number Actione
Reward Trends DiPforent
07
42
309 1986
Strp 2000
Sxp
Figure 9: (Left) Performance comparison across episode length, showing the impact of action count validation accuracy over training steps. (Right.) Reward comparison: comparison of reward values over training steps, highlighting differences in reward trends.
File
Unique Trajectories
Successful Unique Trajectories
Val ACC (%)
reward
EP
100.00
EP
76
5.26
0.20
EP 10
56
40
71.43
0.88
EP 20
74
71
95.95
2.14
EP 50
0.00
0.0
Table 8: Trajectory success rates and reward distribution diversity for various episode lengths and reward
settings.
File
Total Trajectories
Successful Total Trajectories
Val ACC (%)
EP
100
100
100.0
EP
100
5.0
EP 10
100
59
59.0
EP 20
100
97
97.0
EP 50
100
0.0
Table 9: Total success rates for different episode lengths and reward configurations.
These results provide crucial insight: the highest performance and diversity are achieved not through strict length minimization, but at moderate trajectory length that balances efficiency with sufficient 'exploration budget.' This finding directly informs our interpretation of the results from RQ1, showing that. length-penalizing rewards should be seen as guides against. inefficiency rather than tools for absolute minimization. Furthermore, it sets a crucial context for the following analysis of our Geometric policy, highlighting that its efficiency bias is most powerful when it does not prematurely curtail necessary exploration.
Implications
4.2.3 RQ2. Does using Geometric distribution in the forward policy enhance efficiency in diverse trajectory generation?
We compare GFlowNet variants that differ only in the choice forward policy distribution: the original version using Categorical distribution, and the revised version using Geometric distribution. In both cases,
19

---

Published in Transactions on Machine Learning Research (09/2025)
the backward policy (PB) remains Categorical. To provide clean baseline comparison of the policies' innate exploratory behaviors, both variants were trained with the simple sparse (Base) reward function at the base reward scale. This setup allows us to evaluate the pure effect of the policy distribution, without confounding factors from a complex reward structure.
Validation Accuracy: Categorical Geometric Distributions
Reward Trend: Catagorical vs. Geometric Distributions
Geometric Categorical
Geometric (ours)
Categorical
158
Validation Accuracy
Roward
Stop
Figure 10: (Left) Validation accuracy comparison between Categorical (green) and Geometric (bluc) distribu-
tions. (Right) Reward profiles comparison over training steps.
Distribution
Unique Trajectories Successful Unique Trajectories
Val ACC (%)
Categorical
10
13.33
Geometric
45
36
80.00
Table 10: Success rate based on unique trajectories for Categorical and Geometric distributions.
Distribution
Total Trajectories Successful Total Trajectories
Val ACC (%)
Dreward
Categorical
100
28
0.28
3.38
Geometric
100
0.89
0.88
Table 11: Success rate based on total trajectories for Categorical and Gcometric distributions.
Results and Analysis The Geometric PF GFlowNet introduces an inductive bias toward concise and decisive action sequences- an intuition derived from analyzing human trajectory patterns, particularly structured reasoning lasks like ARC. Unlike the original Categorical forward policy, which samples actions uniformly, the Geometric distribution prioritizes earlier actions, naturally biasing the model toward shorter trajectorics.
As shown in Figure 10 the Geometric PF GFlowNet consistently achieves higher validation accuracy throughout training. Table 10 confirms that, while the Categorical policy produces a larger number of unique trajectories (75 vs. 45), it leads to significantly fewer successful outcomes (10 vs. 36). Likewise, Table 11 shows that the Geometric model outperforms in overall success rate (89% vs. 28%), despite exhibiting lower reward diversity (Dreward = 0.88 vs. 3.38).
This contrast highlights that, although the Categorical policy exhibits broader surface-level exploration, much of it fails to reach valid solutions. In other words, diversity without effectiveness may not equate to useful reasoning. In contrast, the trajectories generated by the Geometrie PF GFlowNet are more likely to be both correct and structurally sound. Notably, successful trajectories under the Geometric policy typically reach the goal within to 4 steps-on well below the episode limit of 10 demonstrating the model's inherent preference for efficient reasoning paths.
These paths exhibit key characteristics commonly associated with human problem-solving behavior: minimal reversals, consistent transformation patterns, and clear subgoal alignment. Although do not claim
20

---

Published in Transactions on Machine Learning Research (09/2025)
full cognitive equivalence, this emergent structure indicates meaningful alignment between the model's exploratory behavior and human-like reasoning strategies.
Implications These results suggest that leveraging distributional biases such as those imposed by the Geometric forward policy-can can steer solution augmentors toward generating concise and interpretable reasoning paths. While the diversity of the Categorical model appears greater numerically, the Geometric PF GFlowNet better balances success and efficiency, especially sparse-reward scttings like ARC.
Moreover, the fact that this model can discover multiple valid reasoning paths for ARC-179 task solvable in just few steps-suggent that even simple inductive biases facilitate structured solution generation. When combined with human-inspired reward functions (Section 4.2.4), this opens the possibility of extending the framework to more complex reasoning domains. Rather than replicating human reasoning in its entirety, the proposed approach reflects an early but promising step toward learning structural features expert reasoning such as brevity, decisiveness, and goal alignment- and embedding them within generative solution frameworks.
4.2.4 RQ3. Why does Geometric forward policy excel in goal-conditioned reasoning tasks?
Having established the superior G-C policy (in Section 4.2.3) and identified effective reward components (DF Cycle in Section 4.2.1 we now evaluate their final, combined effect. For this main experiment, compare all policy combinations using the DF + Cycle reward scheme with the 15x amplified reward scale (as justified in Appendix B.1 We evaluate different. combinations of forward (Pp) and backward (PB) policies:
G-C: Geometric for PF and Categorical for PB.
G-G: Geometric for both Pp and PB.
C-C: Categorical for both Pp and PB.
C-G: Categorical for Pp and Geometric for PB-
Validation Accuracy with different PF-PB Combination
GC
0.8
Validation Accuracy
5000
10000
15000
20000
25200
30300
Step
Figure 11: Validation accuracy for different PP-PE combinations.
21

---

Published in Transactions on Machine Learning Research (09/2025)
Pp-PB
Unique Trajectories
Successful Unique Trajectories
Val ACC (%)
G-G 21
18
85.71
G-C
56
49
87.50
C-G
59
16.95
C-C
75
10
13.33
Table 12: Unique trajectory performance for different PF-Pn combinations.
Pę-PB
Total Trajectories
Successful Total Trajectories
D Dreward
G-G
100
97
2.15
G-C
100
97
2.09
C-G
100
11
0.73
C-C
100
10
3.39
Table 13: Overall success rates for different. Pp-Pn combinations.
Results and Analysis We compare four configurations of forward (Pp) and backward (Pn) policies. Both G-G and G-C achieve the highest total success rate (97.0%), but the G-C pairing yields a significantly higher number of successful unique trajectories (49 vs. 18), indicating better structural gencralization. C-G and C-C perform poorly, both in accuracy and trajectory success.
This pattern suggests that the forward and backward policies contribute differently to learning dynamics. The forward policy (Pp) is responsible for generating exploratory trajectories, while the backward policy (PB) adjusts the reverse probabilities to match the desired flow based on reward signals. When both policies are heavily biased (c.g., G-G), the backward policy may no longer act as a corrective mechanism. Instead, it may reinforce the forward bias, narrowing the trajectory space and reducing diversity.
In contrast, the G-C setup preserves high success while also producing wider range of successful solutions. This indicates that an asymmetric policy design-where Pp is biased for efficient exploration and P3 remains uniform to stabilize flow alignment. achieves better reasoning diversity without sacrificing performance. The observed D Dreward values support this: G-G has slightly higher entropy (2.15) than G-C (2.09), but the diversity of correct solutions is much higher G-C.
We also observe that C-C has the highest reward entropy (3.39) but fails to achieve meaningful performance, indicating that reward distribution diversity alone does not imply reasoning quality. High entropy may reflect exploratory breadth, but it must be interpreted alongside success metrics. Ultimately, reward diversity without trajectory effectiveness may signal unstructured exploration, not reasoning robustness.
Insights These findings reinforce the importance of decoupling exploration and correction roles in bidi- rectional learning. The forward policy drives goal-directed trajectory generation, while the backward policy should facilitate stable reward-aligned flow learning. We hypothesize that if both policies impose strong, aligned biases (as in G-G), the resulting feedback loop can distort the reward structure and impair the model's ability to generalize across diverse solutions.
Moreover, successful unique trajectory count emerges as valuable proxy for structural reasoning diversity. Models such as G-C not only succeed frequently but also do so via a wider variety of valid solution paths. This suggests that asymmetric policy pairing enables the model to learn broader reasoning structures key capability in ARC-like tasks where flexible generalization crucial.
Finally, reward distribution diversity, as captured by entropy, offers useful information about the model's exploratory behavior, but does not alone indicate reasoning effectiveness. Reward entropy should be interpretod in conjunction with success metrics to evaluate the utility- not. just the scope- of the model's exploration.
22

---

Qualitative Insights. The fine-tuned model learns to generate syntactically valid DSL expressions such as [rotate, xor, polormmask submit] rather than vague or irrelevant text (e.g., 'Let's analyze the pattern...") This indicates that the model acquired procedural understanding rather than just output Published in Transactions on Machine Learning Research (09/2025)
4.3 Downstream Evaluation on Large Language Models
To assess whether GFlowWeel-generatee trajectories provide langible utility in reasoning models, conduct downstream evaluation using LLM. Our objective is to determine whether exposure to these synthetic trajectorics can teach an LLM to reason through ARC-style tasks using DSL-based action sequences.
Experimental Setup. We fine-tune LLaMA 3.1 Instruct, a powerful language model, on approxi- mately 10k GFloweoot-eenerated trajectorics per ARC task. The target output is the complete action sequence required transform the input into the correct output. This is stricter evaluation than final oulput prediction, it tests whether the model has internalized procedural reasoning in DSL format.
Dataset and Tasks. We select 7 ARC tasks of varying difficulty and structure, covering geometric transformations, color-based logic, and compositional reasoning. Each task contains a mixture of high-level and low-level operations. For each task, 100k trajectories were generated, and 10k were sampled uniformly for fine-tuning.
Results. The results in Table 14 demonstrate a clear performance improvement. The baseline model fails all tasks, generating only natural language or malformed DSL. The fine-tuned model, however, correctly solves two lasks and produces valid DST in several others. Qualitalively, the model demonstrates awareness of transformation steps such as rotation, reflection, and conditional masking.
Table 14: Downstream performance of LLaMA 3.1 8B Instruct on ARC tasks before and after fine-tuning with GelowNNe--generalee trajectories.
Task # (ID)
Baseline (Pretrained) Tine-Tuned w/ GFlowNet Aug.
(3c910459)
140 (6150a2bd)
150 (67a3c6ac)
X
155 (68b16354)
X
X
179 (74dd1130)
X
241 (9dfd6313)
x
X
380 (ed36cef7)
X
Overall Accuracy
0 (0%)
2 7 (28.6%)
matching. Notably, the two successful tasks involve spatial transformations, suggesting that GFlowNet augmentation helped capture geometric reasoning structures.
Implications. These findings provide strong proof-of-concopt evidence that GFlowWeet-cceerate trajecto- ries can serve adequate supervision for training reasoning-capabll LLMs. Even in this limited experiment, trajectory-level augmentation enables the L.M to generalize in domains where it previously failed. This sup- ports the broader thesis that generating diverse, structured reasoning traces can improve model generalization and interpretability.
4.4 Comparison to Trajectory Augmentation Baseline
While many augmentation strategies exist for input-one output pairs in vision and NLP tasks, there is no established baseline for augmenting solution trajectories in ARC. Existing efforts such as ReARC (Hodel 2024) primarily augment the input grids, associating cach augmented instance with single canonical solution. However, this style of augmentation does not capture the diversity of valid reasoning paths that can lead the same outcome. Consequently, adopt ReARC-style augmentation proxy baseline for comparison.
23

---

Published in Transactions on Machine Learning Research (09/2025)
Experimental Setup. To fairly evaluate the utility of our framework, compared LLMs fine-tuned two distinct forms of augmented data, while keeping all model and training configurations fixed. Both models used the LLaMA 3.1 8B Instruct backbone and were fine-tuned with identical LoRA settings (r=16, a= 32). 32)..
ReARC-style Augmentation: ~70,000 examples were generated via input grid perturbations, each paired with a single canonical solution sequence (e.g,, a rotation or reflection). This provides high variability in the input space but minimal variability in solution paths.
GFlowNeet-based Augmentation: 100,000 trajectories were synthesized across ARC problems using our framework. Unlike the baseline, these data contain multiple diverse trajectories per task, better reflecting the exploratory nature of ARC reasoning.
Results. The AelloWeet-uugmeetted model solved 2 of 7 tasks (28.6%), specifically Problem 140 (6150a2bd) and Problem 179 (74dd1130). By contrast, the ReARC baseline solved only task (14.3%), despite being trained on seven times more examples. Morcover, the bascline consistently produced the same action sequence in of 7 (asks, indicating severe overfitting to canonical solutions. In contrast, the GFlowNet-trained model generated more diverse set of valid sequences, enabling it to generalize to problems requiring geometric transformations.
Table 15: Downstream performance of LLaMA 3.1 8B Instruct ARC tasks with ReARC-style GFlowNet-based augmentation.
Task # (ID)
ReARC-style Aug.
GFlowNet Aug.
87 (3c9b0459)
140 (6150a2bd)
150 (67a3c6ac)
155 (68b16354)
X
179 (74dd1130)
241 (9dfd6313)
X
X
380 (ed36ccl7)
X
X
Overall Accuracy
/ 7 (14.3%) / (28.6%)
Analysis. This experiment shows that diversity in solution paths more valuable than diversity in input grids. While the ReARC baseline introduced many surface-level variations, its reliance single fixed solution sequence limited both its learning signal and generalization. In contrast, our GFlowNet framework, even with significantly fewer examples, exposed the model to rich intermediate states and trajectory structures, enabling it to learn the process of problem solving rather than memorizing outcomes.
Conclusion. This controlled comparison highlights that trajectory-level augmentation, which enriches the diversity of solution paths, can outperform input-driven baselines in ARC. By generating diverse and meaningful solution trajectories, our framework doubled downstream accuracy compared to the canonical augmentation approach, despite using fewer data. These findings strengthen the case for trajectory-level augmentation a more effective training signal for reasoning-intensive intensive tasks such as ARC.
5 Conclusion
In this study, we proposed novel GFlowNeet-bssed framework for solution augmentation ARC problems. By leveraging Geometric forward policy, Categorical backward policy, and human-inspired reward models, method discovers structurally efficient and diverse solution trajectories in sparse-reward environments.
24

---

Published in Transactions on Machine Learning Research (09/2025)
5.1 Summary and Implications
Our experiments demonstrate that:
The combination of Geometric forward policy (Pş) with human- inspired reward shaping (discounting, length regularization, and cycle penaltics) substantially improves success rates over categorical baselines, showing that efficiency-baseed heuristics and structural guidance are critical for effective learning.
Policy and Reward Design:
Our analysis reveals that the best performance and diversity emerge at moderate trajectory lengths, not strict minimization. Moreover, pairing a Geometric Pr with Categorical backward policy (Pg) balances directed exploration and flow correction, yielding both high accuracy and diverse successful trajectories.
Exploration-Polic Trade-off:
Downstream Validation: Fine-tuming LLaMA 3.1 8B with our GF GFlooNee--geeertee trajectories led to a significant improvement (0%-+2.6%) In controlled experiment, our 10k diverse trajectorics outperformed 70k ReARC-style input-augmented examples (28.6% vs. 14.3%), highlighting the superiority of trajectory diversity over mere input augmentation.
Efficiency and Test-Time Augmentability: Our framework can generate over 100k trajectories per task in under hour single GPU (2,000/min). This logarithmic yield of unique solutions enables efficient augmentation without redundancy and makes real-time, test-time augmentation feasible.
Practical Significance: Rather than claiming cognitive alignment, our findings show that GFlowNet can be guided toward syntactic features of efficient problem solving brevity, structure, and di- versity-throughe policy and reward design. This positions GFlowNet as a scalable, expert-aligned solution generator for reasoning- intensive tasks like ARC.
Rather than claiming full cognitive alignment, work shows that GFlowNet can be guided to reflect aspects of efficient problem-solving- such as brevity and structural barity-throue reward design. These findings suggest that carefully structured inductive biases (i.e., policy and reward) can guide GFlowNet toward expert-aligned solution augmentation, highlighting its utility for complex reasoning tasks such as ARC.
5.2 Limitations and Future Work
While our proposed framework yields promising results, several limitations remain:
On-Policy Training Sensitivity: The current approach relies on on-policy training, which is sensitive to the quality of initial trajectory samples. Early poor samples may cause high variance in learning. Future work will explore hybrid training schemes that combine the robustness of off- off-policy updates with the stability of on-policy guidance.
Restricted Task Scope: This study focuses on ARC tasks involving full-grid transformations. Extending the framework handle tasks that require localized sub-grid manipulations poses a significant challenge due the exponentially growing action space. We plan to address this by leveraging ARCLE (Tee et al., 2024) and developing submodules capable of supporting fine-grained selection and transformation operations.
Adaptive Exploration Depth and Reward Weighting: Our findings highlight the critical role of episode length. The current framework uses a fixed length, but future work should explore adaptive methods for determining the optimal exploration depth dynamically based on task complexity. Fur- thermore, as natural extension, exploring adaptive reward weighting and training-phase scheduling could help to dynamically manage the trade-offs between different reasoning heuristics in more complex open-ended domains.
25

---

Published in Transactions on Machine Learning Research (09/2025)
5.3 Final Remarks
Our work demonstrates the polential of GelowNet a powerful mechanism for automated solution augmen- tation in reasoning- intensive tasks. By incorporating numan-inspired reward structures and asymmetric policy design, we guide the model to gencrate conciso, diverse, and high-quality solutions an essential capability in sparse-reward environments.
inseesie
Rather than functioning purely as a solver, GFlowNet here serves as an expert-aligned solution generator. This opens opportunities for integrating our approach into broader reasoning pipelines, where diverse solution candidates required for further verification, ranking, learning. Our framework's scalability and downstream effectiveness position it as a practical component in such pipelines, especially under compute- limited test-time settings. We believe that future advances in exploration strategies, structural bias modeling, and downstream integration will further enhance the applicability of GFlowNet to Dystem-2-leves tasks, ultimately contributing to more general and interpretable artificial intelligence systems.
Acknowledgements
This work was supported by NRF (Reinforcement Learning-Based Program Synthesis Techniques for Solving Abstract Visual Reasoning Problems by Identifying Patterns and Combining Concepts: RS- 2024-0045111,,,, Developing Abstraction and Reasoning Capability for AI Model; 25-22024-04444000) ITTP (Enhancing AT Model Reliability Through Domain-Specific Automated Value Alignment Assessment; RS- 4SS-2224004445087, Development of Artificial Complex Intelligence for Conceptually Understanding and Inferring like Human; 0002000,,,,,,,, and GIST (KH0330, Future-leading Specialized Research Project) grants funded by the Ministry of Science and ICT, Korea.
References
Samuel Acquaviva, Yewen Pu, Marta Kryven, Theodoros Sechopoulos, Catherine Wong, Gabrielle Ecanow, Maxwell Nye, Michael Tessler, and Joshua B. Tencnbaum. Commuicating Natural Programs to Humans and Machines. In NewrIPS Dalasels and Benchmarks, 2022.
Emmanuel Bengio, Moksh Jain1, Maksym Korablyov, Doina Precup, and Yoshua Bengio. Flow Network Based Generative Models for Non-Tterative Diverse Candidate Generation. In NewrIPS, 2021.
Yoshua Bengio. System 2 Deep Learning: Higher-Level Cognition, Agency Out-of-Distribution Generalization, and Causality. Invited talk at LJCAL, 2021. https://ijcai 211. wtt.:caaii...oooiiiieaaee arg/invited-talkes
Yoshua Bengio, Salem Lahlou, Tristan Deleu, Edward J. Hu, Mo Tiwari, and Emmanuel Bengio. Nel. Foundations. Journal of Machine Learning Research, 2023.
Kiril Bikov, Mikel Bober- and Soumya Banerjee. AugARC: Augmented Abstraction and Reasoning Benchmark for Large Language Models. In AAAI Workshop, 2025.
François Chollet. On the Measure of Intelligence. 1ix::191100152 2019.
François Chollet. OpenAl o3 Breakthrough High Score on ARC-AGI-Pub, 2024. URL https://arcprizz http:p///oaai----ppp--bbbratttta
François Chollet, Mike Knoop, Greg Kamradt, Walter Reade, and Addison Howard. ARC Prize 2025: Competetion Rules, 2025. URL https://www http:///wwwwk.gggg..ee kaggle. httpa///eooooootiiiiii///rrrrrra 2025/
Karl Cobbe, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training Verifiers to Solve Math Word Problems. arXiv:2110.14168, 2021.
DeepSeek-AL Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Poiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang
26

---

Published in Transactions on Machine Learning Research (09/2025)
Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang. Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingehuan Zhang, Minghua Zhang, Minghui Tang. Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, and S. S. Li. DeepSeck-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501, 12948, 2025.
Jean- Pien-e Falet, Hac Boom Lee, Nikolay Malkin, Chen Sun, Dragos Socrieru, Thomas Jiralerspong. Dinghuai Zhang, Guillaume Lajoie, and Yoshua Bengio. Delta-Al: Local objectives for Amortized Inference in Sparse Graphical Models. In ICLR, 2024.
Piotr Gaifskii Michal Koziarski, Krzysztof Maziarz, Marwin Segler, Jacek Tabor, and Marek Smieja. Diverse and Feasible Retrosynthesis Using GelowNets. Information Sciences, 714(C):122194, 2025.
Jose Orralndez-Orallo and Neus Minaya-Collado. A Formal Definition of Intelligence Based on an Intensional Variant of Algorithmic Complexity. In EIS, 1998.
Michael Hodel. Addressing the Abstraction and Reasoning Corpus via Procedural Example Generation. arXiv:2444.07353 2024.
Edward Hu, Moksh Jain, Eric Elmoznino, Younesse Kaddar, Guillaune Lajoie, Yoshua Bengio, and Nikolay Malkin. Amortizing Intractable Inference in Large Language Models. In ICLR, 2024.
Marcus Hutter. Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Springer, 2005.
Moksh Jain, Emmanuel Bengio, Alex-Hernandez Garcia, Jarrid Rector-Brooks, Bonaventure F. P. Dossou, Chanakya Ekbote, Jie Fu, Tianyu Zhang, Micheal Kilgour, Dinghuai Zhang, Lena Simine, Payel Das, and Yoshua Bengio. Biological Sequence Design with GFlowNets. In ICML, 2022.
Moksh Jain, Sharath Chandra Raparthy, Alex Hernandez- Garcia, Jarrid Rector-Brooks, Yoshua Bengio, Santiago Miret, and Emmanuel Bengio. Multi-Objective GFlowNets. In ICML, 2023.
Hyosoon Jang, Minsu Kim, and Sungsoo Ahn. Learning Energy Decompositions for Partial Inference in GFlowNets. In ICLR, 2024.
Yuxuan Jiang, Zihan Wang, Haozhuo Jiang, Jipeng Zhang, and Zhiyuan Liu. Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing. In EMNLP, 2024.
Greg Kamradt. ARC-AGI-2 + ARC Prize 2025 is Live!, 2025. URL https://aroprizz..rgg hrtp:/prrizee.r/blle.. blog/
Minsu Kim, Sanghyeok Choi, Taeyoung Yun, Emmanuel Bengio, Leo Feng, Jarrid Rector-Brooks, Sungsoo Ahn, Jinkyoo Park, Nikolay Malkin, and Yoshua Bengio. Adaptive teachers for amortized samplers. In ICLRR 2025a.
Sejin Kim and Sundong Kim. System-2 Reasoning via Generality and Adaptation. In NewrIPS Workshop, 2024.
Sejin Kim, Hosung Lee, and Sundong Kim. Addressing and Visualizing Misalignments in Human Task-Solving Trajectories. In KDD, 2025b.
Yunho Kim, Jachyun Park, Heejun Kim, Sejin Kim, Byung-Jun Lee, and Sundong Kim. Diffusion-Baan Basadd Offline RL for Improved Decision-Making in Augmented ARC Task. arXin:2410 11324, 2024.
27

---

Published in Transactions on Machine Learning Research (09/2025)
Hosung Lee, Sejin Kim, Seungpil Lee, Sanha Hwang, Jihwan Lee, Byung-Jun Lee, and Sundong Kim. ARCLE: The Abstract and Reasoning Corpus Learning Environment for Reinforcement Learning. In CoLLAs, 2024.
Seungpil Lee, Woochang Sim, Donghyeon Shin, Wongyu Seo, Jiwon Park, Seokki Lee, Sanha Hwang, Sejin Kim, and Sundong Kim. Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpus. ACM Transactions Intelligent Systems and Technology, 2025.
Shane Legg and Marcus Hutter. Universal Intelligence: A Definition of Machine Intelligence. arXic:0712.3329 2007.
Solim LeGris, Wai Keen Vong, Brenden M Lake, and Todd M Gureckis. H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmark. or arXin:2409001377 2024.
Ailor Lewkowyez, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving Quantitative Reasoning Problems with Language Models, In NewrIPS, 2022.
Wen-Ding Li, Keya Hu, Carter Larsen, Yuqing Wu, Simon Alford, Caleb Woo, Spencer Dunn, Hao Tang, Michelangelo Naim, Dat Nguyen, et al. Combining Induction and Transduction for Abstract Reasoning. ICLR, 2025.
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Ecclos, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d' d'Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals. Competition-Level Code Generation with AlphaCode. Science, (6624)))) 1097, 2022.
Hunter Lightman, Vineet, Kosaraju, Yura Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's Verify Step by Step: Improving Mathematical Reasoning with Process Supervision. In ICLR, 2024.
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems. In ACL, 2017.
Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, and Juanzi Li. Pairwise RM: Perform Best-of-N Sampling with Knockout Tournament. orXio:2501.13000 2025.
Zimu Lu, Aojun Zhou, Houxing Ren, Ke Wang, Weikang Shi, Junting Pan, Mingjie Zhan, and Hongsheng Li. MathGenie: Generating Synthetic Data with Question tackstranslttion for Enhancing Mathematical Reasoning of LLMs. In ACL, 2024.
Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lci Meng, Jiao Sun, and Abhinav Rastogi. Improve Mathematical Reasoning in Language Models by Automated Process Supervision. arXiv:2406.000 06592, 2024a.
Ziyang Luo, Can Xu, Pu Zhao, Qingleng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. WizardCoder: Empowering Code Large Language Models with Evol-Instruct. Tn ICLR, 2024b.
Matthew V. Mahoney, Text Compression as a Test for Artificial Intelligence. In AAAI, 1999.
Nikolay Malkin, Moksh Jain, Emmanuel Bengio, Chen Sun, and Yoshua Bengio. Trajectory Balance: Improved Credit Assignment in GFlowNets. In NewrIPS, 2022.
Meredith Ringel Morris, Jascha Sohl-Dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Alcksandra Fanst, Clement Farabet, and Shane Legg. Position: Levels of AGI for Operationalizing Progress on the Path AGI. Tn ICML, 2024.
28

---

Published in Transactions on Machine Learning Research (09/2025)
Arseny Moskvichev, Victor Vikram Odouard, and Melanie Mitchell. The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domain. Transactions on Machine Learning Research, 2023.
William of Ockham. Philosophicol Wrilings: Selection. Hackett Publishing Company, 1990.
OpenAL. Learning Reason with LLMs. http:://pppaa..cooo///////aaaaaa 2024.
OpenAl. OpenAl o3-mini. attpe:///ppaaa.cco///ddde///paap
2025.
Ling Pan, Dinghuai Zhang, Aaron Courville, Longbo Huang, and Yoshua Bengio. Generative Augmented Flow Networks. In ICLR, 2023.
Marcin Sendera, Minsu Kim, Sarthak Mittal, Pablo Lemos, Luca Scimeca, Jarrid Rector-Brooks, Alexandre Adam, Yoshua Bengio, and Nikolay Malkin. Improved Off-Policy Training of Diffusion Samplers. In NewrIPS, 2024.
Seonghwan Sco, Minsu Kim, Tony Shen, Martin Ester, Jinkyoo Park, Sungsoo Ahn, and Woo Youn Kim. Generative Flows on Synthetic Pathway for Drug Design. In ICLR, 2025.
Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. Synthetic Prompling: Generating Chain-of- Chain-of-Thought Demonstrations for Large Language Models. In ICML, 2023.
Shuaijie She, Junxiao Liu, Yifeng Liu, Jiajum Chen, Xin Huang, and Shujian Huang. R-PRM: Reasoning- Driven Process Reward Modeling. arvi:22503.2 0ate2503.21295, 2025.
Max W. Shen, Emmanuel Bengio, Ehsan Hajiramezanali, Andreas Loukas, Kyunghyun Cho, and Tommaso Biancalani. Towards Understanding and Improving GelowNet Training. In JCML, 2023.
Suyeon Shim, Dohyun Ko, Hosung Lee, Seokki Lee, Doyoon Song, Sanha Hwang, Sejin Kim, and Sundong Kim. O2ARC 3.0: A Platform for Solving and Creating ARC Tasks. In IJCAI Demo, 2024. URL
https://o2arc.com com
Noah Shinn, Beck Labash, Ashwin Gopinath, Ishita Dasgupta, Auca D. Dragan, and Trevor Darrell. Reflexion: Language Agents with Verbal Reinforcement Learning. In NeurIPS, 2023.
Kashun Shum, Shizhe Diao, and Tong Zhang. Automatic Prompt Augmentation and Selection with Chain-of- Thought from Labeled Data. In EMNLP, 2023.
Simon Strandgaard. ARC-Interactive -listtoy--aaaaaeee 2024. URL httpss//gittaab..cm/mnneeey
Yuxuan Wan, Wenxuan Wang, Yiliu Yang, Youliang Yuan, Jen-tse Huang, Pinjia Hc, Wenxiang Jiao, and Michael Lyu. LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models. In EMNLP, 2024.
Ke Wang and Zhendong Su. Automatic Generation of Raven's Progressive Matrices. In IJCAI, 2015.
Xuczhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Sell-Consistency Improves Chain of Thought Reasoning in Language Models. In ICLR, 2023.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of- Chauu-of-Thought Prompting Elicits Reasoning in Large Language Models. In NewrIPS, 2022.
Chenxi Whitehouse, Monojit Choudhury, and Alham Fikri Aji. LLM-powered Data. Augmentation for Enhanced Cross-lingual Performance. In EMNLP, 2023.
Zhengxuan Wu. AacyARC, 2021. URL httpps://gttuu...o///raaaggggggt
Shunyu Yao, Dian Yu, Zhao Jeffrey, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. In NewrIPS, 2023.
29

---

Published in Transactions on Machine Learning Research (09/2025)
Fangxu Yu, Lai Jiang, Haoqiang Kang, Shibo Hao, and Lianhui Qin. Flow of Reasoning: Efficient Training of LLM Policy with Divergent Thinking. arXin:2466.05077 2024.
Eric Ze,ikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. STaR: Self Taught Reasoner Bootstrapping Reasoning with Reasoning. In NewrIPS, 2022.
David W. Zhang, Corrado Rainone, Markus Peschl, and Roberto Bondesan. Robust Scheduling with GFlowNets. ICLR, 2023a.
Dinghuai Zhang, Nikolay Malkin, Zhen Liu, Alexandra Volokhova, Aaron Courville, and Yoshua Bengio. Generative Flow Networks for Discrete Probabilistic Modeling. In ICML, 2022.
Dinghuai Zhang, Hanjun Dai, Nikolay Malkin, Aaron Courville, Yoshua Bengio, and Ling Pan. Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets. In NewrIPS, 2023b.
Yufeng Zhang, Xuepeng Wang, Lingxiang Wu, and Jinqiao Wang. Enhancing Chain of Thought Prompting Large Language Models via Reasoning Patterns. In AAAI, 2025a.
Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The Lessons of Developing Process Reward Models in Mathematical Reasoning. arziv:2501.0733 2025b.
30

---

Published in Transactions on Machine Learning Research (09/2025)
A Extended Background Details
A.1 GFlowNet Training Mechanism and Flow Matching
Flow Matching Condition. GFlowNets maintain a flow F(s' + s) for each directed edge (5',3) in a Directed Acyclic Graph (DAG), ensuring no probability mass lost or gained within each state:
F -1) = Σ F(s
(10)
where F(s' + s) is the flow from state s s. This condition (Equation 10 ensures that each state's inflow equals its outflow. In practice, F(s' + 8) is factored through forward policy Pr(s |s)).
Trajectory Balance (TB) Loss. Building on Flow Matching, the TB loss (Malkin et al. 2022 global consistency between forward and backward paths:
ensures
z II Pr(st st-1) = R(x) II PB(s1-11 st);
(11)
where Z is trainable constant and R(x)) the reward function for trajectory x. Minimizing:
a
(TB(O) [og Zo +
Σ log Pr(se 1)-1; log R(x)
Σ
log PB(st-1 sc: 0)]
(12)
aligns the probability of forward-sampled trajectories with their corresponding reward- sewarr-proportion. flows.
Training Steps. We outline the GFlowNet training procedure:
Forward Sampling: From initial state so, sample actions a, PF(- st-1). Collect the resulting trajectory (8))
Reward Computation: Evaluate R(x), e.g. if it solves an ARC puzzle, else 0.
Backward Sampling: Use PB to reconstruct or partially revisit states from 8n to 8p-
4. TB Loss Computation: Compute LTB via Equation equation 12
5. Parameter Update: Optimize 0 to minimize LTB, adjusting both Pp and Pn accordingly.
By iterating these steps, GFlowNets learn to focus on high-reward trajectorics while maintaining diverse distribution of solutions.
A.2 Extended ARC Details
Although this work focuses on whole-grid transformations, many ARC puzzles require partial selection (e.g. coloring only a sub-region). This drastically increases the action space because selecting subsets of a 30 x 30 grid can be combinatorial. We note:
Dataset Complexity: Some tasks have grid dimensions smaller than 30 x 30, but the upper limit still poses a challenge.
Potential Approaches for Partial-Selection: We can adopt hierarchical policies that first select a region, then transform it. This approach can be integratod into GetowNets by factoring the forward policy into multiple steps (Appendix of (Lee et al. 2024
Sparse Rewards and OOD Issucs: ARC test grids often deviate from training grids (different shapes, new patterns), demanding robust generalization. GFlowNets' ability to maintain multiple solutions is beneficial, but design of the reward function must handle rarely-seen corner cases.
31

---

Published in Transactions on Machine Learning Research (09/2025)
Reasoning Dimensions in ARC. Following the framework of Lee al. (2025) consider three core cognitive dimensions of reasoning assessed by ARC:
Compositionality: The ability to combine simple building blocks (e.g., functions rules) into more complex transformations. ARC tasks often require models to compose multiple abstract operations.
The ability to extrapolate beyond observed examples and generate novel input-output mappings consistent with the inferred rules. ARC evaluates this by testing generalization to new input grids.
Productivity:
Logical Coherence: The ability to maintain consistent internal logic throughout the reasoning process. This includes applying learned transformations in a rule-consistent manner across diverse contexts.
A.2.1 Formal Problem Definition for Whole-Grin Grid ARC
Formally, each ARC task is function mapping æ X' to y E. We define a sequence of transformations 01y.... 0T operating on the entire grid produce y-
(1++)) f(x((), (a(tt)aa) az) with x0()_ x, x(T)
We use a binary reward R(x) = if (T) matches the target output else Further details on Markov Decision Process (MDP) formulations, action definitions, and potential expansions can be found in (Lee et al. 2024
B Ablation Studies
The ablation studies focus on identifying which components of GFlowNet contribute most success solving ARC problems. We evaluate the effects of reward scaling, different distribution combinations, and on-policy off-policy learning.
B.1 Effect of Reward Scale
In this study, analyze the impact of varying reward scales on GFlowNet's performance. By adjusting the magnitude of rewards, we observe changes in learning efficiency, trajectory diversity, and reward distribution, aiming to identify the optimal reward scale for effective solution generation.
Experiment Setup As described in the hyperparameter settings, the base rewards were set at 15 for correct solutions, 0 for incorrect ones, and 10 for solutions ending with the submit action without reaching the maximum episode length. To examine the effects of different reward scales, we conducted experiments with the following settings:
Low Scale: Reduced base rewards to 3 (by dividing the base reward by 5) for correct solutions and 2 for submit-ended trajectories.
Moderate Scale (Base): Maintained the default base rewards of 15 for submit-ended trajectories,
10 for trajectorics that ended due to the maximum episodo limit, and 0 for incorrect trajectorics.
High Scale: Increased base rewards to 225 for correct solutions and 150 for submit-ender trajectories (by multiplying the base reward by 15).
Very High Scale: Significantly increased base rewards 1500 for correct solutions and 1000 for submit-ended trajectories (by multiplying the base reward by 100).
32

---

Published in Transactions on Machine Learning Research (09/2025)
Validation Accuracy Comparison with Different Reward Scales
1.0
Validation Accuracy
0.6
Very High Scale (r*100)
High Scale (r*15)
Moderate Scale (Base)
Low Scale (r/5)
0.4
0.0
5000
10000
15000
20000
25000
30000
Step
Figure 12: Performance comparison across different reward scales, showing the impact of reward values on validation accuracy over training steps.
Results and Analysis Tables 16 and 17 summarize the effects of different reward scales on success rates for unique and total trajectories. A moderate reward scale (Base) demonstrated a balanced improvement in solution diversity and success rate, achieving 71.43% success rate for unique trajectories and 59.0% for total trajectories.
The effect of excessively high scaling (e.g., 100r) led to 100% success rate for both unique and total trajectorics. However, it also significantly reduced solution diversity, producing only single unique trajectory. This outcome suggests that while very high rewards lead to success, they may limit exploration. To confirm this effect, we calculated the diversity metric Dreward, which showed a reduction in reward distribution diversity at very high rewards, supporting the conclusion that excessive scaling discourages exploration.
Conversely, a lower reward scale (r/5) resulted in no successful trajectories, indicated by 0% success rate for both unique and total trajectories. This lack of success demonstrates that low rewards do not sufficiently reinforce correct solutions, causing the model to struggle to distinguish high- from low-quality trajectories.
Figure 12 illustrates the learning curves for each reward setting, showing that lower reward scales lead to noticeably poorer initial performance due to insufficient reinforcement. In contrast, moderate scaling enables broader exploration of the solution space, allowing the model to generalize effectively across different ARC lasks.
Furthermore, the diversity metrics, including trajectory diversity Diraj and reward distribution diversity Dreward, show that moderate reward scaling promotes greater exploration of successful trajectories. This finding suggests that well-scaled reward signal euhances the model's ability to discover correct solutions while encouraging diverse trajectory exploration.
These findings suggest that while moderate reward scaling enables GFlowNet balance efficient learning and exploration, excessively high rewards yield diminishing returns in terms of diversity. Although high rewards increase task completion rates, they may reduce exploration and solution diversity.
Conclusion
33

---

Published in Transactions on Machine Learning Research (09/2025)
Reward Scale
Unique Trajectories
Successful Unique Trajectories
Val ACC (%)
Low Scale
100
0.00
Moderate Scale (Base)
56
40
71.43
High Scale
74
53
71.62
Very High Scale
1
100
Table 16: Success rale and reward distribution diversity based on unique trajectories across different reward scales.
Reward Scale
Total Trajectories
Successful Total Trajectories
Val ACC (%)
reward
Low Scale
100
0.5139
Moderate Scale (Base)
100
59
59.0
0.8817
High Scale
100
75.0
0.6739
Very High Scale
100
100
100.0
0.0
Table 17: Success rate based on total trajectories across different reward scales.
Conversely, low reward scales do not provide sufficient reinforcement for effective learning, Optimal reward scaling is thus essential for supporting both accuracy and diversity, meeting the ARC task's requirements for generalization and high-quality solutions.
Statistical tests (c.g., chi-squared and Fisher's exact tests) confirmed a significant difference in success rates and diversity metrics between moderate and very high reward scales, reinforcing that balanced reward scaling is critical for effective exploration and efficient learning in complex tasks.
B.2 Action Number Performance Comparison
In this ablation study, we investigate the impact of varying the number of actions (3, 5, and 10) on learning efficiency, exploration capacity, and trajectory diversity, further validating the insights obtained from the primary experiments.
Experiment Setup To solve ARC Task 179, which features "Diagonal Flip" transformation, a minimum of three actions is required. While alternative solutions exist, the shortest solution involves three transformations: rolating the grid by 90 degrees, performing a horizontal flip, and executing the submit action. Starting with this minimal action set, we incrementally increased the number of actions to observe how each configuration affected validation accuracy and trajectory diversity over a maximum episode length of 10. This setup allowed us to explore the relationship between action granularity and the model's learning and exploration capabilities.
Results and Analysis As shown in Figure 13 models with three and five actions achieved high validation accuracy, with the threc-action configuration reaching high performance relatively early in training. This indicates that a smaller action space allows more focused exploration, leading to efficient learning. Notably, the five-action configuration achieved near-optimal accuracy, outperforming both the four- and ten-action configurations in terms of learning efficiency. In contrast, the ten-action configuration, despite its expanded action space, failed improve accuracy, suggesting that an excessive number of actions may impede effective exploration within the same training duration.
The reward trend graph further illustrates the differences across action configurations. The five-action setup shows stable and high rewards, indicating that this configuration balances action gramularity with exploration capacity. The four-action configuration stabilizes at a moderate reward level after an initial exploration phase. In contrast, the ten-action configuration exhibits consistently low rewards, suggesting that GelowNet struggles optimize effectively in larger action spaces without additional guidance. This finding aligns with prior research, suggesting that overly large action or state spaces can dilute exploration, leading suboptimal learning.
34

---

Published in Transactions on Machine Learning Research (09/2025)
Validation Accuracy Comparison with Different Number Actions
Reward Trands for Different Actions
Aerton
1980
Strp 2000
2009
35.05
100
Figure 13: (Left) Performance comparison across number of actions, showing the impact of action count validation accuracy over training steps. (Right) Reward comparison: comparison of reward values over training steps for each action configuration, highlighting differences in reward trends.
While expanding the action space increases trajectory diversity, it does not necessarily correlate with higher validation accuracy. For example, although the ten-action configuration showed greater trajectory diversity, it suffered from reduced learning efficiency and failed to achieve high rewards. This suggests that, while larger action spaces introduce more potential trajectories, they may also complicate exploration, especially without. additional guidance.
The calculated Reward Distribution Diversity Dreward values provide further insight. As seen in Tables 18 and 19 Dreward for the three- and four-action configurations zero, indicating uniform rewards due to limited trajectory diversity. In contrast, the ten-action configuration has Dreward of 0.4439, and the five-action configuration yields 0.6739, reflecting greater reward diversity the action space expands. Although increased diversity reflects broader exploration, it does not necessarily lead to improved performance, as shown by the ten-action configuration' 's low success rate and accuracy.
File
Unique Trajectories
Successful Unique Trajectories
VaI_ACC (%)
Total Trajectories
D_reward
n(a) 3
100.00
100
0.0
n(a)=
100.00
100
0.0
n(a) 5 (Base)
41
26
63.
100
0.6739
n(a) 10
95
11
11.58
100
0.4439
Table 18: Summary of unique and successful trajectories, success rates, total trajectories, and reward distribution diversity for each action configuration.
File
Successful Trajectories
ACC (%)
D r__eeard
Comments
uía)= 3
100
100.0
0.0
Minimal exploration with consistent success
nía)=
100
100.0
0.0
Limited diversity but high success rate
n(a) = (Base)
81
81.0
0.6739
Balanced exploration and success High diversity, low success rate
n(a)= 10
11
11.0
0.4439
Table 19: Total success rales, reward distribution diversity, and additional comments trajectory diversity and performance for each action configuration.
The thrce-action and four-action configurations both achieved 100% success rates for unique trajectorics, indicating that each generated trajectory successfully completed the task, though with limited diversity. Conversely, the ten-action configuration showed the highest mumber of unique trajectories (95) but had a low success rate (11.58%), suggesting that large action space dilutes effective exploration, leading lower task completion rates. The five-action configuration balanced exploration and success, yielding 41 unique trajectories with success rate of 63.41%.
35

---

Published in Transactions on Machine Learning Research (09/2025)
For the five- and ten-action configurations, statistical tests (chi-squared and Fisher's exact tests) confirmed statistically significant. difference in success rates, with the five-action configuration performing better among unique trajectories. This reinforces the idea that an optimal action space size is essential for effective learning.
These findings suggest that balancing action space size is critical for effective exploration and learning. Larger action spaces may require additional mechanisms, such as hybrid training approaches or managed replay buffers, to selectively sample high-quality data for focused exploration (Shen al., 2023 Sendera al, 2024). In this study, we implemented off-policy mechanisms to address this challenge. Future rescarch could develop models capable of handling extensive search spaces more effectively, potentially by incorporating adaptive exploration techniques that adjust to the complexities of larger action spaces.
B.3 Generated Trajectories and application to other tasks
To further analyze GFlowNet's trajectory generation and evaluate its applicability to other tasks, conducted task-specific experiments selected ARC lasks. These experiments aimed to examine the diversity of generated trajectories, their success rates, and the consistency of reward-based learning across different. task types.
illustrates multiple trajectories generated for two ARC tasks: Task 150 and Task 179. Each trajectory represents a unique sequence transformations (e.g., rotations, flips) applied to the input grid to generale the correct output:
Trajectory Visualization and Analysis Figure 14
Task 150 (67a3c6ac): The generated trajectories include efficient sequences (e.g., fewer steps) and redundant solutions with repeated actions. This demonstrates GFlowNet's exploration of diverse paths to the same output.
Task 179 (74dd1130): While successful trajectories generated, Task 179 highlights the tendency of some solutions to include unnecessary repetitive actions, suggesting room for further optimization.
These examples showcase GFlowNet's ability to discover a range of action sequences, balancing diversity and correctness trajectory generation.
hau
Cutzei
Lif Note
FITt
FON Towe
VfP
Lehoeey
Lebiooe
Vife
Tny?
7G
VHC
Tua22
Lift 4050
Lett LeftAccee
FelD
Twp
4
Teg'4
(a) Task 150
(b) Task 179
Figure 14: Visualization of generated trajectories for two ARC tasks. Each trajectory represents sequence of transformations applied to the input grid to achieve the correct output. Task 150 (a) and Task 179 (b) demonstrate diverse action sequences, including rotations and flips, leading to successful solutions.
Task-wisc Results and Observations Table 20 summarizes the results of our task-specific analysis, including the number of unique trajectooiis, their success rates, and the reward diversity (entropy).
36

---

Published in Transactions on Machine Learning Research (09/2025)
Task #
Unique Trajectories (Total)
Successful Unique (Total)
Success Rate (Unique, Total, %)
Reward Diversity (Entropy)
87 (3c960459)
(300)
(300)
100.00 (100.00)
140 (6150w2hd)
(300)
3 (300)
100.00 (100.00)
0.00
150 (67a366ac)
(300)
(30)
10.00 (10.00)
0.37
155 (08616354)
(300)
8 (300)
100.00 (100.00)
0.00
241 (91566313)
(300)
99 (280)
94.29 (90.31)
0.37
380 (nd36eeff)
(300)
(300)
100.00 (100.00)
Table 20: Task-wise analysis of trajectorics, success rates, and reward diversity. Unique values are shown with their corresponding total values in parentheses.
Unique Trajectories: For tasks like 380, 155, and 140, the model consistently generated only few successful trajectories, indicating that these tasks allow minimal varialion in the solution space.
Tasks such as 380, 155, and 87 achieved a 100% success rate, demonstrating that GFlowNet effectively explores correct solutions. However, for task 150, only 10% of unique trajectories were successful, suggesting a more complex solution space.
Success Rate:
Reward Diversity: Tasks with lower reward entropy (c.g., 380 and 140) reflect a solution space with limited diversity. In contrast, tasks like 87 exhibit higher entropy, indicating greater variely of valid solutions.
B.4 Off-Policy Training Analysis
Experiment Setup
We explored off-policy training using replay buffer with various sampling methods:
Priority Sampling (PRT): Prioritizes high-reward experiences.
Epsilon-Greedy Sampling: Randomly samples with probability and high-reward trajectories otherwisc.
Fixed Ratio Sampling: Samples fixed mix (e.g., 8:2 or 9:1) of high- and low-reward trajectories.
Validation Accuracy Comparison with on/off-polic
egreedy fixed, fiee_ratti
On-policy
Val Dalldation Accuracy
Mnwe Vwwmm ww www www Www
20000
40000
60000
50000
100000
120000
Step
Figure 15: Validation accuracy comparison between on-policy and off-policy training methods. On-policy training converges much faster than off-policy sampling methods.
37

---

Published in Transactions on Machine Learning Research (09/2025)
Results and Analysis Off-policy training with PRT sampling achieved lower variance across seeds but required significantly more steps (approximately 120,000) to converge, compared to on-policy training which reached near-perfect accuracy within 1,000 steps. This trade-off indicates that while off-policy methods enhance stability, their slower convergence makes on-policy training more practical for ARC problems.
(Additional Analysis) In particular, because ARC problems are highly sparse in rewards and often solved with short sequences, quickly stumbling upon a good trajectory is crucial. Off-policy learning, which replays a mix of older experiences, excessively re-emphasize suboptimal early trajectories unless carefully managed. On-policy methods, by contrast, adapt more rapidly to newly discovered successful paths leading to faster improvement.
38

---

Published in Transactions on Machine Learning Research (09/2025)
C
Algorithm
Algorithm GFlowNet Architecture for ARC Task
Input:
ARC environment with input 2k, forward policy parameters 0r, backward policy parameters 0g; total flow Ze, reward function r(s), max episode length T, Replay buffer BUFFER (optional)
Output: Trajectory
4 80 f- It
forcach erample do
foreach training step do
T [s] ro +-0 0
log total t- 0
+-0
while T and - terviis_dneee do ++1+1 4 OF 0B) Split = into 2F, 2B
Step 1: Forward Pass pF + Softmax(zF)
Sample X, Geometric(pg.) // X; 14.......... a" + arg min; X;
log + log FoomPMF(Xx)) s' + enw.ste((ss.)) T
Step 2: Backward Pass PB <- Softmax(zn)
Sample a' ~ Categorical(ps)
logPB PB logps(a')
Step 3: Reward
r(s') + Tfimal if cycle is detected then
r(s') + -(())-AC(TT
end
log total +- log total logpF PF
log PB total + log total PB
Store BUFFER if off-policy (optional) s
end
loss +- -log Ze + log total logglt Optimize 0F, AB minimize loss
total logr 2
Off-policy updates using BUFFER (optional)
Update sampling model (optional)
end end
return 7
39

---

Published in Transactions on Machine Learning Research (09/2025)
D Comparison of ARC Trajectory Datasets and Generation Methods
In this section, we compare ARC trajectory datasets and generation methods according to their supervision type and augmentation stratogy. We organize existing approaches into four main categories: (1) Human- written trajectories collected through annotation or crowdsourcing (e.g., ARCTraj (Kim et al. 2025b),
ARC-Interactive (Strandgaard 2024), H-ARC (LeGris et al. 2024)); ): (2) Rule-based trajectories derived from
hardcoded logic or symbolic programs (e.g.; SOLAR (Kim al. 2024 ); (3) Model-generated trajectories
produced via generative models such GelowNet (Ours); and (4) Other augmentations such as I/O
pair-based synthesis (e.g., RE-ARC (Hodel) 2024), BARC (Li et al. 2025). BabyARC (Wu, 2021) and
task/label-basee formulations (e.g., LARC (Acquaviva et al. 2022), ConceptARC (Moskvichev et al. 2023
This categorization helps clarify the trade-offs between trajectory diversity, scalability, and generalization. In particular, our Gllow method (3) stands out by generating diverse and plausible reasoning paths directly from input-output examples, without relying on either human supervision or ground-truth programs. This makes our approach especially suitable for ARC, where solvers must generalize to unseen tasks at test. time without access to task-specific demonstrations.
Additionally, unlike other methods, our method uniquely supports the generation of multiple distinct trajectories for each input grid, enabling a richer and more flexible reasoning process. To support this claim, we provide a detailed empirical and structural comparison with other methods below. Table 21 compares these datasets in terms of trajectory format, augmentation capabilities, and multi-trajectory support per grid. then presents quantitative comparison of trajectory scale and diversity across selected methods.
Table 22
Task Count
Data Generation
Trajectory Format
Multi-Traj per Grid
Method
Ours
7
SOLAR (im al.
2024
20
(1 Traj/Grid)
ARCTraj Kim et al.
2025b)
400
(Human Traj)
ARC-Interactive
Strandgaard
2024
400
(Human Traj)
H-ARC LeGris et al.
2024
800
(Human Traj)
x
RE-ARC Hodel 2024
400
(1/O Grid)
X
BARC Li et al.
2025
400
x
(New Task)
x
ARC (Wu
2021
800
x
(New Task)
X
LARC
Acquaviva et al. 2022)
400
x
(Task Description)
X
CRICepLARC MMokvichev et al.
2023
160
X
(Task Label)
Table 21: Comparison of data augmentation and trajectory support across ARC-AGI methods. Ours is the only method that augments multiple trajectories per input grid.
Among the existing datasets, human-collected trajectories such as ARCTraj (Kim et al., 2025b), ARC- Interactive (Strandgaard 2024), and H-ARC (LeGris et al., 2024) offer valuable insights into how people solve ARC tasks. However, these datasets typically cover a large number of tasks, but each task only has a single input grid, limiting their capacity to explore diverse solving behaviors from different inputs. Moreover, the cost of collecting multiple trajectories per input grid is prohibitively high in human studies.
In contrast, our method is the only approach that generates multiple trajectorics per input grid through data augmentation. Despite using only seven target tasks, we synthesize 1,601 valid trajectories with 162 unique ones (Table 22 This results in a trajectory density of 228.7 and trajectory diversity of 23.1, both of which
40

---

Published in Transactions on Machine Learning Research (09/2025)
are significantly higher than those of other methods. As shown in Table 22 our trajectory diversity is even comparable to the trajectory density of human-collected datasets, which typically ranges from 19.7 to 26.7. Since diversity is upper-bounded by density, this suggests that our method achieves at least as much diversity as those human-collected datasets.
These comparisons (Tables and 22 highlight the effectiveness of our augmentation method in producing a rich set of diverse solving trajectories from compact task set, without requiring large-scale human annotation.
Valid Traj.
Unique Traj.
Target Tasks
Grids per Task
Trajectory Density
Trajectory Diversity
Method
Ours
1,601
162
7
1
228.7
23.1
SOLAR Kim et al.
2024
10,000
10,000
20
500
1.0
1.0
ARCTraj (Kim
2025b)
10,672
400
1
26.7
ARC-Interactive Strand
2024
8,374
400
1
20.9
H-ARC (LeGris
2024
15,744
1
19.7
Table 22: Comparison of trajectory density and diversity across human-collected and augmented trajectorics. Trajectory Density is calculated as the number of valid trajectories divided by the number of input grids. In contrast, Trajectory Diversity is the number of unique (non-duplicate) trajectories per input grid. Notably, the diversity of Ours (23.1) is comparable to the trajectory density of human-collected datasets such as ARCTraj, ARC-Interactive, and H-ARC. Since diversity upper-bounded by densily, this suggests that the diversity of Ours least on par with, if not superior to, those datasets.
41

---

Published in Transactions on Machine Learning Research (09/2025)
E
Efficiency and Scalability of GFlowNet Augmentation
GFlowNNet-bssed augmentation is not only effective, but also computationally efficient and structurally scalable. We report the time required to generate 100K valid trajectories for each ARC task used in our downstream experiments. As summarized in Table 223 most tasks complete this process in under an hour, with generation rates exceeding 2K trajectories per minute on average. Given that the ARC Prize evaluation protocol allows only 6 minutes per task, this level of speed implies that even test-time augmentation is feasible, yiclding thousands of trajectories within a fraction of the allotted time.
This level of efficiency demonstrates that our method can serve as a practical augmentation tool even in scenarios with tight compute budgets or real-time constraints. Notably, even the slowest tasks still maintain generation rales above 1,180/min, reinforcing the method's robustness across task variability.
Table 23: Time and throughput required to generate 100,000 trajectories per task. Our framework consistently achieves weweshhuuuand-e----nnee generation rates, enabling fast and scalable augmentation even under strict time constraints.
Task # (ID)
Time Required (for 100k)
Generation Rate (trajectories/minn)
87 (3c9b0459)
53 min
1,880/min
140 (6150a2bd)
52 min
1,920/m
150 (67a3c6ac)
50 min
2,000/min //00/min
155 (68b16354)
27 min
3,700/mi //00/mmin
179 (74dd1130)
46 min
2,170/ min
241
(9dfd6313)
83 min
1,200/mii /mi00/min
380 (cd36eef7)
84 min
1,180/min
Average
56.4 min
2,000/min
Beyond raw throughput, our framework also demonstrates structural scalability in the form of trajectory diversity. To assess how diversity grows over time, we measured the number of unique successful trajectories function of the total number of trajectories generated.
As shown in Figure 16 all tasks exhibit a consistent logarithmic decay pattern in the rate of discovering new unique solutions. During the early stages, new solution strategies are discovered rapidly, but the rate of discovery decreases logarithmically as the solution space becomes progressively saturated. This behavior aligns with the long-tailed structure of human reasoning, where common strategics emerge carly, and rarer patterns require deeper exploration. Notably, while all tasks follow this logarithmic trend, there are substantial variations between different problems, with some tasks (like Problem 150) reaching saturation much faster than others (like Problems 86 and 140), reflecting the inherent complexity differences across ARC tasks.
This logarithmic scaling property has important implications. First, it allows practitioners to make informed tradeoffs between diversity and compute budget. Even small-scale generation (e.g., 10k 20k samples) captures a significant portion the solution space, while longer runs offer diminishing but non-negligible improvements. The logarithmic nature of the decay means that the marginal benefit of additional sampling decreases predictably, enabling efficient resource allocation.
Second, this structure suggests that GFlowNet-based augmentation is well-suited for dynamic test-time settings. Depending on resource availability, one can determine how much augmentation is 'enough' based on the task complexity or diversity needs. The problem-specific variations observed in Figure [6b] indicate that some tasks may benefit. from exlended sampling while others reach effective saturation more quickly, allowing for adaptive resource allocation strategies.
Third, our framework is well-aligned with the real-time constraints of ARC-like benchmarks. For instance, the ARC Prize evaluation protocol allots approximately 6 minutes per task to solvers, including any
42

---

Published in Transactions on Machine Learning Research (09/2025)
Rate of New Unique Sequence Discovery
Finel Uniqueness Rate Among Successful Trajectories
1p hobion
26.5%
25.0%
New Unique Sequences Discovered
Ico
12.3%
10
26%
PB7 PM00 Problem ID
F50
P179
Culmulative Successful Trajectory
(a) Final uniqueness rates vary significantly across tasks, ranging from 2.6% to 26.5%, indicating different levels of solution diversity potential.
(b) Rate of new unique sequence discovery follows a logarithmic decay pattern, with notable variations in decay rates across different problems.
Figure 16: Trajectory diversity analysis across ARC tasks. (a) The final uniqueness rate among successful trajectories shows significant lask-dependent variation. (b) The rate of discovering new unique sequences decreases logarithmically over time, with problem-specific decay characteristics reflecting varying solution space complexity.
computation performed during test time. Within this limited window, our method can generate thousands of candidate trajectories per task, thanks to its throughput of over 2,000 trajectories per minute.
This enables practical test-time augmentation, where diverse solution candidates can synthesized and evaluated on the fly. Even allocating just 2-3 minutes of the 6-minute window for trajectory generation yields tens of thousands of diverse paths, offering valuable input to downstream symbolic solvers neural rerankers. The logarithmic discovery pattern ensures that even short generation runs capture the most promising solution strategies, making our augmentation pipeline not only practical offline but also feasible for real-time AGT scenarios requiring rapid reasoning under constraints.
43


Reviews:
[{"review_id": "DPINuA8Kwh", "reviewer": "Reviewer_VQao", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n- *Clarity of Presentation*: The paper is generally well-written and clearly structured. Technical details are sufficiently presented.\n\n- *Geometric Forward Policy*: The inclusion of a geometric forward (greedy) selection strategy in the ablation studies is interesting. This approach, when coupled with the updated reward function, demonstrates the generation of more diverse solution pathways, producing a non-trivial empirical contribution.\n\n- *Use of ARC-AGI Dataset*: Using the ARC-AGI benchmark—where grid transformations often require complex, whole-grid operations—provides a challenging and appropriate testbed for evaluating the proposed method. This choice improves the relevance of the evaluation.\n\n\n**Weaknesses:**\n- *Overstatement of Human Trajectory Insights*: The insights drawn from human-generated solution trajectories appear largely syntactic—centered on shorter trajectory lengths and the use of a final ``submit'' action. While the work draws inspiration from human strategies, the reliance on surface-level features does not convincingly reflect deeper strategic alignment with human reasoning. A more appropriate phrasing might emphasize that the requirement for diverse solutions implicitly encourages shorter, less cyclic trajectories, facilitating broader state space exploration. This framing remains intuitive and accurate without overstating the human-inspired nature of the method.\n\n- *Overclaimed Interpretations*: The claim that the observed properties (e.g., trajectory brevity and diversity) reflect meaningful alignment with human reasoning strategies is somewhat overstated. A more measured interpretation would acknowledge that the approach mirrors certain syntactic features of human behavior, rather than deeper strategic reasoning. Reframing the contribution in these terms would imporove the credibility of the argument.\n\n- *Assumption that Shorter Trajectories are Superior*: The paper claims that shorter trajectories, as observed in human data, are preferable and lead to more meaningful or diverse solutions. However, this assumption may be limiting. In fact, slightly longer trajectories could potentially uncover more nuanced and diverse solution strategies. This observation is explored in Appendix B.3, and incorporating such insights into the main body of the paper would strengthen the argument and provide a more balanced perspective.\n\n- *Limited Reward Structures Explored*: The reward functions used in the study are relatively simple, and the distinction between the DF and TLR reward types is not clearly stated—both appear to promote trajectory shortening. Introducing a more nuanced reward structure, such as an explicit exploration bonus or a diversity-promoting metric, could have enhanced the diversity and effectiveness of the solutions. Also, leveraging deeper structural patterns from human trajectories—such as clustering similar problem types and extracting shared strategic representations—could lead to the the design of more meaningful and generalizable reward signals. In the absence of such analyses, the contribution remains somewhat incremental.\n\n- *Limited Practical Applicability*: The use of trajectory length as the key feature may lead to the  problem of state space explosion, particularly in tasks that require subregion-level analysis rather than whole-grid transformations. As a result, the presented pipeline may not generalize well to a broader range of transformation tasks and limit its practical applicability. Using alternate features in the reward might mitigate this issue and improve generalisability of the framework.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a trajectory synthesis framework based on Generative Flow Networks (GFlowNets) for the automatic generation of diverse solution trajectories in tasks from the ARC-AGI benchmark. The framework is evaluated on tasks which may be solved via **whole-grid transformations**, as opposed to those requiring subregion-based transformations.\n\nThe reward function guiding the GFlowNet is inspired by empirical patterns observed in human solution strategies and is structured to incentivize the following properties: (i) shorter trajectories, (ii) absence of cycles within the trajectory, and (iii) completion with a valid submission via a terminal \"submit\" action. To improve the diversity and efficiency (in terms of length of successful trajectories) of the generated trajectories, the authors introduce a geometric forward policy, which they show to outperform the traditional categorical forward policy in producing shorter, yet effective, solution paths.\n\nExperimental results demonstrate that the proposed framework—using a modified loss function that integrates trajectory balancing, a trajectory length regularizer, and a composite reward signal (including penalties for cyclic behavior and binary success rewards)—achieves higher performance on a subset of ARC-AGI tasks. The framework effectively balances the trade-off between task success and trajectory diversity.\n\nAn ablation study shows that among individually applied reward components, discounted rewards yield the highest performance, followed by the trajectory length loss. However, combining both discounted and length-based reward components leads to the best overall results. In particular, increasing the constraints imposed by the reward function imporves task accuracy at the cost of reducing the diversity of generated trajectories. Finally, the combination of the geometric forward policy with the standard categorical backward policy proves most effective in producing a wide range of successful and yet diverse solution trajectories.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Clarifications and suggestions for imporvement**:\n- *Clarification on Reward Function Usage (RQ2, RQ3)*: In the experiments addressing RQ2 and RQ3—where different policy strategies are evaluated—it would be helpful to clarify whether the updated (dense) reward function or the original sparse (binary) reward function was used during training. If the updated reward was applied throughout, including comparative performance graphs against the sparse reward baseline would improve consistency.\n\n- *Support for Theoretical Claim (Page 11, Paragraph 2)*: The statement, “This theoretical bias toward shorter, high-probability trajectories is further supported by empirical evidence,” would benefit from either a citation or empirical substantiation. For example, including a graph showing the distribution of human solution trajectory lengths on the ARC-AGI benchmark would make the claim more concrete and reinforce the connection between human strategy and the proposed method.\n\n- *Clarification on Cycle Constraint (Table 5)*: In Table 5, it would be informative to clarify whether the successful and unique trajectories generated under the DF + TLR reward violated the cycle constraint at all. This would help assess the marginal utility of the cycle constraint relative to the trajectory length constraint. Given that the cycle constraint alone does not appear to yield strong performance, it is worth questioning whether the length constraint alone is sufficient to generate diverse and non-redundant solution trajectories.\n\n- *Integration of Appendix Insights*: The discussion in Appendix B.3 regarding the relationship between trajectory length and solution diversity contains some valuable insights. Integrating this analysis into the main body of the paper would enhance the clarity and impact of the findings, particularly in reinforcing the argument that strict length minimization may not always be optimal.\n\n- *Scope of Contribution*: While the work takes a meaningful step toward better understanding diverse trajectory generation in the ARC-AGI setting, the contribution currently feels somewhat limited in scope. In particular, the absence of downstream validation—e.g., demonstrating how the generated trajectories can be used to improve the performance of a learning model—leaves the contribution feeling incomplete. Incorporating such downstream utility would significantly strengthen the practical relevance and completeness of the work.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mD0hgXZ5Q1", "reviewer": "Reviewer_XpcR", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n\n1. A key strength of this work is its empirical exploration as to whether GflowNets can be employed to generate synthetic solution pathways for ARC-AGI training set tasks.\n2. The paper also offers empirical insights through comprehensive experiments in applying GflowNets to ARC-AGI solution augmentations.\n\n**Weaknesses**:\n\n1. In the introduction, the paper emphasizes literatures that explore LLM-based problem solving. However, to the best of my knowledge, the manuscript does not involve any language-based reasoning nor the use of LLMs. Is it appropriate to frame the narrative of the research in this manuscript as reasoning (and mainly cite multiple literature on language-based reasoning)?\n2. The paper lacks a comparison between GflowNets and other forms of generative modeling or RL approaches in generating such synthetic solutions.\n3. To the best of my knowledge, the purpose of ARC-AGI is to test the generalization performance to unseen tasks. However, the manuscript does not provide empirical results on the test set tasks. Although the authors have stated this as a limitation/future work, I think that this is a necessary component, as the manuscript explicitly focuses on ARC-AGI only (as stated in the manuscript title).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This manuscript explores the use of GFlowNets for automatically generating diverse, human-aligned reasoning trajectories on the ARC-AGI dataset.\n\nThe main contribution of the manuscript is the application of GFlowNets as a means towards solution augmentation for ARC-AGI. As an ARC-AGI task can be solved through multiple approaches and trajectories, casting solution augmentation itself as a reward-proportional sampling problem and applying GflowNets is a sound approach.\n\nThe technical contributions of the manuscript are: (1) a geometric forward and categorical backward policy pairing that biases exploration toward succinct action sequences while keeping backward flow corrections stable, and (2) a human-inspired reward scheme that combines discounting, trajectory-length regularization and cycle penalties to emphasize brevity and intentionality in the generated traces .\n\nThe manuscript provides experimental results on whole-grid ARC tasks, showing that GflowNets are capable of generating diverse and valid solutions for a given task. The manuscript also provides explorations over various human-aligned reward fomulations and forward/backward policy selection.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "It would benefit the quality of the work if the following are taken into consideration:\n\n1. Reduce the emphasis on natural language-based reasoning works. Or, if it is not possible, provide support by conducting experiments on language-based reasoning or LLMs.\n2. Provide empirical comparisons with other approaches to assess the relative effectiveness of the proposed approach.\n3. Provide at least a minimal amount of empirical results on the downstream utility.", "broader_impact_concerns": "Currently, there are no broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "NnDl7nS7vl", "reviewer": "Reviewer_43UG", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n\nThe MDP design and algorithm are reasonable.\n\nThe paper is easy to read.\n\n**Weaknesses**:\n\nI think this work has a critical problem. The ARC-AGI benchmark is meant to evaluate the reasoning capability of AGI; it is not a problem we need to solve like the travelling-salesman problem (TSP). \n\nGFlowNets are indeed powerful for combinatorial optimisation and latent sampling. One can certainly build a specialised model that excels at ARC-AGI, but that would be meaningless when ARC-“AGI” is intended as a benchmark for general intelligence.\n\nFor these reasons, I do not understand why this research is necessary.\n\nIn addition, the paper only proposes an augmentation method; it does not apply the augmented dataset to LLMs. The authors should at least show that the framework actually improves an LLM’s reasoning capability when the augmented solution paths are provided as in-context examples.\n\n---\n\nOne minor comment:\n\nGFlowNets are designed to generate diverse solutions (terminal states), not diverse trajectories. Each problem has exactly one solution, so why do we need a GFlowNet to generate diverse trajectories?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper formulates Markov-decision-process of Generative Flow Networks (GFlowNets) for generating diverse solution trajectories for the Abstraction & Reasoning Corpus (ARC-AGI). Because ARC tasks are provided only as input–output grid pairs, the latent sequence of operations that produced the solution is normally unobserved. The authors argue that a GFlowNet, equipped with (i) a carefully designed action/state space and (ii) a reward that penalises long sequences via geometric-length sampling, can learn a policy that yields not just a single high-reward path but a distribution over plausible paths. The paper also introduces a “geometric front-policy” that explicitly controls expected trajectory length.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Include LLM reasoning experiments that leverage the augmented dataset.\n\n2. Provide a more compelling AGI-oriented motivation for this research.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)