Multi-View Encoders for Performance Prediction in LLM-Based Agentic Workflows
Authors: Patara Trirat
Wonyong Jeong
Sung Ju Hwang
Venue: ICLR 2026 Poster (2026)
Keywords: Agentic Workflows
Performance Prediction
Multi-View Encoding
Unsupervised Pretraining
Large Language Models
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: This paper introduces Agentic Predictor, a lightweight framework that uses multi-view encoding and unsupervised pretraining to efficiently predict performance in LLM-based agentic workflows, reducing costly trial-and-error evaluations.

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across diverse tasks, but optimizing LLM-based agentic systems remains challenging due to the vast search space of agent configurations, prompting strategies, and communication patterns. Existing approaches often rely on heuristic-based tuning or exhaustive evaluation, which can be computationally expensive and suboptimal. This paper proposes **Agentic Predictor**, a lightweight predictor for efficient agentic workflow evaluation. Agentic Predictor is equipped with a *multi-view workflow encoding* technique that leverages multi-view representation learning of agentic systems by incorporating code architecture, textual prompts, and interaction graph features. To achieve high predictive accuracy while significantly reducing the number of required workflow evaluations for training a predictor, Agentic Predictor employs *cross-domain unsupervised pretraining*. By learning to approximate task success rates, Agentic Predictor enables fast and accurate selection of optimal agentic workflow configurations for a given task, significantly reducing the need for expensive trial-and-error evaluations. Experiments on a carefully curated benchmark spanning three domains show that our predictor outperforms several strong graph-based baselines in both predictive accuracy and workflow utility, highlighting the potential of performance predictors in streamlining the design of LLM-based agentic workflows.

Published as a conference paper at ICLR 2026
MULTI-VIEW ENCODERS FOR PERFORMANCE PREDIC- TION IN LLM-BASED AGENTIC WORKFLOWS
Patara Trirat', Wonyong Jeong', Sung Ju Hwang!?
'eeepAuto.ai, KAIST (patara, young, sjhwang) @ddepauto.... ai
ABSTRACT
Large language models (LLMs) have demonstrated remarkable capabilities across diverse tasks, but optimizing LLM- basedded agentic systems remains challenging due to the vast search space of agent configurations, prompting strategies, and communication patterns. Existing approaches often rely on heuristic-based tuning or exhaustive evaluation, which can be computationally expensive and suboptimal. This paper proposes Agentic Predictor, lightweight predictor for efficient agentic workflow evaluation. Agentic Predictor is equipped with a multi-view workflow encoding technique that leverages multi-view representation learning of agentic systems by incorporating code architecture, textual prompts, and interaction graph features. To achieve high predictive accuracy while significantly reducing the number of required workflow evaluations for training predictor, Agentic Predictor employs cross-domain unsupervised pretraining. By learning to approximate task success rates, Agentic Predictor enables fast and accurate selection of optimal agentic workflow configurations for given task, significantly reducing the need carefully curated for expensive trial-and-ene -ar--rr evaluations. Experiments outperforms several benchmark spanning three domains show that our predictor strong graph-based baselines in both predictive accuracy and workflow utility, highlighting the potential of performance predictors in streamlining the design MLMMedded agentic workflows.
INTRODUCTION
Large language models (LLMs) have catalyzed the development of agentic systems capable of executing complex, multi-step tasks autonomously (Hong et al., 2024; Wu et al., 2024: Xi et al., 2024; Mialon al., 2024). These systems, often constructed through meticulous manual engineering. reasoning, tool invocation, and memory management
integrate components such as Chain-of-Thouud enable sophisticated behaviors for orchestrating intricate workflows (Xi et al., 2025; Ke et al., 2025; Gridach et al., 2025; Plaat et al., 2025). However, the handcrafted nature of these systems imposes limitations on scalability, adaptability, and rapid deployment across diverse domains.
To address these limitations, recent trends have shifted towards automated design methods for agentic systems (Hu et al., 2025a: Shang et al., 2025; Zhang et al., 2025b; Zhuge et al., 2024; Liu et al., 2024b; Hu et al., 2025b; Yuan et al., 2025). Automated methods typically employ search algorithms to discover optimal workflow configurations by systematically exploring a vast design space. Instead of relying human intuition, these approaches generally involve iterations of candidate generation, evaluation, and refinement. While promising, these methods exhibit significant drawbacks, chiefly the high computational costs associated with the extensive validation steps needed during the exploration and evaluation phases of the search. Each candidate configuration must undergo rigorous evaluation, often through expensive, repeated interactions with LLM APIs, rendering the search prohibitively costly and time-conen time-consuming.
In this paper, we argue that purely search-based automated design methods are inherently inefficient and propose a predictive approach to significantly accelerate workflow evaluation. Specifically, we advocate for predictor-basee agentic workflows. similar to performance predictors in neural architecture search (White et al., 2021), thereby reducing the need for extensive validation.
framework that can rapidly estimate the performance of candidate

---

Published as a conference paper at ICLR 2026
As depicted in Figure 1, instead of fully evaluat- ing every candidate, a predictive model can es- timate the quality and viability of agentic work- flows, thus guiding the search process far more efficiently. By reducing costly ground- executions or environment interactions during the search process, rnddiction-base approaches promise significant improvements in both search efficiency and solution quality. However, build- ing high-quality predictor for agentic work- flows introduces two fundamental challenges.
Ageniic Syetn Baliting Stoce Elancs Sance
Cananets
Rumdmo Envinonment
AMM of
Fowclacck
Prompta
Cutl
Candicane
Wordfow
Evalustion
Gandidate
Pass FFal
10 Eano feedd kilw Gunwrikon (eaitiiy Appissred
Canskoon
Wordfoww
Predictor
Pass /Fa
Connections
IN wwweeneeoweee Weelllen Uae enaice (Ohes)
Figure 1: Comparison between (a) execution- based and (b) prediction- baaed candidate evaluation for agentic workflow generation. Execution-barte -based methods rely on costly runtime or LLM calls, while our prediction-based approach offers faster, scal- able evaluation via a learned predictor.
Workflow Heterogeneity. Agentic workflows exhibit considerable heterogeneity; subtle vari- ations in configuration can lead to dramatically different performances. Specifically, workflows can vary widely in communication structure, prompting strategies, tool invocation patterns, and reasoning styles, making it challenging to learn unified predictive model. Moreover, agentic systems differ significantly across tasks, domains, and toolsets, resulting in diverse and complex workflow configurations that are difficult to model uniformly (Xu et al., 2024; Qiao et al.. 2025).
Scarcity of Labeled Data. The availability of labeled data for training effective prediction models severely limited due to the prohibitive cost of generating performance labels through exhaustive validation. Constructing a large, diverse set of labeled workflows with known execution outcomes particularly expensive, creating a data bottleneck for supervised learning approaches. Moreover, gathering large-scale, high-quality labels for agentic workflows (e.g., success rates and execution outcomes) is often infeasible, further limiting the amount of supervised training data available for learning accurate predictors.
To tackle these challenges, we present Agentic Predictor, multi-view encoder framework for performance prediction in LLM-based agentic workflows. To address workflow heterogeneity, Agentic Predictor uses multi-view workflow encoders that jointly model complementary signals- structural (e.g.. agent topology), behavioral (c.g.. tool usage), and semantic (e.g., prompts) prompts)-capturnins capturing the diverse, task-depnndent characteristics of workflow configurations. To mitigate label scarcity, introduce cross-domain unsupervised pretraining, denoted Agentic Predictor+, which leverages abundant unlabeled workflows from related domains. We pretrain the multi-view encoders with contrastive and reconstruction objectives, then fine-tune limited labeled data, yielding robust and transferable representations for prediction. The main contributions of this paper are as follows.
We propose multi-view encoders and cross-domaie unsupervised pretraining that jointly capture the heterogeneous facets of LLM-based agentic workflows, yielding higher predictive performance, better generalization, and effective predictor training under limited labels.
We introduce Agentic Predictor, unifying these components for the underexplored problem of performance prediction in heterogeneous, label-scarce M.---based agentic workflows, thereby reducing trial-and-error costs and accelerating development.
We empirically demonstrate that, averaged across three domains, Agentic Predictor improves prediction accuracy by up to % and utility by up to 5.87 % over strong baselines.
2 RELATED WORK
Automated Generation of Agentic Workflows. Recent advancements (Xi et al., 2025; Ke et al., 2025; Gridach et al., 2025; Plaat et al., 2025) in agentic workflows have led to the development of various frameworks aimed at enhancing multi-agent collaboration for complex tasks (Guo et al., 2024: Trirat et al., 2025; Niu et al., 2025; Trirat & Lee, 2025a). MetaGPT (Hong et al., 2024) and ChatDev (Qian et al., 2024) use predefined multi-agent structures to address coding challenges, while Agent Verse (Chen et al., 2024) introduces iterative collaboration where agents discuss, execute, and evaluate tasks. LLM-Debate (Du et al., 2024) employs multiple expert agents that engage in debates over several rounds to derive final answers. However, these systems often rely on static configurations, which limits their adaptability to diverse queries across different tasks and domains.
2

---

Published as a conference paper at ICLR 2026
To optimize agentic workflows, GPTSwarm (Zhuge et al., 2024) and G-Designer (Zhang al., 2025a) apply variants of the REINFORCE algorithm to optimize workflow structures represented as directed acyclic graphs (DAGs), while DyLAN (Liu et al., 2024b) dynamically selects agents based on task requirements. ADAS (Hu et al., 2025a) and AFlow (Zhang et al., 2025b) further leverage powerful LLMMs (e.g.. (alaude-..5-oomtte and GPT-4) iteratively generate task-specific multi-agent systems. Similarly, AgentSquare (Shang et al., 2025) proposes a modular design space for automatic LLM agent search, enhancing adaptability to novel tasks. Despite their effectiveness, these methods typically require numerous LLM calls, resulting in significant computational and financial overheads, making them less practical for real-world applications.
Rather than manually designing a fixed workflow (Qian et al., 2024;
Table I: Comparison between and existing frameworks
for prediction-based workflow generation.
Chen et al., 2024; Du et al., 2024) or paying repeated inference costs to synthesize one per query (Zhuge et al., 2024; Liu et al., 2024b), Agen- tie Predictor presents lightweight performance predictor to rapidly estimate the quality of candidate agentic workflows, enabling broad exploration without exhaustive evaluations. Among recent efforts, FL.RA-BBnnhh (Zhang et al., 2025c) advocates GNN-based predictors and releases a benchmark that models workflows as a single- view graph where prompts are node features. A complementary direction, MAS-GPT (Ye et al., 2025), fine-tunes LLMs to directly generate workflows in a single call. In contrast to these directions, Agentic Predictor differs in three respects: representation (multi-view encoding of agent topology, code, and system prompts vs. single-view view graphs), learning (cross-domain unsupervised pretraining to mitigate label scareity, rather than no pretraining recipe or supervised LLM fine-tuning), and efficiency (a compact predictor for fast evaluation without repeated LLM calls). Table summarizes these distinctions.
Framework
Representation
Multi-View
Unsupervised Pretraining
Predictor
Agnostic
Search
MAS- CAS--PP(YY Ye et al., 2025)
FLORA Hench (Zhang 20252)
Agentlc Predictor (Ques)
Performance Predictors for NAS. Neural architecture search (NAS) has spurred the development of performance predictors that aim to reduce the significant computational cost of evaluating candidate architectures. PRE-NAS (Peng et al., 2022) employs predictor-assisted evolutionary strategy to estimate model performance, thereby alleviating the need for exhaustive training. BRP-NAS (Dudziak al., 2020) integrates graph convolutional networks to forecast hardware-ane aware performance metrics, improving the practicality of NAS under resource constraints. CAP (Ji et al., 2024) introduces context-aware neural predictor, leveraging self-supervised learning generate expressive and generalizable representations of architectures, thus enabling more effective search space exploration. FlowerFormer (Hwang al., 2024) advances architecture encoding through flow-aware graph transformer, yielding improved prediction accuracy. A unifying trend among these methods is the emphasis on learning more informative representations to guide the search process. Building on this insight, we propose the Agentic Predictor framework, which approaches performance prediction from a representation-centre perspective. By incorporating multi-view representations conditioned on workflow configurations, Agentic Predictor facilitates accurate performance estimation and efficient exploration of the agentic workflow space.
3 METHODOLOGY: AGENTIC PREDICTOR
3.1 PROBLEM FORMULATION
Let an agentic workflow be denoted as W (VCE,P.C), where V === represents the set of N agents, & denotes the set of edges defining the connections between agents, and = {p:) N denotes the system prompts for each agent i. C represents the complete code specifying the logic and structure of the workflow. Thus, the workflow W is represented directed acyclic graph (DAG).
Given a task description T. the workflow W autonomously executes agents in topological order, where the i-th agent receives the task description T along with the outputs y from its predecessor agents. Formally, the input to agent i is defined as X; {T] U {y, 02 E M, where A\(i)) denotes the set of predecessor agents of agent i, and yj is the output of agent j. The output yı of agent ė is generated by querying an LLM: y; = LLM(X,, pt). ). After executing all agents, the final response of the agentic workflow is defined as r (LW, T), where fLLM represents the overall execution process of the given LLM. Generally, this process is repeated for the evaluation of r, which incurs significant computational and financial overhead.
3

---

Published as a conference paper at ICLR 2026
pretraining flow
Agentic System Building Blocks (Search Space)
(b) Muiti-View Workflow Encoding
Graph Encoder
Ager Aggregation Layer
Dec
Cec
Prompes
Random
Sampling
Code Encoder
con
Prompt Encoder
(b) Prefraining Phase
Modols
Search Method
Task Encoder
(inino Slrais)
Performance Predictor
Datasets
(Workflow, Perf.)
Pair Datasets
(c) predictor training flow
Connections
shared or predictor-giided search flow
Figure 2: Overview of our Agentic Predictor framework. (a) multi-view workflow encoder is designed to encode set of agentic workflows from graph, code, and prompt aspects into unified representations, which serve as features for training the predictor. In the (b) pretraining phase, the encoder learns these representations using cross-domain unsupervised pretraining objectives. In the (c) predictor-guides search phase, performance predictor is trained small (workflow configuration, performance) dataset to classify configurations as pass or fail, and subsequently guides the search toward promising configurations.
unlabeled workflows spanning diverse tasks and domains,
In contrast, this paper aims to design a predictive model M that efficiently estimates the final performance of an agentic workflow W on given task description T, without requiring costly LLM invocations. Therefore, we treat the workflow W and task description T as inputs to the predictor M, which outputs the estimated performance é. Formally, é MM (M T'), where θ denotes the learnable parameters the performance predictor.
Learning Objective. Given the workflow W, task description (or query) T, and performance predictor Me parameterized by e, we aim to find the optimal that minimizes the error between the estimated performance é and the ground-trudh performance Formally, solve
min (w..m(((eee))
where .),,)) is loss function that quantifies the discrepancy between the ground truth and the predicted performance. L can be either the cross-entropy loss the mean squared error loss, depending on whether the prediction task formulated as classification or regression problem.
3.2 FRAMEWORK OVERVIEW
We present an overview of our Agentic Predictor framework in Figure First, the multi-view workflow encoder integrates key aspects of agentic workflow- graph structures (V, 2), code implementations C, and system prompts P-into a unified representation F. Integration is achieved via modality-specific encoders followed by aggregation layer that consolidates features across modalities. Second, when labeled instances are scarce, we refine these representations with unsuper- vised objectives, reconstruction and contrastive learning. improve generalization and adaptability across diverse tasks and configurations. Third, dedicated performance predictor MM is trained on labeled set (often small) comprising workflow configurations W, task descriptions T, and observed performance outcomes e. Finally, with the trained predictor, perform predictor-guides search that efficiently ranks and selects promising workflow configurations without incurring expensive LLM calls. Because Agentic Predictor is search-agnostic, we deliberately do not commit specific search algorithm within the framework.
3.3 MULTI-VIEW WORKFLOW ENCODING
Motivated by recent findings in the NAS literature (White et al., 2020; Akhauri & Abdelfattah, 2024; Trirat & Lee, 2025b), which show that architecture representations strongly influence predictor performance, we argue for expressive, comprehensive representations tailored to agentic workflows. Because agentic workflows differ fundamentally from traditional neural architectures, conventional
4

---

Published as a conference paper at ICLR 2026
graph-based encodings alone are insufficient. Although DAGs naturally capture explicit inter- communication and dependencies, they omit crucial implicit signals such as tool-usage patterns, code structure, computational complexity, and the nuanced semantics present in agent prompts. To address these limitations, we propose a multi-view encoding scheme that integrates complementary representations multiple granularities, with each view capturing distinct yet essential aspects of LLM-based agentic workflows.
Graph View explicitly models structural dependencies and direct interactions among agents, emphasizing inter-agent communication channels. We denote the graph view as g = (V.E).
Code View implicitly encodes program-level semantics, control and logical sequence. computa-
tional complexity. and patterns of tool usage inherent in workflow implementations C.
Prompt View provides semantic embeddings that capture agent roles, behavioral specifications,
and broader contextual guidance embedded within system and instruction prompts P.
Our rationale for adopting this multi view framework is that aggregating heterogeneous information
sources reduces representation bias, thereby improving both robustness and predictive accuracy.
3.3.1 ENCODER NETWORKS
We now detail the components of our proposed multi-view workflow encoding method for perfor- mance prediction in agentic workflows using neural networks. Let Enc(-) denote encoder function that maps a candidate workflow-composed of (G, P) intinto d-dimensional Euclidean space, i.c., Enc(.) (g.C. P) + R.. Given the heterogeneous nature of workflow configurations, we design three specialized encoder networks, each responsible for learning a representation corresponding to distinct view. These view-specific representations are then aggregated into shared latent space, denoted by Z = Enc , C, P). where Z E Rd. This continuous latent representation is used to train the performance predictor Me (see $3.5). The individual encoders for each view are integrated into a unified architecture as described below.
Graph Encoder. We employ graph neural network (GNN) layers to encode graph-based represen- tations. The workflow is modeled as DAG in which each edge encodes a unidirectional message channel. Rather than relying on a single graph, we adopt a multi-graph approach that integrates node features from multiple views, including agent-specific definitions and function-cal call implementations at each agent node. We instantiate this multi-graph representation with three graph views constructed from workflow W. In the prompt graph Grooppe node features are obtained by pooling the em- beddings of each agent's system and instruction prompts. In the code graph Geode- node features correspond to the function-call code associated with each agent. In the operator graph Goperator. node features encode operator types and their definitions. All three graphs share the same node and edge set, where edges can be derived from abstract syntax tree as suggested by Zhang et al. (2025c).
We then obtain specifce node embeddings Horompt = GP Hoode = (Gedee) and H Hoperator = (GNNNG in R. Nxd xd stack them along a view dimension to form X E RNN Xdd with V = 3. and apply cross-view self-attention block with residual connection and layer normalization x LN(MHA(X, X, X) + X), where MHA is multi-head attention applied across views for each node (the sequence axis is the view axis; topology is unchanged).
Next, view-attention pooling module computes per-node attention weights with I-layer multi- layer perceptron (MLP) and tanh nonlinearity, followed by a softmax over views, and produces weighted sum across views H View.AttnPool(X] E Nxd Finally, graph readout Gpool aggregates node embeddings into a single graph representation. Zg poool (H) G Gooo (View Attn (CrooooolCsrsaapppp Autnh Autn(X)) which preserves edge directionality from the up- stream GNN while capturing cross-view contextual dependencies at the node level before graph-level summarization. Here, CrossGraphAAtn enriches each node with multi-view contextual dependen- cies, while ViewAttnPoo highlights which views are most informative.
Code Encoder. While Goode and Goperator primarily encode structural information derived from the different workflow graphs, this code encoder is designed to provide a complementary, holistic representation of the entire workflow code. To model the workflow-level embeddings, L-layer MLP to extract latent semantic features, enabling the model to learn intricate computational logic and tool interactions at global level. The code representation is computed Ze MT)Pc(C).
an
5

---

Published as a conference paper at ICLR 2026
Prompt Encoder. Instead of encoding agent prompts solely as node-level features (Zhang al., 2025c), we use a separate L-layer MLP to encode the entire workflow instruction prompt holistically, This approach captures role descriptions, behavioral intents, and global cuntet-rresulting in richer and more semantically informed representations. The prompt encoding is Zp = ML.Pp(P).
Aggregation Layer. The representations from the graph, code, and prompt encoders Zg. Ze. and Zp-are are concatenated and passed through final MLP layer. This aggregation mechanism adaptively integrates information across all views, enabling the model emphasize the most contextually relevant aspects. The final output of the encoder Enc(-) is computed as Z MLP((Zg, Ze, Zp)).
Consequently, the aggregation layer acts learnable fusion module. The attention- graph encoder produces Zg with cross-view interactions already embedded, and the downstream fusion assigns task-dependent importance the graph-, code-, and prompt-level representations, rather than treating all views equally informative. These encoders learn not only from different workflow perspectives but also at varying levels of granularity, specifically, at the graph level for agent interactions, the code level for logical structures, and the prompt level for agent-specific instructions.
3.3.2 DECODER NETWORKS
The decoder is a generative module that reconstructs G. C, and P from the latent variables Z to encourage learning generalizable representations of agentic workflows. It consists of stack of MLP layers. For simplicity, the decoder outputs the modality-specifie inpur embedding vectors of g, C. and P. Accordingly, we parameterize Dec(-) with an MLP and define MZg( Ć MZe( and P (Zp). This decoder is used only during pretraining for self-supervised reconstruction of speelit embeddings from Z and is not part of the encoding path at inference time.
3.4 CROSS-DOMAIN UNSUPERVISED PRETRAINING
In real-world scenarios, labeled performance data for agentic workflows are scarce due costly evaluation. To enable data-efficient training without label leakage, we optionally adopt two-phase strategy. Rather than directly supervising the encoder with performance labels, we first perform cross-domain unsupervised pretraining to obtain rich and generalizable workflow representations Z. No performance labels (e.g., success/failure) are used in this stage. The resulting representations improve sample efficiency for downstream prediction, in line with observations in NAS (White et al., 2020; Yan et al., 2020; 2021; Akhauri & Abdelfattah, 2024; Trirat & Lee, 2025b). When sufficiently many labels available, direct supervised learning of the predictor remains feasible.
Multi- Pretraining. We train the multi- encoder on M unlabeled workflow configurations by minimizing combined loss comprising reconstruction and contrastive objectives: Crec (Z,,,,))
M M || - + IIC, -IC,-C,,, C, P 2 and Leon + ΣM llg logg Here, C, denote the input graph, code, and prompt embeddings, respectively, while denotes reconstructions via modality-speciffe decoders. Notably, the graph branch reconstructs its own embedding target with dlop-gradient whereas code and prompt/text reconstructed in input space. The contrastive loss is instantiated cross-modale with in-batch sampling. For each configuration i, positives (Z,, Z are the index-aligned embeddings of the configuration across two different views (e.g., Gi with C,). while negatives are all other configurations within the batch. We symmetrize the objective by swapping anchor/target and average over the three view pairs (G,C), (G, P), and (C, P). This learning objective encourages the encoder to capture structure- and content-aware signals without observing performance outcomes. Thus, the total loss function Lene Lrec + Lcon.
5 exp(sim( (.).
3.5 PERFORMANCE PREDICTOR
Following the unsupervised pretraining of the multi-view encoder, we introduce lightweight performance predictor to guide exploration of the large agentic workflow space. This phase enables efficient identification of high-pon high-performin configurations with minimal supervision, using only a small set of labeled workflow perfoommreeoo pairs. As shown in Figure 2(c), our predictor operates learned workflow embeddings, enriched with task-specific context, to form joint representation F used for performance prediction and downstream search.

---

Published as a conference paper at ICLR 2026
Task Encoder. To capture task-specific characteristics, incorporate Task Encoder that gener- ates high-level semantic embeddings from natural-language task descriptions. These embeddings, derived from pretrained language models (e.g., T5 BERT), provide global context that helps differentiate tasks with similar surface form but distinct functional requirements. The task embed- ding is concatenated with the multi-view workflow representation, forming FF [Z, T), where Z is the encoded workflow and T is the task embedding. For workflow content itself' (prompts and code), we pair this with lightweight, domain-specifee encoders within the multi-view backbone to balance representational capacity with efficiency. This separation of modalities followed by fusion captures complementary compositional and contextual signals and supports generalization across heterogeneous tasks with varying operational goals and constraints.
The performance predictor is lightweight prediction head (eg. an ML.) trained on a limited set of labeled data (Xirain Vrain), where each Xman corresponds to F and Viman is the performance label binary success/failure or scalar score). We instantiate the objective to match the label type. For binary labels, we use a binary cross-entropy loss, i.e., Sped E [er log é + (1 - eş) log(1 - ē.)]. where é is the predicted success probability. For nu- mean squared error, i.e., Spred = N Ei (s, ) By operating
N
meric labels, semantically rich pretrained embeddings, the predictor attains strong accuracy in the low-data regime, enabling label-efficient search.
Integration with Workflow Optimization. With the trained predictor in place, we can perform predictor-guide search to efficiently explore the workflow configuration space. Rather than evaluating each configuration via full execution, we embed candidates into their joint representations FF and score them using the predictor. The top-scoring candidates are selected for evaluation. This substantially reduces computational cost by focusing on the most promising regions of the search space. A simple yet effective instantiation of this strategy uses random search to sample K workflow candidates from the full configuration space, and then ranks them using the learned predictor. We select the top-k configurations, averaged across samples in the benchmark, for evaluation. This predictor-as-ranker setup transforms random search into label-efficient guided procedure without requiring complex heuristics. Since our main contribution is the performance predictor rather than the optimization algorithm, we focus evaluation prediction accuracy and ranking quality (i.e., workflow utility) in the following section. Additional results on workflow optimization appear in $B.6.
4 EXPERIMENTS
We conduct a comprehensive evaluation of the proposed Agentic Predictor framework from mul- tiple perspectives. guided by the following questions: (Q1) How does Agentic Predictor perform as a predictor of agentic workflow performance compared to relevant baselines?
(Q2) How do different design choices of Agentic Predictor affect its predictive accuracy? (Q3) Is the pretraining phase helpful for maintaining prediction quality under varying numbers of labels? (Q4) Does Agen- tic Predictor maintain strong predictive performance under out-of-distributio shifts? (Q5) How does Agentic Predictor compare against few-shot LLM--based workflow performance predictors?
4.1 SETUP
Benchmarks. To evaluate performance predictors for agentic workflows, use FLORA-Bench (Zhang et al., 2025c), the only publicly available benchmark (to the best of our knowledge) that enumerates di- verse workflows across multiple domains and LLM backbones. It spans five datasets covering three core areas: code generation (HumanEval (Chen et al., 2021), MBPP et al., 2021)), mathematical problem solving (GSM8K (Cobbe et al., 2021), MATH (Hendrycks et al., 2021b)), and general reasoning (MMLU (Hendrycks et al., 2021a)). Table summarizes the bench- marks. Here, GD and AF denote the independently developed G-Designer (Zhang et al., 2025a) and AFlow (Zhang et al., 2025b) agentic frameworks, respectively. We emphasize structural and procedural diversity over raw difficulty; even for datasets often considered solved by prompting (e.g.. GSM8K), predicting success across workflow variants remains nontrivial. For each dataset, we randomly split instances into training (80%), validation (10%), and test (10%) sets.
Table 2: Summary of benchmark statistics.
Code Generation (GD/AF)
Math (GD/AF)
Reasoning (GD/AF)
Domains
workflows
739/ 38
300/42
189
Avg. # nodes
5.96/6.11
6.06/5.49
5.97/6.58
tasks
233 233
782/
2,400 2,400
samples
30,683 /7362
12,561 / /4,059
453,600 / 72,000
7

---

Published as a conference paper at ICLR 2026
Table 3: Performance comparison between Agentic Predictor and baselines. The best and second-best results are highlighted in bold and underlined, respectively. GD is G-Designer, and AF is AFlow.
Domain Midel
ColeGi
CodeAy
MaihGD
MathAF
Rra
BeasonAP
Average
Acturacy
Uuiliy
Accursscy
Unility
Accurssyy
Unilty
Assurasy
Tdiity
Accurscy
Uniliy
Accurasy
Luitty
Accumiiyy
Unilit
MLP
(10.00)
83.55
(10103)
76.16
(181.50)
75,12
[41.35)
73.54
(40.30
63.22
(201-4)
64.13
[10.31)
73.33
(A0.20)
(10.00)
71 11.54
(.1
62,41
(10.06)
75.45
(10.63)
58.45
(10-34)
74
(10. (10.44)
72,46
GCN
(60.04)
85.25
(20:10)
19:31
(20.34)
84.25
181.18)
72.75
(40.17)
64.12
(20
05.10
(20142
76.19
(41.58)
06:51
(20.03)
12.21
(20.85
59.18
(80.14)
57.13
(20.65)
91.83
(20.19)
75
(10.81)
72.10
CAT
(40.25)
(21.56)
OCN-11
(80.07)
81.31
(29.74)
18
(2)
$3.72
12358)
77.35
(50.74)
61.15
(0)
66.00
(=0.31)
75.04
(85.47)
(20 09)
23
(2)
50.10
(80.14)
10.25
(21.90)
(50.29)
67611
(80.57)
72.50
Grgo: Transformer
(10:19)
85.24
(10.4))
10.20
(HURS)
$4.71
74.00
240.70)
63.25
64.97
36)
(1022)
75.45
(10.9))
06.45
(10.05)
72.2
[11
60.92
(10.77)
85.93
(11.7))
30.00
(10-13)
77.97
(11.013
72.88
DNNGNN
$3.45
76.11
90117
One For All
(40.09)
85.76
(s0.12)
75M55
(20.34)
$1.05
($1.39)
(20.1)
(2))
(20.23
75.21
(25.44
04
(20.12)
(21.25)
00.15
(80.15)
(21.95)
67.6
(20,19)
76.33
(+1.00)
72.1N
Agestic Prodicur
(A0.05)
85.35
(.0:35)
$1.42
(30.47)
85.62
[63.46]
80.08
(40.17)
66.20
(10.21)
67.88
(10225
79.56
(10:47)
74
(10.01)
75.13
(10.45)
63.86
(10.00)
87.96
(30.4)
91 47
(10.16)
79.97
(10.33)
76.33
best beselinz
10.09
41.36
13:12
14,44
0.35
(5 Improvmenn)
(0 111)
(3.528)
(1,079)
(5.025))
(2.095)
(335%)
(4.0854)
(5. (6.18))
(1.19%)
(0.6755)
7896
-00.99% 1990
(2,098)
(3.7996)
Evaluation Metrics. To ensure fair and consistent comparison, we strictly adhere to the official evaluation protocols specified by the benchmark.
Accuracy quantifies how well model predicts agentic workflow performance. It is defined as
accuracy Diss Σ, [ (ê;; = e,), where |Dist] is the size of the test split, and é; and es denote the predicted and ground-truth performance, respectively. 1(+) is the indicator function, which returns if é; e;, and 0 otherwise.
Utility evaluates the consistency between the workflow rankings predicted by the model and the ground-truth rankings, emphasizing the model's ability to determine the relative order of different workflows. First, we calculate the ground-truth and predicted success rates of a workflow W; by averaging e and é across all tasks in Diest Then, we rank the workflows and extract the top-k workflows according to the respective scores, resulting in two ordered sets: H = (W:) and A = (- Formally, utility= = Σ 1(W) E H.).
Baselines. Since there is no direct baseline method specifically designed for performance prediction in agentic systems, we adopt comparison baselines from the benchmark paper. Some of these methods have previously been used as performance predictors for NAS (White et al., 2021). The selected baselines include one naive MLP and several strong graph-based models: GCN (Kipf & Welling, 2017), GAT (Veličković et al., 2018), GCN-II (Chen et al., 2020), Graph Transformer (Shi et al., 2021), Dir-GNN (Rossi et al., 2024) and One For All (Liu et al., 2024a).
Implementation Details. For all methods, we follow the same setup as suggested by Zhang et al. (2025c). Specifically, we use 2-layer backbone with hidden dimension of 512, set dropout to 0.5, and use a batch size of 5 512. Models are optimized with the Adam optimizer (Kingma & Ba, 2014) using learning rate of 1 X 10-4 and weight decay of 5 10-4. Training is conducted for 200 epochs on a single NVIDIA A 1100SSX44800CK GPU, and the best checkpoint is selected by the highest accuracy on the validation subset. Our framework is encoder-agnostic by design. To ensure controlled comparison, we reuse the all Mil1MMnnLMM LM-L6 L6-v2 (Wang et al., 2020) text encoder and adopt CodeRank Embed (Suresh et al., 2025) for code, both via Gurevych, 2019) with default hyperparameters. Text inputs are truncated to 256 tokens and encoded into 484-dimensional vectors, while code inputs (function-level nodes and full-workflow files) are tokenized and truncated to model limits (up to 8, 8.192 tokens) producing 768-dimensional vectors. All embeddings are finally mapped into a unified 512-dimensional space using a 2-layer MLP (I- = 2).
(Reimers &
4.2 MAIN RESULTS (Q1)
We report all experimental results for agentic workflow performance prediction, averaged over three runs with different random seeds on the same dataset. Table 3 presents the main performance scores and standard deviations for all datasets. Our proposed framework, Agentic Predictor, consistently outperforms baseline methods across the three task domains. For accuracy, Agentic Predictor achieves top results in each domain- 85.62% 79.56%, and 87.96%, respectively -yyediiellyyelddingg the highest overall average accuracy of 79.97%. This corresponds to improvements of 2.05% to 6.90% over the comparison baselines. Utility scores show a similar pattern. Agentic Predictor attains the highest utility in code generation (81.42%) and math problem solving (74.08%). well as
8

---

Published as a conference paper at ICLR 2026
Table 4: Results of ablation study different input view variations.
Variations
Code Generation
Math Problem
Common Reasoning
Average
Code
Graph
Text
Accuracy
Utility
Accuracy
Utility
Accuracy
Utility
Accuracy
Utility
82.04
75.66
75.70
68.52
83.19
91.51
80.31
78.56
84.44
77.22
79.14
67.99
87.00
91.03
83.53
78.75
79.87
70.34
76.60
68.45
71.04
74.84
69.94
83.72
73.97
75.86
70.18
86.88
86.14
82.15
76.76
82.27
77.28
76.03
66.66
53.21
70.82
65.72
82.45
74.64
75.70
67.83
69.47
70.55
75.87
71.01
85.62
80,08
79.56
74.08
87.96
91.47
84.38
81.88
Table 5: Results of ablation study on different input graph variations.
Variations
Code Generation
Math Problem
Commong Reusoning
Average
Single View
Multi
Accuracy
Utility
Accuracy
Udility
Accuracy
Utility
Accuracy
Unility
82.58
78.52
78.57
67.51
86.95
90.14
82. 70
78.72
84.44
77.22
79.14
67.99
87.00
91.03
83.53
78.75
near-best score in reasoning tasks (91.47%), second only to GCN (91.82%) On average, it achieves the highest utility score of 76.33%, representing improvements of 3.79% to 5.87% over the baselines. These results demonstrate that Agentic Predictor not only enhances predictive accuracy but also improves downstream utility across diverse agentic workflows, highlighting its robustness and generalizability. The consistent performance gains further underscore the advantages of leveraging multi-view encoding for heterogeneous agentic workflows.
4.3 ADDITIONAL ANALYSES
Ablation Study (Q2). To substantiate our contributions on specific design of multi-view workflow encoding in Agentic Predictor, we conduct ablation study two main components using the AFlow subset: multi-view encoder and multi-graph encoding techniques. According to the results in Table 4, find that incorporating all three input views-coe code, graph, and text-results in the best overall performance across all tasks. Specifically, the full model configuration achieves the highest average accuracy (84.38%) and utility (81.88%), ), underscoring the complementary value of each modality. Notably, the removal of any single view leads to a consistent drop in performance. demonstrating the synergistic role of multimodal inputs in prediction capabilities of Agentic Predictor.
Furthermore, results in Table 5 reveal the significance of multi-graph encoding. When multiple graphs are used instead of a single graph, the model shows a clear performance improvement, particularly in code generation (accuracy improves from 82.58% to 84.44%) and reasoning tasks (utility rises from 90.14% 91. 91.03%). This supports our hypothesis that different graph perspectives enrich structural context and lead to more robust representations. Together, these findings validate the architectural choices in Agentic Predictor, demonstrating that both multi-view and multi-graph designs are integral to its superior performance.
Effects Pretraining Phase (Q3). Since acquiring a large amount of ground-trute labels from agentic workflows is expen- sive, we examine whether cross-domain unsupervised pretrain- ing (denoted Agentic Predictor+) benefits settings where labeled instances are limited. We vary the label ratio from 0.1 to 0.5, selecting labeled samples from the training split of all datasets in the benchmark. We pretrain the proposed multi-view encoder on the remaining 50% (M) of the training set with a batch size of 32 for 20 epochs. See more details in $B.5. On average, the results shown in Figure 3 indicate that Agentic Predictor+ consistently outperforms all baseline mod- els across all label ratios, demonstrating the effectiveness our unsupervised pretraining strategy. The gains are especially pronounced in low-label regimes: at a 0.1 label ratio, Agentic Predictor+ maintains an accuracy above 73%, while other models drop closer to 70%. These findings underscore the importance of leveraging cross-domain structure through pretraining for generalizable workflow performance prediction, especially when direct supervision is limited.
-LPP - -ON GAT GCN-II
Graph Transformer
o For Agent Predictor
Agent Predictoo+
78
72
70
0.5
0.4
0.3
0.2
6.1
Label Ratio
Accuracy comparison between Agentic Predictor and the baselines across varying label ratios.
Figure

---

Published as a conference paper at ICLR 2026
Out-of-Distribution (OOD) Performance (Q4). We evaluate OOD robustness under syssemm generalization (training on one agentic framework and testing on another) and cross- gen- eralization (training one task domain and testing on disjoint domains). As shown in $B.4, Agentic Predictor consistently generalizes beyond in-distribution memorization, maintaining strong performance and preserving relative workflow rankings across both settings. For example, when trained on AFlow and tested on G-Designer, Agentic Predictor improves average accuracy from 59.52% (best baseline) to 62.05% and utility from 55.33% to 58.49%. Similar gains hold in the reverse direction and under cross-domain splits.
Comparison with LLM Predictors (Q5). We evaluate 5-shot, prompt-based LLM classi- fiers (temperature 0) using the standardized LLM-PP template (Jawahar et al., 2023) with GPT-4.1, Claude Sonnet, and Gemini 2.5 Flash. As shown in Table 9, these prompt-only L.MM predictors substantially underperform our graph-based model, indicating that they struggle to exploit the struc- tured nature of agentic workflows. Agentic Predictor achieves 84.9 94.97% accuracy and 81.37% utility, far exceeding the second-best GPT-4.1 at 62.86% and 58.92%, while also avoiding the considerable latency and monetary overhead of LLM inference. Overall, few-shot LLMs serve as a useful baseline but remain less effective and less economical for large-scale agent search.
4.4 RESOURCE COST
We further examine the efficiency of Agentic Predictor measured by computation time and memory usage. As shown in Table 6, our framework remains competitive with
Table Computation cost comparison.
sippelo
MI.
0.185
0.03
0.022
3,225
standard GNN baselines despite its higher model capac- ity and richer imput features, requiring only 0.054ms and
OUN4
4,55
(185
3.545
Crugh
-odeers 0.49GB to score a workflow at inference- orders of mag- nitude faster and cheaper than few-shot LLM predictors. A full run of Agentic Predictor involves a one-time cost of
CRN
1.63
0407
0403
3,5%
C271-11
DnsQpeRitee APD
Nia
169-30
Apras Predice
2.790
1.2 1100 GPU-hours (200 supervised epochs plus 20 optional pretraining epochs), with modest memory requirements that fit on a single 16 GB GPU. In contrast, LLM-based scoring costs about $21 per 1,000 candidates (=$0..221 per sample with Gemini 2.5 Flash), implying break-even point after only 110-120 evaluations assuming a $2/hr A A100 rate. As realistic searches involve thousands of candidates and the trained predictor is reusable across tasks and frameworks, this modest one-time cost is quickly amortized, making Agentic Predictor far more economical than repeated LLM calls while offering near-zero marginal latency and higher accuracy.
Full experimental results on different underlying LLMs, various GNN backbones, LLM classifier comparison, and outto--distiibution test are reported in Tables 7 8, 9, 10, 11 and 12, respectively. An additional evaluation of performance predictors used as reward function for agentic workflow optimization, and case study findings are also provided in $B.6 and $C.
5 CONCLUSIONS
This paper introduces Agentic Predictor, a novel framework for efficient prediction of agentic work- flow performance that leverages a multi-view predictive approach. By integrating multi-view graph structures, code semantics, and prompt embeddings into unified representation, Agentic Predictor captures the diverse characteristies of agentic systems. Moreover, it employs cross-domain unsuper- vised pretraining to mitigate the challenge of limited labeled data, thereby enhancing generalization across varied tasks. Through comprehensive experiments spanning three domains, Agentic Predictor consistently outperforms strong baselines in predictive accuracy and workflow utility.
Limitations and Future Work. While Agentic Predictor exhibits strong performance, it has certain limitations. The current predictor focuses on binary success metrics, constrained by the available benchmark, which may overlook more nuanced aspects of workflow behavior. Evaluating on new, independently curated agentic benchmarks is an important direction for future work. Additionally, adapting to highly specialized domains may still require some labeled data. Future work includes expanding to multi-objective optimization (c.g., balancing accuracy and cost), incorporating richer views such as temporal traces and user feedback, and exploring human-in -aman-in-theeloop workflows for real- time refinement. These directions aim to make Agentic Predictor more generalizable and interactive in complex, real-world settings.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was conducted as part of the research activities at DeepAuto.ai. This work was supported by Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (RS-2019-1119007, Artificial Intelligence Graduate School Meta-leaming Applicable to
Program (KAIST)), IITP grant funded by MSIT Real-world Problems), Center for Applied Research in Artificial Intelligence (CARAI) grant funded by DAPA and ADD (UD190031RD) and National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2023-00256255)
REPRODUCIBILITY STATEMENT
We facilitate reproducibility by providing an anonymous repository with all source code at ht htspps / thub. cwm hamm/DeepAuto-AAe
ctor. Algorithm provides the complete pseudocode of the proposed framework. For experimental consistency, the random seed for each run 2", where is the running index starting from 0.
REFERENCES
Yash Akhauri and Mohamed S Abdelfattah. Encodings for prediction-based neural architecture search. In Forty-first International Conference on Machine Learning, 2024.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang. Carrie Cai, Michael Terry, Quoe Le, al. Program synthesis with large language models. arXiv preprint arXiv:2 108.07322, 2021.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained code. arXiv preprint arXiv:2107.03331 1374444 2021.
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In Proceedings of the 37th International Conference Machine Learning,
pp. 1725-1735, 2020. 8
Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang. Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In The Twelfth International Conference on Learning Representations, 2024. 2,
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. In Forty-first International Conference on Machine Learning, 2024. 2,
Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, Volume 1 (Long and Short Pupers), pp. 2368 2378, 2019.
19
Lukasz Dudziak, Thomas Chau, Mohamed Abdelfattah, Royson Lee, Hyeji Kim, and Nicholas Lane. Brp-nas: Prediction-bain based nas using gens. In Advances neural information processing systems, pp. 10480-104900, 2020. 3
Mourad Gridach, Jay Nanavati, Khaldoun Zine El Abidine, Lenon Mendes, and Christina Mack. Agentic ai for scientific discovery: A survey of progress, challenges, and future directions. arXiv preprint (5l::250030877999 2025. 1
11

---

Published as a conference paper at ICLR 2026
Taicheng Guo, Xiuying Chen, Yaqi Wang. Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: a survey of progress and challenges. In Proceedings of the Thirty- Thirt International Joint Conference on Artificial Intelligence, pp. 8048-8057, 2024.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations, 2021a. 7
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021b.
Sirui Hong. Mingehen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang. Ceyao Zhang. Zili Wang. Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schnidhuber. MetaGPT: Meta programming for a multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations. 2024. 1,2
Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems. In The Thirteenth International Conference on Learning Representations, 2025a. 1.3
Yue Hu, Yuzhu Cai, Yaxin Du, Xinyu Zhu, Xiangrui Liu, Zijie Yu, Yuchen Hou, Shuo Tang, and Siheng Chen. Self-evolving multi-agent networks for software development. In The Thirleenth International Conference on Learning Representations, 2025b.
Dongyeong Hwang, Hyunju Kim, Sunwoo Kim, and Kijung Shin. Flowerformer: Empowering neural architecture encoding using a flow-aware graph transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6128661866666 6137, 2024. 3
Ganesh Jawahar, Muhammad Abdul-Mageed, Laks VS Lakshmanan, and Dujian Ding. Llm perfor- mance predictors are good initializers for architecture search. arXiv preprint arXiv:2310.16712, 2023. 10, 16
Han Ji, Yuqi Feng, and Yanan Sun. Cap: a context-aware neural predictor for nas. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pp. 4219-4221 4227, 2024. 3
Zixuan Ke, Fangkai Jiao, Yifci Ming, Xuan-Phi Nguyen, Austin Xu, Do Xuan Long, Minzhi Li, Chengwei Qin, Peifeng Wang, Silvio Savarese, et al. A survey of frontiers in 11m reasoning: Inference scaling, learning reason, and agentic systems. arXiv preprint 2544:7250 2025. 1,2
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 14X2, 69800, 2014.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. In The Twelfth International Conference on Learning Representations, 2024a.
Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. A dynamic LLM-powered agent network for task-oriented agent collaboration. In First Conference Language Modeling, 2024b. 3,3
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. GAIA: benchmark for general AI assistants. In The Twelfth International Conference on Learning
Representations, 2024. 1
Boye Niu, Yiliao Song. Kai Lian, Yifan Shen, Yu Yao. Kun Zhang, and Tongliang Liu. Flow: Modu- larized agentic workflow automation. In The Thirteenth International Conference on Learning Representations, 2025. 2
12

---

Published as a conference paper at ICLR 2026
Yameng Peng, Andy Song, Vie Ciesielski, Haytham M. Fayek, and Xiaojun Chang. Pre-nas: predictor- assisted evolutionary neural architecture search. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1066-1074, 2022.
Aske Plaat, Max van Duijn, Niki van Stein, Mike Preuss, Peter van der Putten, and Kees Joost Batenburg, Agentic large language models, survey, arXiv preprint arXiv:2503.23037 2025. 1,
Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong. Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. ChatDev: Communicative agents for software development. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 15174-15186, 2024. 2,3
Shuofei Qiao, Runnan Fang, Zhisong Qiu, Xiaobin Wang. Ningyu Zhang, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Benchmarking agentic workflow generation. In The Thirteenth International Conference Learning Representations, 2025. 2
Nils Reimers and Iryna Gurevych. Sentence-bert. Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing, 11 2019. 8
Emanuele Rossi, Bertrand Charpentier, Francesco Di Giovanni, Fabrizio Frasca, Stephan Günnemann, and Michael M Bronstein. Edge directionality improves learning on heterophilic graphs. In Learning on graphs conference. pp. 25-1. PMLR, 2024. 8
Yu Shang, Yu Li, Keyu Zhao, Likai Ma, Jiahe Liu, Fengli Xu, and Yong Li. Agentsquare: Automatic Ilm agent search in modular design space. In The Thirteenth International Conference Learning Representations, 2025. 1,3
Yunsheng Shi, Zhengjie Huang, Shikun Feng. Hui Zhong, Wenjing Wang, and Yu Sun. Masked label prediction: Unified message passing model for semi-supervised classification. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pp. 1548-1554, 2021. 8
Tarun Suresh, Revanth Gangi Reddy, Yifei Xu, Zach Nussbaum, Andriy Mulyar, Brandon Duderstadt, and Heng Ji. CoRNStack: High-quality contrastive data for better code retrieval and reranking. In The Thirteenth International Conference on Learning Reprexentations, 2025. 8
Patara Trirat and Jae-Gil Lee. MONAQ: Multi-objective neural architecture querying for time-series
analysis resource-constrained devices. In Findings of EMNLP, 2025a. 2
Patara Trirat and Jae-Gil Lee. Pasta: Neural architecture search for anomaly detection in multivariate
time series. IEEE Transactions on Emerging Topics in Computational Intelligence, 2eclliience,944222224-9999 (4):2924-2 2025b. 4, 6
Patara Trirat, Wonyong Jeong, and Sung Ju Hwang. AutoML-agent: A multi-agent LLM framework for full-pipeline autoMI. In Forty-second International Conference on Machine Learning, 2025. 2
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018. 8
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang. and Ming Zhou. Minilm: Deep self- attention distillation for task-agnostic compression of pre-trained transformers. Advances in neural information processing systems, 33:5776-5788, 2020. 8
Colin White, Willie Neiswanger, Sam Nolen, and Yash Savani. A study encodings for neural architecture search. In NeurlPS, pp. 20309-20319, 2020. 4, 6
Colin White, Arber Zela, Binxin Ru, Yang Liu, and Frank Hutter. How powerful are performance predictors in neural architecture search? In Advances in Neural Information Processing Systems, 2021. 1,
13

---

Published as a conference paper at ICLR 2026
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang. Shaokun Zhang. Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. AutoGen: Enabling pext-gen LLM applications via multi-agent conversations. In First Conference on Language Modeling, 2024. I
Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Dingwen Yang. Chenyang Liao. Xin Guo, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, and Yu-Gang Jiang. AgentGym: Evolving large language model-based agents across diverse environments, 2024. 1
Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: A survey. Science China Information Sciences, 181101 2025. 1,2
Frank F. Xu, Yufan Song, Boxuan Li, Yuxuan Tang, Kritanjali Jain, Mengxue Bao, Zora Z. Wang, Xuhui Zhou, Zhitong Guo, Murong Cao, Mingyang Yang, Hao Yang Lu, Amaad Martin, Zhe Su, Leander Maben, Raj Mehta, Wayne Chi, Lawrence Jang, Yiqing Xie. Shuyan Zhou, and Graham Neubig. TheAgeniCompany: Benchmarking Ilm agents on consequential real world tasks, 2024.
Shen Yan, Yu Zheng, Wei Ao, Xiao Zeng, and Mi Zhang. Does unsupervised architecture representa- tion learning help neural architecture search? Advances in neural information processing systems, 2020.
Shen Yan, Kaiqiang Song, Fei Liu, and Mi Zhang. Cate: Computation-awar neural architecture encoding with transformers. In International Conference on Machine Learning, pp. 11670-11681. PMLR, 2021. 6
Zhilin Yang. Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pp. 2369-2380, 2018. 19
Rui Ye, Shuo Tang, Rui Ge, Yaxin Du, Zhenfei Yin, Siheng Chen, and Jing Shao. MAS-GPT: Training LLMs to build LLM- based multi-agent systems. In Forty-second International Conference Machine Learning, 2025. 3
Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Dongsheng Li, and Deqing Yang. Evoagent: Towards automatic multi-agent generation via evolutionary algorithms. In NAACL, 2025. 1
Guibin Zhang, Yanwei Yue, Xiangguo Sun, Guancheng Wan, Miao Yu, Junfeng Fang, Kun Wang, Tianlong Chen, and Dawei Cheng. G-designer: Architecting multi-agent communication topologies via graph neural networks. In ICML, 2025a. 3, 7, 16
Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. AFlow: Automating agentic workflow generation. In The Thirteenth International Conference
on Learning Representations, 2025b. 1, 3, 7, 16, 18
Yuanshuo Zhang, Yuchen Hou, Bohan Tang. Shuo Chen, Muhan Zhang, Xiaowen Dong, and Siheng Chen. FLORA: GNNs as predictors of agentic workflow performances. In The Fourth Learning on Graphs Conference, 2025c. 3, 5, 6, 7, 8, 15, 16, 18
Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. GPTSwarm: Language agents optimizable graphs. In Forty-first International Conference on Machine Learning, 2024. 1,3
14

---

Published as a conference paper at ICLR 2026
A PSEUDOCODE OF AGENTIC PREDICTOR
We present the pseudocode of the proposed Agentic Predictor framework in Algorithm below.
Algorithm Overall Procedure Agentic Predictor
Initialization: Multi-View Encoder Enc(.) and Performance Predictor Model Me
Input: User Instruction (or Task Description) T E T and Training Data Dirain
1: Multi-View Graph Construction ($3.3)
2: Construct node-aligned view set G = {g. = (V,E,
{prompt, code, operator)
where X, is the view-specific node features.
3: & Cross-Domain Unsupervised Pretraining ($3.4, Optional)
4: Sample M unlabeled workflows Wi, W2; WM from multiple domains
5: for each W, = (G,, C. P.) do
6:
Z₁ f Enc(9,,C P.)
b Encode multiview graph, code, and prompts
7: (G., C,. P.) + Dec (Z:)
Decode reconstructions
8: end for
9: Lene = Free + Leon
b Minimize total pretraining loss
10: Training Performance Predictor ($3.5)
11: Obtain (small) labeled dataset {(W,, 7j, ez)] from D
12: for each (W, do
Zj Enc(W))
Encode workflow b. Encode task description Form joint representation Predict performance
14:
T. - (TskEn)
F MLP((Z,. T,
16:
êj -F))(F))
end for
18: Train Me using binary cross-entropy loss Lepea(ejj éj). where {e, 1 Tredictor-Guide Candidate Ranking
20: Sample K candidate workflows {W
21: for each W& do
22:
ZE +- Enc(Wi)
Encode workflow b Encode task b Predict score
23: FR + ( T))
24:
éx < (F(FF)
25: end for
26:
Rank all {Wk) by predicted scores ék
27:
return top-k ranked workflows for final evaluation
B ADDITIONAL EXPERIMENTAL RESULTS
This section provides complementary studies that further characterize our approach: robustness when the agent-controller LLM backbone varies ($B.1) I); an ablation over multiple GNN backbones ($B.2) 2);
comparison to few-shot LLM predictors ($B.3); and out-of-distribution (OOD) generalization evaluations ($B.4).
B. PERFORMANCE ON DIFFERENT LLM BACKBONES
As shown in Table 7, assess whether predictor performance is robust when the agentic work- flows are driven by different LLMs. Concretely, we replicate our evaluation while swapping the controller LLM among GPT-40-mini, DeepSeek, Qwen 7B, and Mistral 7B, holding the training data construction, multi-view encoder, and evaluation protocol fixed. Except for the Mistral 7B case, Agentic Predictor exhibits stable performance and preserves the relative ranking of workflows across these backbones, indicating that it captures structural and behavioral regularities of agentic programs rather than idiosyncrasies of any single L.M.
B.2 PERFORMANCE ON DIFFERENT GNN BACKBONES
Our main experiments use 2-layer GCN (hidden size 512) following the standard setup in FLORA- Bench (Zhang al., 2025c), enabling controlled comparison to baseline predictors. To test
15

---

Published as a conference paper at ICLR 2026
Table 7: Results on different backhones driven agentic workflows.
Domain
GPT----mini
DeepSeek
Qwen 7B
Mistral 7B
Model
Accuracy
Utility
Accuracy
Utility
Accuracy
Utility
Accuracy
Utility
MI.P P
83.88
76.16
85.89
71.72
84.25
80.52
89.23
85.07
GCN
82.94
80.40
86.56
75.69
86.71
84.48
92.58
88.48
GAT
83.03
80.25
84.42
75.18
86.98
84.26
92.62
88.72
GCN-II
82.81
79.48
84,34
75.68
85.17
82.71
90.94
85.89
Graph Transformer
79.83
73.06
84.65
92.80
88.87
Dir-GNN
84.85
79.81
85,38
71.27
86.30
84.50
91.87
88.47
71.92
73.23
80:42
Agentic Predictor
85,33
81.42
88.39
76.64
86,99
85.02
92.33
88.69
architecture sensitivity, we conduct an ablation over five diverse GNN backbones-GCNN GAT, GCN-II, Graph Transformer, and Dir-GNN-whin keeping the prompt and code views fixed. As presented in Table All backbones yield comparable predictive accuracy and replicate the same trends, reinforcing that the performance improvements stem from the multi- encoding and pretraining rather than specific GNN design. These results support the architecture-agnosti nature of the Agentic Predictor.
B.3 COMPARISON WITH L.M PREDICTORS
We compare against few-shot, prompt-based LLM classifiers implemented with a standardized LLte-P-style template (Jawahar et al., 2023) with 5-shot and temperature set using GPT-4.1, Claude Sonnet, and Gemini 2.5 Flash. The results in Table 9 are consistent with prior findings on FLORA-Bench (Zhang et al., 2025c) (which evaluated DeepSeek-v3), these prompted LLMs underperform even simple MLP predictor and substantially trail our graph-based approach. A likely reason is that prompted LLM classifiers do not exploit the structured execution patterns and tool-usage dynamics present in agentic workflows. Beyond accuracy, prompted LLM inference incurs a per-sample monetary and latency cost, whereas our predictor amortizes cost at training time, In our setup, generating predictions for up to 1, 1,000 samples per task with LLM prompting required approximately $300, implying considerably higher expense at full-benchmark scale. By contrast, the learned predictor scales large candidate sets with constant per-sample computational cost at inference. Overall, while few-shot LLMs provide useful baseline, they are less effective and less economical for large-scale agent search.
B.4 OUT-OF-DISTRIBUTTI (OOD) GENERALIZATION PERFORMANCE
We study two factors that enable OOD robustness. First, the multi-view encoder jointly represents workflows via graph, code, and prompt views, all of which are architecture-annstic This design allows unseen agents and tools to be incorporated as long as their implementations and textual descriptions are available; the graph encoder embeds novel entities through structural and attribute signals without relying fixed IDs. Second, cross-domain unsupervised pretraining diverse unlabeled workflows equips the encoder with priors over common structural and behavioral motifs (e.g., tool invocation patterns and reasoning flows), improving robustness unseen configurations.
Regarding evaluation, following RQ3 in L.ORA-Bench (Zhang et al., 2025c), we perform two levels of OOD generalization. Cross-system generalization: train AFlow (Zhang et al., 2025b)) and test on another (e.g., G-Designer (Zhang et al., 2025a)) as well as cross-domain generalization: train on one set of downstream tasks (e.g., math) and on disjoint tasks (e.g., coding) not observed during training. As presented in Table 10, Table 11 and Table 12, across both settings, Agentic Predictor maintains strong performance and preserves relative workflow rankings, indicating that it generalizes beyond in-distribution memorization.
agentic framework (e.g.,
B.5 EFFECTS OF PRETRAINING PHASE (FULL RESULTS)
Since acquiring a large amount of ground-trun truth labels from agentic workflows is expensive, we examine whether cross-domain unsupervised pretraining (denoted Agentic Predictor+) benefits settings where labeled instances are limited. We vary the label ratio from 0.1 to 0.5, selecting labeled samples from the training split of all datasets in the benchmark. Concretely, construct
16

---

Published as a conference paper at ICLR 2026
Table 8: Results different GNN backbones of Agentic Predictor.
Domain
Code Generation
Math Problem
Common Reasoning
Average
GNN Backbone
Accuracy
Utility
Accuracy
Utility
Accuracy
Utility
Accuracy
Utility
GCN
85.62
80.08
79.56
74.08
87.96
91.47
84.38
81.88
GAT
83.74
73.11
75.86
67.03
86.95
87.20
82.19
75.78
GCN-IT
84.71
73.83
76.68
68.41
86.76
86.04
82.72
76.09
Gruph Transformer
83.22
78.17
76.64
70.03
89.50
82.25
Dir-GNN
84.62
79.64
80.26
75.03
87.93
94.77
84.27
83.15
Table 9: Comparison between Agentic Predictor and LLM-based few-show classification.
Domain
Code Generation
Math Problem
Common Reasoning
Average
Model
Accuracy
Utility
Accuracy
Utility
Accuracy
Udility
Accuracy
Utility
GPT-4.1 (-$59)
62.42
57.00
67.08
52.97
59.10
66.79
62.86
58.92
Claude Sannet (~5202)
56.72
51.65
64.62
57.32
44.50
41.25
55,28
50.07
Gemini 2.5 Flash (-521)
60.52
58.94
51.60
55.21
59.20
63.17
57.10
59,11
Agontic Predictor
84.40
78.84
80.10
77.61
90.40
87.67
84.97
(a) Code Generation.
(b) Math Problem.
(c) Reasoning Tasks.
(d) Average.
(e) Code Generation.
(l) Math Problem.
(g) Reasoning Tasks.
(h) Average.
Figure 4: Comparison of accuracy (upper) and utility (lower) between Agentic Predictor and the baselines across varying label ratios.
unlabeled corpus by pooling all workflow configurations from the remaining training splits of the FLORA- -ench across Code, Math, and Reasoning tasks and both AFlow and G-Designer frameworks, sampling uniformly over the pool without additional deduplication or domain re-weighting, This yields M = 232, 104 distinct samples (+= 6.56% Code, 2.86% Math, 90.58% Reasoning). No validation and test workflows or labels are included to avoid leakage. We pretrain the proposed multi-view encoder with a batch size of 32 for 20 epochs.
Following the average results in the main text, provide comprehensive comparison of accuracy (top row) and utility (bottom row) across three task domains- code generation, math problems, and reasoning-under varying label ratios from 0.5 to 0.1 (Figure 4).
Across all settings, our proposed framework, Agentic Predictor, and its pretrained variant, Agen- tic Predictor+, consistently outperform baseline models, especially in low-resource scenarios. In the code generation domain (Figures 4a, 4e), Agentic Predictor+ achieves superior accuracy and notably higher utility as the label ratio decreases, outperforming all graph- and non- non-graph baselines. Similarly, for math problems (Figures 4b, 4f). Agentic Predictor+ maintains a stable accuracy even as labeled data diminishes, while significantly improving utility, indicating better performance in label-scarce conditions. In reasoning tasks (Figures 4c, 4g), although accuracy deltas narrow between models, Agentic Predictor+ sustains strong utility across all label ratios, highlighting its robustness in generalization. When averaged across domains (Figures 4d, 4h), Agentic Predictor+ shows clear advantages in both metrics under limited supervision. The utility improvements are particularly prominent, suggesting that our pretrained encoder captures transferable representations that enhance decision making, even when fine-tuning data is sparse. These findings validate the efficacy of the unsupervised pretraining phase and highlight the importance of cross-domain datasets for pretraining.
decision-making,
17

---

Published as a conference paper at ICLR 2026
Table 10: Results when train on AFlow and test G-Dessiner.
Domain
Code Generation
Math Problem
Common Reasoning
Average
Model
Accuracy
Utility
Accuracy
Utility
Accuracy
Utility
Accuracy
Utility
GCN
56.76
54.29
49.64
51.95 51.92
61.37
54,13
55.92
53.45
GAT
57.25
56.05
48.29
48.71
57.03
53.12
54,19
52.63
GCN-II
64.16
62.67
48.85
50.56
65.55
52.76
59.52
55.33
Gruph Transformer
60.83
47.73
48.87
54,81
One For All
58.97
53.25
50.60
51.02
63.84
55.22
57.80
53.16
Agentic Predictor
65.02
64.91
53.62
52.83
67.51
57.74
62.05
58.49
Table 11: Results when train on G-Designer and test on AFlow.
Domain
Code Generation
Math Problem
Common Reasoning
Average
Model
Accuracy
Utility
Accuracy
Utility
Accuracy
Utility
Accuracy
Utility
GCN
58.21
$7.33
67.57
54.63
57.51
53.37
61.10
55.11
GAT
59.29
66.34
52.70
56.07
50.38
60.57
GCN-II
58.75
61.17
67.32
52.96
55.93
$2.19
60.67
55.44
Graph Transformer
60.52
61.44
58.97
57.49
56.50
54.86
58,
One For All
62.01
54.57
58.72
61.23
59.40
54,17
60.04
56.66
Agentic Predictor
60.94
59.75
69.11
63.02
58.56
56.73
62.87
59.83
B.6 WORKFLOW OPTIMIZATION RESULTS
In the main experiments, we demonstrate the feasibility and robustness of predicting agentic work- flow performance. However, it remains an open question whether such predictions effectively contribute to improving efficiency and to what extent they may introduce performance degradation in agentic workflows. To investigate this, we evaluate whether using Agentic Predictor as a predictor enhances the optimization of agentic workflows compared to alternative baselines. Specifically, we measure the performance improvement (or loss) incurred when using performance predictors.
To ensure fair comparison, we adopt the same experimental setup as Zhang et al. (2025c), which provides unified platform for optimizing agentic workflows and evaluating their performance. During the optimization process on each benchmark, predictor is used to estimate the performance of candidate agentic workflows. These predicted performance values are treated as rewards to guide the optimization. Upon completion of the optimization, the quality of the resulting workflows is assessed based on their accuracy score on held-out test tasks.
We compare Agentic Predictor against four baselines: (I) the ground truth bascline, which directly evaluates agentic workflows to obtain ground truth performance scores (as done in the original AFlow (Zhang al., 2025b)); (2) two strong GNN-based predictors GCN and GAT; and (3) random baseline, which assigns random performance scores as rewards. This experiment is conducted across five benchmarks: MATH, GSM8K, MBPP, HumanEval, and MMLU.
As in Table 13, Agentic Predictor consistently outperforms the random, GCN, and GAT baselines, achieving an average accuracy score of 74.43% significantly higher than random (62.56%). GCN (68.422 (68.4 and GAT (71.0%%) Notably, as a predictor incurs zero search cost compared to the ground- truth's 's cost of $39.83, this result underscores the effectiveness and efficiency of Agentic Predictor as reliable predictor for optimizing agentic workflows. Note that the search cost is 0 because the predictors do not incur any LLM inference cost. Note that the search cost when using the performance predictor is effectively zero because the predictor incurs no L.M inference calls (i.e., no downstream task executions task queries) to decide whether the current workflow has failed. The lightweight LLM update applicd after each pass/fail decision, whose cost is about 0.005 0.01 per update round with GPT-4. I -mini on the existing workflow, is excluded from the reported costs for all methods.
In real-world deployments, Agentic Predictor can be combined with any workflow generator (e.g.. AFlow). In such cases, the overall cost decomposes into (1) the candidate -egerrattonn LLM cost (shared across all search strategies) and (2) the evaluation cost. Our predictor reduces (2) by replacing most candidate evaluations with cheap, yet more accurate predictions, while incurring only a one-time training cost (see $4.4). This analysis applies equally to the other predictors as well.
18

---

Published as a conference paper at ICLR 2026
Table 12: Results cross-domain OOD test.
Domain
Code-Math
Cale-Reasom
Mall-Reasen
Math-Code
Reass-Cole
Reawo-MMath
Average
Model
Arroracy
Uniliy
Accuracy
Unilly
Accuracy
Utiliy
Acturacy
Unilit
Accurary
Urility
Accuracy
Unility
Acruracy
Unility
GCN
54.07
52.61
50.07
48.75
32.56
14.49
50.65
GAT
49.42
53.73
51,02
47.40
33.79
44.12
49.55
GCN
56.02
46.49
53.64
39.68
51.55
38.13
57.93
65.68
Graph Trnndeuu
47.67
56.18
53.71
57.85
47.90
43.63
$4.00
56.20
60.92
52.37
41.77
52.91
51,00
53.21
Ose Fox All
36,61
61.11
50.33
39.82
44.92
45.88
65.40
56.24
63.36
50.60
38.05
45.27
49.78
49.82
Apenne Prodecto
57.17
61.33
54.22
62.99
53.86
61.75
59.88
00.25
61.00
$4.52
62.90
52,09
$8.37
$8.87
Table 13: Workflow optimization performance based on the selected workflow across methods.
Methods
Math Problems
Code Generation
Reasoning
Average
MATH
GSM8K
MBPP
HumanEval
MMLU
DROP
HotpotQA
Score
Search Cost (S)
Groun Truth (AFlow)
87,38
94,53
73.22
97.20
83.10
84.25
69.94
84.23
39.83
Random
78.40
75.23
67.84
76.34
42.87
80.42
16.86
62.56
0.00
GCN
86.16
68.23
97.46
82.33
19.14
68.42
0.00
GAT
80.11
86.22
68.62
97.71
57.00
85.83
21.47
71.00
0.00
Agentic Predictor
81.89
92.68
68.42
98.73
79.70
86.25
13.37
74.43
0.00
B.7 TRANSFERABILITY
Considering that the MMLU benchmark encompasses various reasoning tasks, further investigate the transferability of predictors trained on MMLU datasets to determine whether they can be used optimize similar reasoning tasks, specifically DROP (Dua al., 2019) and HotpotQA (Yang et al., 2018) 8). As reported in Table 13, the workflow optimized using Agentic Predictor achieves competitive performance on these tasks: 86.25% on DROP and 13.37% on HotpotQA, demonstrating notable transferability. While performance on HotpotQA is lower than the baselines, the results remain broadly comparable, indicating that the workflows optimized via Agentic Predictor maintain substantial effectiveness when transferred to closely related reasoning tasks. This highlights the practical potential of Agentic Predictor for broader applicability in workflow optimization scenarios.
CASE STUDY C
This section presents qualitative results from the workflow optimization process using Agen- tic Predictor as the reward function across three domains.
C.1 CODE GENERATION
The code generation workflow on the HumanEval dataset demonstrates that the initial solution generation step often required subsequent refinement through explicit review and revision cycles. By systematically reviewing the initially generated code, and conditionally revising based on feedback from automated tests, the workflow substantially improved the final solution's correctness. This iterative approach effectively balanced computational cost and performance, resulting in solutions that were consistently more robust and accurate compared to single-step generations.
Workflow for Code Generation (HumanFval
from typing import Literel
import workspace. Humankvel. workflows. template cemmlaate.opeeee operater as operatoz import workspace. HumanEval, workflows. round_19, 1t..dd_11.popppp. prompt__csseen from www.pppp.povvder....o Im proviiereeeee...e des_deesstrrry Amport create_l inn_taaacc meLagpt. utils, www.iille.ost_aaaeeeo cost__anager import CostManager
DaLaseL Type Literal "uuaanDaal", "MBPP", "GSMBK", "MAT"" "BotpotOA", "DROP", "XMEU")
class Workflow: def init__ self, name: str, Lim_eonCig, dateset: DatasetIype,
19

---

Published as a conference paper at ICLR 2026
-> None:
self.name datases
self.dateser inntttnnl (1l__oonfig)
self.llm create_ I instannce
CostXanager
self.custem
salf-austem_coddegnneeee salf.tast = operator. reat (self.1lm)
operater. aat.attce.ccccccccceeaaaaaaaaaaa (se1..
adyne def _ (self, problen: str, entny_ str):
Implementation of the workflow
1. Generete initiel solution using custam_sode_generssee
2. Review the solution
custom operetor.
3. Test the solution; if test fal 8, revise using auston operato" and retest.
**.
Step 1: Gonerate inftial solution initial_selution await httticccutttnoceeeeeeeeeeee ipt.oerattepoooooeepproooooo entry_point- entry_peint, instruction="";
Step Review the solution improve quality
reviewed awalt self.custon settteatttttttttttttttttttttaaat (inputtinitiill__ee.... iooouuttinooot reaponse" J, instructions
prampt__cottom. REVIEN REVNEN PROKEE)
Step Test the zeviewed solution
test_result await self.test (prohlemmpročeem soluticn-zevievedd response' cotry.oointsantey.ooin:
If test fails, revise solution based on test Teedback and retest once
test_resuit(' :'ssultt:
reviaed awalt sell.custoo wwc.cuuttmmmnuuiii (input=reviewed responae" *\n" test_result["
solutioa'), instructien-premp.. cwtpppp_.ssooo..vVee.. ROVISE PROKET) test_result await self. (problem solutiennzevisedl" response' entry sen=y_oointtontxy_ooli
REVIGG_PROK22T)
rinal_solution revised[' reaponse"] test_resul= reaulty) else reviewed
reapocse']
else:
tinal_solution zeviewed," responsee)
Finsl_sclutien, self. 11m. htw.mmmottmmmneeeeeeee... www..m/oggg..tttll.cooom
C.2 MATH PROBLEM
In addressing mathematical problems using the MATH dataset, the workflow leverages an ensemble strategy by producing multiple candidate solutions, subsequently selecting the most consistent one via self-consistency ensemble step. The selected solution then further refined through additional review process. This combined ensemble and review mechanism significantly enhanced solution quality, highlighting the value of ensemble techniques in solving complex mathematical reasoning tasks, while maintaining controlled computational budget.
Workflow for Math Problem (MATH)
from Lyping import Literal
import workspace. expraaceeexx.....o.. www...ww..........o......... workflows. cemplate.ooerateo import workspane. MATH workflows. ro. pr.ppp prompt_custon
operater as operator
from pentill.ccst_maaagee import bestManaggr
Im provider peevidd__regaatii import create_ 1m_ in_inatance
metagpt.
Dataset Type Literal "HumanSval", "MBPP", "GSMBK", "MATH", "HotpotOA", "DROP", "XMEU"
class Workflow:
def init__ self,
name:
iim_onnig,
datasel: DatasetType,
None:
self.name name
salf.datsset
self.1im create_llm, oreete__ln__nettanee (1 confioo
h.lf.lim.cost_naneee CostXanager
seltom operator, Custoo (self.
self.sc_ensemble operator. ScEasemble 11m)
20

---

Published as a conference paper at ICLR 2026
asyne _call__seell (self, prcblem: atr):
Tmpl inplemttttion the
enaemble review stop
Generate multiple candidate solutions using custom operator with different
instructions
candidates ()
range (31:
responge await self.custom (input-problann instroatiicnprromt.ccee PROMPT
Attempt [1+1].")
candidales ayppsateeeeeeennteeppoeeeee response']]
self-consisteney ensemble select the best solution
ensembl e reault await self.sc_ensem) e lonsseatons datea, probl best sost_solution easemble_result response')
problemaproa emaproa em)
Review and refine the best solution ceviex_response swait self.custee cinpute "AnSelution review:'n". best_solution mnsrruction-premt...omm custon. http:/occtto...oo........ PoV1EN_
finel_colution revies_reaponse respanse')
return simal_scluticn, odl.......et..nnggrr.........
C.3 REASONING TASK
For reasoning tasks on the MMLU dataset, the workflow combines multiple generation techniques, including
solutions with varying prompts and answers produced by specialized answer-generation operators, diversify initial candidate answers. The self-consistency ensemble step effectively selected the most consistent candidate, which was subsequently subjected to rigorous review and format verification steps. This meticulous process, which included conditional regenera- tion and revision to ensure strict adherence to specified answer formats, proved highly effective in enhancing both accuracy and reliability of the final responses.
Workflow for Reasoning Task (MMLU)
typing import Litersl
import workspace, Mtbccee.MMMMi MMUU lowa template operator operator
import workspace, MMUU wwwkflwws. round_11, 19..d__1.prooeeee. proptt as prompt_
from mettapt.previder. wtw.:oiie.......rrrerrreeeeereee from metagpt.utile wtt.:iill..es_maaaaaaaa. import WostMansger
import Lteate__ll_instsece
Datasetlype Literal sumanaval*, "MaPP", "ssM8x*, "*AT"", "HotpotgA", "DSOP", "XMLU")
class Mork[low:
def init__
self, nome: str,
lim_config,
datsset: LatasetType,
None:
self.name
self.dateset datese:
salf.lim create_lln_instomee (ll=_config)
salf.1lm.cost coot__anaagerrr CostWanager
self.custem cperator, ttattorrsssttonsseee.
salf.anawer_oe s_teerttrr_ggeeratee operator wnwitorr..nsweeee. Anwwe (enerrttnetttt (self self.se_ensemble operator, ScEnsemble (self.11m)
asyne def (self, problen: str):
**n
Ioplementation of the workflow with multiple custon answera, multiple AnswerGemerate answers, easemble, zeview,
Step 1: Genezate multiple candidate answezs using custom operator with concise prospt
custom aastomsansxers []
(21:
restom_reaponse await self.custom (input-problem,, instruct pet.ruttioorproopt_coeee
CUSTON_
custom_anawer cuaton_rrsponn response"]
customsanswers. aestoaaawees.eeeeeeeeeeeeee. (pgeeddeessttttooeeeeee...
21

---

Published as a conference paper at ICLR 2026
Add answer with diversity prompt to increase answer variety
bueton_diverse_eee www.__vveeee__eepcecee respense CUSTON_DIVERSB__RRE (RUMPT)
selt.custcm pinput-, Anetrucctiorrrroopt_.cco.o..
mustonaanswrrs-ppa
response"
Step 2: Generate muitiple cendidate answers using AnswerSenerate operator to increase diversity
aanarrgee_answeraaao =
range cncmergen_respone await self. cngwer.neerrr..mm wangwer_gonrrateeeeees (input-problem)) answergen_anewe anavergen_ anaxonger__eepponel "'sponne""answer" answer') heteeeeeeLseeeseeeeeeeeeleaaeee
Step ânsemble all candidate
select the consistent
al Lanawers ovatom austom_anawers anawergen_anakert _anawers response rnseonelereeppone ensemble ansker = ansemble_responnee responae")
self.ss_ensenbler (oltilooooottitooaalaa.eeeeee
Step Review the ensemble eevie__input = problem "Antnswer: ensemble_answes
to formst and correctness
instruaciingprmpp..ooooomm raviex_ respposeesseeeeee swnit self.custam wwwprr/ooooppttttooooooooooooo REVTEW REVTBR_PROMPT) PROMPT)
anviexad_answes Seviex_responae response")
Step If reviewed answer is not in correct format, zegenerate with stricter prompt
if not atw.e,,,tttrrrttttttiiiii ("Answer: Uption "):
strict_regen_! proal "\nP provide the final strict y the format 'Answer: OpLion X',* http input, instruction=
strict_regen_ Sww.STTTRIETSEE ATRNTTTTTTT
prompt_cuatom.sTT reviewed_answer striet_.egen_reesposse (ersppossseeessssssee response"]
Stop 6: Revision stop to refine the reviesed answer for strict format adherence reviaton_inpits problen
revision_respooe respoonee = awalt
REVISION_P30NET
finel_answer revisiot_reeponee reaponse")
Finsl_enswer self, A.f.1la.ccm matr__maaggrrroom
wtw.:i..oottmoor..orttttt.. mangger.totol_cor
D USE OF LARGE LANGUAGE MODELS
In preparing this submission, we employed ChatGPT-5 strictly as a tool for language refinement, including polishing text, improving clarity, and correcting grammatical and typographical errors. Its role was limited to grammar correction, sentence restructuring, and rephrasing for readability. All model-generated content was thoroughly reviewed and revised by the human authors ensure accuracy, originality, and adherence to research-integrity standards. The LLMs did not contribute the core research ideas, experimental design, or any substantive intellectual components of the work. Note that LLMs also served as baselines for LLM-based prediction ($B.3) and case-study (SC) experiments, as described above.
22


Reviews:
[{"review_id": "p8WEo4OP4T", "reviewer": "Reviewer_rPuy", "summary": "This paper proposes Agentic Predictor, a performance prediction framework for LLM-based agentic workflows that aims to reduce expensive trial-and-error evaluations during workflow optimization. The core technical contribution is a multi-view encoding scheme that integrates graph structures, code semantics, and prompt embeddings, combined with optional cross-domain unsupervised pretraining to address label scarcity. Experiments on FLORA-Bench show improvements of up to 12.12% in prediction accuracy and 15.16% in workflow utility over baseline methods across three domains (code generation, math, and reasoning tasks).", "questions": "As shown in weakness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper addresses a practically important challenge—reducing the computational cost of evaluating agentic workflows—and clearly articulates why existing execution-based approaches are inefficient for workflow optimization. The evaluation is thorough, including ablation studies, low-label regime analysis, OOD generalization tests, and workflow optimization experiments, demonstrating the predictor's effectiveness across multiple dimensions. The method consistently outperforms strong GNN-based baselines (GCN, GAT, Graph Transformer, etc.) with notable margins, and the pretrained variant (Agentic Predictor+) shows clear advantages in label-scarce scenarios.", "weaknesses": "1. The core contribution essentially combines existing techniques (multi-graph GNN, cross-view attention, contrastive pretraining) without fundamental innovation, the multi-view encoding is a relatively straightforward ensemble of three modality-specific encoders, and the pretraining strategy follows standard contrastive + reconstruction objectives commonly used in multi-modal learning.\n2. All experiments rely on a single benchmark (FLORA-Bench) with its specific workflow representation format; it remains unclear whether the design choices (three specific views, multi-graph encoding, cross-view attention) would transfer to other agentic frameworks with different architectures, and the OOD tests are still within the same benchmark ecosystem rather than truly novel workflow paradigms.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AhUXCBNAQv", "reviewer": "Reviewer_h3gr", "summary": "The authors present a system to predict how well different workflows of agents built on large language models (LLMs) will perform, before actually running them. They encode workflows using three “views” (structure/graph of agents, code logic, and prompts) and pre-train unsupervised across domains, then fine-tune for prediction of success/utility. The method is inspired by recent work in Neural Architecture Search task that effectively evaluates architectures. They show that this multi-view approach improves prediction accuracy and helps guide workflow design with fewer expensive evaluations.", "questions": "see weaknesses", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The motivation (too many workflows, high cost to evaluate) is convincing.\n- The multi-view encoding idea is intuitive and makes sense: workflows are complex, so capturing various facets (graph, code, prompt) is a logical approach.\n- The pre-training across domains is a good practical touch: many real-world tasks have limited labels, so this should help generalization.", "weaknesses": "In Table 3, it is not clear how the improvement percentages are computed. When comparing Agentic Predictor to the best baseline, the gains appear much smaller. It seems that the authors calculate improvements relative to the simplest baseline (MLP), which could be misleading. Are the other rows in table 3 from the author? Is the MLP the previous state of the art baseline?\nOverall, the paper tends to overstate the magnitude of the improvements. The results are positive but more modest than implied.\n\nMinor comments:\nIn Table 2, please define the abbreviations GD and AF directly in the caption for clarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Jb8S7TZjlW", "reviewer": "Reviewer_MLNd", "summary": "This paper proposes Agentic Predictor, a performance prediction framework for LLM-based agentic workflows. The core approach employs multi-view workflow encoding: graph view (topology and communication dependencies), code view (implementation, tool usage, and computational logic), and prompt view (system/instruction prompt semantics) are separately encoded and fused through an aggregation layer into a unified representation, followed by training a lightweight prediction head (MLP) for performance classification/regression on limited labeled pairs. In label-scarce scenarios, cross-domain unsupervised pretraining (reconstruction + contrastive learning) is adopted to learn generalizable representations, improving low-sample prediction performance. Experiments are conducted on FLORA-Bench subsets covering three domains: code generation, math problem solving, and commonsense reasoning, reporting consistent improvements in accuracy and utility, along with robustness across different LLM/GNN backbones and OOD scenarios. Ablations show that both multi-view and multi-graph encoding contribute gains, with pretraining being particularly effective under low label ratios.", "questions": "1. **Multi-graph construction**: Please formally define G_prompt/G_code/G_operator (whether isomorphic, node/edge features, construction process from W with pseudocode/illustrations).\n2. **Unsupervised pretraining data details**: Scale of M, domain distribution, sampling and deduplication, strategies to prevent overlap with test domains/frameworks; if cross-framework (AFlow/G-Designer) or cross-domain mixing exists, please clarify the partitioning and anti-leakage measures.\n3. **CodeRankEmbed implementation details**: How does the code view convert source code to vectors (tokenization/truncation/aggregation, whether combined with AST/CFG/call graphs), and why not choose some large-scale pretrained general-purpose embedding models?\n4. **Cost clarification**:\n   - Explain the basis for \"search phase LLM cost is 0\" in the paper (is it because candidates come from existing workflow pools rather than LLM generation?).\n   - If LLM generation of candidates is needed in real systems, how would the overall cost change? You may discuss this appropriately.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. **Novel and well-motivated approach**: Decomposes agentic workflows into three complementary views (graph, code, prompt), balancing structural dependencies and semantic signals, fitting the \"heterogeneous, label-scarce\" problem nature.\n2. **Effective pretraining strategy**: Cross-domain unsupervised pretraining significantly improves prediction quality in low-label scenarios, offering practical sample efficiency advantages.\n3. **Efficiency and practicality**: The lightweight predictor serves as a candidate ranker, substantially reducing expensive LLM calls, achieving a \"search-agnostic, composable\" accelerator paradigm.\n4. **Robustness and generality**: Maintains good ranking consistency and accuracy across different LLM/GNN backbones and cross-system/cross-task OOD settings.\n5. **Comprehensive experiments**: Main results + view/multi-graph/pretraining ablations + workflow optimization + case studies provide thorough coverage and clear argumentation.", "weaknesses": "**1. Missing Multi-graph Definition**\n\n- The paper uses G_prompt, G_code, G_operator fused via CrossGraphAttn/ViewAttnPool, but does not clarify how the three graphs are constructed: whether they are isomorphic (sharing V, E), the source and representation of each graph's node/edge features (text embeddings, AST/CFG, operator types, etc.), how they are specifically generated from W={V,E,P,C}, and the boundary with FLORA-Bench \"following\". The absence of formal definitions/pseudocode/illustrations directly impacts reproducibility and contribution assessment.\n\n**2. Insufficient Unsupervised Pretraining Data Description**\n\n- The paper does not specify the scale of unsupervised pretraining data M, source/domain composition, sampling and deduplication strategies, whether there is overlap with test domains/frameworks, and measures to prevent potential label/semantic leakage. These factors directly relate to the credibility and transferability of +pretraining.\n\n**3. Code Encoder Too Black-box**\n\n- Missing details on input representation (specific usage of CodeRankEmbed, whether combined with AST/CFG/call graphs), length normalization/truncation strategies. As the code view is critical to final performance, design details should be supplemented.\n\n**4. Incomplete Cost Analysis**\n\n- While emphasizing zero LLM cost during search, the paper lacks detailed breakdown of one-time training/pretraining time/memory/dollar costs, and comparisons with few-shot LLM classifiers at different scales regarding cost inflection points.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aU6ZLpzSrY", "reviewer": "Reviewer_R5e8", "summary": "The paper presents Agentic Predictor, a framework for predicting the performance of LLM-based agentic workflows without costly runtime evaluations. It introduces multi-view encoders that jointly model workflow graphs, code structure, and prompt semantics, combined with cross-domain unsupervised pretraining to address label scarcity. Experiments on the FLORA-Bench benchmark show consistent improvements in both prediction accuracy and workflow utility over existing graph-based baselines.", "questions": "Can the authors clarify how multi-view information is balanced during aggregation? Is there any learned weighting or attention over views beyond simple concatenation?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The idea of learning to predict workflow performance for LLM agents is interesting and timely, as it could make agent design more efficient.\n2. The proposed multi-view representation is intuitively reasonable and well-motivated from both structural and semantic perspectives.\n3. The experiments are technically thorough within the chosen benchmark, with clear ablations and implementation details that support reproducibility.", "weaknesses": "The architectural novelty is moderate; most encoder components are adapted from prior NAS and graph-representation literature, with limited methodological innovation beyond combining them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)