Improving Parallel Program Performance with LLM Optimizers via Agent-System Interfaces
Authors: Anjiang Wei
Allen Nie
Thiago S. F. X. Teixeira
Rohan Yadav
Wonchan Lee
Ke Wang
Alex Aiken
Venue: ICML 2025 poster (2025)
Keywords: Large language model
agent
parallel programming
performance optimization
Primary area: deep_learning->large_language_models

Abstract: Modern scientific discovery increasingly relies on high-performance computing for complex modeling and simulation. A key challenge in improving parallel program performance is efficiently mapping tasks to processors and data to memory, a process dictated by intricate, low-level system code known as *mappers*. Developing high-performance mappers demands days of manual tuning, posing a significant barrier for domain scientists without systems expertise. We introduce a framework that automates mapper development with generative optimization, leveraging richer feedback beyond scalar performance metrics. Our approach features the Agent-System Interface, which includes a Domain-Specific Language (DSL) to abstract away the low-level complexity of system code and define a structured search space, as well as AutoGuide, a mechanism that interprets raw execution output into actionable feedback. Unlike traditional reinforcement learning methods such as OpenTuner, which rely solely on scalar feedback, our method finds superior mappers in far fewer iterations. With just 10 iterations, it outperforms OpenTuner even after 1000 iterations, achieving $3.8\times$ faster performance. Our approach finds mappers that surpass expert-written mappers by up to $1.34\times$ speedup across nine benchmarks while reducing tuning time from days to minutes.

Improving Parallel Program Performance with LLM Optimizers via Agent-System Interfaces
Anjiang Wei"!
Allen Nie
Thiago S. F. X. Teixeira? 2 Rohan Yaday
Wonchan Lee
Ke Wang* Alex Aiken
Abstract
entists, including physicists, chemists, and biologists, rely on high-performance computing to tackle complex prob- lems. These scientific computations dominate workloads on the world's most powerful supercomputers (Exascale Computing). However, many domain scientists lack exper- tise in computer science, and therefore having difficulties in optimizing their programs because of the complexity and scale of the underlying machines. Even for experts, finding and fixing performance problems resulting from program modifications or when porting to new machine is often time-consee cone-consuming. Any progress on automating performance tuning is of great benefit this domain.
Modern scientific discovery increasingly relies on high-performance computing for complex modeling and simulation. A key challenge in improving parallel program performance ef- ficiently mapping tasks processors and data memory, process dictated by intricate, low- level system code known mappers. Devel- oping high-performance mappers demands days of manual tuning, posing a significant barrier for domain scientists without systems expertise. We introduce a framework that automates map- per development with generative optimization, leveraging richer feedback beyond scalar perfor- mance metries. Our approach features the Agent- System Interface, which includes Domain- Specific Language (DSL) to abstract away the low-level complexity of system code and define structured search space, as well as AutoGuide, mechanism that interprets raw execution output into actionable feedback. Unlike traditional rein- forcement learning methods such as Open which rely solely on scalar feedback, our method finds superior mappers in far fewer iterations. With just 10 iterations, it outperforms OpenTuner even after 1000 iterations, achieving 3.8x faster performance. Our approach finds mappers that surpass expert-written mappers by up to 1.34x speedup across nine benchmarks while reducing tuning time from days to minutes.
Task-based programming (Slaughter et al., 2015: Bauer et al., 2012; Augonnet et al., 2009; Chamberlain et al., 2007: Moritz et al., 2018; Barham et al., 2022) has emerged as promising approach to high performance computing. The paradigm involves decomposing computations into independent tasks that communicate exclusively through their arguments. key advantage of task-based systems is that the performance tuning problem is factored into separate mapping: an assignment of tasks processors and data to particular memories. High-quality mapping, achieved through well-designed mapper (implemented as code), can significantly improve performance, often by an order of magnitude (Galvez et al., 2017).
However, currently writing mappers remains a labor- intensive process, it requires deep knowledge of appli- cations, hardware, and low-level system APIs. In addition, this process is highly application-specifie,, input-specific, and machine-specifie, often taking experts several days of meticulous tuning to achieve high performance. This challenge is especially pronounced for domain scientists, who typically lack the necessary expertise computer sys- tems and code optimization. Automating mapper develop- ment would enable scientists to focus their own domain of expertise while fully utilizing the capabilities of high- performance computing systems.
1. Introduction
Modern scientific discovery depends on advanced software tools for modeling and simulation (Stocks et al., 2024; Wang et al., 2024; Ltaief et al., 2024). Computational sci-
'Equal contribution Staafoord University PIntel 'NVIDIA *Nanjing University. Correspondence to: Anjiang Wei <an- jiang@es. wtteeeccs.stnoo.cccc
In this paper, we introduce a system powered by large lan- guage models (LLMs) to automate both the generation and optimization mapper code. The first challenge stems from the complexity of generating mapper code due to the original low-level programming system, which ex-
Proceedings of the 1nnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
Agent System Interface
Agent System Interface
Iteration 1
Iteration
System
101010
System
101010
101010
101010
Code
101010
Code
101010
DSL
Compiler
DSL
Compiler
LLM
Update
B
GPU2
Agent
X
AutoGuide
AutoGuide
E
Execution
Generative Optimization
Figure I. Iterative mapper refinement with agent-based generative optimization. The system leverages the Agent-System Interface, which consists of the Domain-Specinf Specific Language (DSL) and AutoGuide. The DSL abstracts away the low-level system code, defining search space for mapping strategies, while AutoGuide interprets execution results into actionable guidance. As iterations progress, the mapper evolves improve performance.
poses the agent intricate system APIs, coupled with the problem that feedback messages from the system are often uninformative to the agent. The second challenge involves optimizing mapper performance. Specifically, it consists of (1) defining an appropriate search space and (2) devising efficient methods to find optimal mappers, thereby maximizing parallel program performance.
LLM--poereed agents not only match but often surpass expert-writted mappers, achieving up to 1.34x speedup across nine benchmarks. Since expert-written mappers set the highest standard, surpassing them is a notable accom- plishment. At the same time, our method significantly re- duces mapper tuning time from days to minutes, making perfoormroe mapping more accessible to domain sci- entists. To further highlight the advantage of generative op- timization, compare it against OpenTuner, a reinforce- ment learning-based autotuning framework. Our genera- tive optimizer finds mappers 11x faster than OpenTuner when both run for 10 iterations and still maintains a 3.8 advantage even when OpenTuner runs for 1000 iterations. Furthermore, ablation studies underscore the necessity of the agent-system interface design in achieving these per- formance gains. Our contributions are as follows:
To address the first challenge, System Interface (ASI), as shown in Figure 1, abstrac- tion layer between the agent and the system that simplifies code generation and provides more meaningful feedback to the agent. At the core of ASI a Domain-Spectie Lan- guage (DSL), a high-level interface that encapsulates performance-criticn decisions required to generate a map- per. The DSL abstracts away the complexity of low-level system code with compiler. Additionally, the DSL defines structured search space, enabling systematic exploration of mapping strategies. We also design and implement the AutoGuide mechanism to interpret raw execution output into informative and actionable guidance. This mechanism allows the agent to iteratively optimize the mapper by lever- aging enriched feedback to update its strategy.
propose Agent-
1. Design of an Agent-Systen Interface: We introduce an abstraction layer that simplifies mapper code gener- ation and provides guidance to the agent. The Domain- Specific Language (DSL) defines a search space, al- lowing the agent to explore mapping strategies without dealing with low-level system code. AutoGuide inter- prets execution output into targeted feedback, abling the agent refine mapper code more effectively.
For the second challenge, we adopt the generative opti- mization approach, recent advance in optimization tech- niques. Unlike traditional methods such as reinforcement learning (Ansel et al., 2014), which rely solely on scalar re- wards, generative optimization can utilize richer forms of feedback, such error explanations and actionable sugges- tions expressed in natural language. This agentic optimiza- tion workflow has previously proven to be effective across various domains (Nie et al., 2024; Cheng et al., 2024; Yang et al., 2023; Khattab et al., 2023; Yuksekgonul et al., 2024). Our work is the first to apply such technique to the domain of system optimization.
2. Generative Optimization for Systems: We intro- duce generative optimization improve system perfor- mance, leveraging richer feedback such as error mes- sages and actionable suggestions in natural language. Unlike reinforcement learning methods like OpenTuner Tuner, which rely solely on scalar feedback, our method identi- fies better mappers in far fewer iterations. With only 10 iterations, outperforms Open'unnr by 3.8x even after 1000 iterations.
3. Empirical Evaluation of Performance: Our agent- based solution achieves up to 1.34x speedup across
Our experiments demonstrate that mappers optimized by
2

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
nine benchmarks, surpassing expert-written mappers while reducing tuning time from days to minutes. We highlight the critical role of the agent-system interface through ablation studies, demonstrating its impact on achieving the performance gains.
led with numerical methods, including mixed-integer pro- gramming (AhmadiTeshnizzi et al., 2024a;b) and numerical optimization (Nie et al., 2024). A key advantage of genera- tive optimization is its ability to iteratively refine solutions using diverse forms of feedback. For example, Cheng et al. (2024) applies generative optimization to robotic manipu- lation and game playing, while Yuksekgonul et al. (2024) optimizes prompts and molecular designs. While reinforce- ment learning has been applied to system optimization, the potential of LLM-driven optimization in systems re- mains unexplored. Our work explores whether generative optimization with richer feedback outperforms traditional methods using scalar rewards in system optimization.
2. Related Work
Mapping in Parallel Programming Many parallel pro- gramming systems allow users to make their own mapping decisions, such as Legion (Bauer et al., 2012), StarPU (Au- gonnet et al., 2009; 2010), Chapel (Chamberlain et al., 2007), HPX (Kaiser et al., 2014; Heller et al., 2017), Se- quoia (Fatahalian et al., 2006), Ray (Moritz al., 2018), TaskFlow (Huang et al., 2021), and Pathways (Barham et al., 2022). Several techniques have been proposed to automate mapping, including machine learning mod- els Boyle et al., 2013; Wang & O' O'Boyle, 2009), static analysis (Poesia et al., 2017; Ren et al., 2008), reinforce- ment learning (Ansel et al., 2014; Mirhoseini al., 2017) and auto-tuning (SFX Teixeira et al., 2023). We use an agent- based approach with LLMs and explore larger search space for mappers than traditional methods.
3. Problem Definition
Motivation and Challenges The concrete problem we address is the automated generation of high-performance mappers for the Legion parallel programming frame- work (Bauer et al., 2012). Mappers dictate task scheduling and data placement. A well-designed mapper can achieve orders of cffeers-of-magninn magnitude speedup over naive strategies.
However, automating mapper generation is challenging due to two key factors. First, the complexity of low-level sys- tem code. Implementing a mapper requires writing hun- dreds of lines of intricate C++ code, demanding exper- tise in system internals. Second, the vast search space of mapping strategies. The search space grows exponen- tially with the number of tasks and arguments.
Agentic Frameworks Agents powered by Large Lan- guage Models (LLMs) play a critical role in decision- making, planning, tool integration, and solving complex problems in dynamic environments (Guo et al., 2024). Many agentic frameworks have been developed (Yao et al., 2022; Wu et al., 2023; Li et al., 2023; Hong et al., 2023),
with uses spanning domains such as software engineer- ing (Gur et al.. 2023; Yang et al., 2024b; Jin et al., 2024), robotics (Kannan et al., 2024), hcalthcare (Li et al., 2024). education (Ramirez& & Esparrell, 2024), and knowledge en- gineering (Shao et al., 2024). Our work is the first to apply an agentic workflow to iteratively optimize mapper code, improving the performance of parallel programs.
Search Space and Performance Impact As illustrated in Figure A1, the search space of mappers involves multi- ple decisions, each influencing performance. The first key aspect is processor selection, which determines whether task runs on GPUs, CPUs, or the OpenMP runtime. This choice depends on factors such as task size, GPU mem- ory capacity, and kernel launch overhead. For instance, small tasks may prefer CPUs due to the overhead of launch- ing GPU kernels, while tasks with large memory footprints may run on CPUs when GPU memory is insufficient.
AI for Systems The application of AI to optimize sys- tem design has gained significant traction in recent years. Techniques such as deep learning (Zheng et al., 2020a;b; 2022b) and gradient-boosted trees (Feng et al., 2023) have been used to predict program execution times for per- formance optimization. Reinforcement learning methods have addressed challenges in chip floorplanning (Mirho- seini et al., 2021), autotuning (Ansel et al., 2014), auto- vectorization (Haj-Ali et al., 2020a), and compiler phase ordering (Haj-Ali et al., 2020b). While previous efforts have predominantly relied on traditional approaches for cost prediction and optimization, our work uses the recent advances in generative optimization tackle complex sys- tem challenges.
Another crucial dimension is memory placement, which dictates where data is stored. A mapper must decide whether to place data in the GPU's FrameBuffer for fast access, ZeroCopy memory for CPU-GPU sharing, or CPU system memory for more available storage. Each option presents trade-offs between access speed, memory usage, and data transfer overhead.
Additionally, memory layout further expands the search space, with decisions on Struct of Arrays (SOA) vs. Ar- ray of Structures (AOS), data ordering (Fortran-order vs. C-order), and alignment constraints (c.g., 128-byte align- ment) significantly affecting cache efficiency and perfor-
Generative Optimization Recent work has explored the use of LLMSs for optimization problems traditionally tack-
3

---

Improving Paraliel Program Performance with LLM Optimizers via Agent-System Interfaces
Map task0 to GPO.
woid alicw taak fconet Taaks task, const B.izeIackIngot simput, BlioeTaskOutgut soutput; ventonchrogsserr targets
Task tacko GPU;
Place certain data paco GPD terobopy,
Region ghost_rregio GPU 2CHEM
thic->eeieet_ tor.ttsstortttsiictt. tass); DonairT<2> space input. Point<2> nun_points space.bounds.
Specity layeut in nenozy
+
(aligned to 64 bytes)
baundd. ones;
9
Leyout
C_order SCA Aligne-64
Reat<c2> blocks (xeroes, non blocks onest;
... lines for (PointinRectttrroooes it iblocke) 2t 1- NULL, it++)
code
here
11 Dorine syelie napping stratogy
11
12 def cyclic(Tess taskl:
12
13
task.ipoint;
13
con=iat-2,coordd.>> siies_epaec]
nape Racaine (SPU);
TaskSlice siice;
15
node_idx 1p[C] [ape.size[D]]
15
slica. donean (slies_ic, alice_hili
16
@pa_idx 1p]0] [1pl)
16
elice.proc targeto & torgets.cizo(11:
return node_idx, sp__idk];
17
output. slices sericer.push_basee pash_back(sliae)
IntexTackMap tocki cyclts
(a) An example mapper in Domain-Specific Language (DSL.
(b) Code snippet from C++ mapper
Figure 2. Comparison of DSL mapper and C++ mapper. The DSL's declarative, high-level design abstracts away the complexity of low-level C++ code, serving as the core of the Agent-System Interface. The highlighted boxes illustrate how the same functionality, which requires extensive C++ system code, can expressed concisely just a few lines DSL.
mance.
significantly lower than that of C++. Figure 2a provides an example of a DSL, mapper, highlighting the key fea- tures of our DSL. In contrast, Figure 2b shows snippet from C++ mapper, emphasizing the intricacy of lowelevel implementation details. Across the benchmarks, using the DSL results in an average lines of code reduction of 14x. This substantial reduction makes DSL a more suitable tar- get for LLM code generation, as it abstracts away the com- plexities inherent in low-level systems. As we will show in Section 5.2, LLMs generate DSL code more effectively, despite DSL having no examples in LLM training corpora, whereas C++ is widely represented.
Finally, an important idiom in ing is launching tasks over partitioned data. Index map- ping determines how data partitions and task executions are distributed across multiple processors. For consistency, we can represent data partitioning as a tensor of data par- titions, the machine as a tensor of processors, and tasks operating on the partitioned data as a tensor of tasks. The way data and task indices are mapped to processor indices affects inter-processor communication, key factor in per- formance (Unger al., 2022: Zheng al., 2022a).
comput-
4. Our Approach: Agent-System Interface
Next, we describe the DSL's design, emphasizing its declarative nature and structured search space. Section 3 details the performance impact of each decision.
4.1. Domain-Specifie Language Design
A key challenge in automating mapper generation with a coding agent the complexity of low-level system code, which requires intricate C++ implementations. To address this, we design high-level Domain-Specifie Language (DSL) as the core of our Agent-System Interface (ASI). The DSL provides structured search space for mapping strategies while abstracting away low-level implementation details. Unlike C++, which demands imperative specifica- tions of mapping policies, our DSL adopts a declarative design, allowing users to specify what to achieve rather than how to implement it. Most critically, the DSL. sepa- rates concerns, enabling multiple aspects of mapping de- cisions to be expressed independently rather than being entangled in low-level system APIs. This design reduces code complexity and naturally provides a search space for the agent to explore. To implement it, we develop a com- piler that translates DSL into the low-level C++ APIs.
The Task statement (Line 2) defines processor selection for each task, choosing between CPU, GPU, or OpenMP. Line 2 specifies that instances of task0 should run on GPUs. This decision is made per task; note that the search space expands exponentially with the number of tasks.
The Region statement (Line 5) controls memory place- ment for data arguments. Line 5 specifies that all tasks us- ing
should place the data in GPU Zero- Copy memory. Other choices include GPU FrameBuffer memory and CPU System Memory. This decision is made per task and per argument, causing the search space to grow exponentially.
The Layout statement (Line 9) defines memory layouts. Line enforces C_order axis ordering, an SOA layout, and 64-byte memory alignment for all data used by all tasks mapped to all processors. Alternative choices include F F_order, AOS, and various alignment strategies. This is per-task, per-data, per-processor decision.
As illustrated in Figure 2, the complexity of DSL code is
4

---

Improving Paraliel Program Performance with LLM Optimizers via Agent-System Interfaces
Input
Generative Optimization
Mapper Agent
Raw Execution Output
AutoGuide
ServerSpec
TaskDecision
LayoutDecision
Explain
1000000 U900000
AppSpec
def layout_deds _el__deceeee denision()
def _ask_decision(taskk
Suggest
RegionDecision
IndexTaskMapp
X
Execution
def region_nap(reglons)
patpo_ttailnapiiddee specs)
tasks,
Application
Feedback
MapperCode
Server
Figure 3. Agent Optimization Process. The mapper agent takes server specifications and application-spee sppeiific information input, generates mapper code, and executes alongside the application on the server. Raw exccution feedback is enriched using the AutoGuide mechanism, and the mapper is iteratively refined by L.M optimizer to improve performance.
AutoGuide
Case
Raw Execution Output
Explain
Suggest
Execution Error:
Assertion failed: stride does not match expected value.
Memory layout is unexpected.
Adjust the layout constraints or move tasks to different processor types.
Case
Performance Metrie:
Execution time is 0.03s.
Move more tasks to GPU to reduce execution time.
Case 2
N/A
Table AutoGuide Feedback Mechanism. AutoGuide mechanism interprets raw execution output from the runtime system, providing more informative error explanations and suggestions for mapper modifications. It is implemented via keyword matching. Additional examples are shown Table A3.
The IndexTaskMap statement (Line 19) controls index mapping using a customized function. Line 12 defines the mapping function that establishes the correspondence be- tween two index spaces: the task index space (represented by task. ipoint) defined in the application code (e.g.. for loops) and the processor space of the distributed ma- chine (represented by Ma Machin ne (GPU) ). The DSL. allows users to express arbitrary arithmetic mappings between the two index spaces. This decision applies to each task group launched by parallel for loops.
possible mappers, w is an optimization objective, and T is function that takes a mapper and returns S. the feedback from executing the mapper (i.e., the measured performance after running the applica- tion code with the generated mapper), and g, the process graph tracing how the mapper generated. In our setup, mapper performance is deterministic, we carefully con- trol all sources of randomness in the environment. If the parameter space were numerical, this online optimization problem could be addressed using bandit algorithms (Lat- timore Szepesvári, 2020), reinforcement learning (Sut- ton & Barto, 2018), or Bayesian optimization (Snoek al., 2012), but these methods are less efficient when the param- eter search space is large and discrete (i.e., text).
as input, (f. g) = (())
Our DSL designed to express wide range of high- performance mapping strategies, including all of the most important decisions. While there may be cases where cer- tain optimizations are not directly expressible, we have not encountered any. Despite being more constrained than general-purpose C++, the DSL. has been proven to effec- tive: all mappers discovered by our agent that outperform expert-written written C++ implementations are expressible within the current DSL.
In this online optimization problem, we leverage the DSL to structure the parameter space to improve the efficiency of optimization. Here, represents
program code, while w and f are expressed as text. We adopt generative op- timization, leveraging LLMSs optimizers given the ob- jective in text form. This emergent optimization behavior has been recently observed and applied across various do- mains (Yang et al., 2024a; Cheng et al., 2024; Yuksekgonul et al., 2024; Patel et al., 2024).
4.2. Generative Optimization via AutoGuide
We formulate mapper generation an online optimiza-
tion problem. Given triplet (O,w, T), where O is set of
5

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
Optimization Process We present the optimization pro- cess in Figure 3. The agent takes two inputs: server spec- ifications and application metadata. Server specifications detail the hardware configuration, including the number of CPUs and GPUs per node, as well as the total node count. Application metadata provides information on task names and the associated data arguments accessed by each task. These imputs define the structured search space explored by the agent during optimization. The agent, using the given inputs. generates mapper code that is executed alongside the application code on the server. Raw execution feed- back from the runtime is augmented with the AutoGuide mechanism and fed back to the LLM, iteratively refining the agent for improved mapper code generation.
5.1. Speedup of Application Performance
Benchmarks Our evaluation utilizes a suite of 9 bench- marks, including 3 scientific computing workloads and 6 well-known matrix multiplication algorithms. Circuit is simulation benchmark that models electrical circuit be- havior by simulating currents and voltages across inter- connected nodes and wires (Bauer et al., 2012). Sten- cil simulates a 2D grid where each point's value is up- dated based on stencil pattem determined by its neigh- bors (Van der Wijngaart & Mattson, 2014). Pemnant mod- els unstructured mesh Lagrangian staggered-grid hydro- dynamics, commonly used for simulating compressible flow (Ferenbaugh, 2015). The remaining six benchmarks Canmon's 's SUMMA, PUMMA, Johnson's 's, Solomonik ss and COSMA - are well-known parallel matrix multiplica- tion algorithms (Cannon, 1969; Van De Geijn & Watts, 1997; Choi et al., 1994; Agarwal al., 1995; Solomonik & Demmel, 2011; Kwasniewski et al., 2019). Parallel matrix multiplication remains an active research topic due to its central role in high-performance computing and scientific simulations (Yadav et al., 2022). Furthermore, improving matrix multiplication performance has broad impact, as it accelerates numerous downstream machine learning work- loads (Jangda al., 2022; Zheng et al., 2025). We discuss these matrix multiplication algorithms in more detail in Ap- pendix A.3. This benchmark suite provides both depth with its representative matrix multiplication algorithms and va- riety with its range of scientific computing workloads.
Coding Agent Our mapper agent improves mapping de- cisions by iteratively generating DSL code. A high-level schema of the mapper agent is shown in Figure The Python program in the Trace (Cheng et al., 2024) framework, where decom- pose the task of generating monolithic mapper into in- dependent code segments. This decomposition allows the agent to decide what code to generate for each segment separately. This approach is effective because our DSL de- sign eliminates unnecessary dependencies between map- ping decisions. Our modularization strategy aligns with least-to-me mostt most prompting (Zhou et al., 2022).
mapper agent is implemented
In this experiment, we evaluate the performance of the mappers with the following baselines.
Expert- Mappers.
These mappers are manually developed by domain scientists who spend years master- ing computational science Writing mappers in parallel programming frameworks is another challenge, and tuning them for specific applications can take days.
AutoGuide The AutoGuide feedback mechanism is de- signed based on three key motivations: (1) generative op- timization benefits from natural language feedback rather than relying solely on scalar values, (2) raw execution out- put from the runtime system is often too uninformative to effectively guide the agent's decisions, and (3) domain heuristics known to systems researchers can be naturally expressed in language (e.g., most tasks run faster on GPUs than CPUs). To address these needs, AutoGuide helps the agent by explaining opaque error messages and suggest- ing mapper modifications. As shown in Table 1, it in- terprets uninformative execution output into actionable in- sights, with additional examples in Appendix A.5. The implementation relies on keyword matching over the raw execution output. An ablation study in Section 5.3 demon- strates its effectiveness in our experiments.
Randomly Generated Mappers. These mappers were randomly generated with 10 different random seeds, sam- pling from the entire search space of each application. We report the average performance.
Agent-Optimized Mappers. Using Trace (Cheng et al., 2024), we evaluated the Trace and OPRO (Yang et al., 2023) search algorithms, running 10 iterations per appli- cation. To account for stochastic output, we repeated the process 5 times and report the average. The best mapper from Trace across runs is also reported.
OpenTuner Mappers. OpenTuner (Ansel et al., 2014) is program autotuning framework that uses reinforcement learning to optimize performance based on sealar feed- back. We provided execution time as feedback, with a high penalty for failures.
5. Evaluation
Experiments are conducted on one node with two Intel 10-core E5-2640 v4 CPUs, 256G main memory, and four NVIDIA Tesla P100 GPUs. We use gpt-4o-2024-08-00
6

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
Circuit
Stencil
Pennant
Norm. Throughput
1.5
Norm. Throughput
Norm. Throughput
1.0
0.5
0.0
0
4
6
2
8
2
Iterations
Iterations
Iterations
Cannon's
SUMMA
PUMMA
Norm. Throughput
1.5
Norm. Throughput
Norm. Throughput
1.0
0.5-
0.0
4
6
2
8
2
4
Iterations
Iterations
Iterations
Solomonik's
Johnson's
COSMA
Norm. Throughput
1.5
Norm. Throughput
Norm. Throughput
1.0
0.5
0.0
6
8
2
4
6
Iterations
Iterations
Iterations
Expert Design
Random
Trace
OPRO
OpenTuner
Best Trace
Figure 4. Performance Comparison. Normalized throughput for benchmarks, comparing expert mappers, random mappers, the
average optimization trajectories of Trace, OPRO, and OpenTuner in 10 iterations across 5 runs, and the best mappers found by Trace.
Generative Optimizer vs Traditional RL (1k iters)
mance, which includes both the correctness and efficiency of the generated mappers. If the generated code has any syntax or runtime issues, its throughput is recorded as 0. We report the best mappers found by Trace, and the av- erage optimization trajectories Trace, OPRO and Open- Tuner over iterations across 5 runs.
13X
Relative Throughput
Trace Tuner
3.8X
11x
5X
All the best mappers found by Trace match pass the expert-written mappers, underscoring the effec- tiveness of agent-based generative optimizer. In our con- text, reporting the best-performing mapper is appropriate. Mapper optimization is an offline process, and in practice, it standard to run the optimizer multiple times and deploy the best result. Once identified, the mapper can be reused across repeated executions on the same application, input, and hardware, incurring further search cost.
1X
10
10 Iterations (Log Scale)
10
10
Figure 5, Comparison of Trace (generative optimizer) and Open- Tuner (traditional RL) over 1K iterations (averaged across all 9 benchmarks)
Random mappers consistently exhibit low performance across all applications, emphasizing the critical role of mapping decisions. For each application, we generate 10 random mappers by sampling from the full DSL-defined search space, totaling 90 mappers across 9 applications. Among them, 74 (82.2%) raise runtime errors due to invalid
Results In Figure 4, we use normalized throughput our performance metric, where higher values indicate bet- ter performance. The throughput normalized relative to the expert-written mappers, providing a clear baseline for comparison. Our focus is on measuring end-to-end perfor-
7

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
Mapping Strategy
Code Generation Target
Success Rate
2
3
4
6
8
10
C++ (single trial)
X
X
X
0%
DSL (single trial)
V
V
v
V
V
v
V
80%
C++ (iterative refine)
X
X
X
X
X
X
x
X
0%
DSL (iterative refine)
v
100%
Table Code Generation Success Rates. Success rates for generating code across 10 mapping strategies described in natural language. The test evaluates whether the generated code compiles and passes execution tests. Generating DSL code significantly outperforms generating C++ for both settings. Symbols indicate results: fails to compile, compiles but fails the test, and passes the test.
mapping decisions. The runtime system enforces correct- ness by rejecting such mappers during execution, resulting in throughput of
across GPUs. For additional context, examples of Trace mappers are presented in Appendix A.8.
5.2. Ablation Study of DSL for Code Generation
When comparing optimization trajectories, Trace performs similarly to OPRO, and significantly outperforms Open- Tuner. To further compare the agent-based optimizer with traditional reinforcement learning, we extended Open- Tuner's optimization iterations from 10 to 1000, as shown in Figure 5, where the x-axis is the log-scale of itera- tions and the y-axis represents relative throughput (aver- aged across all benchmarks). Notably, Trace achieves 3.8x speedup over OpenTuner even when OpenTuner is run for 1000 iterations. When both are limited to 10 itera- tions, Trace outperforms OpenTuner by 11x, demonstrat- ing its ability to quickly identify high-performance map- pings. This highlights the superiority of Trace (genera- tive optimizer) over OpenTuner (traditional reinforee- ment learning). Moreover, Trace completes the entire op- timization process in just 10 minutes per application, re- ducing mapper development time from days to minutes.
In Section 5.1, we demonstrate the overall effectiveness of our approach. Here, we conduct an ablation study on the DSL, the core of the Agent-System Interface. Since suc- cessful generation is the foundation of optimization, this subsection focuses on how well the DSL helps LLMs gen- erate syntactically and semantically correct mappers compared to C++, rather than directly optimizing perfor- mance.
Experiment Setup We designed 10 mapping strategies, described natural language, to evaluate whether LLMs can generate correct code in both the DSL and the orig- inal low-level C++. The strategies are detailed in Ap- pendix A.7. To ensure fair comparison, identical prompt materials (documentation, examples, and starting code) were provided for both the DSL and C++. Success rates are measured based on whether the generated code passes predefined test cases, with results reported for single trials and iterative refinement, where the LLM is allowed up to iterations to improve the code using compiler feedback. The evaluation is conducted with the DSPy (Khattab al., 2023) framework.
To offer more comprehensive view of performance vari- ations, we present additional statistics, including the mean, standard deviation, worst, median, and best normalized throughput for both our method and OpenTuner. These statistics are derived from five runs for each benchmark, as detailed in Appendix A.4.
Case Analysis The largest performance gain achieved by Trace over the expert mapper is observed in Circuit, with a speedup of 1.34x. This improvement is primarily due to memory placement: the best mapper allocates two data collections to GPU FrameBuffer memory, while the ex- pert mapper places them GPU ZeroCopy memory. De- spite slight increase in inter-GPU communication costs, Trace reduces task execution time due to faster memory ac- cess, resulting in higher overall performance. For matrix- multiplication algorithms, the greatest speedup is seen in COSMA, with Trace achieving a 1.31x speedup over the expert mapper. This is attributed to Trace's more efficient index mapping functions, which reduce inter-GPP com- munication by better distributing partitioned submatrices
Results Table 2 shows that DSL achieves significantly higher generation success rates than C++ in both the single-trial and iterative refinement settings. This demon- strates the effectiveness of DSL's design in abstracting sys- tem complexity and providing a high-level interface that enables LLMS to tackle complex system challenges in code generation. Incorporating iterative refinement with compiler feedback further improves success rates, resolv- ing four compilation errors in C++ and two in the DSL. However, the gap between DSL mappers and C++ map- pers remains substantial. Notably, these results are strik- ing given that the DSL is a low-resource language with no pre-training or fine-tuning data, while C++ code is widely present in LLM training corpora.
8

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
Norm. Throughout
Circuit
Norm. Throughput
COSMA
Norm. Throughput
SUMMA
1.5
1.0
0.5
0.0
2
8
6
2
Iterations Expert Design
Iterations
Iterations
5-Shot
Execution + Explain
0-Shot
Execution
Execution + Explain + Suggest
Figure 6. Comparison of different feedback designs. 0-Shot and 5-Shot are baselines. Execution provides only the exccution output as feedback. Explain provides additional explanations of execution emror. Suggest offers mapper modification suggestions. All feedback automatically generated.
Analysis LLMs perform better with the DSL for two rea- sons. First, the semantic gap between natural language and code is smaller with the DSL than with C++. For example, writing mapper to "align all data to 64 bytes in mem- ory and use Fortran ordering" requires one line Layout A1 Aligm==64 F F_order in the DSL because of its declarative design. In contrast, the C++ mapping API requires a sequence of operations to enforce alignment and ordering, which widens the semantic gap. Second, the DSL reduces the amount of code. As discussed before, LLMs achieve average reduction of 14x in lines code, sim- plifying code generation. These results underscore the im- portance of high-level agent-system interface.
6. Conclusion
In this paper, we introduced system that leverages LLMs to automate mapper generation and optimization. The Agent-System Interface (ASI) simplifies code generation with a Domain-Specifie Language (DSL., which abstracts away the low-level complexity of system code, and en- riches execution feedback through AutoGuide, which in- terprets execution output into actionable guidance. We adopted generative optimization, allowing LLMs to refine mappers using rich textual feedback beyond scalar met- rics. Unlike RL-based methods like OpenTuner, which rely on numerical rewards, our approach incorporates error explanations and targeted suggestions, accelerating search efficiency. Experiments show that agent-generated map- pers outperform expert written ones, achieving up 1.34x speedup across nine benchmarks. Our method, running only 10 iterations, maintains a 3.8x advantage over Open- Tuner even after 1000 iterations. By reducing mapper de- velopment time from days to minutes, our approach ben- efits computational scientists and demonstrates the effec- tiveness of generative optimization in system design.
5.3. Ablation Study of the AutoGuide Feedback
The AutoGuide mechanism provides enriched feedback to the agentic optimizer. We compare with alternative feed- back designs.
Experiment Setup We compare the following baselines. 0-shot and 5-shot have no feedback, allowing the LLM to generate once with either 0 or examples provided. Execu- tion only provides execution feedback, Explain offers additional explanations for execution errors, and Suggest offers mapper modification suggestions. The Trace trajec- tory shown in Figure 4 uses the full AutoGuide mode with all DeccuttionEEplain---iie evaluate benchmarks.
A Acknowledgements
This work was supported part by a Google Research Award. We thank Yuhui Zhang, Genghan Zhang, Qizheng Zhang, Mohammad gonul. Chenyang Yang, Jiaxin Ge, Simon Guo, and Anne Ouyang for the feedback. We thank Ching-An Cheng and Ken Liu for their discussions and feedback.
As an ablation study, we
Fadiheh, Ed Chen, Mert Yuksek-
Results and Analysis Figure 6 demonstrates that the full feedback mechanism consistently outperforms all re- duced feedback variants. The Shotot and 5-shot results per- form the worst, underscoring the importance of feedback- based iterative refinement. This highlights the value of an agentic workflow, showing that performance improvements are not solely driven by prompting the LLM but are direct result of the iterative refinement the workflow design.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.

---

Improving Paraliel Program Performance with LLM Optimizers via Agent-System Interfaces
References
Cheng, C.-A,, Nie, A., and Swaminathan, A. Trace is the autodiff: Generative optimization with rich feedback, execution traces, and llms. arXiv preprint urXiv24406.66188, 2024.
Agarwal, R. C., Balle, S. M., Gustavson, F. G., Joshi, M., approach to paral- lel matrix multiplication. IBM Journal of Research and Development, 39(5):575-555 1995.
and Palkar, P. A
Choi, J., Walker, D. W., and Dongarra, Pumma: Par- allel universal matrix multiplication algorithms on dis- tributed memory concurrent computers. Concurrency: Practice and Experience, 6(7):543 1994.
AhmadiTeshnizi, A., Gao, W., Brunborg, H., Talaei, S., and Udell, M. OptiMUS--033 0.3: Using large language models to model and solve optimization problems at scale. Sub- mitted, 2024a. URL https:/ /tt./viivvvv.////////a/////aaala wwtp/iioorr/aaaaa 2407.19633
Exascale Computing. DOE Explains Exascale Computing, 2025. https: htttp:://ww.ennrrry. energy gow /sci/ssiaee ence/ doe eeeeeeaaansxasccaaleee comput
AhmadiTeshnizz, A., Gao, W., and Udell, M. OptiMUS: Scalable optimization modeling using MIP solvers and large language models. In International Conference on Machine Learning (ICML), 2024b. URL https: //arxiv.oro 2abb/24402.1012
Fatahalian, K., Horn, R., Knight, T. J., Leem, L., Hous- ton, M., Park, Y., Erez, M., Ren, M., Aiken, A., Dally, W. J., and Hanrahan,
Sequoia: Programming the memory hierarchy. In SC '06: Proceedings of the 2006 A ACM/EEEE Conference Supercomputing. volume O of SC '06, pp. 83-es, New York, NY, USA, 2006. Associa- tion for Computing Machinery. ISBN 0769527000.
Ansel, J., Kamil, S., Veeramachaneni, K., Kelle-Kelley, J. Bosboom, J., O' O'Reilly, U.-M., and Amarasinghe, S. Opentuner: An extensible framework for program auto- tuning. In Proceedings of the 23rd international con- ference on Parallel architectures and compilation, pp. 303-316, 2014.
Feng, S., Hou, B., Jin, H., Lin, W., Shao, J., Lai, R., Ye, 7 Zheng, L., Yu, C. H., Yu, Y., et al. Tensorir: An abstrac- tion for automatic tensorized program optimization. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, pp. 804-817, 2023.
Augonnet, C., Thibault, S., Namyst, R., and Wacrenier, P.- A. Starpu: a unified platform for task scheduling on het- European Con-
erogeneous multicore architectures. ference on Parallel Processing, pp. 863-874. Springer, 2009.
Ferenbaugh, C. R. Pennant: unstructured mesh mini-app for advanced architecture research. Concurrency and Computation: Practice and Experience, 27(17):4555- 4572, 2015.
Augonnet, C., Clet-Ortega, J., Thibault, S., and Namyst, R. Dwta-Aware Task Scheduling on Multi-Accelerator based Platforms. In 16th International Conference on Parallel and Distributed Systems pp. 291-298, Shangai, China, December 2010. IEEE. URL https: //al. inria.fr/inrt 1t..aaaaa0aii0053333.
Galvez, J. J., Jain, N., and Kale, L. V. Automatic topology mapping of diverse large-scale parallel applications. In Proceedings of the International Conference on Super- computing, pp. 1-10, 2017.
Barham, P., Chowdhery, A., Dean, J., Ghemawat, S., Hand, S., Hurt, D., Isard, M., Lim, H., Pang, R., Roy, S., et al. Pathways: Asynchronous distributed dataflow for ml. Proceedings of Machine Learning and Systems, 4:430- 449, 2022.
Guo, T. Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N. V., Wiest, O., and Zhang, X. Large language model based multi-agents: A survey of progress and challenges. arXiv preprint arXiv:2402.01668 2024.
Bauer, M., Treichler, S., Slaughter, E., and Aiken, A. Le- gion: Expressing locality and independence with logi- cal regions. In SC' 12: Proceedings of the International Conference on High Performance Computing, Network- ing, Storage and Analysis, pp. 1-11. IEEE, 2012.
Gur, I. Furuta, H., Huang, A., Safdari, M., Matsuo, Y., Eck, D., and Faust, A. A real-world webagent with plan- ning, long context understanding, and program synthe- sis. arXiv preprint arXiv:2307. /2856 2023.
Cannon, L. E. A cellular computer implement the Kalman filter algorithm. Montana State University, 1969.
Haj-Ali, A., Ahmed, N. K., Willke, T., Shao, Y. S., Asanovic, K., and Stoica, I. Neurovectorizer: End-to- end vectorization with deep reinforcement learning. In Proceedings of the 18th ACM/IEEE International Sym- posium on Code Generation and Optimization, pp. 242- 255, 2020a.
Chamberlain, B. L., Callahan, D., and Zima, H. P. Parallel programmability and the chapel language. The Interna- tional Journal of High Performance Computing Applica- tions, 21122221133,, 2007.
10

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
Haj-Ali, A., Huang. Q. J., Xiang, J., Moses, W., Asanovic, K., Wawzzynek, J., and Stoica, I. Autophase: Juggling his phase orderings in random forests with deep rein- forcement learning. Proceedings of Machine Learning and Systems, 2:70-81, 2020b.
Lattimore, T. and Szepesvári, C. Bandit algorithms. Cam- bridge University Press, 2020.
Li G., Hammoud, H.. Itani, H., Khizbullin, D., and Ghanem, B. Camel: Communicative agents for" mind" exploration of large language model society. Advances in Neural Information Processing Systems, 36:51991- 52008, 2023.
Heller, T., Diehl, P., Byerly, Z., Biddiscombe, J., and Kaiser, H. Hpx-an open source c++ standard library for parallelism and concurrency. Proceedings of OpenSuCo. 5, 2017.
Li, J., Wang, S., Zhang, M., Li, W., Lai, Y., Kang, X., Ma, w., and Liu, Y. Agent hospital: A simulacrum of hospital with evolvable medical agents. arXiv preprint Stlv:2405.229555 2024.
Hong, S., Zheng, X., Chen, J., Cheng, Y., Wang. J., Zhang, C., Wang, Z., Yau, S. K. S., Lin, Z., Zhou, L., et al. Metagpt: Meta programming for multi-agent collabora- tive framework. arXiv preprint wrXiv:2308.00355 2023.
Ltaief, H., Alomairy, R., Cao, Q., Ren, J., Slim, L.,, Kurth, T., Dorschner, B., Bougouffa, S., Abdelkhalak, R., and Keyes, D. E. Toward capturing genetic epis- tasis from multivariate genome-wide association stud- ies using mixed seeeddprecisio kernel ridge regression. In SC24: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1- 12. IEEE, 2024.
Huang, T.WW., Lin, D.-L., Lin, C.-X., and Lin, Task- flow: A lightweight parallel and heterogeneous task graph computing system. IEEE Transactions Parallel and Distributed Systems, 33(6):1303-132) 2021.
Jangda, A., Huang, J., Liu, G., Sabet, A. H. N., Maleki, S., Miao, Y., Musuvathi, M., Mytkowicz, T., and Saarikivi, O. Breaking the computation and communication ab- straction barrier in distributed machine learning work- loads. In Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 402 416, 2022.
Mirhoseini, A., Pham, H., Le, Q. V., Steiner, B., Larsen, R., Zhou, Y., Kumar, N., Norouzi, M., Bengio, S., and Dean, J. Device placement optimization with reinforce- ment learning. In International conference on machine learning, pp. 1430-2439, PMLR, 2017.
Jin, H., Huang, L., Cai, H., Yan, J., Li, B., and Chen, H. From iims to lim-based agents for software engineer- ing: A survey of current, challenges and future. arXiv preprint ar Xii:22408.024799 2024.
Mirhoseini, A., Goldic, A., Yazgan, M., Jiang. J. W., Songhori, E., Wang, S., Lee, Y.-.,, Johnson, E., Pathak, O., Nazi, A., et al. graph placement methodology for fast chip design. Nature, 594(7862):207-212 2021.
Kaiser, H., Heller, T., Adelstein Lelbbchh,, B., Serio, A., and Fey, D. Hpx: A task based programming model in global address space. In Proceedings of the 8th Interna- tional Conference on Partitioned Global Address Space Programming Models, pp. 1-11, 2014.
Moritz, P., Nishihara, R., Wang, S., Tumanov, A., Liaw, R., Liang, E.,, Elibol, M., Yang, Z., Paul, W., Jordan, M. I., et al. Ray: A distributed framework for emerging {AI} applications. In 13th USENIX Symposium on Operat- ing Systems Design and Implementation (OSDI 18), pp. 561-577, 2018.
Kannan, S. S., Venkatesh, V. L., and Min, B.-C. Smart- Ilm: Smart multi-agent robot task planning using large language models. In 2024 IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS), pp. 12140-12147, IEEE, 2024.
Nie, A., Cheng, C.-A., Kolobov, A., and Swaminathan, A. The importance of directional feedback for Ilmebased optimizers. arXiv preprint arXiv:2405. 16434, 2024.
O' O'Boyle, M. F. P., Wang, 2., and Grewe, D. Portable map- ping of data parallel programs to opencl for heteroge- neous systems. In Proceedings of the 2013 IEEE/ACM International Symposium Code Generation and Op- timization (CGO), CGO '13, pp. 1-10, USA, 2013. IEEE Computer Society. ISBN 9781467355247, doi: 10. 10.11099CC0022013.44449993 URL https: /doi. 2013 6494993.
Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., San- thanam, K., Vardhamanan, S., Haq, S., Sharma, A., Joshi, T. T., Moazam, H., et al. Dspy: Compiling declar- ative language model calls into self- emprviiio pipelines. 2023.
arXiv preprint
Kwasniewski, G., Kabié, M., Besta, M., Vondele, J., Solcà, R., and Hoefler, T. Red-blue pebbling re- visited: near optimal parallel matrix-matrix multiplica- tion. In Proceedings of the International Conference for High Performance Computing. Networking. Storage and Analysis, pp. 1-22, 2019.
Patel, B., Chakraborty, S., Suttle, W. A., Wang, M., Bedi, A. S., and Manocha, D. Aime: Ai system op- timization via multiple I1m evaluators. arXiv preprint 11X0v:4410.33131, 2024.

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
Poesia, G., Guimarães, B., Ferracioli, F., and Pereira, F. M. Q. Static placement of computation on heterogeneous devices. Proceedings of the ACM on Programming Lan- guages, 1(OOPSLA), October 2017.
Sutton, R. S. and Barto, A. G. Reinforcement learning: An introduction. MIT press, 2018.
Unger, Jia, Z., Wu, W., Lin, S., Baines, M., Narvaez, C. E. Q. Ramakrishnaiah, V., Prajapati, N., McCormick, P., Mohd- Yusof, J., et al. Unity: Accelerating {DNN} training through joint optimization of algebraic transfor- mations and parallelization. In 16th USENIX Sympo- sium on Operating Systems Design and Implementation (OSDI 22), pp. 267-284, 2022.
Ramirez, A. B. and Esparrell, J. A. F. Artificial intelli- gence (ai) in education: Unlocking the perfect synergy for learning. Educational Process: International Jour- nal, 13(1):35 2024.
Ren, M., Park, J. Y., Houston, M., Aiken, A., and Dally, J. A tuning framework for coftware-managed mem- ory hicrarchies. In Proceedings of the 17th Interna- tional Conference on Parallel Architectures and Compi- lation Techniques, PACT '08, pp. 280-291, New York. NY, USA, 2008. Association for Computing Machin- doi: 10.1145/1454115. cry. ISBN 9781605582825,
Van De Geijn, R. A. and Watts, J. Summa: Scalable univer- sal matrix multiplication algorithm. Concurrency: Prac- tice and Experience, 9(4):255-274 1997.
Van der Wijngaart, R. F. and Mattson, T. G. The parallel research kernels. In 2014 IEEE High Performance Ex- treme Computing Conference (HPEC), pp. 1-6. IEEE, 2014.
1454155. URL
1454115. 1454155.
SFX Teixeira, T., Henzinger, A., Yadav, R., and Aiken, A. Automated mapping of task-based programs onto dis- tributed and heterogeneous machines. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1- 13, 2023.
Wang, X., Liu, S., Tsaris, A., Choi, J.-Y., Aji, A. M., Fan, M.. Zhang, W., Yin, J., Ashfaq, M., Lu, D., et al. Or- bit: Oak ridge base foundation model for earth system predictability. In SC24: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1-11. IEEE, 2024.
Shao, Y., Jiang, Y., Kanell, T. A., Xu, P., Khattab, O., and Lam, M. S. Assisting in writing wikipedia-lite articles from scratch with large language models. arXiv preprint mrX1::2402.14207, 2024.
Wang, Z. and O' Boyle, M. F. Mapping parallelism to multi- cores: A machine learning based approach. In Proceed- ings of the 14th ACM SIGPLAN Symposium on Prin- ciples and Practice of Parallel Programming, PPoPP '09, pp. 75-84, New York, NY, USA, 2009. Associa- tion for Computing Machinery. ISBN 9781 9781605583976. doi: 14445///500111114141119 org/10, 1541
Slaughter, E., Lee, W., Treichler, S., Bauer, M., and Aiken, A. Regent: high-productivityy programming language for hpc with logical regions. In Proceedings of the Inter- national Conference for High Performance Computing. Networking, Storage and Analysis, pp. 1-12, 2015.
WtRLhttss:/ doi.
Wu, Q., Bansal, G., Zhang, J., Wu, Y., Zhang, S., Zhu, E., Li, B., Jiang, L., Zhang. X., and Wang, C. Autogen: Enabling next-gen 11m applications via multi-agent con- versation framework. arXiv preprint 23X.v:2308.08/5. 2023.
Snoek, J., Larochelle, H., and Adams, R. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25, 2012.
Solomonik, E. and Demmel, J. Communication-opiinaa parallel 2.5 d matrix multiplication and lu factorization algorithms. In Euro-Par 201 Parallel Processing: 17th International Conference, Euro-Par 2011, Bordeaux, France, August 29- 99-September2 2, 201 Proceedings, Part 11 17, pp. 90-109. Springer, 2011.
Yadav, R., Aiken, A., and Kjolstad, F. Distal: the dis- tributed tensor algebra compiler. In Proceedings of the 43rd ACM SIGPLAN International Conference on Pro- gramming Language Design and Implementation, pp. 286-300, 2022.
Yang, C., Wang. X., Lu, Y., Liu, H. Le, Q. V., Zhou, D., and Chen, X. Large language models as optimizers. arXiv preprint arXiv:2309.0322 00409, 2023.
Stocks, R. Vallejo. L. G., Fiona, C., Snowdon, C., Palethorpe, E., Kurzak, J., Bykov, D., and Barea, G. M. Breaking the million-electron and eflop/s barriers: Biomolecular-scale ab initio molecular dynamics using mp2 potentials. In SC24: International Conference for High Performance Computing, Networking. Storage and Analysis, pp. 1-12. IEFE, 2024.
Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. Large language models as optimizers. In International Conference on Learning Representations, 2024a.
12

---

Improving Paraliel Program Performance with LLM Optimizers via Agent-System Interfaces
Yang, J., Jimenez, C. E., Wettig, A., Lieret, K., Yao, S., Narasimhan, K., and Press, O. Swe-agent: Agent- computer interfaces enable automated software engi- neering. arXiv preprint arXiv:2405.15793 2024b.
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. and Cao, Y. React: Synergizing reasoning and acting in language models. arXiv preprint (rXiv:2210.0362)) 2022.
Yuksekgonul, M., Bianchi, F., Boen, Liu, S., Huang, Z., Guestrin, C., and Zou, J. Textgrad: Automatic" differen- tiation" via text. arXiv preprint wtw.:::22000049999000 2024.
Zheng, L., Jia, C., Sun, M., Wu, Z., Yu, H., Haj-Ali, A., Wang, Y., Yang, J., Zhuo, D., Sen, K., al. An- sor: Generating (figg-Performance) tensor programs for deep learning. In 14th USENIX symposium on op- erating systems design and implementation (OSDI 20), pp. 863-879, 2020a.
Zheng, L., Li, Z., Zhang, H., Zhuang, Y., Chen, Z., Huang, Y., Wang, Y., Xu. Y., Zhuo, D., Xing, E. P., al. Alpa: Automating inter-and (Intra-Operator) parallelism for distributed deep learning. In 16th USENIX Sympo- sium on Operating Systems Design and Implementation (OSDI 22), pp. 559-578, 2022a.
Zheng, S., Liang, Y., Wang, S., Chen, R., and Sheng, K. Flextensor: An automatic schedule exploration and opti- mization framework for tensor computation on heteroge- neous system. Proceedings of the Twenty-Fip Inter- national Conference on Architectural Support for Pro- gramming Languages and Operating Systems, pp. 859- 873, 2020b.
Zheng, S., Chen, R., Wei, A., Jin, Y., Han, Q., Lu, L., Wu, B., Li, X., Yan, S., and Liang, Amos: enabling au- tomatic mapping for tensor computations on spatial ac- celerators with hardware abstraction. In Proceedings of the 49th Annual International Symposium Computer Architecture, pp. 987, 2022b.
Zheng, S., Fang. Zheng, X., Hou, Q.. Bao, W., Zheng, N., Jiang, 7., Wang, D., Ye, J., Lin, H., et al. Tilelink: Generating efficient compute-communicatio overlap- ping kernels using tile-centric primitives. arXiv preprint arXiv:2503.20313. 2025.
Zhou, D., Schärli, N., Hou, L., Wei, Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q. et al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205. 10625, 2022.
13

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
A. Appendix
A.1. Illustration for Mapping
We show an illustration for mapping in Figure Al.
Application Tasks
Mapper
Machines
Node 0
task tack regionciees, 1800-, region
Processor Selection (tasks)
BPU
CPU
CPU
GPU
Trmee
Syren
CPU
CPU
GPU
CPU
GPU
task [C; regiorde,, 88,888t Bile, D: region-e, B0r, o..
300 GB/s
for i range
for J range rangel for rangetabls
Memory Placement (data)
GPUs
FrameBufler
Network
Eli,j]
D1K,jl;
ZeroCopy
25 GB/s
CPUs
SycMem
Data
Node
GPJ
GPU
CPU
Index Mapping
CPJ
TaaTe
CPU
CPU
GPU2
GPUS
Node
Pamma
(tasks, data)
GPUB
GPUL
GPJ
Sufe:
CPU
CPU
regtonc14N0,
regt
(processor)
Node
32GB
1 TB
Figure A1. Mappers decide the placement of each task in the task graph to processors, the placement of data to memory, and how the itcration space of data partitioned and mapped to different processors.
A.2. DSL Grammar
Terminals: TaskName, Regionnanannnr var, int
Grammar Rules:
Program
Statement+
Statement
TaskMap DataMap DataLayout FuncDef IndexTaskMap
TaskName var
TaskMap
Task TaskName Proct
DataMap
Region TaskName RegionName Proc Memory+
Proc
CPU
GPU CMP
Memory
SYSMEM FBMEM ZCИEM
DataLayout
Layout TaskName RegionName Proc Constraint
Constraint
SOA AOS C_order F_orderr Align int
FuncDef
def var (vart) FuncStmt+
FuncStmt
Expr return Expr
Expr
var var (Expr+) Machine (Proc) Expr. Expr Expr Op Expr (Expr)
Expr [Expr] #Expr Expr Expr Expr
14

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
A.3. Parallel Matrix Multiplication Algorithms
2D Algorithms Cannon's (Cannon, 1969) introduced systolic communication pattern with tiled data partitioning for distributed matrix multiplication. PUMMA (Choi al., 1994) and SUMMA (Van De Geijn & Watts, 1997) extended this approach by supporting non-square matrices and improving communication efficiency through pipelining. They are called 2D algorithms because they partition the matrices into 2D tiles and then map them onto the processor space.
Non-2D Algorithms Johnson's (Agarwal et al., 1995) introduced a 3D algorithm that partitions the input matri- ces into 3D tiles and uses additional memory per processor to reduce communication compared to 2D algorithms. Solomonik's (Solomonik & Demmel, 2011) balances between 2D and 3D approaches by using extra memory further minimize communication. COSMA (Kwasniewski al., 2019) takes different approach by optimizing the processor grid and parallelization strategy based on the input size and the machine size.
A.4. Additional Performance Statistics
In our setting, reporting the best result across multiple runs is appropriate, the best mapper is the one that is desired by the user. Mapper search is offline optimization process, and it is feasible run the optimizer multiple times to select the highest-performing mapper. Once identified, this mapper can be reused without incurring additional search cost, as the deployment scenario (application, input, and hardware) remains fixed.
That said, additional statistics on performance varations can provide a more complete picture. Here we include the mean, standard deviation, worst, median, and best normalized throughput across five runs for each benchmark of our approach Trace and Open
Benchmark
Mean
Std Dev
Worst
Median
Best
Circuit
1.33x
0.01
1.31x
1.33x
1.34x
Stencil
1.01
0.01
1.00x
1.01x
1.02
Pennant
1.03
0.02
1.00x
1.03x
1.04x
Cannon
1.09x
0.00
1.08
1.09
1.09
SUMMA
0.86x
0.48
0.00x
1.07
1.09
PUMMA
0.57x
0.55
0.00x
0.66x
1.09
Johnson
0.98
0.17
0.68x
1.06x
1.07
Solomonik
0.52
0.41
0.00
0.61
1.09
COSMA
1.25x
0.03
1.23x
1.23x
1.31x
Table AI. Normalized throughput of our framework Trace.
Benchmark
Mean
Std Dev
Worst
Median
Best
Circuit
0.97x
0.16
0.81
0.99
1.20
Stencil
0.00x
0.00
0.00
0.00x
0.00
Pennant
0.00x
0.00
0.00x
0.00x
0.00x
Cannon
0.00x
0.00
0.00x
0.00x
0.00x
SUMMA
0.00x
0.00
0.00x
0.00x
0.00x
PUMMA
0.00x
0.00
0.00x
0.00x
0.00
Johnson
0.00x
0.00
0.00x
0.00x
0.00x
Solomonik
0.00x
0.00
0.00x
0.00x
0.00x
COSMA
0.00x
0.00
0.00
0.00x
0.00x
Table A2. Normalized throughput of OpenTuner.
Our method achieves relatively stable performance across most benchmarks. The higher variance and occasional 0.00x worst-case throughput observed in SUMMA, PUMMA, and Solomonik are due invalid mapper configurations in the search space (e.g., violating cuBLAS layout constraints). The runtime enforces correctness by rejecting such configurations during execution. While the generative optimizer typically learns to avoid these cases through the AutoGuide mechanism, occasional failures within the 10-iteration budget are still possible. In practice, such failures can be mitigated by repeating the optimization and selecting the best best-perfonning mapper. In contrast, Open'unee despite running the same number of iterations, fails to generate valid mappers for 8 out of 9 benchmarks. This highlights the difficulty of exploring the search space using traditional reinforcement learning methods.
15

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
A.5. Examples of Feedback Configurations
We give examples for the raw execution oulput and enriched feedback in Table A3. explanations of emrors and suggestions for mapper modifications.
The enhanced feedback includes
AutoGuide
Case
Raw Execution Output
Explain
Suggest
Compile Error:
Syntax error, unexpected
There should be no colon in function definition.
casel
N/A
expecting {
Compile Error: Index Map'ssss function undefined
Define the IndexTaskMap function first before using it.
case2
N/A
Include mgpu = Machine (GPU)
case3
Compile Error: mgpu not found
N/A
in the generated code.
Execution Error: Assertion failed: stride does not match expected value.
Memory layout is unexpected.
Adjust the layout constraints or move tasks to different processor types.
case4
Execution Error: DGEMM parameter
Memory layout is unexpected.
case5
Adjust the layout constraint.
number 8 had illegal value
Ensure that the first index of mgpu ends with $ mgpu a ze [0], and the second element ends with mgpu s ze [1].
MadeM statements cause error.
Execution Error:
Slice processor index out of bound
case6
Instancelimin statements cause error.
Execution Error: Assertion "event.exists()" failed
Avoid generating Instance statements.
case7
Performance Metric: Execution time is 0.03s.
Move more tasks to GPU to reduce execution time.
case8
N/A
Try using different TndexTaskMap SingleTaskMap statements to maximize throughput.
Performance Metric: Achieved
case9
N/A
throughput 4877 GFLOPS
Table A3. Raw execution output and AutoGuide (error explanations and adjustment suggestions) different cases.
16

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
A.6. Trace Agent Code
Trace (Cheng et al., 2024) uses Python decorators like @bundle annotate Python programs. It allows to design an LLM code generation agent if were writing Python program ourselves. We first set up end-to-end runnable Python program that can generate valid mapper program by randomly making decisions over the search space. We show the high-level structure of our Trace Mapper in Figure A3. Figure A2 shows how we incorporate the feedback from the execution to update the agent. At each optimization step. Trace will execute corresponding execution flow build up graph. Then it will make a call to an LLM to perform an update to any function that is decorated with @bundle (t (trainable-rue) The DSLMapperGenere providing search space specified by the DSL, where an LLM optimizer can make decisions along the pre-designed axes. We note that this type of design only enabled by recent developments like Trace and is much challenging to do using older LLM-based frameworks.
coaMapperGeneratee
and collect the
structured the same way as
policy Mapperkgent ()
params policy-garemetezze
optimizer trsce.Optinizer (perams))
app GeLXpplicationIntol
test GetKaapprrvalluater (app)
in zaage (iterations):
+ Forward pass
try:
11
mapper po
12
Seedback (stx) containa performance (mapper)
13
feedback
14 except franetxecutionB.ror as e:
15 feedback str(e)
16 target = excepttion_nnde
17
18 1 Backward serorr.eer___eee sero__eedbackk) ggtimizer.baciwen (target, feedback)
and update
19
21 optimizez.step
Figure A2, We show how use Trace to incorporate the feedback from the execution update the agent, with Pytorch-like syntax.
17

---

Improving Paraliel Program Performance with LLM Optimizers via Agent-System Interfaces
import opto. trace as trace
class MapperAgent (czace.Module).
Strace. bundle (trainable-Tzue)
def task_decision (self, tasks):
gtrace. bundle (tarraabeeeee def degioo__declsiottseeff,, =egions):
10
11
12
strace. bundle (trairable-Truse def Layout_decisson
13
14
15
Strace.bundie (traiaaberra def
(self, tasks)
18
19
20
Strace bundle (trainetttaa def wtt.ttaaapeeciaiiiooo.o index_tasks)
21
22
23
Strace.bundie (traaaableear)))
def single_task_nmp_deeee (self, single_tasks):
def generate_mapper (self);
28
29
Generate the final mapper code by combining all code statements.
30
31
taak stateenttte selfitaas__eciisionnseellll tasks)
32
region_statenerte f. regiireeiionddecisii (sel regiens)
33
layout_statenents
self. w helf.leyour_decision ()
instance__liiit_stateee
het.:ee______tteeeeeeeeoeeeeee.. (self.tasks)
index_taak__a__stateeeee self self.index_te _a (self htt.aaampp.ccccaoooeeeeeee.eeeed
self. hell
36
self. task deci (self wtt.eeeef..nnnnneaaaaa_eee
37
38
code_statements
39
task_statements
40
regi.on_statements
41
layout_statementt
instance__limit__sateee
index_task tasssk_map_tttaee statements
single_task_sstccmmntt
45
46
Combine all code atatements and function definitions into single atring code_list = sode_statements
47
48
mapper_code str_join (! (nn"), \n"), +code_list)
49
return mapper_code
Figure A3. High-level structure of the Trace-based agent template, where functions annotated with @bundle (trainable=Trus) define the search space that the LLM optimizer updates during mapper generation. Note: This agent serves as a shared starting point for ALL tasks. For each task, produce mapper from this starting agent and then ask LLMs "optimize" this agent (by changing functions that are t ra trainable) to produce mappers that are optimal for the particular task.
18

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
A.7. Mapping Strategies
Strategy 1:
Map the tasks of @elculate_new_coo
aetribuue_ccharee update_ _ooltaee onto
GPUs in this way: linearize the 2D GPU processor space into ID, then perform ID block mapping from launch domain to the linearized ID processor space.
CPU; for any task, run
if supported
Region +02U FEMEN; for any task, any regicn, if mapped onto Geu, use FEMEN default Rogion SYSMEM; mapped ento CPU, use SYSMEX defau's t
Layout SOA c_orda;;
acpu Machine (CPU); (G2U);
agpu
Above 1a rixed
11 def linearblock (Taek Lazk)
12
return mapo (p) mepp.stze(1), taak.ipoint (0)
14
15 IndexIaskMap sel.aulte..noo.crreetss...iteeee
update veltagee linearblock;
Strategy 2: Place ghost/shared regions (rp shared and rp ghost) onto GPU zero-copy memory
Task CPU; # for task, run GPU if supported
Region GPU FIMEM; & for task, any zegion, if mapped ento GPU, PBMEM default Region + CPU SYSNEM) if mepped onto CPU, use SYSMEX as defauit
Layout SOA C_order;
mcpu Mach i ne (CPU);
9 -gpu Machine (GPU);
10
11
Above is fixed
13
Region sp_shared GPO ZONEX;
on rP__hoott CPU SCMEM;
Strategy 3: Use Array Of Struet (AOS) data layout for all data instead the default SOA
Task CPU; for task, run GFU if supported
Region raxam; t for task, any region, mapped GPU, FaMEn defoult Region CPU SYSMEM; if mapped anto CPU, use SYSMEX as de foult
acpu Machine (CPU);
agpu (GPU);
Above fixed
11 Tayout A08;
Strategy 4: Use Fortran ordering of data layout for all data instead of the default C order
GPU, CPU; for any Lask,
it susported
Region raxam; for any task, any region, mapped ento GPU, use FAMEM as defauls on CPC SYSMEM; if mapped ento CPU, use SYSMEM as de fault
ncpu Machine (CPU); mgpu Machine (GPU)
Abeve is fixed **...*.***
11 Tayout F_order;
19

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
Strategy 5: Align all the regions 64 bytes while using the Fortran ordering of data
Task GPU, For any task, zun on GPU if supported
Region GPU PRMEM; & for any task, any region, if mapped onto GPU, use FRMEM as de teullt Region SYSMEM; mapped CPU, SYSNEX defauit
scpu Machine (CPU);
agpu Machine (GPU);
Above is fixed
10
Layout A1ign=-61 F_ordez;
Strategy 6: Place the task calculate new new currents onto CPU
Iask GPU; for task, run GPU if supported
Region CPC FEмEM; & for task, any region, if mapped onto CPU, use FBKEM as de detult Region CPU SYSMEM; if mapped onto CPU, use SYSNEX as defauit
mepu Machine (CPU);
mgpu Machine (GPU);
10 Layout s0A C_order;
11
la fixed
13 Task selculate_nev_cuzzeen CPU;
Strategy 7: Collect all the memory used by task calculate_nev. setcuurre_eecurrenttss
CPU; for any taak,
if susported
Region GPU FEМEM; for any task, any region, mapped onto GPU, use FRMEM as de default Region CPU SYSMEM; if mapped ento CPU, SYSMEX default
mepu Machine
agpu Machine (GPU))
Tayout s0A C_ordar;
10
11 Abeve is fixed
12 tollectMemory
aaleulatt_now_ccureett
Strategy 8: Ensure that at most 4 tasks of calculate_ ptt__eeeeeeeeeeeeee can be run at the same time
Task GPU, CPU; for any task,
GPU if supported
any task, any zegion, mapped ento GPU, FBMEM defaule Region CPU SYSMEM if mapped ento CPU, use SYSMEN as defauit
Region FIMEM;
ne (CPU); mgpu Machine (CPU);
Layout x 80A C_order;
11 amsaansam Above is fixed casssaaase InstanceTiiit latealaatteeeeerrrreeeeeeeeeeeee 4;
Strategy 9: Map the second region argument of task distribute_charge onto GPU's Zero-Copy memory
Task GPU,CPU; # for Laak, run GPU if supported
Region raxaM; + for task, any region, mapped GPU, FaMEM defoult Region v CPU SYSMEM; if mapped anto CPU, use SYSNEX as de fault
mcpu Machine (CPU); mgpu Machine (GPU);
Layout SOA C_order;
10
11 Above ia fixed
Region distribute_chasee ZUMEN;
20

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
Strategy 10: Map the tasks of calculate_ lettiateeeee_uurrettteeee curreets,distribuut heaprrrbuee_carrreuuueaeeeeeeeee
GPUs in 1D eyclic man- GPU, CPU; for any tack, run GPU supporsed
ner: perform cyclic distribution over both the node and processor dimensions.
Region GPU FAMEM; + foz any task, any zegion, mapped onto GPU, use FRMEM as default Region SYSMEM; mapped CPU, SYSMEM defauit
mcpu Machine (CBU); mgpu = Machine (CPU);
Layout soA C_order;
10
Above is fixed
12 def cyclicld (Tesk task] {
ip task.ipoint;
14
cyclic over node, eyalic over spu
13
return mgpu[ip[0] mggu.sizel0), 1p10] mgau.size[0] appu.sizel1] [1]];
16
17
18 IndexTaskMap gellulate__eww_orreee wwstribate_chhreeeeeeeeeeee updata_voltages eyclicid;
21

---

Improving Paralicl Program Performance with LLM Optimizers via Agent-System Interfaces
A.8. Generated Mapper Examples
Here provide examples of generated mappers for subset of problems. The mappers, written in DSL, are produced by the mapper agent. While the LLM is responsible for creating and refining the mapper agent, the agent itself is implemented in Python, and it generates mappers as DSL programs. For the Circuit Simulation benchmark, the optimized mapper (Figure A5) is more concise than the initial version (Figure A4), with an additional constraint for byte alignment in the data layout. In contrast, for Solomonik's ss algorithm, the initial mapper is relatively simple (Figure A6), whereas the final optimized mapper adopts more complex and detailed index mapping strategy (Figure A7).
1 Task * GPU, CPU;
2 Task setpllate_nw__urrreee G2U;
3 Task update_voll volltages GPU;
4 Region
GPU FEMEX;
Region
SOCKMEM, SYSMEM;
Region all_times
РBМEM;
Region all_nodes GPU FBMEN;
Region al1_wi GPU PRMEM;
Reglon ghost_ GPU FEMEM;
Region xp_all_nodes GPU FEMEM;
11 Region all_privast GPU FEMEM,
12 Region all_shared GPO FRMEM;
Region rp_shared GPU PIMEM; Region Ip_wires GPU FBМEм; Region TP_ghost_ranges C_order AOS; mgpu Machine (GPU);
PAMRM;
Layout
2d Machine (GPU);
20 def same_point (Task task)
21 return m 2d (*task.parent prw.cecoc.......cccrr.r.......r. (m (m_2d) );
22
Figure A4, For the Circuit task, show the mapper produced by the mapper agent at iteration 2.
Task GPU, OMP, CPU;
Task Aicculate_ne__current G2U;
3 Task modate_voltasee GPO;
4 Reglon GPU FOMEM;
C_order AOS Aliga==128; 6 mgpu Kachine (GPU);
5 Layout
2d Machine (GPU);
def same same_point (Task task)
point
return parllaaaaaaaartt..o..
(m_2d)
Figure A5, For Circuit task, we show the mapper produced by mapper agent iteration
22

---

Improving Paraliel Program Performance with LLM Optimizers via Agent-System Interfaces
GPU, OMP, CPU;
Region GP3 FBMEM;
Reglon SOCKMEM, SYSMEM; F_order 808;
4 Layout
5 mggu Kachine (GPU);
def blockid (Task task) {
ip task. task. ipoint; return ngpu[ip[0] sipu.size(0], ip[3] mipu.size ;
IndexTaskMap task_2 block1d;
m_2d
Machine (GPU);
15 def same point (Task task)
return m_2d[+tast .pooesst (m_2d) );
Figure A6. For Solomonik's 's algorithm, we show the mapper produced by the mapper agent iteration 2.
1 Task * GPU, OMP, CPU;
2 Region * GPO FBMEM;
3 Region
SOCKMEM, SYSMEM;
c_order SOA No_Align; mgpu Machine (GPU);
Layout
def Linearize3D (Task task)
ip tasktt.kpp...... taskkippint; linearize - ipl0 ip[1] ip[2];
return (linearie mgpu.size[0] [0], linearize & size
11
IndexTaskMap task linearize30;
def Linearize2D (Task task) {
ip task. ipoint;
linearize ip[0]
ip[2];
return mepu [linearize mgpu.size(0), linearize & mgpa. size
20
21 IndexTaskMap task 1 linearize20;
22
IndexTaskMap task_2 2 Linearize2D;
IadexTaskMap task_3 linearize2D;
IndexTaskMap task_5 linearize20;
md Kachine (GPU);
def same_point. (Task task)
return paretttakk.parettt.mm processor (d] )
29
Figure A7. For Solomonik's 's algorithm, we show the mapper produced by the mapper agent at iteration 10.
23


Reviews:
[{"review_id": "YdRYtIcaAr", "reviewer": "Reviewer_kSxf", "summary": "The paper introduces an innovative framework aimed at automating the process of optimizing parallel program performance using large language models. The proposed system employs a Domain-Specific Language to simplify the generation of mapping code and uses a mechanism called AutoGuide to turn raw execution feedback into actionable insights for the optimization agent. This system leverages generative optimization to find high-performance mappers efficiently, achieving superior results compared to existing tools like OpenTuner, even after fewer iterations.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review! We address the concerns below:\n\n**Q1: Performance metrics could be extended**\n\n\nWe appreciate your suggestion and welcome the opportunity to clarify our evaluation methodology and expand the reported statistics.\n\nIn our setting, reporting the best result across multiple runs is appropriate, as the best mapper is the one that is desired by the user. Mapper search is an offline optimization process, and it is feasible to run the optimizer multiple times to select the highest-performing mapper. Once identified, this mapper can be reused without incurring additional search cost, as the deployment scenario (application, input, and hardware) remains fixed.\n\nThat said, we agree that additional statistics provide a more complete picture of performance variations. In the revised version, we will include the mean, standard deviation, worst, median, and best normalized throughput across five runs for each benchmark. The extended results are as follows:\n\n### Our Framework (Normalized Throughput)\n| Benchmark  | Mean   | Std Dev | Worst  | Median | Best |\n|------------|--------|---------|--------|--------|------|\n| Circuit    | 1.33× | 0.01 | 1.31× | 1.33× | 1.34× |\n| Stencil    | 1.01× | 0.01 | 1.00× | 1.01× | 1.02× |\n| Pennant    | 1.03× | 0.02 | 1.00× | 1.03× | 1.04× |\n| Cannon     | 1.09× | 0.00 | 1.08× | 1.09× | 1.09× |\n| SUMMA      | 0.86× | 0.48 | 0.00× | 1.07× | 1.09× |\n| PUMMA      | 0.57× | 0.55 | 0.00× | 0.66× | 1.09× |\n| Johnson    | 0.98× | 0.17 | 0.68× | 1.06× | 1.07× |\n| Solomonik  | 0.52× | 0.41 | 0.00× | 0.61× | 1.09× |\n| COSMA      | 1.25× | 0.03 | 1.23× | 1.23× | 1.31× |\n\nWe additionally report OpenTuner results.\n\n### OpenTuner (Normalized Throughput)\n| Benchmark  | Mean   | Std Dev | Worst  | Median | Best |\n|------------|--------|---------|--------|--------|------|\n| Circuit    | 0.97× | 0.16 | 0.81× | 0.99× | 1.20× |\n| Stencil    | 0.00× | 0.00 | 0.00× | 0.00× | 0.00× |\n| Pennant    | 0.00× | 0.00 | 0.00× | 0.00× | 0.00× |\n| Cannon     | 0.00× | 0.00 | 0.00× | 0.00× | 0.00× |\n| SUMMA      | 0.00× | 0.00 | 0.00× | 0.00× | 0.00× |\n| PUMMA      | 0.00× | 0.00 | 0.00× | 0.00× | 0.00× |\n| Johnson    | 0.00× | 0.00 | 0.00× | 0.00× | 0.00× |\n| Solomonik  | 0.00× | 0.00 | 0.00× | 0.00× | 0.00× |\n| COSMA      | 0.00× | 0.00 | 0.00× | 0.00× | 0.00× |\n\nOur method achieves relatively stable performance across most benchmarks. The higher variance and occasional 0.00× worst-case throughput observed in SUMMA, PUMMA, and Solomonik are due to invalid mapper configurations in the search space (e.g., violating cuBLAS layout constraints). The runtime enforces correctness by rejecting such configurations during execution. While the generative optimizer typically learns to avoid these cases through the AutoGuide mechanism, occasional failures within the 10-iteration budget are still possible. In practice, such failures can be mitigated by repeating the optimization and selecting the best-performing mapper.\n\nIn contrast, OpenTuner, despite running the same number of iterations, fails to generate valid mappers for 8 out of 9 benchmarks. This highlights the difficulty of exploring the search space using traditional reinforcement learning methods.\n\n\n**Q2: Experiments on multi-node systems**\n\nThank you for raising this point. Our system supports multi-node execution, as this capability is already provided by the underlying runtime system. There is no fundamental technical limitation in our approach that prevents generalization to multi-node systems. The only technical issue we encountered in scaling up our experiments is an engineering issue, not a limitation of our method. Specifically, the cluster we currently use permits interactive execution on a single node, but multi-node runs require submitting jobs through the SLURM scheduler and waiting for resource allocation and job execution. While this can be addressed through SLURM-specific customization, it is orthogonal to the core contributions and novelty of our system. For users with direct multi-node access (e.g., without queueing), our system runs seamlessly without modification. We plan to extend our evaluation to larger-scale experiments as part of future work.", "claims_and_evidence": "1. The experiments are conducted on a single-node setup with specific hardware. Whether the approach can achieve similar results on larger, more heterogeneous systems with diverse hardware remains questionable.\n\n2. Average performance across multiple runs is reported, but reports on other percentile of performance are missing. This makes it hard to judge the consistency and reliability of the speedup. Additionally, how is the tail performance being impacted?", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "This paper focuses on experimental validation and does not include formal proofs for its theoretical claims.", "experimental_designs_or_analyses": "The authors evaluate their approach using nine established benchmarks from the Legion framework, which is appropriate for assessing parallel program performance. A few potential issues:\n\n1. Experiments on a single-node hardware design\n2. Performance measured only on average is not sufficient, tail performance should also be reported", "relation_to_broader_scientific_literature": "The paper advances parallel programming by synthesizing high-level DSL design, reinforcement learning autotuning, and LLM-driven code generation, incorporating natural language feedback and a modular, agent-based approach.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "### Strengths\n\n1. The system handles complex parallel computation systems and scales efficiently with larger programs and datasets.\n2. The DSL abstracts away the complexities of low-level programming, making it easier for LLMs to generate correct mapping code and improving code generation success rates.\n\n### Weaknesses\n\n1. The current system works best for parallel programs that align with the DSL’s design. For non-standard or highly unique system architectures, the framework may need further customization.\n2. While the AutoGuide mechanism is powerful, it still relies on raw execution output which might not always provide the necessary insights, especially in cases of complex or obscure errors.\n3. Performance metrics could be extended.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BYDciXUtrT", "reviewer": "Reviewer_vUWr", "summary": "This paper proposes a system powered by large language models (LLMs) to automate both the generation and optimization of mapper code. Specifically, it introduces a Domain-Specific Language (DSL) that provides a high-level interface encapsulating all performance-critical decisions required for mapper generation. The authors then implement the AutoGuide mechanism, which interprets raw execution outputs into informative and actionable feedback. This mechanism enables the agent to iteratively optimize the mapper by leveraging enriched feedback to refine its code generation strategy. Finally, the authors apply generative optimization to further enhance the generated code.  \nEvaluation results demonstrate that the proposed method outperforms OpenTuner even after 1,000 iterations, achieving a 3.8× performance improvement.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful and constructive feedback.\n\n**Q1: What is your DSL grammar?**\n\nIn the revision, we will include a complete description of the DSL syntax in the Appendix, covering its constructs for task placement, memory allocation, layout specification, and index mapping, as shown below.\n\n\n```\nTerminals:\n  TaskName, RegionName, var, int\n\nGrammar Rules:\n  Program         → Statement+\n  Statement       → TaskMap | DataMap | DataLayout | FuncDef | IndexTaskMap TaskName var\n  TaskMap         → Task TaskName Proc+\n  DataMap         → Region TaskName RegionName Proc Memory+\n  Proc            → CPU | GPU | OMP\n  Memory          → SYSMEM | FBMEM | ZCMEM | SOCKMEM\n  DataLayout      → Layout TaskName RegionName Proc Constraint+\n  Constraint      → SOA | AOS | C_order | F_order | Align == int\n  FuncDef         → def var(var+): FuncStmt+\n  FuncStmt        → var = Expr | return Expr\n  Expr            → var | var(Expr+) | Machine(Proc) | Expr.Expr | Expr Op Expr |\n                    (Expr) | Expr[Expr] | *Expr | Expr ? Expr : Expr\n```\n\n\n**Q2: Why only report the best point for the proposed approach rather than full optimization curve?**\n\nPlease kindly note that we report both the best and average optimization trajectory over 10 iterations across 5 runs in Figure 4. Also, we want to clarify that reporting the best-performing mapper is appropriate in our context. Mapper optimization is an offline process, and in practice, it is standard to run the optimizer multiple times and deploy the best result. Once identified, the mapper can be reused across repeated executions on the same application, input, and hardware, incurring no further search cost.\n\n**Q3: Is the DSL expressive enough to cover all mapper code generation problems?**\n\nOur DSL is designed to express a wide range of high-performance mapping strategies, including all of the most important decisions. While there may be cases where certain optimizations are not directly expressible, we have not encountered any. Despite being more constrained than general-purpose C++, the DSL has been proven to be effective: all mappers discovered by our agent that outperform expert-written C++ implementations are expressible within the current DSL.\n\n**Q4: Server specifications and application information are mentioned as inputs, but details are missing.**\n\nWe will clarify these details in the revised version. The server specification includes the number of GPUs and CPUs, and whether the OpenMP runtime is enabled. The application specification includes the list of tasks defined in the application and the data regions accessed by each task. These inputs define the structured search space explored by the agent during optimization. We will also include an example of such input specifications in the revised version for completeness.\n\n\n**Q5: The authors present Code Generation Success Rates (Section 5.2). However, in the main evaluation (Section 5.1), only the performance results are reported.**\n\n\nIn the main evaluation (Section 5.1), we focus on measuring end-to-end performance, which includes both the correctness and performance of generated mappers. If the generated code has any syntax or runtime issues, its throughput is recorded as 0. Therefore, the performance numbers in Section 5.1 implicitly reflect code generation success, i.e., incorrect mappers yield zero performance.\n\n\nSection 5.2 isolates the code generation aspect to better analyze the effects of the DSL on LLM generation success. This section does not aim to evaluate performance directly but rather investigates how often the LLM produces syntactically and semantically correct mappers given natural language descriptions. It complements the main results by demonstrating that using the DSL significantly improves generation success compared to C++, which underpins the performance improvements seen in Section 5.1.\n\n**Q6: Are all random mappers correct?**\n\nNo, not all random mappers are correct. For each application, we generate 10 random mappers by sampling from the full DSL-defined search space, totaling 90 mappers across 9 applications. Among them, 74 (82.2%) raise runtime errors due to invalid mapping decisions. The runtime system enforces correctness by rejecting such mappers during execution, resulting in a throughput of zero.", "claims_and_evidence": "1. This paper argues that the proposed DSL simplifies mapper code generation and provides valuable guidance to the agent. The results demonstrate that the DSL requires less code and enhances the optimization process.  \n\n2. The paper claims that the agentic framework achieves up to a 1.34× speedup across nine benchmarks, outperforming expert-written mappers while reducing tuning time from days to minutes. As shown in Fig. 4, the performance gains are indeed significant.", "methods_and_evaluation_criteria": "1. One key contribution of this work is the proposed DSL. However, the authors do not describe its grammar; instead, they illustrate it with a single example. Furthermore, it remains unclear whether the proposed DSL is expressive enough to cover all mapper code generation problems.  Another unclear part is that the authors mention they use server specifications and application information as inputs but did not provide more details.\n\n2. Regarding the evaluation (Fig. 4), why does the paper report only the best results of the proposed method rather than presenting the full optimization curve?", "theoretical_claims": "NA", "experimental_designs_or_analyses": "In the ablation study, the authors present Code Generation Success Rates. However, in the overall evaluation, only the performance results are reported. Additionally, I am curious about the Random Mapper— is it possible that all these random mappers are correct?", "relation_to_broader_scientific_literature": "It is important for improving system performance.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "1. This paper is well-written and easy to follow.\n\n2. The studied problem is interesting and important.", "other_comments_or_suggestions": "NA", "questions_for_authors": "1. What is your DSL grammar.\n\n2. Why only report the best point for the proposed appraoch?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SZJaHpVtkD", "reviewer": "Reviewer_edWw", "summary": "The paper proposes a system for automatically generating and optimizing parallel program mappers. Particularly, it tries to do this via using  a generative optimization approach aided by an \"agent-system interface\" which uses a DSL to allow LLMs to write code at a high-level. \nEmpirical results show that it can sometimes even beat expert-written mappers.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful and constructive review.\n\n**Q1: How much domain expertise (about the benchmark domain and the LLM capabilities) is needed to expand the ASI and the prompt for the benchmark? More practically, do we need experts who write the mappers to design DSL encoding appropriate priors?**\n\n\nOur framework is designed to minimize the need for domain or system-level expertise when adding new benchmarks. Specifically, for all benchmarks in our evaluation, the agent begins from a randomly initialized mapper and improves it through generative optimization, without any handcrafted priors or expert-written strategies.\nTo incorporate a new application into the benchmark, only two inputs are needed:\n1) **Application metadata**, including task names and the list of data arguments each task accesses\n\n2) **Hardware specification**, including the number of CPUs and GPUs per node and the total number of nodes.\n\n\nThese inputs are typically available from the application code and the machine, requiring no understanding of the DSL or the runtime system. As a result, application developers do not need to write or understand mappers, provide mapping hints, or possess any knowledge of the DSL.\n\n\nThat said, our system can also **support optional injection of expert knowledge** via the `AutoGuide` module. This module can provide customized interpretations of execution failures or application-specific heuristics (e.g., mapping large tasks to GPUs if the developer already knows which tasks are large). While our experiments do not use any such guidance, we believe this optional flexibility is valuable in practice, especially for developers who already have insight into performance bottlenecks and want to speed up the optimization process.\n\n**Q2: The authors can motivate the choice of benchmarks better, particularly for general readers unfamiliar with the domain knowledge of these tasks.**\n\nThanks for the suggestion! We will improve the paper to provide more context and motivation for our benchmark selection. Among the 9 benchmarks, 6 are well-known matrix multiplication algorithms (Cannon, SUMMA, PUMMA, Johnson’s, Solomonik’s, and COSMA). Parallel matrix multiplication remains an active research topic due to its central role in high-performance computing and scientific simulations [1]. Furthermore, improving matrix multiplication performance has a broad impact, as it accelerates numerous downstream machine learning workloads [2,3]. The remaining 3 applications (Circuit, Stencil, and Pennant) represent diverse scientific computing workloads beyond matrix multiplication. Together, this benchmark suite offers both depth (through representative matrix-multiplication algorithms) and breadth (through diverse HPC workloads). We will expand the background and motivation accordingly in the revision.\n\n\n**References:**\n\n[1] Yadav et al. “DISTAL: The Distributed Tensor Algebra Compiler”. 2022\n\n[2] Jangda et al. “Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads”. 2022\n\n[3] Zheng et al. “TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitives”. 2025", "claims_and_evidence": "The authors claim that Agent-System Interface (an abstraction layer between the agent and the system) simplifies code generation and provides more meaningful feedback to the agent. This includes the DSL design and AutoGuide mechanisms. Both of these are supported by evidence from Sections 5.2 and 5.3. Performance results (against OpenTuner and Human Baselines) show strong performance of the system.", "methods_and_evaluation_criteria": "The proposed methodology is well-motivated and explained clearly. The ASI design with customized DSL and explain-suggest based feedback is novel within the domain context and likely helpful for future work. For evaluation, the paper uses 9 HPC benchmarks and measures speedup achieved by the proposed approach.", "theoretical_claims": "none", "experimental_designs_or_analyses": "The paper uses various relevant baselines in the experiments. The ablations are also well directed to properly evaluate individual contributions of the ASI block, providing evidence for improvements from both DSL design and AutoGuide.", "relation_to_broader_scientific_literature": "The paper builds upon prior HPC / autotuning literature and promotes generative optimization as an alternative to prior reinforcement learning approaches.", "essential_references_not_discussed": "To the best of my knowledge, relevant works are discussed appropriately.", "other_strengths_and_weaknesses": "Strengths:\n- Comprehensive experiments on standard HPC benchmarks, with clear baselines and ablations.\n- Agent-System Design with proper ablations\n\nWeaknesses\n- Since only 9 benchmarks are considered, it is not clear how to estimate the generality of the approach \n- It is possible the DSL and relevant prompts bake in a lot of human prior which can lead to inflated estimation of underlying model capabilities. Authors should also attempt to study sensitivity to the design of this ASI", "other_comments_or_suggestions": "The authors can motivate the choice of benchmarks better, particularly for general readers unfamiliar with domain knowledge of these tasks.", "questions_for_authors": "How much domain expertise (about the benchmark domain and the LLM capabilities) is needed to expand the ASI and the prompt for the benchmark? More practically, do we need experts who write the mappers to design DSL encoding appropriate priors?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes an agent-based framework for automating mapper generation in parallel programs using LLMs, incorporating a custom DSL and execution-guided feedback (AutoGuide). The approach shows promising performance improvements across nine benchmarks, outperforming both OpenTuner and expert-written mappers within a small number of iterations.

All three reviewers recommend weak accept. They commend the novelty of the Agent-System Interface, clear experimental results, and practical utility for HPC. Key concerns include limited theoretical analysis, the scope of DSL expressiveness, and evaluation restricted to single-node hardware and a small benchmark suite. The authors’ rebuttal addressed most points thoroughly, adding statistical performance metrics, clarifying DSL design and search space coverage, and justifying the benchmark choices. Reviewers acknowledged these improvements and maintained their scores.

Overall, while some scalability and generalisation concerns remain, the paper makes a solid and novel systems contribution.
Decision: Accept (poster)