Rethinking Code Similarity for Automated Algorithm Design with LLMs
Authors: Rui Zhang
Zhichao Lu
Venue: ICLR 2026 Poster (2026)
Keywords: Algorithm Similarity
Automated Algorithm Design
Large Language Model
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: The rise of Large Language Model-based Automated Algorithm Design (LLM-AAD) has transformed algorithm development by autonomously generating code implementations of expert-level algorithms. Unlike traditional expert-driven algorithm development, in the LLM-AAD paradigm, the algorithm's ideas are often implicitly embedded in the generated code. Therefore, assessing algorithmic similarity directly from code, distinguishing genuine algorithmic innovation from mere syntactic variation, becomes essential. While code similarity metrics exist, they fail to capture algorithmic similarity, as they focus on surface-level syntax or output equivalence rather than problem-solving behavior.

We propose BehaveSim, a novel method to measure algorithmic similarity through the lens of problem-solving trajectories (PSTrajs)—sequences of intermediate solutions produced during execution. By quantifying the alignment between PSTrajs using dynamic time warping (DTW), BehaveSim distinguishes algorithms with divergent logic despite syntactic or output-level similarities. We demonstrate its utility in two key applications: (i) Enhancing LLM-AAD: Integrating BehaveSim into existing LLM-AAD frameworks (e.g., FunSearch, EoH) promotes behavioral diversity, significantly improving performance on three AAD tasks. (ii) Algorithm analysis: BehaveSim clusters generated algorithms by behavior, enabling systematic analysis of problem-solving strategies—a crucial tool for the growing ecosystem of AI-generated algorithms. Data and code of this work are open-sourced at https://github.com/RayZhhh/behavesim.

Published as a conference paper at ICLR 2026
RETHINKING CODE SIMILARITY FOR AUTOMATED ALGORITHM DESIGN WITH LLMS
Rui Zhang
Zhichao Lu
Department of Computer Science
Department of Computer Science City University of Hong Kong zh chao. luêcityu edu. hk
City University of Hong Kong
rui, zhang- csêmy. cityt.ittuuuu. hk
ABSTRACT
The rise of Large Language Model-based Automated Algorithm Design (LLM- AAD) has transformed algorithm development by autonomously generating code implementations of expert-level algorithms. Unlike traditional expert-driven algo- rithm development, in the LLM-AAD paradigm, the main design principle behind algorithm is often implicitly embedded in the generated code. Therefore, assess- ing algorithmic similarity directly from code, distinguishing genuine algorithmic innovation from mere syntactic variation, becomes essential. While various code similarity metrics exist, they fail to capture algorithmic similarity, as they focus on surface-level syntax or output equivalence rather than the underlying algorithmic logic.
We propose BehaveSim, novel method to measure algorithmic similarity through the lens of problem-solving behavior sequence of intermediate solutions produced during execution, dubbed as problem-solving trajectories (PSTrajs). By quantifying the aligmment between PSTrajs using dynamic time warping (DTW), BehaveSim distinguishes algorithms with divergent logic despite syn- tactic or output-level similarities. We demonstrate utility two key appli- cations: (i) Enhancing ILM--AD: Integrating BehaveSim into existing L.MM- AAD frameworks (c.g., FunSearch, EoH) promotes behavioral diversity, signif- icantly improving performance three AAD tasks. (ii) Algorithm analysis: BehaveSim clusters generated algorithms by behavior, enabling systematic anal- ysis of problem-solving strategies crucial tool for the growing ecosystem Al-generated algorithms. Data and code of this work are open-sourced at https::/ggitiii com www.:/cchc..caccc/aaaaoaa hww.hhhhhhhhhwwiiimmmm
INTRODUCTION
The emerging paradigm of Large Language Model-based Automated Algorithm Design (LLM- AAD) (Liu et al. 2026) has garnered significant interest for its potential to autonomously generate code implementations of expert-level algorithms. This approach integrates a LLM into an iterative search framework (e.g., evolutionary algorithm), where the L.LM proposes candidate algorithms and the search routine governs the overall process. Crucially, LLM-AAD inverts the traditional, expert-driven workflow: instead of first articulating algorithm's design logic and then implementing it in code, the core ideas are often implicitly encoded in the generated algorithm implementation. Consequently, as LLM-AAD sees broader applications, the ability to assess algorithmic similarity directly from code distinguishing genuine innovation from mere syntactic variation becomes essential.
Measuring similarity between code snippets has been extensively studied in software engineering, with widespread applications diverse tasks such as code retrieval (Keivanloo al. 2014), clone detection (Roy et al., 2009), and program classification (Mou et al., 2016). Existing code similarity measurements can be broadly classified into two main categories. Methods the first category analyze programs in static setting without executing them, i.e., calculating similarity based on features derived from their code. Some representative features used by methods in this category
"Corresponding Author

---

Published as a conference paper at ICLR 2026
Code Breadth-fistt Search
Code Depeh-first Search (Itentive)
Code Degth-first Search (Recursive)
(a)
def hrf((rot)):
AST
def dFs(root):
def dfa(root)::
Lf not root:
relurn ()
Sim.
1f not root:
neturn
Sim.
If not roat:
return []
Binary Tree
[roo:]
0.79
- [root]
0.44
Left def(
res [
res - []
right dfs(node.rightt)
while 1:
while
return
node L.p00(0)
node L.ppp()
[root.value]
res, www.oddpoooeeee.ooooo vol)
Token
res. we..opppernddeee vol)
Token
right
if node, left:
Sim.
node left:
Sim.
left
cropenddCodee left)
1. opppprpprdde left)
node right:
0.82
node r ighe:
0.23
cappend(rode.
dappppppdddd right)
return res
retum res
Order:
[AB.DERG]
[AC.G.BE..] -AC0
[.CGFBED]
Trajectory:
[A]- [A.B] [ABC]- (AB.C.D))
[]] - [AC]
Original Amm
Internediote Resals
Final Outpot
Insertion Sort:
[4. 1. 5]
12.
4.
3,
0.
2,
3,5]
2. 2. 4, 5]
[1. 5.
Rubble Sort:
[4, 1,
P.
11, 3,4,
Figure 1: Examples demonstrating existing code similarity metrics are insufficient for measuring algorithmic similarity. (a) Existing code similarity metrics, on the one hand, find the breadth-first search (BFS) and depth-first search (DFS) algorithms highly similar, despite the two algorithms being inherently different in their traversal logic. On the other hand, they also find that a re-implementation of DFS (Code 3) based on recursion is completely different algorithm from the original iteration- based implementation (Code 2), despite the two codes essentially representing the same algorithm. (b) Merely checking the output of two algorithms cannot distinguish between distinct algorithms. Both insertion sort and bubble sort algorithms yield identical sorted arrays, despite being inherently different algorithms.
include token-based (Papineni et al., 2002). structure-based (Ren et al. 2020), and embedding- based (Dong et al. (222). In contrast, methods in the other category assess similarity by executing the codes and comparing the generated outputs over suite of test cases (Roziere al. 2020).
However, fundamental limitation arises when applying existing code similarity metrics to assess algorithmic similarity: they primarily evaluate surface-level syntax or output equivalence rather than capturing the underlying algorithmic logic. For example, in the binary tree traversal problem (Figure a)), existing static feature-based metrics cannot differentiate between algorithms with syntactically similar implementations but divergent underlying logic. Similarly, existing execution- based metrics (Figure conflate algorithms like insertion sort and bubble sort, as they produce identical outputs despite differing in the sorting logic. These cases collectively demonstrate that accurate assessment of algorithmic similarity must account for an algorithm's 's problem-solving behavior specifically, the approach it employs to solve given task.
In this work, we introduce BehaveSim, a tangible method for measuring similarity between algorithms from the behavioral perspective. Specifically, propose define behavior similarity based on the goblemm-solving trajectory (PSTraj) of an algorithm, where a PSTraj is a sequence of intermediate or partial solutions generated by algorithm two PSTrajs by aggregating over the pairwise distances between elements from these two PSTrajs via dynamic time warping (DTW) (Senin) 2008). A pictorial illustration is provided in Figure showing that BehaveSim can differentiate "lookalike" algorithms with distinct mrollem-solving behaviors. In essence, BehaveSim provides a new angle to quantify the novelty of algorithm from the perspective of behavior similarity (to existing algorithms).
problem. Then, measure the differences between
Furthermore, we demonstrate two direct use cases (UCs) of BehaveSim:
[UCI] Improving the performance of existing LLM-AAD methods: Recent advances have suggested that maintaining diversity among candidate algorithms is crucial for guiding the search towards high-quality algorithms (Romera-Pareee provides a new way to control diversity by encouraging algorithms with distinct problem- solving behaviors. Empirically. we demonstrate that both FunSearch Ghomera-Paredes et al.) (2024) and EoH (Liu et al., 2024) with BehaveSim significantly outperform their original counterparts and other existing state-of-the-art methods on three AAD tasks.
al. 2024, Wang et al., 2024a). BehaveSim
[UC2] A new tool for quantitative algorithm analysis: As LLM-AAD is gaining popularity, an increasing number of Al-generated algorithms expected in the near future. Being able to
2

---

Published as a conference paper at ICLR 2026
Algorithm
Suppe5
Step-15
Step-25
destinntion
dist.
):
iF unvisited,dio size:
return natiiintttnnnnnnne
dist. dist..to.ime to.viisttt
dist_.not [cur_nocccood unvisited]
Index rP, wengninneiit_.oouuugg
next node unvistted index_ return next node
Metri: DEG Token CodBLBB Embed
Metric: Behaviot Similarit
A Stniing Cit
Sim: 1.0 10 88
34
98
Stm.
.99
def select__nxt__noe node(
cur node, destinelionnnndd, unvisited, dist_rot
):
IF not unvisited.eoo. size: return destination node dist to unvisit
dist_mnce [cur_.ooe unvisited] ni: ne.argsacciss unvi51))) nexi_node unvistied] return next_
Algorithm: 2
Stepe5
Step-15
Stop-25
Figure 2: Problem-solving behaviors on the traveling salesman problem (TSP) for two algorithms with highly similar codes. The only distinction in their implementations lies in the use of argmin n () and a argmax (), which leads to profoundly different behaviors: Algorithm I chooses the nearest neighbor node of the current node, while Algorithm 2 always steps the farthest node away from the current node. Nevertheless, existing similarity metrics still assign them a high degree of similarity, failing to reveal their behavioral differences.
automatically and quantitatively analyze algorithms becomes critically important to ensure the sustainable development of LLM-AAD. Taking algorithms generated by an existing LLM-AAD method as an example, we demonstrate that BehaveSim can organize generated algorithms into clusters with similar behavior, facilitating the discovery and analysis of distinct problem-solving strategies.
In summary, our primary contributions are as follows.
1. We demonstrate the necessity of measuring algorithmic similarity through the perspective of their problem-solving behaviors.
2. We propose BehaveSim, tangible method for measuring behavioral similarity based on the problem-solving trajectory. We demonstrate its significance and effectiveness from both methodological and empirical perspectives.
3. We demonstrate the effectiveness of BehaveSim in two use cases: (i) enhancing existing LLM-AAD methods by promoting behavioral diversity during search, and (ii) providing a quantitative tool for analyzing algorithms based on their problem-sol behaviors.
REVISITING CODE SIMILARITY
This section provides an overview of existing code similarity metrics and empirical study evaluating their effectiveness in measuring behavioral similarity.
2.1 EXISTING CODE SIMILARITY METHODS
common approach in existing code similarity methods first derives features from code implementa- tions and then measures similarity based on those features.
Token-based methods treat source code as natural language text. Methods such as BLEU (Papineni al. 2002) and ROUGE (Lin) 2004) parse the source code into sequence of tokens and match their similarit using N-Gram. In particular, CrystalBLEU (Eghbali & Pradel| 2022) and CodeBLEU (Ren
al. 2020) improve N-Gram by emphasizing programming Linguage-specitn tokens.
Structure- and Graph-based methods calculate the similarity between higher-level representations of code implementations. These include abstract syntax tree (AST) (Gabel al. 2008 Ren et al.
3

---

Published as a conference paper at ICLR 2026
2020), data flow graphs (DFG) (Ren et al. 2020). control flow graphs (Zhao & Huang 2018 and program dependency graphs (PDG) (Lu et al. 2023)
Embedding-basee methods represent code in learned vector space (Maveli et al. 2025 Güinherr
ct al. 2023). These approaches typically employ pre- pre-trained models to capture semantic aritis
to compare token-level embeddings (Maveli al. 2025 Zhou et al. 2023 Zhang al. 2020 For
example, CodeBERTSccor (Zhou et al.] 2 computes the cosine similarity between the embeddings
encoded by fine-tuned CodeBERT (Feng et al. 2020) and calculates the similarity based on the F' score of the best matching token pairs.
Execution trace-based methods measure the similarity between low-level execution traces (Pei et al.) 2020), which typically comprise dynamics of variable values at executed lines (Ni et al. 2024 fine-grained program-state changes at the instruction level (e.g.. x86 instructions) (Pei al. 2020
or variable attributes (e.g., addresses and sizes) (Wang et al. 2024b).
2.2 LIMITATION OF EXISTING METRICS FOR ALGORITHMIC SIMILARITY
A Methodological Perspective. While existing static code similarity methods (i.e., token-, structure- graph-, and embedding- based methods) are effective at software engineering tasks (e.g., clone detec- tion, code retrieval), they struggle capture the dynamic problem-solving behavior of algorithms. As illustrated in Figure the only difference between Algorithm and Algorithm 2 lies in the minor modification of function (argmin () vs. argmax ( ). yet this leads to substantial discrepancy in their behavior: Algorithm 1 controls the step toward the nearest neighbor node, while Algorithm 2 always chooses the farthest node. Despite sigmificant differences in their behavior, existing code similarity metrics yield high similarity score. Notably, both their DFG and AST similarities are 1.0, indicating identical structural features in their code. This reveals that they likewise struggle to measure the underlying behavior of algorithms.
On the other hand, execution-tres ceaciioonntteeeeeeee trace-based methods (Pei et al., 2020; Ni et al. 2024 Wang al. 2024b) track the dynamics for multiple variables and function calls, which contain rich and useful information for general-purpose program analysis. However, when characterizing mroblem-solving behavior, the large number of low-level state changes may introduce excessive information unrelated to the algorithm's problem-solving dynamics, potentially obscuring similarity calculations. this regard, BehaveSim be viewed specialized, focused execution trace of intermediate solutions, rather than an exhaustive record of all program variables' state changes.
An Empirical Perspective. In addition, we curate a dataset to systematically assess whether existing code sim- ilarity metrics effectively capture algorithmic similarity. Algorithmic similarity can be characterized along three dimensions: (1) textual appearance, (2) outputs/results, and (3) procedural behavior (i.e., the steps taken to de- rive results). This yields eight possible similarity types, as detailed [able[[ We exclude trivial cases in which all three dimensions dissimilar, as well as infeasible scenarios in which a be- haviorally similar algorithm produces divergent outputs. Consequently, our dataset construction focuses on the four remaining non-trivial types (Type-1 Type-4).
Table 1: Eight possible types of algo- rithmic similarity. "V" indicates simi- larity in given dimension (textual ap- pearance, results, or behavior), while "X" indicates dissimilarity. After excluding trivial cases, the dataset focuses on the four non-redundant types (Types 1 1-4).
either entirely similar or entirely
Text
Result
Behavior
Type-1 Type-2 Type-3 Type-4
/
X
X
J
X
Accordingly, our dataset consists of four distinct types of algorithm pairs, designed to disentangle similarities in text, results, and behavior. Below, we provide brief explana- tions and representative examples for each type. Detailed specifications for individual data pairs are presented in Appx. $
Not Interssted
/
J
X
X
X
V
X
V
Reasible
X
Type 1: High text similarity, different behaviors, similar results. These pairs consist of algorithms that textually similar but exhibit different behaviors. A typical example is matrix multiplication, where subtle differences the order of i, 1, k lead to divergent internal computations, despite high degree of code-level overlap.
4

---

Published as a conference paper at ICLR 2026
Table Average similarity on four types of data calculated by various metries.
Method Type
Method Name
Type-1
Type-2
Type-3
Type-4
ROUGE
0,95
0.96
0.70
0.47
Based on Token
BLEUU
0.83
0.94
0.42
0.16
CrystalBLEU
0.97
0.99
0.68
0.51
Based on Structure
AST
0.96
1.00
0.76
0.57
Combine Token and Structure
CodeBLEU
0.97
0.94
0.91
0.75
CodeBertSccre
0.84
0.97
0.60
0.38
Based on Embedding
0,99
0.99
0.90
0.84
Baeen--Emheddding-O..e
0.94
0.93
0.87
0.73
Based on Execution Results
1.00
0.00
1.00
1.00
Exe-Tracee
Trace+BLEU
0.86
0.95
0.61
0.54
Based on Execution Trace
Trace+Jina- Emb
1.00
1.00
0.87
0.77
0.99
1.00
0.91
0.78
Based on behavior (Ours)
BchaveSim
0.56
0.73
1.00
0.46
Type 2: High text similarity, different behaviors and results. These algorithms similar in
text, but a small, critical change (e.g., a parameter or a conditional statement) not only alters their behavior but also yields different final outputs.
Type 3: Low text similarity, similar behaviors and results. This category contains algorithms implemented diverse ways (e.g., an iterative vs. a recursive implementation of Depth-First Search) but that produce the same behavior and results.
Type 4: Low text similarity, different behaviors, similar results. Algorithms of this type are distinct not only in their code implementation but also in their behavior. However, they produce the same final output. For example, both quicksort and bubblesort yield sorted array, but their behavior when swapping each subarray is completely different.
We evaluate six categories of code similarity metrics on the curated dataset: Token-level metrics, including ROUGE, BLEU, and CrystalBLEU; @ Structure-level metrics, such as AST similarity; @ Hybrid metrics, such as CodeBLEU, which combines token and structural features via a weighted average; Embedding-
metrics, including CodeBERTScore (Zhou et al., 2023) and cosine similarity of two embedding models, Mina-Code-Embeddiin (Günther et al. 2023) and Qwen3- Embedding-OO.6 (Zhang et al. 2025): : G Execution- based metries, which evaluate whether two methods. We trace the internal variable dynamics of algorithms using pysnooper, converting the traces into tokens and embeddings, and measure their similarity via N-Gram methods (e.g., BLEU) and embedding models UJinaCCode- Sodde-Emeddiing.
algorithms yield identical results the same set of inputs. ① siccution-trace-beee
We report the average similarity value each type of data in Table 2 with results on individual data pairs within each type listed Appx. $ A We observe from the results that:
Both static similarity methods (token-, structure-, and embedding- -aased methods) and execution-traccebasee methods consistently yield high similarity for Type-1 and Type-2 data, which have similar code. However, they fail to measure Type-3 pairs, which have similar behavior but dissimilar code, indicating they cannot distinguish algorithmic behavior.
The Execution- based method correctly identifies Type-3 and Type-4 pairs as having identical results. cannot distinguish between Type-1 and Type-4 pairs, as both produce identical final outputs despite exhibiting different behaviors, suggesting that simply checking outputs is insufficient for evaluating behavioral similarity.
Although drecution-trace-basee methods incorporate dynamic execution information, they still exhibit high similarity on Type- I and Type-2 pairs, as execution traces may contain excessive noise information (e.g., variable state changes) that obscures the underlying wroblem-solving logic.
In summary, the results demonstrate that existing algorithmic similarity metrics consistently fail to distinguish between problem-solving behaviors.
5

---

Published as a conference paper at ICLR 2026
3 BEHAVESIM: MEASURING BEHAVIORAL SIMILARITY BETWEEN ALGORITHMS
Scope of Algorithms. This work focuses a general class of algorithms that progressively refine solutions through successive updates, as opposed to one-shot solution generation. Formally, we define an iterative algorithm mapping at iteration t. This formulation encompasses a broad spectrum of important algorithms,, including search and sorting algorithms, optimization methods (e.g.. gradient descent, heuristics), and many machine learning training procedures.
where 7+++ f(x, with iy representing the solution
Problem-solving Trajectory (PSTraj). To measure behavioral similarity between algorithms, essential to first establish a quantitative representation of problem-solving behavior. We represent this behavior as a problem-sole solving trajectory (PSTraj) on a problem instance. Specifically, a PSTraj consists of the intermediate solutions generated by iterative algorithm during problem-solving Iterative algorithms start from an initial solution 2g and progressively produce a solution or partial solution at each step. For example, in convex optimization, an intermediate solution may be complete but suboptimal vector; in binary tree traversal, a partial solution might refer to a subset of nodes that does not yet constitute full path. We collect these progressively generated intermediate solutions into a sequence T (To, 21,- -i,.... ".p....-,rr)) ITy), where T denotes the total number of iterations. We refer this sequence as the algorithm's is PSTraj. This sequence provides a tangible quantification for characterizing an algorithm's 's problem-solvin behavior.
Pairwise Distance Between Two Solutions. We first define distance measure between individual solutions before measuring the distance between two PSTraj. The calculation of pairwise distance depends on the solution types of the problem. The pairwise distance between different types of solutions is defined follows.
Categorical, ordinal, permutation solutions. For these types, we adopt the edit distance dedit (zr, y) as the pairwise distance. To ensure comparability across problem instances, the distance is normalized as: d(x, y) deait (z, y) ddmx where dmax is the maximum possible edit distance for the given problem.
Discrete and continuous solutions. For these cases, we employ the Euclidean distance deue (z. y) = || vll2, normalized by ppoblem-specific upper bound of distance D (e.g.. the possible maximum distance the domain): d(x. y) I| yllz/D.
Similarity Between PSTrajs. Given the pairwise distances between solutions, define the distance between two trajectories using the Dynamic Time Warping (DTW) (Senin 2008). DTW is particularly suitable because it aligns trajectories of potentially different lengths or with temporal shifts, enabling it to capture local similarities of two trajectories. Formally, given two trajectories X (I1...., Fm) and Y (y1,.... ... ..)). the DTW distance is defined as:
DX
min TEA((..)
Σ (FJ)em
d(z;, W;),
where Alm, n) denotes the of all possible alignment paths between the two trajectories. We define Simpstraj by normalizing the DTW distance by the length of the shorter trajectory:
Simpsnay = DTW (T, Y)
min (i J
We note that alternative methods for calculating trajectory distance are also feasible in this scenario. For example, we may calculate the mean pairwise distance for respective solutions. Further discussion on the choices of trajectory distances is provided in Appx. $B
BehaveSim. Given a target problem (e.g., traveling salesman problem) and pl, a probability distribution its problem instances, we let:
1 ~ pl be the problem instance sampled from the distribution Pl,
8 ~ S, be the starting point (or initial solution) sampled from Sy, which is the distribution of feasible starting points on specific problem instance 7,

---

Published as a conference paper at ICLR 2026
I,) be the PSTraj generated by algorithm A on instance I from start-poing
Simpstaa (71, T2) be the similarity score between two PSTrajs.
We define the behavioral similarity BehaveSim between two algorithms, A1 and A2, as the expected similarity of their problem-solving trajectories:
BehaveSim(A1, A2) Pr [E [SimpsTry (T(A1, I, T(22, s))]]
Critically, this expectation is calculated over two nested levels of variation: (i) the distribution of problem instances / for given problem distribution P', and (ii) for each instance, the distribution of possible starting points Sy. Moreover, we highlight two additional considerations in the practical utilities of our method:
Processing PSTraj via truncation or sampling. It is sufficient to select a subset of PSTraj represent the algorithms' behavior. Shorter PSTraj can save computational cost for Be- haveSim calculation. BehaveSim controls trajectory granularity via O Truncation (removing the last proportion & of the PSTraj): or @ Sampling (keeping one solution every steps). We analyze the influence of these parameters in Appx. $
BehaveSim for algorithms with stochastic internal state. Stochastic intermediate states do not invalidate the concept of behavioral similarity, but we need new ways to measure the pairwise distance. ① If the transition F++11 = fA(I, | {) involves a random noise &, we obtain multiple samples to approximate the distribution (z+1(z) and similarly obtain (2) for algorithm B. The pairwise distance can then be defined as divergence (()++1(z), (y)) @ Fixing the random seed is another way to reduce the randomness. Sampling multiple solutions at each step is more robust because it compares behavioral distributions, but it incurs greater computational overhead. Fixing random seeds ensures reproducibility, but may lead to biased calculations of behavioral similarity. Depending on the specific context, either approach can be selected accordingly.
3.1 VISUALIZATION OF BEHAVESIM
First, we consider a continuous optimization problem of minimizing two-variable Rosenbrock function, where each solution is a vector of two real numbers. Figure visualizes the problem- solving trajectories of three different optimization algorithms, namely SGD, BFGS, and Conjugate Gradient (CG), this problem. For each pair of algorithms, we report the average similarity score across two random initial starting points. As shown in the left two plots (SGD vs. BFGS), the trajectories are less similar, leading to lower similarity scores. In contrast, the right two plots (CG vs. BFGS) show more similar trajectories, leading to higher similarity scores.
Second, consider combinatorial optimization of the traveling salesman problem, where each solution (or partial solution) corresponds to a permutation of cities that defines the visiting order. As shown in Figure b), we compare the problem-solving behavior of three expert-designed algorithms and visualize their partial solutions at steps 5, 25, and 50. The starting city is marked with red star. We find that the similarity between Algorithm 1 and Algorithm is higher than that between Algorithm 1 and Algorithm 2, as reflected in both the similarity results and intuitive observations, indicating that our approach can handle both continuous and permutation solution types and can reflect intuitive behavioral similarity between algorithms.
3.2 EMPIRICAL EVALUATION OF BEHAVESIM
We empirically evaluate BehaveSim on the code similarity benchmark dataset described in Sec. 2.2 As evidenced by the results in Table BehaveSim accurately captures behavioral similarity, achieving similarity score of 1.0 on Type-3 instances instances-ccorrectlyy identifying algorithms with identical problem- solving logic despite syntactic differences. Furthermore, BehaveSim assigns significantly lower similarity values to Types 1, 2, and 4, demonstrating its robustness in distinguishing behavioral divergence. Notably, it remains unaffected by superficial textual overlap (Types and 2) or output equivalence (Type 4), validating its focus problem-solving dynamics rather than surface-level features. These results collectively underscore the effectiveness BehaveSim for a quantitative evaluation of behavioral similarity in algorithm analysis.
7

---

Published as a conference paper at ICLR 2026
Trajectory Dist.:
Mean Pairwise
DTW
Trajectory Dist.: Mean Pairwise DTW
Behavioral Sim.:
0.64
0.76
Behavioral Sim.:
0.82
0.95
33
18
SGD
SCD SFF5
8PG5
BFGS
(b)
Starting City City
Route
Algorithm
Algorithm 2
Stepr-5
Step-25
Step-50
Step-
Step
Step--0
Algorithm
Trajectory Mean Pairwise
Algol & Algo2
Behavioral
0.61
Trajectory Dist.: Mcan Pairwise DIW
Algol Algo3
Behavioral Sim.:
0.98
0.98
Figure (a) Comparison of the problem-solving behavior for optimization algorithms. Each plot shows the trajectories of two algorithms on the Rosenbrock problem, each initialized at two different locations. Behavioral similarity reported using both mean pairwise distance and DTW. (b) Comparison of TSP algorithms. Each plot shows the partial solution at different steps (5, 25, 50). The red star denotes the starting city.
USE CASES OF BEHAVESIM
This section demonstrates two direct use cases BehaveSim in LLM-AAD and algorithm analysis. BehaveSim enables direct control over the behavioral diversity of generated algorithms, potentially enhancing search performance. BehaveSim also provides behavioral perspective for algorithm analysis, offering new insights beyond traditional code-level analysis.
4.1 INTEGRATION IN LLM-AAD METHOD
BehaveSim can be integrated into existing LLM-AAD methods to enhance the behavioral diversity of the candidate algorithms. A brief review of related work on LLM-AAD is provided in Appx.gD D In this work, we explore two such integration strategies and apply them to two representative LLM-AAD methods.
Method.
First, we propose a niche-batt bache-based integration, which we demonstrate by combining with Fun- Search Shomera-Paredes al., 2024). The key idea is to augment its multi-island database with a behavioral diversity management strategy, in which a new candidate algorithm is assigned to an island based on BehaveSim score rather than inheriting from its parent algorithm. We refer this combination as FunScarch=-BhhavSSin
helper objective augment the primary fitness score in EoH LLu et al. 2024), in which we use the BehaveSim with the best algorithm found as the second objective during the search. We refer this combination as EoH+BehaveSim. The implementation details of both methods are elaborated in Appx.
Second,
BehaveSim
We evaluate approach three AAD tasks, including Admissible Set Problem (ASP) (Romera-Paredee et al, 2024). Traveling Salesman Problem (TSP) (Matai et al. 2010)), and Circle Packing Problem (CPP) Novikov et al. 2025). We compare search method against their
AAD Tasks.
8

---

Published as a conference paper at ICLR 2026
Fadkaad Beavvsin
- FerSeach
0:40
ParScadde
Bdaaeesn
Rebalie Cop Botoknoos Opsimmnn
0.8
Relseve Cop to Bops Bog-Knooo Known Optinum
2339
-D- Passara
000
2,335
Raslandh
0.55
2.900
Ball
6.50
Edit
0.30
2392
006
2535
6:15
2.900
6938
19000
400
5/6
160
600
506
1290
1500
#Evelarted Algorihme
#livilisted
greithmms
ASvallsttd Algorithme
(a) ASP
(b) TSP
(c) CPP
Figure 4: Convergence the top-10 10 algorithm sets produced by different search methods on three AAD tasks. The y-axis reports the relative gap the best-known optimum (lower is better). Markers denote the mean over three independent runs; shaded regions indicate standard deviation.
original counterpart, FunSearch and EoH. The maximum number of evaluations is set to 2,000 for TSP and CPP, and 10,000 for ASP due to its higher complexity. Each candidate algorithm is evaluated with 50-second timeout. All methods interact with the GPT-5-Nand via API. More detailed settings on individual AAD tasks are provided in Appx. and Appx. S
Table 3: Performance of the top-1 and top- algorithms obtained by each AAD method. For ASP and TSP, metrics are the relative gap to the best-known optimum (%, lower is better). For CPP, the metric is the sum of radii (higher is hetter). All results reported mean standard deviation over three runs. Better results are highlighted in bold.
ASP Performance (L)
TSP Performance
CPP Performance
Top-1
Top-10
Top-1
Top-10
Top-1
Top-10
FunSearch
5.84
6. 6,159.39
25. 75.22
27. 27.4641.21
2. 2.6259
20.044
Behavein
5.24+100
5.28
17.37=0.28
18.05
2.6293
20.002
2.6249
EoH EoH+BchaveSim
3.99 10.20
1.08
18.86 10.20
19.68.0.42
2.5724
2.553510.0550
5..3
5.59
16.55_2.90
17.20
2.62632200054
2.6234.20.000
Results. presentss the convergence curves of the performance of top-10 algorithms obtained by FunSearch-BBhhaeSSin EoH-BehaveSim, and their original counterparts. Complementary results for the top-1 and top-10 performances are reported in Table 3 The performance is calculated by the relative gap the best-known optimum, with lower values indicating better performance.
We find that both counterparts, achieving the best results on both top-1 and top-10 algorithms across these tasks. These results indicate that coupling search methods with behavioral similarity improves both efficiency and final performance ILLM-AD applications, underscoring the effectiveness of encouraging behavioral diversity. Due to the space limit, please refer to Appx. F ablation studies and analysis.
and EoH+BehaveSim outperform their respective original
4.2 ALGORITHM ANALYSIS
BehaveSim offers a novel behavioral perspective for analyzing algorithms. To demonstrate this analytical utility, evaluate 30 algorithms randomly sampled from the final database checkpoint of a FunSearch-BBehaveSim run on the TSP. Figure visualizes the hierarchical clustering dendrograms

---

Published as a conference paper at ICLR 2026
Code11
Code12
If Jest www.ssttttttoooo... (6003)
diatonees dLabars eatrutareet mode, atrizited meha]
dust
rade]
for rode atrlarites
noracitoed ctat-20rat sts.attt---attton clatinnes wttcop/cduseeeeeeee..
(eouuiing uww.itefoonisstteoooo.o nak] anngreaatnnng nin ment "p.irteuut stinkk, remoining)
coeet peeeaaaaen poeait by wax)) dece srac aptiler)
swicit slvves geggenn (eess)
atropogic volue scores oppe sccfeee scorr) (ngoacionn))
Two lighte similar code, di diferrcc fixi soceiiin
Clussering Using Our Behavioral Similarity
Chastering Using CodeBLEU
200 00300 S
select peat mode 31
Cudoó
rotet
Codo8
Jestumos
pereettahttee extieel share Latttiaff smeer oade sieel
carnen, stares instanrustvic coremmneaee unit sates]
Is an eepeedooss hel asses gestimes munel)) mussl)
correts, caveet a rp op-ctacarreil distance), caaaraeeeet disaraao
sovre 8.7 rert newerr 8.1 wer furere crpttcccenee)
ruture
ttaly mat wwt.ruutussssteeneee schertiela)
seedl current (cumert distreas en cament) (aee ourert ein carnens) Lugarnet
an ailee
allaero
gesoatedlss dhe
(destirectos distenses
dest) Cacs
dest)
Tiereiiee gariten always edecs the neighher.
recaiiing more Ieconddiin aid mndes)
retpr/iettittteeeee notrece)
The sigrestees daway unsgro bis weight dhe concest wuught, with whech daveiibez daway sclect the seares acapitor is this inseace
comtine soae nureet anigt meral thee, Junner
retur modeago baaao crgecac acorat)
Figure 5: Clustering results based on two similarity measures. (i) Code 11 and Code 12 are clustered together by CodeBLEU, yet they exhibit distinct behaviors due to differences in their logic for computing the final score. (ii) Code 6 and Code 8 differ in code structure but display consistent behaviors. This is because a bug in Code 6 sets the enttureeptttttia ials" field to an all-zero array, making its score solely determined by the distance neighbors. Similarly, Code 8 assigns dominant weight to the distance term, which overrides other factors. As result, both Code 6 and Code 8 essentially follow the strategy of selecting the nearest unvisited neighbor.
produced by BehaveSim and CodeBLEU. In these visualizations, higher linkage distance (i.e., branching point) denotes a lower degree of similarity between the algorithms.
We observe significant discrepancy in their clustering results. Notably, we find that Code 6 and Code 8 exhibit consistent behavior, as evidenced by their low-level merger in the left dendrogram (highlighted by the blue circle). However, they are distinct in code implementation, reflected by the significantly higher branching point between them in the right dendrogram. Qualitative analysis of Code 6 and Code 8 reveals that their behavioral alignment stems from shared underlying logic: both inherently select the nearest node. Conversely, although Code 11 and Code 12 exhibit high structural similarity according Co (highlighted by the red circle in the right dendrogram), they diverge significantly in practice. This discrepancy originates from their distinct scoring mechanisms, which ultimately dictate their problem-solving behaviors.
These observations underscore the contributions of behavioral perspective in algorithm analysis: it not only identifies the specific code segments responsible for critical behavioral shifts but also uncovers convergent design logic that might be obscured by implementation differences.
5 CONCLUSION
This paper introduces BehaveSim, a novel method measuring algorithmic similarity from a behavioral perspective. Through empirical comparisons with existing metrics, we conclude that BehaveSim can distinguish algorithms with disparate behaviors, whereas existing methods cannot. Two use cases of BehaveSim are demonstrated, showcasing the effectiveness maintaining behavioral diversity in the algorithm search and its application in analyzing algorithm behaviors.
Moving forward, algorithmic similarity and novelty can be assessed along multiple dimensions beyond problenn behavior, such as time and space complexity. These perspectives are also crucial for understanding algorithmic properties. In this work, we focus solely on behavioral similarity, leaving other dimensions for future exploration.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
We sincerely thank Prof. Qingfu Zhang for his valuable feedback and constructive suggestions that significantly improved this work. We also gratefully acknowledge the LLM4AD team at the Optima Group, City University of Hong Kong, for insightful discussions and continuous support.
REFERENCES
Yihong Dong. Jiazheng Ding, Xue Jiang, Ge Li, Zhuo Li, and Zhi Jin. Codescore: Evaluating code generation by learning code execution. ACM Transactions on Software Engineering and Methodology. 2025.
Aryaz Eghbali and Michael Pradel. Crystalbleu: precisely and efficiently measuring the similarity of code. In International Conference on Automated Software Engineering, 2022.
Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng. Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. Codebert: A pre-trained model for programming and natural languages. In Findings of the association for computational linguistics, 2020.
Mark Gabel, Lingxiao Jiang, and Zhendong Su. Scalable detection of semantic clones. In Interna- tional Conference on Software Engineering, 2008
Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, Moham- mad Kalim Akram, Susana Guzman, Georgios Mastrapas. Saba Sturua, Bo Wang, et al. Jina embeddings 2: 8192-token DrXiv:2310.19923, 2023.
text embeddings for long documents. arXiv preprint
Iman Keivanloo, Juergen Rilling, and Ying Zou. Spotting working code examples. In International Conference Software Engineering, 2014.
Chin-Yers Lin. Rouge: A package for automatic evaluation of summaries. Text summarization branches out, 2004.
Fei Liu, Tong Xialiang, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang, Evolution of heuristies: Towards efficient automatic algorithm design using large language model. In International Conference on Machine Learning, 2024.
Fei Liu, Yiming Yao, Ping Guo, Zhiyuan Yang. Xi Lin, Zhe Zhao, Xialiang Tong, Kun Mao, Zhichao Lu, Zhenkun Wang, et al. A systematic survey on large language models for algorithm design. ACM Computing Surveys, 58(8): 12, 2026.
Jiahao Liu, Jun Zeng, Xiang Wang, and Zhenkai Liang. Learning graph-based code representations for source-level functional similarity detection. In International Conference on Software Engineering 2023.
Jinzhu Mao, Dongyun Zou, Li Sheng, Siyi Liu, Chen Gao, Yue Wang, and Yong Li. Identify critical nodes in complex network with large language models. arXiv preprint urXiv:2403.0334 2024.
Rajesh Matai, Surya Prakash Singh, and Murari Lal Mittal. Traveling salesman problem: an overview of applications, formulations, and solution approaches. Traveling salesman problem, theory and applications, 1(1):1-25, 2010.
Nickil Maveli, Antonio Vergari, and Shay B Cohen. What can large language models capture about code functional equivalence? In Findings of the Association for Computational Linguistics: NAACL, 2025.
Lili Mou, Ge Li, Lu Zhang, Tao Wang, and Zhi Jin. Convolutional neural networks over tree structures for programming language processing. In AAAI conference artificial intelligence, 2016.
dean-Baptise Mouret and Jeff Clune. Illuminating search spaces by mapping elites. arxiv preprint 2015.
arXiv: 1504. 04909, 1504.04999
11

---

Published as a conference paper at ICLR 2026
Ansong Ni, Miltiadis Allamanis, Arman Cohan, Yinlin Deng, Kensen Shi, Charles Sutton, and Pengcheng Yin. Next: Teaching large language models to reason about code execution. In International Conference Machine Learning, 2024.
Alexander Novikov, Ngån Vū, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wag- ner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al. Alphae- volve: coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506. 13131, 2025.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Annual Meeting of the Association for Computational Linguistics, 2002.
Kexin Pei, Zhou Xuan, Junfeng Yang. Suman Jana, and Baishakhi Ray. Trex: Learning execution semantics from micro-traces for binary similarity, arXiv preprint arXiv:2012.086880 2020.
Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. Codebleu: a method for automatic evaluation code synthesis. arXiv preprint arXiv:2009.1022 2020.
Bernardino Nomera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. 2024.
Nature, 625(7995):468
Chanchal K. Roy, James Cordy, and Rainer Koschke. Comparison and evaluation of code clone detection techniques and tools: A qualitative approach. Science of Computer Programming, 74(7): 470-495, 2009.
Baptiste Roziere, Marie-Anm Lachaux, Lowik Chanussot, and Guillaume Lample. Unsupervised translation of programming languages. In Advances in neural information processing systems, 2020.
Pavel Senin. Dynamic time warping algorithm review. Information and Computer Science Department University of Hawaii Manoa Honolulu, USA, 855(1-33):40, 2008
Parshin Shojaee, Kazem Meidani, Shashank Gupta, Amir Barati Farimani, and Chandan K Reddy. Llm-sr: Scientific equation discovery via programming with large language models. In Interna- tional Conference on Learning Representations, 2025.
Niki Van Stein and Thomas Bäck. Llamea: A large language model evolutionary algorithm for automatically generating metaheuristics. IEEE Transactions on Evolutionary Computation, 29(2): 331-345, 2024.
Evan Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, Will Song, Vaskar Nath, Ziwen Han, Scan Hendryx, Summer Yue, and Hugh Zhang. Plamning in natural language improves 11m search for code generation. In International Conference on Learning Representations, 2024a.
Huanting Wang, Zhanyong Tang, Shin Hwei Tan, Jie Wang, Yuzhe Liu, Hejun Fang, Chunwei Xia. and Zheng Wang. Combining structured static code information and dynamic symbolic traces for software vulnerability prediction. In International Conference on Softwar Engineering, 2024b.
Shunyu Yao, Fei Liu, Xi Lin, Zhichao Lu, Zhenkun Wang, and Qingfu Zhang. Multi-objective evolution of heuristic using large language model. In AAAI Conference on Artificial Intelligence. 2025.
Yiming Yao, Fei Liu, Ji Cheng, and Qingfu Zhang, Evolve cost-aware acquisition functions using large language models. In International Conference on Parallel Problem Solving from Nature, 2024.
Haoran Ye, Jiarui Wang, Zhiguang Cao, Federico Berto, Chuanbo Hua, Haeyeon Kim, Jinkyoo Park, and Guojie Song. Reevo: Large language models as hyper-heuristir with reflective evolution. In Advances in neural information processing systems, 2024.
12

---

Published as a conference paper at ICLR 2026
Rui Zhang, Fei Liu, Xi Lin, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. Understanding the importance evolutionary search in automated heuristic design with large language models. In International Conference Parallel Problem Solving from Nature, 2024.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations, 2020.
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05/ 76, 2025.
Gang Zhao and Jeff Huang. Deepsim: deep learning code functional similarity. In ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2018.
Shuyan Zhou, Uri Alon, Sumit Agarwal, and Graham Neubig. Codebertscore: Evaluating code generation with pretrained models of code. In Empirical Methods in Natural Language Processing, 2023.
A DETAILED RESULTS ON ALGORITHMIC SIMILARITY DATASET
We provide similarity results for individual data pairs for each data type. Please refer to Table and Table & for detailed data pair specifications and results. Table T Table
Table 4: Comparison of similarity metrics on Type 1 cases, where code is syntactically similar, but execution paths are different while yielding similar final results. Case I: Two matrix multiplication algorithms with different loop orders (ijk jik). Case 2: Two BFS binary one expanding the left child first, the other expanding the right. Case 3: Two DFS binary tree traversal algorithms, one expanding the left child first, the other expanding the right.
traversal algorithms,
Method
Case 1
Case 2
Case 3
ROUGE
0.89
0.98
0.98
BI.UU
0.65
0.93
0.92
CrystalBLEU
0.92
0.98
1.00
AST
0.88
1,00
1.00
CodeBertScore
0.61
0.96
0.96
CodeEmbedding
0.97
0.99
1,00
CodeBLEU 0.96
0.96
0.99
BehaveSim
0.32
0.69
0.68
Table 5: Comparison of similarity metrics on Type 2 cases, where code is syntactically similar, but both execution paths and final results are different. Case Two graph traversal algorithms differing only by min () vs. max () to select the next node (nearest vs. farthest). Case 2: Two online bin packing algorithms with slight hyperparameter variations. Case Another pair of online bin packing algorithms with slight hyperparameter variations.
Method
Case 1
Case 2
Case 3
ROUGE
0.95
0.97
0.97
BLEU
0.92
0.95
0.95
CrystalBLEU
0.99
0.99
0.99
AST
1.00
1.00
1.00
CodeBertScore
0.96
0.98
0.96
CodeEmbedding
0.99
1.00
0.99
CodeBLEU
0.88
0.97
0.97
BehaveSim
0.70
0.90
0.59
13

---

Published as a conference paper at ICLR 2026
Table 6: Comparison of similarity metrics Type 3 cases, where code is syntactically dissimilar, but execution paths and final results are similar. Case 1: Recursive vs. iterative BFS. Case 2: Recursive vs. iterative Bubble Sort. Case 3: Recursive vs. iterative Insertion Sort. Case 4: Two implementations of Merge Sort. Case 5: Two equivalent implementations of First Fit for bin packing. Case 6: Two equivalent implementations of Best for bin packing.
Method
Case
Case
Case
Case
Case 5
Case 6
ROUGE
0.74
0.61
0.57
0.51
0.88
0.91
BLEU
0.55
0.23
0.23
0.14
0.72
0.65
CrystalBLEU
0.75
0.57
0.58
0.62
0.88
0.70
AST
0.80
0.67
0.76
0.55
0.85
0.92
CodeBertScore
0.71
0.43
0.51
0.30
0.79
0.87
dine
0.91
0.88
0.89
0.82
0.96
0,95
CodeBLEU 0.85
0.87
0.88
0.89
0.98
0.99
BehaveSim 1.00
1.00
1.00
1.00
1.00
1.00
Table Comparison of similarity metrics on Type 4 cases where code and execution paths are dis- similar, but final results are similar (Part 1 of 2, Cases 1-9). Cases 1-14 involve pairwise comparisons of different sorting algorithms.
Method
3
5
7
9
ROUGE
0.54
0.48
0.56
0.49
0.55
0.48
0.65
0.41
0.54
BLEU
0.12
0.22
0.17
0.20
0.21
0.10
0.31
0.12
0.18
CrystalBLEU
0.38
0.70
0.56
0.68
0.62
0.40
0.68
0.37
0.51
AST
0.64
0.60
0.70
0.57
0.56
0.66
0.64
0.50
0.69
CodeBertScore
0.31
0.52
0.37
0.32
0.35
0.49
0.46
0.31
0.43
Cobedddddding
0.65
0.83
0.88
0.65
0.86
0.83
0.90
0.81
0.88
CodeBLEU 0.74
0.79
0.88
0.73
0.72
0.76
0.75
0.69
0.71
BehaveSim
0.64
0.49
0.79
0.69
0.44
0.42
0.46
0.45
0.64
Table 8: Comparison of similarity metrics on Type 4 cases (Part 2, Cases 10-18, continued).
Cases 10- 14 involve sorting algorithms. Cases 15-18 involve shortest path algorithms.
Method
10
11
12
13
14
15
16
17
18
ROUGE
0.44
0.37
0.38
0.47
0.52
0.42
0.65
0.29
0.25
BLEU
0.07
0.08
0.09
0.14
0.22
0.14
0.38
0.07
0.05
CrystalBLEU
0.44
0.58
0.61
0.46
0.54
0.48
0.53
0.25
0.39
AST
0.45
0.39
0.48
0.65
0.64
0.50
0.73
0.49
0.42
CodeBertScore
0.40
0.39
0.42
0.25
0.44
0.28
0.49
0.30
0.24
Eobeddngd
0.81
0.80
0.83
0.85
0.83
0.88
0.88
0.78
0.76
CodeBLEU
0.73
0.72
0.79
0.68
0.66
0.75
0.85
0.77
0.77
BehaveSim
0.44
0.51
0.36
0.78
0.29
0.37
0.35
0.00
0.05
B CHOICE OF TRAJECTORY DISTANCE MEASURES
This experiment investigates the effects of different trajectory-similanie measures behavioral- similarity assessment. We consider four measures: mean pairwise distance, Dynamic Time Warping (DTW) distance, Edit Distance with Real Penalty (ERP), and the average cosine similarity between trajectory segments. Figure illustrates the similarities among trajectories computed using these methods.
We observe that the mean pairwise distance, DTW, and ERP suggest that CG and NelderMead- Adaptive are similar, and the cosine similarity indicates that CG and Newton-CG more
14

---

Published as a conference paper at ICLR 2026
2
1o*
Mcasure
Sim.
Mean Pairwise
10"
0.73
1o
DTW
0.96
10
CO
Newton
10"
Cosine
0.37
10
10"
Measure
10
Mean
Pairwise
0.62
DTW
0.93
10
Co
CG
CG
to
ERP
0.81
Nelde Addpiiive
Nelda Mead--dagiiv
Medde
10
Cosine
2
10
-2
Figure 6: Demonstration of problem- trajectories on the Rosenbrock function.
Patiiie
Mora Pasteli
0.95
1.4
0.2
0.91
0.5
Codne
4.4
4033
-02
Cuinn:
-055
40.3
.0.33
-04
D7W
Casne
Mom
Pararto
Drw
Castet
(a) Kendall's Tau (KTau)
(b) Spearman Rank Correlation
Figure Rank correlations of the similarities calculated by four trajectory similarity measures.
comparable. This implies that different trajectory similarity measures capture various aspects of the trajectories.
We also evaluate the PSTrajs of eight algorithms on the Rosenbrock function: CG, Newton CG, SGD, Adam, BFGS, L-BFGG-B, MelderMead, and NelderMead- Adaptive. For each pair of algorithms, we compute their BehaveSim using the four trajectory similarity measures, then report Kendall's Tau (KTau) and Spearman's 's rank correlation coefficient quantify the correlation. The correlation matrices are shown in Figure 7a
The results show that mean pairwise distance, DTW, and ERP exhibit high correlations with one another, while their correlations with cosine similarity are comparatively low. This suggests that mean pairwise distance, DTW. and ERP similarity primarily captures the directional consistency of trajectory segments and may therefore be particularly relevant in specific application scenarios.
interchangeable alternatives. In contrast, cosine
15

---

Published as a conference paper at ICLR 2026
C INFLUENCES OF PSTRAJ TRUNCATION AND SAMPLING PARAMETERS ON BEHAVESIM
BehaveSim controls trajectory granularity via (i) Truncation (removing the last proportion k of the trajectory) and (ii) Sampling (keeping one solution every n steps), We analyze the sensitivity of similarity scores these parameters, with results shown in Table
We observe that the metric is robust to light-to-moderate processing. Truncating 10-30% of the trajectory yields scores nearly identical the whole trajectory, and deviations only become noticeable at aggressive ratios (40-50%). Similarly, sampling intervals of 1-2 steps have minimal effect, while larger intervals (3-5) introduce expected variance. This confirms that BehaveSim stable under reasonable parameter choices.
Table 9: Influence of trajectory sampling strategies similarity scores.
Settings
Type-1
Type-2
Type-3
Type-4
Full-Trajectory (k=0, n=0)
0.56
0.73
1.00
0.46
k =0.1, n
0.55
0.73
1.00
0.47
-0.2, n
0.55
0.73
1.00
0.48
k 0.3, n = 0
0.57
0.73
1.00
0,52
0.4, π
0.60
0.73
1.00
0.58
= 0.5, n = O
0.65
0.75
1.00
0.64
k O. k
0.57
0.72
1.00
0,52
0.56
0.74
1.00
0.59
= O, K
0.56
0.72
1.00
0.65
0.56
0.82
1.00
0.72
π
0.62
0.82
1.00
0.74
D RELATED WORK ON LLM-AAD
The integration between large language models (LLMs) with search methods has become a prevailing
paradigm in automated algorithm design (AAD) (Liu et al. 2024 Zhang et al., 2024). leading
notable advances across a spectrum of AAD applications, including mathematical discovery (Romera
Paredes et al.] (024). combinatorial optimization (Liu et al., 2024: Ye
2024). Bayesian
optimization Yao et al. 2024). black-box optimization Van Stein & Bäck 2024). and science
discovery (Shojaee et 2025).
Notably, recent advances have emphasized the importance of diversity in algorithm search (Romera: Paredes et al. 20244] Novikov et al. 2025). For example, methods such as FunSearch (Romera- Paredes al. 2024) and LLM-SR (Shojace et al. 2025) adopt multiple-island-based program database to enhance diversity. AlphaEvolve (Nvikovv et al. 2025) combines the multiple-islan basdd model with MAP elite (Mouret & Clune 2015) to further improve diversity. PlanSearch (Wang al.) 2024a) performs search in idea and pseudo-code spaces to increase diversity. Other methods incorporate algorithmic similarity metrics into the search process. For instance, MEoH (Yao et al. (2225) embeds AST distances into multi-objective algorithm search, Mao et al. (2024) clusters algorithms within sub-populations according to their embedding distance. In this paper, we introduce combining BehaveSim with a search method to promote the behavioral diversity during search.
E IMPLEMENTATION DETAILS OF OUR SEARCH METHOD
E.1
IMPLEMENTATION DETAILS OF SONSEARCH++BEAVEER
Our search pipeline begins with a predefined "template algorithm" (detailed in Appx. $ G that provides task information for LLMs, including the algorithm's 's inputs and outputs, task descriptions, and argument formats. To initialize the database, Nini randomly generated algorithms are clustered based behavioral similarity and subsequently allocated different islands. The search process
16

---

Published as a conference paper at ICLR 2026
(a) Multi-island- Algo Database
(b) Select Algo from Island
Island
Sel. Prob.
Sel. Prob.
DO
Islands
Tra
Tra
Trai
Trai
Score
Length
Clusters
Island 2
Score-0.1
Score-0.4 0,4
Score 4
Selected Algo
C
DB
Algos
<P
Trai
Trai
Trai
Truj
Len
Len
Len
Island Nist
(0) Create Prompt
(d) Sample & Eval
(e) Register
oo
Generate new
Sim-0.5
algorithm inspired
oo
by given examples
Traj Score=0.4
Sim=0.6
S-0.4 0.4
LLM
Sim=0.3
Figure 8: Overview of TunSearch-BBhaveSSin (a) We utilize multi-island algorithm database, where each island contains clusters of algorithms grouped by identical scores. (b) In the selection phase, an island is chosen from the database. Within the selected island, clusters are prioritized by score, with bigher scores given preference. Once cluster selected, an algorithm is chosen, favoring shorter implementations to promote conciseness. (c) The selected algorithms are used to create a prompt, which guides the LLM to generate a new algorithm.
(d) The newly generated algorithm is evaluated to determine its fitness score and problem-solving trajectory. (e) Finally, the evaluated algorithm is assigned to the island with the highest behavioral similarity and placed into the appropriate cluster.
then proceeds iteratively: algorithms are selected from a database serve as examples for the LLM to generate new algorithms. These new algorithms are evaluated to determine their fitness score and behavioral trajectory, and are then registered back into the appropriate island in the database. The search terminates after a fixed number of Neval evaluations.
Algorithm Database. The algorithm database preserves a population of diverse algorithms obtained in the search phase. Inspired by prior works such as FunSearch TRomera-Parede et al., 2024) and AlphaEvolve (Novikov al., 2025). our database adopts a multiple-island- promote diversity. As shown in Figure B(a), the database consists of fixed size of Nisi islands, each of which groups algorithms with similar problem-solving behaviors as measured by our BehaveSim. Within each island, algorithms are further grouped into several clusters, each containing algorithms with identical fitness scores.
population to
For database initialization, instead of cloning single seed algorithm to each island (as in FunSearch). we first sample Nini 100 algorithms. We then perform clustering based on their BehaveSim similarity and register these initial algorithms into the Nia islands.
Algorithm Selection. The algorithm selection phase selects two different algorithms from the algorithm database as few-shot examples. We propose two selection strategies, S1 and S2, specifically:
Inter-island Selection (S1): To ensure the communication of two distinct islands, S1 strategy chooses two distinct islands in the algorithm database. We subsequently obtain an algorithm from each of them.
Intra-island Selection (S2): As adopted in FunSearch, we randomly choose one island and select two distinct algorithms within it.
17

---

ceciinance-disismllan mechanism during search to Published as a conference paper at ICLR 2026
To balance the utility of two strategies S1 and S2, we hyperparameter Pst. which determines the probability of adopting the S1 strategy in the selection phase.
As shown in Figure 8 (b), the selection of an algorithm from given island is two-step process. First, we select a cluster within the island, giving preference to clusters with higher fitness scores. Second, within the chosen cluster, we select algorithm that prioritizes shorter implementations (e.g., fewer lines of code) to promote conciseness and simplicity.
Sampling and Evaluation. As depicted Figure g once two algorithms are selected, they formatted into a prompt for the LLM. Following two-shot prompting approach inspired by FunSearch, we sort the examples by their fitness scores in ascending order. The prompt demonstrates that the second algorithm performs better than the first, and then instructs the LLM to generate potentially better algorithm. We perform two queries to the LLM for each prompt to obtain two new candidate algorithms.
Algorithms sampled from the LLM are sent to sandbox (implemented as a separate process isolated from the main process) for secure evaluation, in which record fitness score representing its performance of solving the specific problem, and a trajectory t indicating its problem-solving behavior. We discard infeasible algorithms that either raise errors during evaluation or exceed the timeout limit.
Register Algorithms into Algorithm Database. After evaluation, feasible algorithms are registered in the database. A key distinction from previous methods is that do not simply register the new algorithm back to its source island. Instead, we use BehaveSim to place it on the most similar island in the database. Given a newly evaluated algorithm with trajectory and a database of Nia islands, where fj is the trajectory of the j-th algorithm in island I; the target island is determined by:
surgetisslnd arg max
BehaveSim(t, tj)
xE[1,Nul
4
This equation calculates the average BehaveSim between the new algorithm's trajectory and all trajectories within each island and selects the island with the highest average similarity. The new algorithm is then placed in the appropriate cluster within the target island, or a new cluster is initialized none exists for its fitness score.
Restarting Islands. To prevent the search from getting stuck in local optima and manage the database size, we periodically restart a portion of the islands. Every two hours, we identify the Naa2 islands with the lowest-scoring best algorithms. We discard all algorithms within these islands and re-initialize them by randomly importing the best algorithms from the surviving Nia/2 2 islands. This mechanism ensures continuous influx of new ideas and prevents the database from becoming over-bloated with suboptimal solutions.
E.2 IMPLEMENTATION DETAILS OF GOH+BEHAVESIN
For the integration of EoHl-BehaveSim, we propose to augment primary performance objectives with BehaveSim to the best algorithm found (smaller is better) as another objective during the search. The integration is inspired by the success of MEoH (Yao et al. (025). which models the AAD problem as multi-objective optimization and employs enhance both diversity and search efficiency.
Parent Selection. As shown in Algorithm]] I the dominance matrix D and dissimilarity matrix S first calculated based on the dominance relationships and BehaveSim, and S' is calculated by the element-wise multiplication of S and D. Then, the cominance-dissimille by the column-wise sum of S', based on which we calculate æ, which indicates the probabilities of sampling each candidate algorithm.
vector is calculated
Population Management. As shown Algorithm algorithms in the old population are sorted based on their Dominance-dissimilarig and the top-N algorithms survive.
18

---

Published as a conference paper at ICLR 2026
Algorithm 1: Parent Selection
Algorithm 2: Population Management
Input: Population P: Population size N; Parent selection size d
Input: Population P; Population size N
Output: New population P
N" size(P)
Output: Selected parents P parent
S f- N' x N' matrix filled with zeros
D an N' x N' matrix filled with zeros
+ an N N matrix filled with zeros
for <<< to do
D - an N X N matrix filled with zeros
for j < 1 to N do
for i < 1 to do
if * then S(i.j] <
for j <- to N do i + then
BehaveSim Pf Pb])
S i + BehaveSim P P())
if Pl p then D
7
if P < Pi then
Diijj
10
S' + S
V + ColumrwiseSum((5"
S' S
k DesseningSortedlnddxxss(t)))))))
10 f ColumnwiseSum(5*)
13 P <
11 t Softmax(v)
for i + to N do
P Pprrent 4- Sample P,
P' - P PPUP]]
13 return parent
16 return P
F EXPERIMENT DETAILS AND ANALYSIS
F.1 SETTINGS FOR AAD TASKS AND METHODS
Admissible Set Problem (ASP). ASP aims to maximize the size of the set while fulfilling the
criteria below: (1) The elements of the set are vectors belonging to {0, 1, 1, (2) Each vector has the same number of non-zero elements but unigue support. (3) For any three distinct vectors, there is a coordinate in which their three respective values are {0, 1, 1, {0, 1,2]. 3. {0, 1, 2]. Following prior 15 and 10 in this work.
works Gkomera-Paredee et al. 2024), we
Partial Solutions in ASP. The objective is to design priority function that scores candidate vectors. At each step, the highest-scoring vector is added to the set, forming partial solution. The distance between two partial solutions defined as the edit distance between their current sets.
Traveling Salesman Problem (TSP). TSP aims to find a route that minimizes the total distance traveled by salesman who must visit each city exactly once before returning to the starting point. We investigate the constructive heuristic design for TSP (Matai et al. 2010). Specifically, we adopt an iteratively constructive framework that starts from single node, iteratively selects the next node until all nodes have been selected, and then returns to the start node. The task is to design a heuristic for choosing the next node to minimize the route length. We generate five TSP instances, each comprising 50 cities, for training.
Partial Solutions in TSP. At each step, the algorithm selects the next city, yielding a progressively constructed route. partial solution is an ordered list of visited cities. The distance between two solutions is the edit distance between their routes, with similarity aggregated across five instances.
Settings for LLM-AAD Methods. For FunSearch, use 10 islands for the database. Each prompt presents two reference algorithms, and we sample two algorithms per prompt. For EoH, the population size is set to 20. To ensure fairness, EoH is terminated based on the total evaluation budget instead the maximum number of generations. All methods are evaluated under identical evaluation budgets and stopping criteria for fair comparison.
19

---

Published as a conference paper at ICLR 2026
0.4
0.325
0.3
0.275
FunSearch island
PunSearch sland Disance
0.200
PunSearch Inter island Distance
0.1
PunnSearhh Inter island Distance
Curs litto island Disunce
Intro island Distance
Ours inter idlan Distane
Oter inter island Distance
Databmec Chockponnt
Databuae Chockpoins
(a) ASP
(b) TSP
Figure 9: Comparison of intra- and inter-island distances for FunSearch and our method. Each box shows the distribution of intra-island distances across 10 islands each checkpoint, while the curves track the average inter-island distance. Lower distances indicate similar problem-solving behaviors; higher distances indicate more diverse behaviors.
F.2 ANALYSIS OF DIVERSITY
Motivations. core challenge in the search process is to balance exploration across diverse solution spaces with exploitation within promising regions. We hypothesize that method, with optimized algorithm database, can achieve more effective balance than FunSearch. To verify this, we analyze the behavioral diversity of algorithms within and across islands. We compare method against FunSearch under an identical multi-island database setup, where the only difference is the population management strategy. At each database checkpoint, we sample N 50 algorithms from each of the 10 islands to serve as prototypes.
Intra-island distance. This metric quantifies the behavioral coherence within an island, reflecting local exploitation. For set of representative trajectories Pa (t.....,ty from an island k the distance is:
Dintra (Pb)
Σ ISKIJN
(1 J-BehhveSim(t,.t)) t,,ty E Ph
(I)
(2)
Inter-island distance. This metric measures the behavioral separation between islands, indicating global exploration. For two distinct islands and the distance is:
Dinter (Pk Pt)
Σ &EP tEP.
Σ (1 (tha
N2
Figure / visualizes these metrics, where boxes show the distribution of intra-island distances over all islands, and the curves track the average inter-island distance. We observe that:
Our method yields consistently lower intra-island distances (red boxes vs. green). This indicates that each island maintains more behaviorally coherent population, enabling focused exploitation.
Simultaneously, our method achieves higher inter-island distances (red curves vs. green). This demonstrates that the islands are more behaviorally distinct, promoting global explo- ration across diverse problem-solving strategies.
These results reveal that our method achieves superior balance between exploitation and exploration, crucial factor for robust discovery of novel, high-performing algorithms.
20

---

Published as a conference paper at ICLR 2026
0.18
TusSendrBellossi (Pa 0.0)
TunSerrhBBobweee (Pa.
0.16
Budseuud-BBlvveSiii (Pa Chasering
Ielatire Copp to Best-ooon Oytnum
FunSearch BeloveSim
0.14
Muneeurc-CodeBL...
0.08
0.06
0.04
500
2000
4000
6000
8000
10000
#Evallaeed Aim
Figure 10: Ablation study ASP. We evaluate the effect of inter-island selection probability Pa and the clustering operation in the initialization of the algorithm database.
F.3 ABLATION STUDY
To better understand the contribution of each component in framework, conduct ablation studies on the ASP task. We focus on two factors: (i) the probability pa of adopting the inter-island selection strategy (S1) versus the intra-island strategy (S2), and (ii) the effect of clustering within islands during algorithm database construction. The results are shown in Figure T0
Effect of Inter-Island Selection (Ps1). We vary Psi from 0 0.5. When Pst (purely intra-island selection), the method degenerates into variant similar to FunSearch, which restricts algorithm mix- ing within single island. Increasing Pst consistently improves performance, with the best results obtained Ps1 0.5. This demonstrates the importance of promoting cross-island communication: algorithms from different islands capture diverse problem-solving behaviors, and combining them enhances exploration of the search space.
Effect of Clustering. We also evaluate a variant without clustering during database initialization. As shown Figure [0, the absence of clustering significantly degrades performance, even when inter-island selection is enabled (ps1 = 0.5).
G TEMPLATE ALGORITHM
We initialize all methods except for EoH with identical template algorithms to ensure faimess. The template algorithms employed in the ASP and TSP tasks are shown in the following listings.
Code I: Template Algorithm for ASP
import math import numpy as np
def priority tuplelint, ...l, int, wi int) -> float:
=**
Returns the priority with which we want to add 'el' to the set. Args:
el: the unique vector has the same number of non-zero elenents.
length of the vector.
10
w number of con-zero elements.
11
return 0.
Code 2: Template Algorithm for TSP
import numpy np
21

---

Published as a conference paper at ICLR 2026
def select_next_node
current_ncde: int, dostination_nnde:: int,
unvisited_nodes: nd.ndar distance_matrix: nd.nday int:
Design novel algorithm to select the next node in each step. Args:
current_node: the current node.
13
destination_node ID of the destination node.
14
unvisited_nodes: Array of IDs of unvisited nodes.
distance distance_matrix: matrix: Distance matrix of nodes.
17
Return:
18
ID of the next node to visit.
next_nede undesited [0] next_node
Code 3: Template Algorithm for CPP
import numpy as np import math
def pack circles int) -> adarrayy
Pack n circles in unit square to maximize sum radii.
Args:
n: Number of circles to pack
Returns:
11
Numpy array of shape (n, 3) where each row is (x, y, radius) All values should be between 0 and 1
12
Circles must not overlap
15
Important: Set "ali" random seeds to 2025, including the packages such as scipy sub-packages) involving random seeds.
16
np. rppdem.mm.see seldd (2025)
if == 0:
19
return np. zeros (0, 3))
20
21
circles = np. zoros ( (n, 3))
23
Place first eirele in the center circles [0.5, 0.5, 0.5]
24
25
for i in range (1, n) 0 best_pos (0.5, 0.5)
28
29
30
Generate candidate positiens near existing circles and boundaries
candidates []
32
for j in cange(i): cx, cy, r circles[i]
33
for angle in np. np.1inspace (0, 2*np.pi, pi, 20): dx cos (angle)
36
dy np.sin (angle)
37
new_x ex (r + 0.01) + dx new_y oy + (r 0.01) + dy if 0 <= new <= and 0 <= new_y <= 1: candidates. (new_x, x, new_y))
38
22

---

Published as a conference paper at ICLR 2026
42
Add boundary points
43
for b in np, linsieee (0, 1, 20):
44
candidatos. extend (I (b, 0), (b, 1), (0, b), (1, b)l)
Evaluate each candidate
47
for (ex, cy) candidates:
48
# calculate maximum possible radius current_r min 1 cx, cy, 1 cy)
Check against existing circles if > 0:
52
53
existing circles(:i, :2]
radii circles(:i, 21
distances np.sqrt (eexssting:; 0] **2 (existing
[:, -] x*2)
56
min_dist np.min (distances - radii)
57
eurrent_r min (current_r, min_dist)
if current_r max_r:
60
max_r current_r best_pos - (cx, cy)
61
circieslil [best_pos(0] best pos [1], max r
64
65
return circles
H USE OF LARGE LANGUAGE MODELS
Large Language Models are employed in ways in this work. (i) We use LLMs to aid or polish writing in this paper. (ii) This work investigates LLM-based automated algorithm design. Therefore, LLMSs are used in the experiments.
23


Reviews:
[{"review_id": "nzUBbEmqrN", "reviewer": "Reviewer_SVva", "summary": "This paper proposes BehaveSim, a new metric for measuring algorithm similarity from a behavioral perspective rather than focusing on code structure or final outputs. The key idea is to represent an algorithm by its problem-solving trajectory, which records intermediate solutions generated step by step. The similarity between two algorithms is then defined as the resemblance between their trajectories, measured through DTW. The authors demonstrate that BehaveSim can distinguish algorithms that are behaviorally different but structurally similar, and that integrating this measure into LLM-AAD improves search performance by promoting behavioral diversity.", "questions": "[1] Could the authors elaborate on how BehaveSim would handle non-iterative algorithms or those with stochastic internal states (e.g., randomized search or Monte Carlo methods)? \n\n[2] Would the DTW-based comparison still be meaningful in these contexts, and if not, how might the behavioral similarity concept be adapted?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "[1] The paper addresses an important and overlooked gap by redefining algorithm similarity from the behavioral viewpoint. This is a clear and original contribution.\n\n[2] The distinction among code-level, behavior-level, and result-level similarity is well presented, and the taxonomy of four types of algorithm pairs is intuitive and pedagogically useful.\n\n[3] The idea of representing algorithm behavior through trajectories and computing DTW-based similarity is theoretically grounded and applicable to both continuous and discrete problems.", "weaknesses": "[1] BehaveSim is currently designed for iterative algorithms only. It cannot yet handle recursive, dynamic programming, or machine-learning-based algorithms. This limits its generality.\n\n[2] Several heuristic parameters, such as trajectory truncation, normalization constants, and distance scaling, are not systematically analyzed. Their influence on stability and reproducibility is unclear.\n\n[3] The benchmark for similarity evaluation mainly includes synthetic or classical algorithm examples. Broader testing on more diverse algorithmic domains would strengthen general claims.\n\n[4] The integration details of BehaveSim into the multi-island FunSearch framework are brief in the main text, making reproduction difficult without reading the appendix.\n\n[5] Some recent semantic or execution-trace-based code similarity methods are discussed only briefly without quantitative comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SbduD6fDcX", "reviewer": "Reviewer_rhhD", "summary": "This paper introduces BehaveSim, a novel similarity metric for measuring algorithm similarity from a behavioral perspective rather than code structure.The paper demonstrates that existing code similarity metrics (token-based, AST-based, embedding-based, execution-based) fail to capture algorithmic behavioral differences. To solve this problem, the paper proposes measuring similarity via problem-solving trajectories - sequences of intermediate solutions generated during algorithm execution, compared using Dynamic Time Warping (DTW).", "questions": "See weakness.", "limitations": "", "rating": 2, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The curated dataset with 4 algorithm pair types (varying code/behavior/result similarity combinations) provides rigorous validation. The results clearly demonstrate that BehaveSim achieves 1.0 similarity on Type-3 pairs (same behavior, different code) while existing code metrics fail, and correctly identifies behavioral differences where code metrics show high similarity.", "weaknesses": "1. The evaluation methodology does not use any AI models or AI-related methods. BehaveSim is essentially a general algorithm comparison technique based on execution traces and DTW, which appears equally applicable to comparing human-written code. The source of code (LLM-generated versus human-written) seems irrelevant to the core methodology, raising questions about whether this is fundamentally a software engineering contribution rather than an AI/ML contribution suitable for ICLR.\n2. The method only applies to iterative algorithms such as sorting, search, and optimization, excluding ML algorithms and non-iterative approaches. This significantly limits the practical applicability and generalizability of the approach.\n3. The similarity dataset contains only two AAD tasks, and all code exampels are hand-crafted, well-known algorithms. The experimental scope is very limited. Besides, the paper does not demonstrate whether BehaveSim can distinguish novel, unseen LLM-generated algorithms, which is critical for the claimed contribution to \"algorithm discovery.\" This represents a significant gap between the evaluation (known algorithms) and the application (discovering novel algorithms).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "89TintRWuM", "reviewer": "Reviewer_KsNQ", "summary": "This paper introduces BehaveSim, a novel similarity metric designed to measure algorithm similarity from a behavioral perspective, rather than code-level or output-level similarity. The authors argue that existing metrics (token-, AST-, embedding-, or execution-based) fail to capture the problem-solving behavior of algorithms—especially in the context of LLM-based Automated Algorithm Design (LLM-AAD), where generated code can differ syntactically yet implement equivalent ideas.\nBehaveSim represents each algorithm as a problem-solving trajectory, i.e., a sequence of intermediate solutions generated during execution. The similarity between two algorithms is then defined as the resemblance between their trajectories, computed via Dynamic Time Warping (DTW). Experiments show that BehaveSim better differentiates between algorithms that have similar code but distinct behavior (e.g., BFS vs DFS, insertion sort vs bubble sort), and vice versa.\nThe paper further demonstrates two applications: (1) improving behavioral diversity in LLM-AAD search (enhancing FunSearch performance on ASP and TSP tasks), and (2) clustering algorithm behaviors for interpretability and discovery.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Novel Perspective:\nThe paper identifies a clear conceptual gap between code similarity and algorithmic behavior similarity, proposing an elegant behavioral abstraction based on execution trajectories. This reframing is insightful and well-motivated in the context of LLM-generated algorithms.\n\nConcrete Implementation (BehaveSim):\nThe definition of behavioral trajectories and the use of DTW distance provide a simple yet powerful operationalization of behavioral similarity. The methodology is well formalized, reproducible, and extensible.\n\nComprehensive Benchmark:\nThe authors curate a systematic dataset with four categories (Type-1 to Type-4) decoupling code-, behavior-, and result-level similarities, offering a rigorous evaluation against existing metrics such as CodeBLEU, CodeBERTScore, and execution-based scores.\n\nStrong Empirical Results:\nBehaveSim achieves intuitive and consistent performance across all dataset types (e.g., correctly scoring 1.0 for Type-3 pairs with equivalent behaviors). Integration with FunSearch also improves both convergence and final performance on ASP and TSP benchmarks, validating practical relevance.\n\nInterpretability and Analysis:\nThe algorithm clustering experiment (Fig. 5) compellingly illustrates how BehaveSim distinguishes semantically similar but syntactically different implementations, supporting new interpretability avenues in algorithm discovery.", "weaknesses": "Scope Limitation:\nBehaveSim applies only to iterative algorithms producing discrete trajectories. Many LLM-generated algorithms, including stochastic, differentiable, or recursive paradigms, are excluded. This significantly restricts generality.\n\nMetric Design Choices:\nThe use of DTW on normalized edit or Euclidean distances is heuristic; there’s limited justification for why DTW best captures “behavioral similarity.” Ablation on alternative measures (ERP, cosine, etc.) is included but not theoretically grounded.\n\nDependence on Trajectory Definition:\nDefining what constitutes a “partial solution” may require manual instrumentation of each algorithm, limiting scalability and automation for arbitrary code. This makes BehaveSim less plug-and-play for general LLM evaluation pipelines.\n\nComparative Baseline Gaps:\nAlthough the work compares to standard code metrics, it lacks comparison to semantic or dynamic program analysis methods (e.g., symbolic execution traces, graph-based semantic embeddings). These could offer a fairer baseline.\n\nModerate Empirical Gains:\nWhile improvements in AAD tasks (Table 3) are consistent, they are modest (~10–15% relative gap reduction) and limited to small benchmark scales. Broader tests on complex algorithmic synthesis domains would strengthen impact.\n\nOverclaiming “Novelty” for AAD:\nIntegrating diversity measures into LLM-AAD (FunSearch + BehaveSim) is conceptually incremental to existing multi-island or MAP-Elite-based diversity frameworks. The true novelty lies more in behavioral similarity than in AAD improvement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WVamuCthBI", "reviewer": "Reviewer_TTPL", "summary": "This paper notes existing code similarity metrics (e.g., CodeBLEU) fail to reflect algorithm similarity in LLM-AAD. It proposes BehaveSim, which measures similarity via algorithms’ problem-solving trajectories. Using DTW to align trajectories and normalize distances, BehaveSim outperforms traditional metrics in validation. It enhances LLM-AAD performance and aids algorithm analysis, with limitations on non-iterative algorithms.", "questions": "1. Regarding the differences in trajectory lengths among different iterative algorithms, what methods are adopted in the paper to avoid biases in BehaveSim's similarity calculation? \n\n2. How is the hyperparameter p_s1 (probability of inter-island selection) tuned in the paper, and what empirical evidence supports that p_s1=0.5 is the optimal value for balancing exploration and exploitation?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The perspective of the paper is quite novel, as it calculates similarity based on the problem-solving trajectories of algorithms. The paper argues that traditional code similarity metrics fail to reflect true algorithm similarity, so it proposes BehaveSim, which quantifies behavioral similarity by analyzing sequences of intermediate solutions (trajectories) generated during algorithm execution.", "weaknesses": "The proposed behavioral similarity metric, is only designed for iterative algorithms (e.g., sorting, optimization algorithms) and cannot be directly applied to other types of algorithms like machine learning models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)