CORE: Collaborative Optimization with Reinforcement Learning and Evolutionary Algorithm for Floorplanning
Authors: Pengyi Li
Shixiong Kai
Jianye HAO
Ruizhe Zhong
Hongyao Tang
Zhentao Tang
Mingxuan Yuan
Junchi Yan
Venue: NeurIPS 2025 poster ()
Keywords: Floorplanning
Evolutionary Algorithms
Reinforcement Learning
Primary area: reinforcement_learning

Abstract: Floorplanning is the initial step in the physical design process of Electronic Design Automation (EDA), directly influencing subsequent placement, routing, and final power of the chip.
However, the solution space in floorplanning is vast, and current algorithms often struggle to explore it sufficiently, making them prone to getting trapped in local optima. To achieve efficient floorplanning, we propose **CORE**, a general and effective solution optimization framework that synergizes Evolutionary Algorithms (EAs) and Reinforcement Learning (RL) for high-quality layout search and optimization.
Specifically, we propose the Clustering-based Diversified Evolutionary Search that directly perturbs layouts and evolves them based on novelty and performance. Additionally, we model the floorplanning problem as a sequential decision problem with B*-Tree representation and employ RL for efficient learning. 
To efficiently coordinate EAs and RL, we propose the reinforcement-driven mechanism and evolution-guided mechanism. 
The former accelerates population evolution through RL, while the latter guides RL learning through EAs. The experimental results on the MCNC and GSRC benchmarks demonstrate that CORE outperforms other strong baselines in terms of wirelength and area utilization metrics, achieving a 12.9\% improvement in wirelength. CORE represents the first evolutionary reinforcement learning (ERL) algorithm for floorplanning, surpassing existing RL-based methods. The code is available at https://github.com/yeshenpy/CORE.

CORE: Collaborative Optimization with Reinforcement Learning and Evolutionary Algorithm for Floorplanning
Pengyi Li Shixiong Kai?, Jianye Hao' Ruizhe Zhong Hongyao Tang',
Zhentao Tang", Mingxuan Yuan", Junchi Yan College of Intelligence and Computing, Tianjin University
Huawei Noah's 's Ark Lab 3 Dept. of Computer Science and Engineering & MoE Key Lab AI, Shanghai Jiao Tong University
Abstract
Floorplanning the initial step the physical design process of Electronic De- sign Automation (EDA), directly influencing subsequent placement, routing, and final power of the chip. However, the solution space in floorplanning is vast, and current algorithms often struggle to explore it sufficiently, making them prone getting trapped in local optima. To achieve efficient floorplanning, we propose CORE, general and effective solution optimization framework that synergizes Evolutionary Algorithms (EAs) and Reinforcement Learning (RL) for high-quality layout search and optimization. Specifically, we propose the Clustering- Diversified Evolutionary Search that directly perturbs layouts and evolves them based on novelty and performance. Additionally, problem as sequential decision problem with B*-Tree representation and employ RL for efficient learning. To efficiently coordinate EAS and RL, we propose the reinforcement driven mechanism and poolution-guideed mechanism. The former accelerates population evolution through RL, while the latter guides RL learning through EAs. The experimental results on the MCNC and GSRC benchmarks demonstrate that CORE outperforms other strong baselines in terms of wire- length and area utilization metrics, achieving a 12.9% improvement in wirelength. CORE represents the first evolutionary reinforcement learning (ERL) algorithm for floorplanning, surpassing existing RL-based methods. The code is available at https: /gi thub. com/ wwwo///eeeepy/////// CORE..
model the floorplanning
1 Introduction
Electronic Design Automation (EDA) [1, 2] encompasses series of steps from the initial phase of electronic system design final manufacturing and testing. These steps include design, verifi- cation, synthesis, physical design (ftoorplanning. macro placement and routing), packaging, and manufacturing. Previous works have made significant progress macro placement [3-11], whereas research on floorplanning receives less attention. Unlike macro placement, floorplanning requires representations that support the construction of compact layouts, which makes macro place- ment algorithms ineffective for solving floorplanning problems. This is empirically validated in Section 4.2. Floorplanning, as the first step physical design, primarily involves determining the locations and shapes of functional modules, providing foundation for subsequent placement, clock tree synthesis (CTS), and routing. Floorplanning a crucial step in EDA, as directly impacts
"Contact me at lipengyi @ wtw..eeu..unnnnn
'Conesponding author: Jianye Hao (jianye.haod) @ tju.edu.en)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

subsequent placement, CTS, routing, and overall power consumption. However, the floorplanning problem is NP-hard [12], with vast solution space, making very challenging to construct a high-quality layout.
Methods for solving the floorplanning problem can generally be divided into three categories: SA-based algorithms, analytical methods, and reinforcement learning (RL) algorithms, SA-based methods [13, 14] model the problem based on specific floorplan representations, which encode the floorplan layout into a structure that is convenient for optimization, and rely on Simulated Annealing (SA) to search solutions [15, 16]. SA-based methods are reliable in small cases but easily converge to suboptimal solutions in large cases with vast search spaces. Analytical methods [17-21] require modeling the optimization objective as a differentiable function, which is then optimized through gradient descent. Analytical methods face challenges in constructing differentiable functions and are prone to getting stuck suboptimal solutions due to the non-convexity of these functions [22]. For the third category, RI. [23] is class of learning algorithms that has achieved significant progress in various practical tasks. Using RL for floorplanning is an emerging research direction [24-28].
RL-based methods model the problem as Markov Decision Process (MDP) [29] with specific floorplan representations, tackling the task by placing modules one by one and learning the solution through trial and error. RL can leverage fine-grained task feedback, which offers local guidance ability and better sample efficiency. However, RL heavily depends on the MDP modeling, which significantly affects solution quality. Additionally, the limited global exploration capability of RL can also lead to suboptimal solutions.
Previous work often suffers from insufficient exploration of the design space, leading to suboptimal results. We identify two main factors that impact the solution quality: (1) Floorplan Representation: Directly placing modules in continuous space is challenging. Floorplan Representation is used to create a more compact and favorable solution space. A proper floorplan representation should ease the solution-finding process with the purpose of avoiding overlap and meeting optimization objectives. The floorplan representation significantly impacts the feasibility and complexity of floorplan designs, further influences both the execution time and the quality of the results [30]. (2) Limited Exploration Capability: Previous methods often suffer from insufficient exploration capability and are prone to getting stuck in suboptimal solutions.
To address the above challenges, we propose hybrid framework, named Collaborative Optimization with RL and Evolutionary Algorithms (CORE). CORE employs B*eTree representation [31], which offers a favorable solution space, linear transformation time, and compact placement. B*-Tree representation has been proven to outperform other representation methods. For efficient exploration, CORE integrates Evolutionary Algorithms (EAs) and RL. EAs [32] are a class of gradient-free heuristic optimization algorithms that mimic the process of biological evolution by maintaining population. EAs improve the population through iterative variation. Compared to RL, EAs have better global optimization and search capabilities. CORE leverages the respective strengths of EAs and RL to achieve efficient layout optimization. It is worth noting that the integration of EAs and RL, known Evolutionary Reinforcement Learning (ERL) [33-36], has become vast research area. The most related prior works are those that use RL-assisted EAS to solve problems such as the Traveling Salesman Problem [36]. Yet, collaborative framework for tackling more complex combinatorial optimization problems remains largely unexplored. In contrast, CORE is the first attempt within the ERL domain to design novel collaborative optimization framework for complex floorplanning tasks.
CORE consists of three key processes: (1) Evolutionary Process, (2) Reinforcement Process, and (3) Collaboration Process. In the evolutionary process, we propose Clustering Diversified Evolutionary Search (CDES) based on B Tree representation. CDES maintains an initial population of layouts and clusters them into multiple subpopulations based on layout features. Each individual is ranked based weighted combination of its quality score and novelty score. Elite layouts are selected from each subpopulation and a new layout population is generated by perturbing these elites. Through the above iterations, CDES facilitates more comprehensive exploration of the solution space. In the reinforcement process, we employ B*-Tree representation to model the floorplanning problem as a Markov Decision Process (MDP). We then adopt Proximal Policy Optimization (PPO) [37] to learn a policy network that makes step-by-step layout decisions. This process allows us to utilize finer-grained information for policy learning efficiently. To efficiently collaborate EAs and RL, we propose two collaboration mechanisms: (1) deinforcemeen-driveer Mechanism accelerates population evolution by injecting complete layout solutions obtained from RL into the population. (2) Evolution-

---

guided Mechanism maintains an archive of the best layouts discovered by EAs. The layouts that are better than RL's current performance are leveraged guide the RL policy by imitation learning. Through these mechanisms, EAs and RI. can achieve efficient collaboration. In the experiment, we evaluate our framework on two widely used benchmarks, MCNC and GSRC. Our experimental results demonstrate that CORE outperforms other strong baselines in terms of wirelength and area utilization metrics. Specifically, CORE achieves an average improvement of 12.9% in wirelength for each case compared to the best wirelengths achieved by other algorithms
We summarize the contributions as follows: (1) Firstly, we propose an EA-RL hybrid framework CORE and design collaboration mechanisms for efficient optimization. (2) Secondly, we propose the Clustering- based Diversified Evolutionary Search to maintain population diversity, thereby enhancing global optimization efficiency. (3) Thirdly. we model the floorplanning problem as sequential decision problem using the B*-Tree representation. Then we employ PPO to learn policy for floorplanning, aiming to enhance local guidance capabilities and improve optimization efficiency. To the best of our knowledge, we are the first use RL with B*-Tree representation to solve the floorplanning problem. (4) Finally, experimental results demonstrate that CORE significantly outperforms other strong baselines, achieving 12.9% improvement wirelength.
2 Background
2.1
Floorplanning Problem
The core of the floorplanning problem lies in how effectively arrange various functional modules to meet specific design objectives and constraints. The floorplanning problem involves the following three fundamental elements:
{m,, Modules are the basic units in circuit design. denoted as M }. Each module m, is characterized by rectangular shape with di- mensions length 4 and width w;, centered at coordinates (I; yi).
Modules:
Ports: Ports are utilized for input and output signal transmission of current die, denoted as Each port p: is located a specifie position (I;,
P (pI, (PI,P2,.... P2....,.PPp
Netlist: The netlist É describes the connectivity within modules and ports. Each connection E E E represents a net that encompasses a set of modules and ports (64)
{ [m] m2
(e4) (es). .Pi .P ,p2 .p)))
A layout is formed after all the modules are placed. The optimization objectives are minimizing area, Half Perimeter Wire Length (HPWL), other criteria. In these objectives, HPWL serves widely used indicator to assess the quality of a layout and is defined as follows:
HPWI- Σ 6.EE
{((Ee)
[mxxx [2a] - min (225) bEe:
+ max (ub) -min (wb) }
(1)
bee.
2.2 T*-Tree Representation
B*-Tree [31] is an ordered binary tree that represents compact placement. The pro- cess of converting a - to a floorplan illustrated in Figure 1. Each node n; in the B represents a module m,. We tra- verse the B*-Tree using Depth-First Search and determine the topological positions of each module according to the following rules: The root node of the B corresponds to the bottom-left corner, i.e., (0.0). The left child n, of node n, represents module my as nearest right adjacent module to m, i.e.,
5
4
3
3
6
Figure 1: Treeebbased floorplan representation.
z + wy. The right child of node ni represents the first module my above m, with the same x-coordinate, i.e., Jt = T;- To compute the y-cooniinate, the B*-Tree utilizes contour structure [38] doubly linked list of modules that records the contour line in the current compaction direction.
3

---

We choose the B -reTree representation for three main reasons:
Efficient evaluation and update. Since the MDP depends the chosen representation, the computational cost of each state-action transition must be carefully considered. Unlike other representations, the B*-Tree avoids building constraint graphs computing shortest paths, greatly reducing overhead. It also supports incremental evaluation. making it well-suited for sequential
Compact search space. A - encodes fewer permutations compared to most alternative representations, resulting in a more compact search space that facilitates efficient exploration and learning.
Modifiable open-source infrastructure. Effective heuristics such as SA have already been developed for B*-Tree, and robust open-source implementations are readily available, making it easier to integrate and extend.
2.3 Evolutionary Algorithms
Evolutionary Algorithms (EA) [39, 40] are a class of black-box optimization methods. Below is a simplified optimization process: EA maintains population of solutions IP = {s1. $2. s first evaluate all solutions to get where (*x) needs to be defined specifically according to the optimization objective. The solution with higher fitness is more likely be selected parents to produce the next generation with variation operators. The definition the variation operator relies on expert experience tailored specifically the problem. EAs iteratively optimize through the above process until feasible solution is found.
in which evolution is iteratively performed. In each iteration,
the fitness {f(*1), ), (*2)....,,
2.4 Reinforcement Learning
Reinforcement Learning (RL. [23] is a category of learning algorithms primarily employed for solving sequential decision problems. RL involves modeling the problem as a Markov decision process [29], defined by tuple AS A. P, R, y, T") At each step t, the agent uses a policy to select an action a, ~ π (-1ss) E A according to the state 8; E S and the environment transits to the next state 8t+11 according to transition function P(-|st, az) and the agent receives reward ra R(se, a4). The return is defined as the discounted cumulative reward, denoted by R. Σ "r'"r, where [0, 1) is the discount factor and T is the maximum episode horizon. The goal of RI is to learn an optimal policy x* that maximizes the expected return. PPO is representative RI. algorithm with convergence guarantees, which achieves significant success in various application scenarios. The key feature of PPO is the clipped objective function, which helps to prevent large policy updates that may destabilize training. This clipping mechanism ensures that the policy does not deviate too far from its previous version, thereby improving stability and convergence.
2.5 Evolutionary Reinforcement Learning
Evolutionary Reinforcement Learning (ERL) refers to family of hybrid algorithms that integrate EAs with RL. It constitutes broad research area encompassing multiple directions [36], including EA-assisted RL, RL-assisted EA, and synergistic optimization of EA and RL. In recent years, several ERL algorithms focus on integrating EAS and RL achieve more efficient policy search. Representative methods such as ERL [41], CERL [42], PDERL [43], CEM-RL [44], ERL-Re² [45], VEB-RL [46], and EvoRainbow [47] show strong performance in various tasks involving games, locomotion, and manipulation. In addition, some works apply ERL-inspired ideas for reward function design, including Evo-Reward [48], Eureka [49], and R* [50]. Moreover, some works explore areas such multi-agent systems [51, 52] and game testing [53]. In contrast, CORE focuses on complex combinatorial optimization problems with constraints. To the best of our knowledge, CORE is the first ERL algorithm successfully applied to solving complex floorplanning problems.
3 Collaborative Optimization with RL and EA
This section introduces our framework Collaborative Optimization with RL and EA (CORE). We begin by introducing the overall framework of CORE, followed by the details of its three key

---

Evolutionary Process
Initial population
Clustering based layout features
Evaluation Each clustered class
Elite Layouts
New Population
Layout
Layout 1
Layout 2
N
Layout 2
Loyout
Pz
N
Variation
Layout
Layout
Loyout
Layouts
3
Learning Process
Convert
Optimize
Policy Gradients
Imitation Learning
Experiences
Value Function
Policy
Decision
Leyout
sequences
Select Better Layouts than RL's
Complete Layout
Inserted Module
Target Placement
SM
Sg
Policy
12
RL Layout
Left/Right
State
C
Action Selection
Interaction Process
Collaboration Process
Figure 2: The framework illustration of CORE.
Clustering based Diversified Evolutionary Search, RL, and EA-RL collaboration mechanisms.
components:
3.1 Overall Framework of CORE
CORE is novel ERL framework based on B*-Tree representation. The principal idea to achieve efficient collaboration between EAs and RL for optimization, leveraging the strong search and global optimization capabilities of EAs along with the powerful learning abilities of RL. This synergy allows for efficient search and optimization within the vast solution space of floorplanning.
The framework illustration shown in Figure 2, which comprises three parts: Evolutionary Process (Circle 1), Reinforcement Process (Circles 2 and 3), and Collaboration Process (Circle 4).
The Evolutionary Process performs variation on the layouts within a population, aiming to search for layouts with higher quality scores and greater diversity.
The Reinforcement Process requires the RL policy to interact with the environment and place modules step by step to construct a complete layout solution (Circle 2). Through these interactions, experiences are gathered to optimize the value function and policy (Circle 3),
The collaboration process includes leveraging RL to accelerate population evolution and using EAs guide RL policy learning.
In the following, we provide comprehensive introductions to the three key processes.
3.2 Clustering-based Diversified Evolutionary Search
Although EAs have strong search and global optimization capabilities due to the redundancy and randomness their populations, relying solely on these properties can reduce diversity and make population get stuck in suboptimal solutions. Improving population diversity effectively solve this problem, as demonstrated by numerous Quality Diversity methods [54-57]. Motivated by this, propose the Clustering based Diversified Evolutionary Search (CDES). Specifically, illustrated in Figure 2, CDES comprises five processes:
Population Initialization: Initialize the population P -3)..... 8x) with size N where each individual s; represents complete layout solution, all modules are planned.
5

---

Concatenate
SoftMax
Graph Network
Embedding
MLP
Zy
Mark
Sp
Transformer
MLP
Quagee
ag
Mean Pooling
5g
Mask
Graph Embedding Ze
Figure 3: The policy architecture of PPO.
2. Population Clustering: We construct the layout features based on the positions of all modules, i.c. F(s,) {Im, 3m.............................. 2mmm Mm J. and then utilize these features to cluster the population into C' clusters using K-Means [58].
Subpopulation Evaluation: Within each cluster, we evaluate the quality and novelty score of each layout. The novelty score (s;)) is defined the negative average of the two highest cosine similarities between the current layout feature and the layout features of other individuals within the same cluster. Considering the top two similarities helps avoid situations where two layouts have the highest similarity to each other, making it challenging rank them. Therefore, we consider the top score P(s;) constructed based on the optimization objectives, such as the negative of HPWL to minimize the wirelength.
enhance discrimination. The quality
4. Elite Selection: We combine the quality and novelty score calculate the final fitness score. defined as follows:
f(x)) aP(s)) + (s:)
where used weight P(s:) and (s.).. A larger f(s:) indicates better performance achieved by the layout and higher novelty simultancously.
5. New Population Construction: We mutate the elite layouts within each cluster generate offspring, forming new population. The variation operation follows the principles of the B+-Tree-based SA [31] and includes three operators: ! Rotation Operator: randomly select a module and rotate it 90 degrees. 2 Exchange Operator: randomly seleet two modules and swap them, 3 Deletion and Insertion Operator: remove module that does not have both left and right children simultaneously, and insert the module into the ww-Tree as the left or right child of the target module.
Step 1 is executed only once at the beginning, while steps 2-4 are executed in loop until the algorithm terminates. With CDES, we can maintain diverse layouts within the population, preserving population diversity and thus avoiding convergence to suboptimal solutions.
3.3 Tree-ree- Reinforcement Learning
As heuristic algorithms, EAs rely on random exploration for layout optimization and often fail to leverage the experiences collected during the optimization process. These characteristics lead to inefficiency. In contrast, RL, as a learning algorithm, can utilize finer-grained information, approximate value functions, and optimize policies. Compared to EAs, RL offers more targeted optimization directions and stronger local optimization capabilities.
To fully exploit the strengths of RI, we model the floorplanning problem as an MDP based on the B*-Tree representation. The problem can be viewed as constructing a binary tree from scratch, wherein different modules are sequentially inserted into the tree. Specifically, we design the following five elements within the MDP (S, A, P, T):
States S: The state comprises information about all modules, ports, and netlist, i.e., s {SM, }.). Each module or port includes coordinates of the starting and ending points, length, width, whether has been placed, whether has left child or right child in the tree, and the layout order (the order of adjustments in the B*-Tree). For ports, the length, width, presence of left/right child, and layout order are all set to O. Netlist represents the connections between modules and ports.

---

The action space comprises three types of actions: choosing the module to be inserted, selecting the target module, and determining whether the inserted module should be placed as the left or right child of the target module, along with the option to rotate the inserted module. Hence, the size of the action space is 4n,m 2.
Actions A:
State Transition P: We adjust the layout according to the B*-Tree to avoid overlaps and obtain the adjusted layout.
Reward R. When we optimize the HPWI,, scale it with 10-1 and use the negative change in HPWL relative to the previous step as the reward, i.e., r; -10 AHHWL. If none of the modules involved in a net e; are planned, the wirelength for that net is not calculated. When we optimize both HPWL and area utilization U E (0, 1]. we define the area ratio as the area of the placed modules divided by the bounding box area, and use its change as the area reward. The final reward combines both terms: r; - -0 AAHPW + AU. Maximum episode horizon T: When all modules are planned, the episode is done, so TT equals 7mm-
After modeling the MDP, we propose a policy architecture to process the complex state information. As depicted in Figure 3, we adapt graph neural network [59] as the initial structure to process the current modules state sM, the port states 8p, and netlist state sg. Then we get the module embedding and port embedding by employing the Transformer [60] to process the output of the graph neural network. We use the mean pooling to obtain the current layout representation. Subsequently, we employ hierarchical conditional dependencies for action selection. Firstly. we concatenate representations of all modules with the current layout representation input to MLP. Then, select the inserted module by applying the action mask to remove actions corresponding to selecting modules on the panel. Secondly, we concatenate the inserted module representation with representations of all modules and the layout representation as input. Following the same procedure, we obtain the target module. Finally, we concatenate the inserted module representation, the target module representation, and the layout representation, and pass them through MLP and mask operator to obtain nour-dimensional input corresponding to the 4-dimensional action space, i.e., whether to rotate and whether to be the left or right child.
Based the MDP modeling and the architecture design described above. employ the PPO algorithm. The choice of PPO over other algorithms is motivated by several factors: 1) its guarantee of monotonic improvement and convergence, its superior parallelization properties, and 3) the elimination of the need to maintain a state-action value function for the excessively large joint action space. For example, in large case with 300 modules, the joint action space amounts to 360,000, making it challenging to approximate the values accurately. In addition to maintaining the policy as described above, PPO also requires the maintenance of state value function. We use separate representation processing module with the same architecture as the policy. The obtained layout representation is then passed through MLP to predict the value of the current state. We update the parameters of the policy network with the following loss:
fcip(() E min TTo(aelse) AAAB GAB (t), clip Tooad (arlst)
(To((ee)s (8e) Tooad (Aelst)' e, +e AAEE ot)
where Ostd is the parameter of the policy before the update, is hyperparameter used constrain the update step size, and AGAE (st: or) is the advantage, which is calculated with Generalized Advantage Estimation (GAE) [61]. The state value network optimized using the following loss:
(v(0) = E (Vo(s)) - vitarget 27
where Ve Va (++) + AGAE (st, ot). To improve the exploration capability, we introduce an entropy loss to encourage the policy for more diverse exploration, defined follows:
(x(T)
Σ (aas: log (alss)
(5)
Based on the definition provided, we can learn a policy with PPO for the floorplanning problem. is worth noting that the action space for the initial module differs from that of subsequent modules because there are no selectable target modules. Therefore, we randomly sample the initial module, while subsequent modules are selected through RL.
7

---

3.4 EA-RL Collaboration Mechanisms
In the previous two subsections, we ex- plore solving the floorplanning problem us- ing EAs and RL independently. However, these two processes are decoupled, and the performance of the final solution equals the best performance obtained by each method separately. To break through this perfor- mance bottleneck, we propose EA-RL col- laboration mechanisms, which comprise I) seaaforrcement-Driven Mechanism and 2) Evolutionary-Guiie Guided Mechanism, as depicted in Figure 4. The Reinforcement Driveen Mech- anism aims to leverage RL to accelerate the evolution of EAs. Specifically, we inject com- plete solutions constructed by RL into the pop- ulation of EAs. Subsequently, based on the defined fitness score, inferior layout solutions are removed to maintain a constant population size. If the solution discovered by RL achieves a higher score, will replace other old elites in the population, thereby guiding population exploration. Conversely, if RL fails to provide better layout solutions, these layouts are promptly discarded without hindering the population's 's evolution.
Reinforcement Driven
Evolutionary Guidance
Correction
Elite replaced
Guidance
New elites
Improvement Paths
EA Solutions
RL solution
Old elites
Better EA solution
Figure 4: The EA-RL collaboration mechanisms.
The Evolutionary-Guus Guided Mechanism is designed to utilize EAs to guide RL learning. For each module using different rotations the root node, store the best layout discovered by EAs into an archive LA Additionally, we also maintain the layouts constructed by RL with different root nodes. Then compare the layouts in L and the RL layout with the same root. If the EA layout is better, we convert into a decision sequence (i.e., state-action pairs) based on the B*-Tree placement rule and store it in DEA- Subsequently, incorporate Imitation Learning loss term into the PPO learning process, defined as follows:
L aa [- log (ac]s:))
(6)
Through this approach, we enable RL to employ imitation learning to adjust the learning direction when the layouts it learns inferior to those discovered by EAs. This allows RL to quickly grasp the superior layout and conduct more efficient local exploration around it. Thus, the final loss PPO in CORE can be defined as:
Cp = Lclip + WEnCE + WVaate + wn.Cu.
(7)
where Wints WValues and wn. are hyperparameters. The pseudocode is provided in Algorithm 1.
4 Experiments
4.1 Benchmarks & Baselines
Our experiments are conducted on two widely used floorplanning benchmarks: MCNC and GSRC. For the MCNC benchmark, we use the ami33 and ami49 designs. The GSRC benchmark includes six designs with module counts ranging from 10 to 300. The MCNC and GSRC benchmarks were designed based on the practical requirements of floorplanning.
industrial scenarios, floorplanning tasks typically involve fewer than 100 modules. In all cases, we reserve 10% of the area as whitespace and ensure a 1:1 aspect ratio for the boundary. We then project the ports to the new boundary. We consider two metrics for floorplanning: HPWL and Area Utilization (AU).
We compare with five competitive methods using different floorplan representations: Corblivar [14], SP-FP [13], B*-Tree SA [31], GoodFloorplan [27], and CBL-RL [28]. SP-FP, Corblivar, and _ Tree SA are open- spenesource. Both GoodFloorplan [27] and CBL-RL [28] are RL-based methods without open-source code. Based on the same setting, results from the literature are included in results tables. It worth noting that they not report area utilization, only HPWL. Therefore, we assume their area utilization to be 90%, which aligns with practical usage requirements. The convergence performance intuitively reflects the effectiveness the algorithm. Previous methods all provide the best convergence performance. To enable fair comparison, we train CORE and the

---

Table I: Convergence Performance Comparison Based on HPWL and Area Optimization. CORE achieves shorter HPWL compared to other strong baselines while maintaining comparable AU.
Corhlivar
SP-FP
B*-Tree SA
GoodFFoorplan
CBI-RI
CORE
Case
HPWL
AU
HPWL
AU
HPWL
AU
HPWL
AU
HPWL
AU
HPWL
AU
n10
43082
0.897
36514
0.916
36044
0.859
41019
0,900
35170
0.911
n30
124918
0.885
108835
111496
0.812
111793
0,900
99964
n50
164044
0.902
137516
0.912
140716
0.851
162600
0,900
124890
0.948
100
271747
0.881
222559
0.911
218976
0.741
309320
0.900
337284
0.900
191349
0.915
n200
535212
0.874
410438
0.912
388562
0.775
558330
0.900
351735
0.900
348956
0.917
n300
764890
0.862
569820
0.910
626997
0.700
690760
0.900
476765
0,900
469242
0.901
ami33
80298
0.830
64315
0.910
81629
0.869
87540
0.900
81767
0,900
52672
0.940
ami49
1649560
0.822
930836
0.911
1180549
0.788
1067590
0.900
1375114
0,900
656068
0,918
Table HPWL Comparison of CORE with macro placement methods.
Method
n10
n30
n50
n100
n200
n300
ami33
ami49
MaskPlace
54105
150706
201743
355918
666599
960487
126360
2410415
ChipFormer
54954
122035
165600
264180
500447
693501
88129
1377496
CORE
35170
99964
124890
191349
348956
469242
52672
656068
available open-source algorithm B*-Tree SA for the same number environment steps, running each algorithm five times and reporting the best convergence performance achieved. The experimental setting is consistent with prior literature. The average performance and error bar over five runs is provided in Appendix C (consistently outperforming the second-best baseline). We provide details on hyperparameter settings, network architecture, and training configurations in Appendix A. is important emphasize that floorplanning requires structural representations to achieve compact, gap-free placement, which makes macro placement algorithms inefficient for this task. We provide comparisons with the state-of-thee ahr macro placement algorithms Appendix
4.2 Performance Evaluation
We compare CORE with other basclines on 8 cases. The experiment results in Table show that CORE typically achieves comparable area utilization while maintaining significantly shorter HPWL. CORE achieves an average im- provement of 12.9% in wirelength compared to the best results obtained by other algorithms across all cases, with 22.6% improvement on the n300 case and a 29.5% improvement on the ami49 case. This demonstrates that CORE has superior capabilities in layout search and op- timization. Besides, we observe that compared to B*-Tree SA, which uses the same floorplan representation, CORE demonstrates significant advantages in both HPWL and area utilization. More- over, the performance gap increases with larger cases. This advantage primarily stems from CORE superior search and optimization capabilities, whereas TreTree SA is more prone to getting stuck in suboptimal solutions. When compared to two methods, GoodFloorplan and CBL-RL, CORE also shows better performance metrics. The improvement stems from both the favorable solution space enabled by the B*-Tree representation and the complementary synergy between EA provide the visualization results in Figure 5, demonstrating that
Figure 5: Visualization of the compact layout.
and RL in CORE. In addition, CORE can achieve compact layout solutions. We also conduct experiments focusing solely on HPWL optimization in Appendix C. The results demonstrate that CORE consistently outperforms other baselines, achieving superior wirelength metrics, further validating its efficiency. Further details can be found in the appendix.
also provide a comparison with macro placement methods. Unlike macro placement, the scale of floorplanning in real-world industry is typically under 100 blocks. Thus current floorplanning benchmarks are designed at this smaller scale. Floorplanning tasks generally rely on compact representations such B Treeree or CBL to minimize HPWL and ensuring gap-free placement.

---

In contrast, macro placement tasks often involve congestion constraints, requiring spacing between macros allow for standard cell placement. This fundamental difference means that compact floorplanning representations are not suitable for macro placement tasks. Likewise, due to the lack of compact structural modeling, macro placement algorithms tend to perform poorly on floorplanning. To support our claim, we compare with two SOTA macro placement algorithms, MaskPlace [4] and ChipFormer [5]. The results are shown in Table 2. CORE, designed for floorplanning, significantly outperforms both MaskPlace and ChipFormer.
4.3 Analysis of CORE
In this section, we delve into each component of CORE assess its significance. CORE com-
Table 3: Ablation study on CORE.
prises the evolutionary algorithm CDES, the RL algorithm PPO, and the collaboration mecha- nisms. We first compare the performance of CORE with CDES and PPO. To intuitively re- flect the algorithm's 's efficiency, we focus solely
Case
CDES
B*-Tree PPO
CORE
n10
35076
34604
34296
n30
104045
99561
98713
n50
126047
124683
123195
on minimizing HPWL in our comparisons. Experimental results in Table 3 show that CORE achieves shorter HPWL compared PPO and CDES. This highlights CORE's effective col- laboration between RL. and EAs, enabling it to surpass the performance of both methods. More hyperparameter analysis and ablation study are provided in Appendix C.
n100
199488
189151
188376
n200
382955
346382
342693
n300
574579
468089
461219
ami33
53773
49903
46849
ami49
679473
642811
627620
In addition, we conduct detailed analysis of CORE's collaboration mechanisms. We
Better Data Number
Success rife
present the changes in the number of layouts maintained by the archive that outperform those constructed by PPO, along with the prob- ability of layouts constructed by PPO being re- tained after injection into the population. The former indicates the extent to which evolu- tionary search can guide RL. Consistently low data volume suggests that EAs are unable to find layouts superior to RL, thereby failing to provide effective guidance. The latter reflects the extent of PPO's 's involvement in population evolution. If layouts constructed by PPO are immediately eliminated upon being added to the population, it indicates that EA layouts are significantly superior. Consequently, PPO layouts cannot be injected into the population, demonstrating that PPO does not contribute to population evolution. In Figure 6, we present the results for the n100 case. We observe that the number of samples in the archive that outperform RL-constructed layouts consistently remains at high level, indicating that EAs continuously provide effective guidance to RL. Additionally, the success rate of PPO layout injections into the population has consistently remained above 0.6, indicating that RL continues to influence the population evolution. The above two mechanisms complement each other, thus achieving better synergy. More experiments on runtime, CDES, and comparisons with ERL provided in Appendix C.
10000
1900
1.9
2000
Figure Analysis experiment on the collaboration mechanisms. The left side reflects the amount of data used for PPO imitation learning, while the right side reflects the success rate of RL-constructed layouts being injected into the EA population.
5 Conclusion
This paper proposes an efficient hybrid framework, CORE, which combines EAs and RL to solve the floorplanning problem. CORE involves three main processes: the evolutionary process, the reinforcement process, and the collaboration process. Specifically, we introduce the clustering-based diversified evolutionary search to ensure robust global optimization capabilities. Additionally, we propose complete reinforcement learning process with Tr-e representation for floorplanning. Furthermore, we design effective collaboration mechanisms for EAS and RI. Our experimental results demonstrate that CORE outperforms strong baselines two widely used floorplanning benchmarks.
10

---

Acknowledgments
This work is supported by the National Natural Science Foundation of China (Grant Nos. 62422605, 624B2101, 92370132). We would like to thank all the anonymous reviewers for their valuable comments and constructive suggestions, which have greatly improved the quality of this paper.
References
[1] K. I. Gubbi, S. Aresh B., D. Sheaves, S. Salehi, S. Manoj P. D., S. Rafatirad, A. Sasan, and H. Homayoun. Survey of machine learning for electronic design automation. In GLSVLSI, 2022.
[2] D. Sánchez, L. Servadei, G. z. Kiprit, R. Wille, and W. Ecker. A comprehensive survey on electronic design automation and graph neural networks: Theory and applications. ACM Transactions on Design Automation of Electronic Systems, 2023.
[3] Y. Shi, K. Xue, S. Lei, and C. Qian. Macro placement by wire-mask-guided black-box optimization. NeurIPS, 2024.
[4] Y. Lai, Y. Mu, and P. Luo. Maskplace: Fast chip placement via reinforced visual representation learning. NeurIPS, 2022.
[5] Y. Lai, J. Liu, Z. Tang, B. Wang. J. Hao, and P. Luo. Chipformer: Transferable chip placement via offline decision transformer. In ICML, 2023.
[6] A. Mirhoseini, A. Goldie, M. Yazgan, J. W. Jiang, M. Songhori, Wang, Y. Lee, Johnson, et al. A graph placement methodology for fast chip design. Nat., 2021.
[7] X. Jiang, Y. Zhao, Y. Lin, R. Wang, R. Huang, al. Circuitnet 2.0: An advanced dataset for promoting machine learning innovations in realistic chip design environment. In ICLR, 2024.
[8] Z. Geng, J. Wang, Z. Liu, S. Xu, Z. Tang, M. Yuan, J. Hao, Y. Zhang, and F. Wu. Reinforcement learning within tree search for fast macro placement. In ICML, 2024.
[9] R. Cheng, X. Lyu, Y. J. Ye, J. Hao, and J. Yan. The policy-gradient placement and generative routing neural networks for chip design. NewrIPS, 2022.
[10] R. Cheng and J. Yan. On joint learning for solving placement and routing in chip design. NeurlPS, 2021.
[11] Y. Lin, S. Dhar, W. Li, H. Ren, B. Khailany, and D. Z. Pan. Dreamplace: Deep learning toolkit-enabled gpu acceleration for modern vlsi placement. In DAC, 2019.
[12] H. Murata, K. Fujiyoshi, S. Nakatake, and Y. Kajitani. Vlsi module placement based on rectangle-packing by the sequence-pair. IEEE TCAD, 1996.
[13] Q. Xu, S. Chen, and B. Li. Combining the ant system algorithm and simulated annealing for 3d/2d fixed-outline floorplanning. ASC, 2016.
[14] Knechtel, E. F. Young, and J. Lienig. Structural planning of 3d-ic interconnects by block alignment. In ASP-DAC, 2014.
[15] S. Kirkpatrick, C. D. Gelatt Jr, and M. P. Vecchi. Optimization by simulated annealing. Science, 1983.
[16] A. B. Kahng and Q Wang. Implementation and extensibility of an analytic placer. In ISPD, 2004.
[17] J. Lin, T. Chen, Y. Chang, W. Chang, Y. Shyu, Y. Chang, and J. Lu. fast thermal-aware fixed-outline floorplanning methodology based on analytical models. ICCAD, 2018.
[18] Lin and J. Wu. F-fm: Fixed-outline floorplanning methodology for mixed-size modules considering voltage-island constraint. IEEE TCAD, 2014.
11

---

[19] F. Huang, D. Liu, X. Li, B. Yu, and W. Zhu. Handling orientation and aspect ratio of modules in clectrostatics-bert based large scale fixed-outline floorplanning. In IEEE ICCAD, 2023.
[20] X. Li, K. Peng, F. Huang, and W. Zhu. Pef: Poisson's 's equation based large-scale fixed-outline floorplanning. IEEE TCAD, 2022.
[21] J. Lin, T. Chen, H. Hsich, Y. Shyu, Y. Chang, and J. Lu. Thermal-aware fixed-outline floorplan- ning using analytical models with thermal-force modulation. IEEE TVLSI, 2021.
[22] M. Kuwano. Stable-Ise based analytical placement with overlap removable length. Proc. of SASIMI, 2010.
[23] R. S. Sutton and A. G. Barto. Reinforcement learning an introduction. Adaptive computation and machine learning. 1998.
[24] Z. He, Y. Ma, L. Zhang, P. Liao, N. Wong, B. Yu, and M. D. Wong. Learn to floorplan through acquisition of effective local search heuristics. In ICCD. IEEE, 2020.
[25] H. Wang, K. Wang, J. Yang. L. Shen, N. Sun, H. Lee, and Song Han. GCN-RL circuit designer: Transferable transistor sizing with graph neural networks and reinforcement learning. In DAC. IEEE, 2020.
[26] A. Mirhoseini, A. Goldie, M. Yazgan, J. W. J. Jiang, E. M. Songhori, S. Wang, Y. J. Lee, E. Johnson, O. Pathak, S. Bae, A. Nazi, J. Pak, A. Tong, K. Srinivasa, W. Hang, E. Tuncer, A. Babu, Q. V. Le, J. Laudon, R. Ho, R. Carpenter, and J. Dean. Chip placement with deep reinforcement learning. Nature, 2021.
[27] Q Xu, H. Geng, S. Chen, Yuan, C. Zhuo, Y. Kang, and X. Wen. Goodfloorplan: Graph convolutional network and reinforcement learning-basse based floorplanning. IEEE Trans. Comput. Aided Des. Integr: Circuits Syst., 2022.
[28] M. Amini, Z. Zhang, S. Penmetsa, Y. Zhang, J. Hao, and W. Liu. Generalizable floorplanner through corner block list representation and hypergraph embedding. In KDD, 2022.
[29] M. L Puterman. Markov decision processes. Handbooks ORMS, 1990.
[30] L. Jain and A. Singh. Non slicing floorplan representations in vlsi floorplanning: A summary. IJCA, 2013.
[31] Y. Chang, Y. Chang, G. Wu, and S. Wu. B*-trees: A new representation for non-slicing floorplans. In DAC, 2000.
[32] M. Mitchell. An introduction to genetic algorithms. MIT Press, 1998.
[33] M. M. Drugan. Reinforcement learning versus evolutionary computation: A survey on hybrid algorithms. Swarm Evol. Comput., 2019.
[34] O Sigaud. Combining evolution and deep reinforcement learning for policy search: survey. arXiv preprint, 2022.
[35] H. Bai, R. Cheng, and Y. Jin. Evolutionary reinforcement learning: A survey. arXiv preprint, 2023.
[36] P. Li. J. Hao, H. Tang, X. Fu, Y. Zheng, and K. Tang. Bridging evolutionary algorithms and reinforcement learning: A comprehensive survey. IEEE TEVC, 2024.
[37] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint, 2017.
[38] P. Guo, C. Cheng, and T. Yoshimura. An o-tree representation of non-slicing floorplan and its applications. In DAC, 1999.
[39] K. A. De Jong. Evolutionary computation: a unified approach. GECCO 2020. ACM, 2020.
[40] Z. Zhou, Y. Yu, and C. Qian. Evolutionary learning: Advances in theories and algorithms. 2019.
12

---

[41] S. Khadka and K. Tumer. Evolution Evolution-guided policy gradient in reinforcement learning. In NeurIPS, 2018.
[42] Khadka, Majumdar, T. Nassar, Z. Dwiel, E. Tumer, S. Miret, Y. Liu, and K. Tumer. Collaborative evolutionary reinforcement learning, In ICML, 2019.
[43] C. Bodnar, B. Day, and P. Lió. Proximal distilled evolutionary reinforcement learning. In AAAI, 2020.
[44] A. Pourchot and O. Sigaud. CEM-RL: combining evolutionary and gradient-based methods for policy search. In ICLR, 2019.
[45] Hao, P. Li, H. Tang. Zheng, X. Fu, and Z. Meng. Erl-reS-2S: Efficient evolutionary reinforcement learning with shared state representation and individual policy representation. In ICLR, 2023.
[46] Li, Hao, H. Tang, Y. Zheng, and Z. Barez. Value-evolutionary-baeed reinforcement learning. In ICML, 2024.
[47] P. Li, Y. Zheng, H. Tang, X. Fu, and J. Hao. Evorainbow: Combining improvements in evolutionary reinforcement learning for policy search. In /CML, 2024.
[48] S. Nickum, A. G. Barto, and L. Spector. Genetic programming for reward function search. IEEE Trans. Auton. Ment. Dev., 2010.
[49] Y. Jason Ma, W. Liang, G. Wang. D. Huang. O. Bastani, D. Jayaraman. Y. Zhu, L. Fan, and A. Anandkumar. Eureka: Human-level reward design via coding large language models. In ICLR, 2024.
[50] Li, HAO, H. Tang, Yuan, Qiao, Dong, and ZHENG. R#: Efficient reward design via reward structure evolution and parameter alignment optimization with large language models. In JCML, 2025.
[51] S. Majumdar, S. Khadka, S. Miret, S. McAleer, and K. Tumer. Evolutionary reinforcement learning for sample-efficient multiagent coordination. In ICML, 2020.
[52] Li, J. Hao, H. Tang, Y. Zheng, and X. Fu, Race: Improve multi-agent reinforcement learning with representation asymmetry and collaborative evolution. In /CML,
[53] Y. Zheng, X. Xie, T. Su, L. Ma, J. Hao, Z. Meng, Y. Liu, R. Shen, Y. Chen, and C. Fan. Wuji: Automatic online combat game testing using evolutionary deep reinforcement learning. ASE. IEEE, 2019.
[54] O. Nilsson and A. Cully. Policy gradient assisted map-elites. In GECCO, 2021.
[55] Mouret and J. Clune. Illuminating search spaces by mapping elites. arXiv preprint, 2015.
[56] Lehman and O Stanley. Novelty search and the problem with objectives. GPTP IX, 2011.
[57] B. Tjanaka, M. C. Fontaine, J. Togelius, and S. Nikolaidis. Approximating gradients for differentiable quality diversity in reinforcement learning. In GECCO, 2022.
[58] M. Ahmed and R. Seraj and S. M. S. Islam. The k-means algorithm: A comprehensive survey and performance evaluation. Electronics, 2020.
[59] W. Hamilton, Z. Ying, and Leskovec. Inductive representation learning on large graphs. NeurlPS, 2017.
[60] A. Vaswani, N. Shazeer, N. Parmar, Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. NeurIPS, 2017.
[61] J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High nigh-dimensional continuous control using generalized advantage estimation. arXiv preprint, 2015.
13

---

Algorithm 1: Collaborative Optimization with RL and EA (CORE)
Initialize RL: the PPO policy network тe, the PPO value network Va, the experience huffer D. Initialize population P P= {#1;,..- { .8 with size N, the a to combine quality scores and novelty scores, the number of clusters C, clite number in cach cluster, The EA layout Archive LA,
EA experience buffer DFA.
repeat
# ( EA Process
Population Clustering: Perform K-Means clustering in the population based on the features F(s.) of cach individual, resulting in the construction of C' clusters.
Subpopulation Evaluation: In each cluster, evaluate the quality score P(s:) and novelty score (s:) of each layout
Elite Selection: Combine quality scores and novelty scores to obtain the final fitness:
f(*.) o P(s:) + (1 - N))((ss)) ) and then select the top clite layouts from each cluster based on the fitness.
New Population Construction: Mutate the elite layouts to create layouts, replacing the non-elite layouts.
# ( Evolutionary- Guided Mechanism
10
Try to add the layouts within the population into the EA archive La (Maintaining the best layouts for cach module acting as the root, based on different rotations). Compare the layouts in La with RL's layouts based on the same
node, and convert the EA better layouts into decision sequences, and then add sequences into DEA. If DEA is not empty, calculate the imitation learning loss Lm. based on the experiences in DEA. # 3 RL Process:
11
12
RL Interaction: RL interacts with the environment collect experiences, which stored in D RL Optimization: Optimize using experiences from D and imitation learning loss Cu..
14
15
# 4 Reinforcement-DDe
Mechanism: Add complete layout constructed by RI. population, calculate the soore f*,, for each individual, and keep the top N individuals to ensure the population size remains unchanged.
until reaches maximum training steps:
A Method Implementation Details
All experiments are carried out on NVIDIA GTX 2080 Ti GPU with Intel(R) Xeon(R) CPU E5-2680
v4 @ 2. 2.
A.1 Baselines and Benchmark
We compare CORE with five baselines: Corblivar [14], SP-FP [13], B*-Tree SA [31], GoodFloor- plan [27]. and CBL-RL [28]. We utilize the open-source implementations of Corblivar", SP-FPt and B*-Tree SA'. For GoodFloorplan and CBL-RL, since the experimental setups are identical, directly use the results from the original papers. Evaluation is conducted on two widely used benchmarks, MCNC and GSRC", All algorithms are trained to convergence over 5 runs, and the best performances are reported.
A.2 Algorithm Pseudocode
The pseudocode for CORE is presented Algorithm 1. The entire process of CORE contains three main parts: (1) Evolutionary Process, (2) Reinforcement Process, and (3) Collaboration Process. To improve clarity, we break down the Collaboration Process into the Evolutionary-GGuded Mechanism and Reinforcement-Drrvee Mechanism.
Specifically, CORE begins with the Evolutionary Process. Here, population individuals are clustered to form subpopulations, evaluated within these subpopulations, elite individuals selected, and a new population is constructed based on these elites. Subsequently, we initiate the Evolutionary-
https///gitub..om/DDXX www..//DDEDNUAA/CCDDOiiiiio
"hatp///taaff.ustc
wtw.://ww...tt......... edu.ce/eeucc//// songch/package.htt
'huts://github.co com/ wtwm/sshley900911/BBS.. Tree
"hut::/vlsicad.ee.
http:/awww.......ccccccc/c////// .miich.cluBBK/MCCCceennn
"hap://vssccad.eem eecs. wwt.scccccc/BBK//SRRCCC...
14

---

Table 4: Details of setting for the policy and value networks.
Model
Selection
Activation Function
ReLU
GNN Type
SAGEConv
The layer number of the GNN
2
GNN input size
10
GNN hidden size
64
GNN output size
64
The layer number of the Transformer
2
The number of the Transformer attention head Transformer input size Transformer hidden size Transformer output size The layer number of the MLP for insert modules MLP input size for insert modules MLP hidden size for insert modules
64
64
128
64
MLP output size for insert modules
The layer number of the MLP for target modules
MLP input size for target modules
192
MLP hidden size for target modules
64
MLP output size for target modules
The layer number of the MLP for rotation and left/right
MLP input size for rotation and left/right
192
MLP hidden size for rotation and left/right
64
MLP output size for rotation and left/right
4
The layer number of the MLP for value prediction
2
MLP input size for value prediction
64
MLP hidden size for value prediction
64
MLP output size for value prediction
Guided Mechanism, aiming to add population layouts into archive LA based their quality scores. The archive maintains the best layouts (If there are multiple optimization objectives, the Pareto frontier is maintained) for each module under different rotations the root node. Then compare the layouts in LA with RL's layouts and convert the layouts that are better (e.g.. Pareto dominance) than RL's into decision sequences. The sequences stored into buffer DEA. If DEA is not empty, calculate the RL. imitation learning loss Lτ, based on the experiences in DEA-
Following this, we proceed to the Reinforcement Process. In this process, PPO interacts with the environment to collect experiences into D. We optimize PPO using collected experiences and the imitation learning loss Lu. Next, activate the Reinforcement Driven Mechanism to introduce fully constructed RL layouts into the population. Individuals in the population are evaluated based on their quality and diversity, yielding f(s)). The top layouts are retained to maintain the population size unchanged. The above process iterates until convergence.
A.3 Network Architecture
This section provides a detailed description our network architectures for PPO. Figure 3, we present the specific process, and here we provide the detailed architectural design.
The PPO mainly consists of three parts:
The graph neural network consists of two layers of SAGE convolutional layers with ReLU activation. Each module or port contains 10-dimension information, so the input dimension for SAGE is 10. The output dimension is 64. The Transformer employs PyTorch's integrated Transformer encoder layer with 2 layers, attention heads, and hidden size of 64. Therefore, each module or port representation is 64-dimension, and the graph representation, obtained by mean pooling across all modules and ports, is also 64-dimensional. The activation function is ReLU.
Representation Module:
15

---

Table 5: Details of hyperparameters.
Parameter
Value
Optimizer
Adam
Learning rate
2.5e 4
Batch size
128
Gamma
0.99
GAE Gamma
0.95
Scale for HPWL
10-4
Population size N
100
Number of clusters C
4
Elite size in each subpopulation
2
The weight to balance quality score and novelty score a in CDES
0.8
The weight to balance area utilization and HPWL in CDES Clip ratio 7
0.1 for U and 0.9 for HPWL
0.1
The weight of entropy loss Wit
0.01
The weight of value network loss Wvaauu
0.5
The weight of imitation learning loss Wtr.
0.1
Table 6: Algorithm Comparison Based HPWL Optimization.
Case
# Modules
Nets
Corblivar
SP-FP
B*-Tree SA
CORE
n10
10
118
43344
36204
34873
34296
n30
30
349
131253
113498
106365
98713
n50
50
485
171832
143920
130937
123195
n100
100
885
266013
244956
223317
188376
n200
200
1585
527828
464062
400560
342693
n300
300
1632
753813
661 199
621663
461219
ami33
33
123
85116
57782
57815
46849
ami49
49
408
1722410
913264
900802
627620
The MLP input for deciding target modules is 228-dimensional, posed of 64 dimensions from module representations and 64 from graph representations. The MLP consists of two layers: input size 128, hidden layer size 64, and output size 1, deter- mining the modules to be inserted. The inserted modules' representations are concatenated with the earlier 128-dimensional representation, yielding 192-dimensional representation. This undergoes another two-layer MLP to select the inserted modules. The target module and inserted module representations, along with the graph representation, are concatenated and processed through two-layer MLP with a hidden size of 64 and an output dimension of 4. This step determines actions such as whether to rotate and insert as the left or right child. All MIPs utilize ReLU as its activation function.
Decision Module:
Value Network: The value network starts with a separate representation module with the structure. It takes modules, ports, and netlists input derive graph representation. The graph representation then is fed into a two-layer MLP for value prediction. The MLP has input dimension of 64, hidden layer size of 64, and an output dimension of ReLU serves as the activation function for the MLP.
A.4 Hyperparameters
This section provides two aspects of hyperparameter selection: one for the evolutionary algorithm CDES and another for the RI. algorithm PPO. Below, we present detailed hyperparameter choices.
For the CDES's hyperparameters, we define them as follows:
Population size N is set to 100.
Number of clusters C' is 4.
The number of clites in each subpopulation is set to 2.
16

---

The balancing parameter a, which balances quality and diversity, is set to 0.8.
hen optimizing solely for HPWL, its weight is 1.0.
When optimizing for both HPWL and area utilization, assign weight of 0.1 area utilization and 0.9 to HPWL.
For the selection of RL's hyperparameters, we define them as follows:
Optimizer: Adam with a learning rate of 2.5 x 10-4
Batch size: 128.
Gamma: 0.99.
GAE Gamma: 0.95.
Clip ratio for PPO: 0.1
The weight of entropy loss WEnt 0.01
The weight of value network loss WValue: 0.5
The weight of imitation learning loss wn.: 0.1
When optimizing solely for HPWI., HPWI. is multiplied by a scaling factor of x 10-1 as the reward each step, i.e., 7t -10- HPWL.
When optimizing for both HPWL and area utilization, the reward is computed by adding the area utilization U and HPWL scaled by 10-4, i.e., 77 = 10-HHPPL
Throughout the algorithmic process of CORE, RL interacts with the environment / times per epoch, ie., complete / layouts. We set to 30, resulting RL completing 30 full episodes of interaction with the environment. Following this, inject I complete layouts into the population. The number of evolutionary iterations equals the game length T multiplied by J, divided by the population size N. This ensures that EAs and RL interact with the environment as equal terms possible from the perspective of environment steps. We train CORE on n200 and n300 for 20,000 epochs, while 10,000 epochs for other cases.
B Parallel Framework
We observe that due the requirements of interaction and training, especially RL requires network forward passes for action selection. The algorithm is time-consuming. This is common challenge faced by all RL-based algorithms.
To demonstrate this, we analyze the proportion of time cost for each component: RL interaction time accounts for approximately 65%, primarily due the neural network processing inputs and selecting actions during each interaction, which is time- time-consuming. Training time accounts for 20.4%, and the evolutionary process accounts for around 15%. We can observe that the main source of time cost the RL interaction.
To mitigate this, develop parallel framework, focusing on parallelizing interactions. This allows us to concurrently interact with multiple environments, thus significantly reducing runtime. Specifically, upon algorithm initiation, we initiate sub-processes. Subsequently, the main process sends the current network parameters of PPO to subprocesses, which initialize their networks based on these parameters and interact with the environment. Subprocesses collect complete experiences and return them to the main process. Once the main process gathers experiences from all subprocesses, it begins training the network parameters. With the aforementioned parallel interaction approach. we can greatly reduce the runtime. For example, hours to 3.78 hours, and on n50, it decreases from 23.6 hours to 8.3 hours. Additionally, we find that parallel sampling during the EA phase is possible. If the EA phase is also parallelized, the time for n30 will decrease from 3.78 hours to 2.54 hours, and for n50, will decrease from 8.3 hours 4.72 hours. but we have not implemented this version yet, leaving for future work.
n30, the time consumption decreases from 11
17

---

Table Analysis of Population Size on HPWL.
Pop size
n50
n100
ami49
50
126147
211774
711931
100
126047
199488
679473
200
124849
198223
680698
Table 8: Analysis of Cluster Number on HPWL.
Cluster Number n50
n100
n200
ami49
132708
229737
417632
790673
2
127841
226644
403261
759635
126047
199488
382955
679473
10
144938
192398
357297
993230
C Additional Experiments
We also compare CORE with Corblivar, SP-FP, and B*-Tree SA in the setting where only HPWL. is optimized. In this context, HPWL provides a more intuitive measure of the algorithm's search and optimization performance. We train all algorithms until convergence and report their convergence performance. The experimental results, shown in Table 6, show that CORE significantly outperforms the other algorithms, further highlighting the efficiency of CORE.
Here we present hyperparameter sensitivity analysis. worth noting that all hyperparameters remain fixed across all cases without any fine-tuning throughout the entire study.
We first analyze CDES independently (excluding the influence of RL), with the objective of minimizing HPWL. The results are shown in Table 7: We observe that larger population sizes generally lead to better performance.
Population Size:
The results are shown in Table 8. For the number of clusters, a setting of 4 performs better on smaller-scale problems, while 10 more effective for larger-scale instances.
Cluster Number:
Novelty Weight:
The results are shown in Table 9, The novelty weight a = 0.8 yields better
performance.
Elite size: The results are shown Table 10. performs better on larger-scale tasks, whereas n=2 works better for smaller-scale cases.
Imitation learning coefficient: The results shown in Table 10. For the imitation learning coefficient, we observe that the algorithm performs best when the coefficient is set to 0.1.
It is worth noting that we did not fine-tune these parameters-acee all tasks, we consistently used population size 100, clusters, elite size 2, 0.8 and w=0.1.
In the main text, we report the best results following prior works. Below, we present the mean and standard deviation across five runs, as well as performance comparison with the second-best floorplanning algorithm. The results are shown in Table 12. We observe that CORE consistently outperforms the best performance of other baselines.
Runtime comparison between CORE and other methods. CORE introduces RI., which results additional time overhead compared to heuristic methods. Thus we design a parallel architecture to improve efficiency. Unfortunately, existing learning based methods for floorplanning have not released open-source implementations. Therefore, compare the runtime and achieved HPWL with the available baselines in Table 13. CORE achieves better HPWL compared to baselines, and requires more time than EA. However, outperforms other learning- based method MaskPlace.
Ablation study on CORE. We present the results of our ablation study in Table 14, which clearly demonstrate that the integration of both mechanisms leads improved performance.
Comparison between CORE and ERL. We additionally implement a parameter-centric ERI. framework. It is important to note that such frameworks typically rely off off-policy algorithms, such
18

---

Table Analysis of on HPWL.
n50
n100
n200
ami49
0.0
138954
234500
408446
837418
0.2
138592
231034
411872
805013
0.5
136496
225504
418917
766239
0.8
126047
199488
382955
679473
1.0
191069
341052
835619
1729777
Table 10: Analysis of clite size on HPWL.
Elite size
n50
n100
n200
ami49
135760
191708
359261
824228
2
126047
199488
382955
679473
5
130018
225533
410475
731270
TD3 SAC, which face inherent challenges when applied discrete action spaces. To ensure a fair and consistent comparison, we adapt the baseline methods to use B* tree-based PPO, which is better suited for discrete structural representations. The corresponding experimental results presented in Table 15: We observe that CORE consistently outperforms the ERL-based methods across all cases.
Comparison between CDES and EA. We compare our proposed CDES with a vanilla EA that does incorporate the clustering method. The results in Table 16 show that CDES achieves significantly better optimization performance than the vanilla EA.
Comparison between B Tree and other representation. We evaluate the effectiveness of the B*-Tres modeling approach against alternative structure Sequence pair. The results Table 17 indicate that the B*-Tree model consistently yields superior performance.
that the network input includes netlist, whose topological structure naturally lends itself to graph neural networks (GNNs). Therefore, the of GNN is essential. We introduce transformer layers primarily for better representation. The ablation study is shown in Table 18. We observe that the Transformer module contributes to better model performance.
Ablation Study the Transformer Module. It is important
19

---

Table 11: Analysis of wyz on HPWL.
WIL
n50
n100
ami33
ami49
1.0
124319
189956
48437
634400
0.1
123195
188376
46849
627620
0.01
124054
189103
47619
631599
0.0
124172
189009
48403
635198
Table 12: Comparison Between CORE and the Best Baseline on HPWL (mean + std).
Method
n10
n:30
a30
n100
a200
a300
ami33
baseline
36044
108835
137516
218976
351735
476765
64315
CORE
35290 294
100733
1113
125132
473
191610 840
348416
1140
470296 701
52930 753
Table 13: Runtime and HPWL Comparison between CORE and baselines.
Case
EA
Maskplace
CORE
n30 n50
1.65
104045
4.15
150706
3.78
199964
3.54
126047
6.37
201743
8.3 124890
Table 14: Ablation study of CORE on different cases.
Method
n30
n50
ami33
ami49
CORE w/ Injection
98888
123938
49061
641670
CORE w/
99302
124172
48403
635198
CORE w/o both
99561
124683
49903
642811
CORE
98713
123195
46849
627620
Table 15: HPWL Comparison between CORE and ERL.
Method
n10
n30
n50
n100
ERL
34645
99911
124418
190343
CORE
34296
98713
123195
188376
Table 16: Comparison between CORE variants with EA and CDES,
Method
n30
n50
ami33
ami49
CORE w/ EA
101252
127929
52604
698285
CORE w/ CDES
98713
123195
46849
627620
Table 17: HPWI. Comparison between Sequence Pair and B* Tree Representations (x (x103).
(x10*)
n100
n200
n300
ami33
Sequence pair
209.8
404.5
556.9
59.0
B tree
191.3
349.0
369.2
52.7
Table 18: Ablation Study on the Transformer Module.
Method
n30
n50
ami33
ami49
w/o transformer
99744
124954
52820
699215
w/ transformer
99561
124683
49903
642811
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We primarily focus on integrating EA and RL address the floorplanning problem, and our claims are supported by experimental results. The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in Appendix B.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
21

---

Answer: [N]]
Justification: Our work focuses on empirical evaluation and does not provide theoretical proofs formal analysis. We leave the development of theoretical guarantees and formal analysis as future work.
Guidelines:
The answer NA means that the paper does not include theoretical results, All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed pseudocode and experimental hyperparameter settings in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
22

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [NA]
Justification: The paper does not include experiments requiring code. We will open-source the code after the public release.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/ sowniides/ooeSSumissstttss
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ /nipp. htppppubbuuddeeeeeeeeepppppppeee t
/CodeSubmic seabbbissiooo sesiiisssiiiiiiieeeeeioo onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The experimental setup and details are provided in detail in both the main text and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Following the experimental settings of prior work, we conduct each experi- ment with 5 independent runs and report error bars (in Appendix C) to reflect statistically significant variability.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
23

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: These details are provided in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: / http://wwrp..cccppcttt///e/e//ee heteliic//hhiieeee ctwiiiieeeelleeesss
Answer: [Yes]
Justification: Yes, the research conducted in this paper fully conforms with the NcurIPS Code of Ethics in all respects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction)
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: This paper presents work whose goal is advance the field of floorplanning. There are many potential socictal consequences of our work, none which we feel must be specifically highlighted here.
Guidelines:

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes, all creators and original owners of the assets used in the paper properly credited and fully respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
25

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
26

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: / //neruurpp.... wcc//onfeeencce/222//11111 for what should or should not be described.
27


Reviews:
[{"review_id": "pYXJgR1KLN", "reviewer": "Reviewer_YTf4", "summary": "This work addresses the floorplanning problem and proposes a hybrid framework aimed at achieving efficient and high-quality solutions. The framework integrates Evolutionary Algorithms (EAs) with Reinforcement Learning (RL) in a synergistic manner. Specifically, the authors introduce a Clustering-based Diversified Evolutionary Search, formulate floorplanning as a sequential decision-making problem using a B*-Tree representation, and employ RL for efficient policy learning. The coordination between EA and RL is designed to combine their respective strengths. Empirical results demonstrate the effectiveness of the proposed approach.", "questions": "How does the runtime efficiency of the proposed method compare to other baseline approaches?", "limitations": "A deeper ablation analysis is needed to fully understand the contribution and sensitivity of each component in the proposed framework.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. Clear and accessible presentation: The paper is well-organized and easy to follow.\n\n2. Sound and novel methodology: The combination of EA and RL is innovative and well-motivated. The use of B*-Tree as a representation mechanism is a clever choice for the floorplanning domain. Both the design of individual components and their integration are thoughtfully executed.\n\nWeaknesses:\n\n1. Potential efficiency concerns: The integration of multiple complex components (EA, RL, B*-Tree) may introduce runtime overhead, which could affect scalability or deployment in real-world settings.\n\n2. Lack of ablation studies: The paper would benefit from more extensive ablation experiments to isolate and validate the contribution of each key component in the framework.", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's valuable review and constructive comments, and we would like you to know that your questions provide considerably helpful guidance to improve the quality of our paper. \n\nWe will try our best to address each of the concerns and questions raised by the reviewer below:\n\n**1. [Re: The integration of multiple complex components (EA, RL, B\\*-Tree) may introduce runtime overhead, which could affect scalability or deployment in real-world settings.]**\n    \nLearning-based methods typically need additional runtime overhead, primarily due to the cost of sampling and training. To address this issue, we design a parallel architecture that significantly reduces the runtime burden (details in Appendix B).\n\nMoreover, the framework can be further accelerated by adopting the statically compiled JAX backend instead of PyTorch, which often yields a speedup of approximately 300%.\n    \nIn practical industrial floorplanning tasks, the number of modules is typically below 50, and cases exceeding this scale are extremely rare. This makes learning-based approaches particularly competitive.\n    \nFurthermore, compared to the current state-of-the-art macro placement algorithm, our method achieves comparable runtime while delivering superior performance.\n    \n|  | EA | Maskplace | CORE |\n| --- | --- | --- | --- |\n| n30 | 1.65 \\|  104045 | 4.15 \\| 150706 | 3.78 \\|  99964 |\n| n50 | 3.54 \\|  126047 | 6.37 \\|  201743 | 8.3 \\| 124890 |\n\n**2. [Re: More extensive ablation experiments to isolate and validate the contribution of each key component in the framework.]**  \n\nWe thank the reviewers for their valuable suggestions. In response, we provide the following ablations:  \n\n **Coordination Mechanism:** The ablation results are presented below. As shown, integrating both of our proposed coordination mechanisms leads to the best overall performance.\n  \n|  | n30 | n50 | ami33 | ami49 |\n| --- | --- | --- | --- | --- |\n| CORE w/ Injection | 98888 | 123938 | 49061 | 641670 |\n| CORE w/ IL | 99302 | 124172 | 48403 | 635198 |\n| CORE w/o both | 99561 | 124683 | 49903 | 642811 |\n| CORE | **98713** | **123195** | **46849** | **627620** |\n\n\n **CDES:** We compare our proposed CDES with a vanilla EA that does not incorporate the clustering method. The results show that CDES achieves significantly better optimization performance than the vanilla EA.  \n\n |  | n30 | n50 | ami33 | ami49 |\n | --- | --- | --- | --- | --- |\n | CORE w/ EA | 101252 | 127929 | 52604 | 698285 |\n| CORE w/ CDES | 98713 | 123195 | 46849 | 627620 |\n**B\\*-Tree Modeling:** We evaluate the effectiveness of the B*-tree modeling approach against alternative structure Sequence pair. The results indicate that the B*-tree model consistently yields superior performance.\n   \n|  x 1e3 | n100 | n200 | n300 | ami33 |\n| --- | --- | --- | --- | --- |\n| Sequence pair | 209.8 | 404.5 | 556.9 | 59.0 |\n| B* tree | 191.3 | 349.0 | 369.2 | 52.7 |\n\n**3. [Re: How does the runtime efficiency of the proposed method compare to other baseline approaches?]**  \nCORE introduces RL, which results in additional time overhead compared to heuristic methods. Thus we design a parallel architecture to improve efficiency. We compare the runtime and achieved HPWL with available baselines:\n|  | EA | Maskplace | CORE |\n| --- | --- | --- | --- |\n| n30 | 1.65 \\|  104045 | 4.15 \\| 150706 | 3.78 \\|  99964 |\n| n50 | 3.54 \\|  126047 | 6.37 \\|  201743 | 8.3 \\| 124890 |\n    \nCORE achieves better HPWL compared to baselines, and requires more time than EA. However, it outperforms other learning-based method MaskPlace.\n    \n\n---\n\nWe hope our replies have addressed the concerns the reviewer posed and shown the improved quality of the paper. **We are always willing to answer any of the reviewer's concerns about our work** and we are looking forward to more inspiring discussions."}, {"review_id": "e8cRl3TK98", "reviewer": "Reviewer_F7Fd", "summary": "This paper proposes CORE (Collaborative Optimization with RL and Evolutionary Algorithm), a hybrid framework designed to address the floorplanning problem by improving search and exploration to avoid local optima. The central idea is to synergize the global search capabilities of Evolutionary Algorithms (EAs) with the local, learning-based refinement of Reinforcement Learning (RL). The paper's main contributions are:\n1. A novel collaborative framework where the EA and RL components mutually guide each other.\n2. An EA component, named Clustering-based Diversified Evolutionary Search (CDES), which uses layout feature clustering and a novelty-based fitness score to maintain a diverse population of solutions, enhancing global exploration.\n3. An RL component models the floorplanning task as the sequential construction of a B*-Tree, a compact and non-slicing floorplan representation.\n4. Two specific collaboration mechanisms: a \"Reinforcement-Driven\" mechanism where high-quality layouts from the RL agent are injected into the EA's population, and an \"Evolutionary-Guided\" mechanism where superior layouts found by the EA are used as expert demonstrations to guide the RL agent via imitation learning.\n\nExperimental results on the MCNC and GSRC benchmarks show that CORE outperforms strong baselines in wirelength (HPWL) and area utilization.", "questions": "1. You run each algorithm five times and reporting the best convergence performance achieved. I am not sure if it is a convention in this area. Why not report the mean and standard deviation?\n2. Your work shows excellent results on HPWL and AU optimization. A key challenge in industrial floorplanning is satisfying a variety of specific hardware design rules, such as boundary alignment or grouping constraints. How might the CORE framework, particularly its B*-Tree representation, be adapted to handle such constraints? Would this require new mutation operators for the EA and a modified reward/MDP formulation for the RL component?\n3. The \"Evolutionary-Guidance\" mechanism relies on converting a complete B*-Tree from the EA archive into a state-action sequence for the RL agent to imitate. Could you elaborate on this conversion process? Since the RL agent builds the tree sequentially, the order of node insertions matters. Is this order uniquely determined from a static B*-Tree, or does it require a specific traversal (e.g., pre-order)? How does this choice of conversion affect the stability and effectiveness of the imitation learning?\n4. The analysis in Figure 6 suggests a healthy, bidirectional collaboration. Do the roles of the EA and RL shift over the course of training? For example, one might hypothesize that the EA's guidance is more critical in the early stages to help the RL agent find promising regions of the search space, while the RL agent's injections are more impactful later for fine-tuning the population. Do your observations support this, or is the collaborative balance relatively stable?\n5. The use of B*-Tree is a key design choice. Did you consider other compact representations like Sequence-Pair or Corner Block Lists? What are the specific advantages of B*-Tree for this hybrid EA+RL framework that made it the preferred choice?", "limitations": "The paper mentions that \"it discuss the limitations in Appendix B\" (Line 647). However, Appendix B describes a parallelization framework to reduce runtime.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n1. The hybrid EA+RL approach is a well-motivated and powerful idea for complex optimization problems like floorplanning. Combining the global exploration of an EA with the local learning of RL is an effective paradigm. The specific two-way collaboration mechanisms (RL-to-EA injection and EA-to-RL imitation learning) are well-designed and novel in this context.\n2. The RL formulation is based on sequentially constructing a B*-Tree. This is a clever and powerful approach, as the B*-Tree representation inherently guarantees compact, non-overlapping layouts, fundamentally shifting the RL problem from direct coordinate placement to making structured decisions within a combinatorial tree space.\n3. The paper is backed by a strong empirical study. The ablation study in Table 2 convincingly demonstrates that the full CORE framework outperforms both the standalone EA (CDES) and RL (PPO) components, providing clear evidence of synergy. Furthermore, the analysis in Figure 6 provides excellent insight into the collaborative dynamics, showing that both guidance mechanisms remain active and effective throughout training. The performance gains in HPWL over strong baselines are substantial.\n4. The paper is clearly written and well-organized. The high-level framework is effectively communicated through diagrams (Fig. 2 and Fig. 4), which are essential for understanding the complex interactions between the different components.\n\n# Weaknesses\n1. The paper's evaluation focuses exclusively on optimizing foundational metrics like HPWL and Area Utilization. It does not address the broader set of specific, complex hardware design rules (e.g., boundary constraints, grouping constraints) that are often critical in modern industrial floorplanning. This narrows the immediate applicability of the proposed method to more complex, real-world design scenarios.\n2. For two RL baselines, the Area Utilization was an assumption taken from the literature, not a direct measurement, which slightly weakens the comparison on that metric.\n3. The CORE framework is significantly complex, integrating a diversity-driven EA, a GNN/Transformer-based PPO agent, and multiple collaboration mechanisms. This results in a large number of hyperparameters (Table 4) and moving parts, which could pose a challenge for reproduction and practical implementation.", "was_revised": true, "final_justification": "The rebuttal addressed most of my concerns.", "rebuttal": "We appreciate the reviewer's valuable review and constructive comments, and we would like you to know that your questions provide considerably helpful guidance to improve the quality of our paper. \n\nWe will try our best to address each of the concerns and questions raised by the reviewer below:\n\n**1. [Re: For two RL baselines, the Area Utilization was an assumption taken from the literature, not a direct measurement, which slightly weakens the comparison on that metric. ]**\n    \nConsequently, for several metrics that were not reported in the original papers, we provide results under idealized assumptions. To mitigate these issues and ensure reproducibility and fair benchmarking, we will release our algorithm along with a Python-based floorplanning platform.\n    \n**2. [Re: You run each algorithm five times and reporting the best convergence performance achieved. I am not sure if it is a convention in this area. Why not report the mean and standard deviation?]**\n    \nWe follow the prior works by reporting the best results, as this highlights the algorithm’s peak performance. For completeness, we also provide the mean and standard deviation in the appendix, as shown in the table below.\n    \n|  | n10 | n30 | n50 | n100 | n200 | n300 | ami33 |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Best baseline | 36044 | 108835 | 137516 | 218976 | 351735 | 476765 | 64315 |\n| CORE | 35290 ± 294 | 100733 ± 1113 | 125132 ± 473 | 191610 ± 840 | 348416 ± 1140 | 470296 ± 701 | 52930 ± 753 |\n    \nWe can observe that CORE still consistently outperforms the best performance of other baselines.\n    \n**3. [Re: A key challenge in industrial floorplanning is satisfying a variety of specific hardware design rules, such as boundary alignment or grouping constraints. How might the CORE framework, particularly its B\\*-Tree representation, be adapted to handle such constraints? Would this require new mutation operators for the EA and a modified reward/MDP formulation for the RL component?]**\n    \nAdding constraints typically does not require changes to the algorithm itself. Instead, we can incorporate the constraints into the reward or fitness function. For example, by embedding grouping constraints as part of the evaluation objective—both in the fitness function for EA and the reward function for RL. we can handle such constraints within the existing optimization framework. In addition, some adjustments based on the specific optimization objectives may also be necessary. For example, constructing subtrees on top of the B* tree can be an effective way to address grouping constraints.\n    \n**4. [Re: converting a complete B\\*-Tree from the EA archive into a state-action sequence for the RL agent to imitate.  or does it require a specific traversal (e.g., pre-order)? How does this choice of conversion affect the stability and effectiveness of the imitation learning? ]**\n    \nBefore answering this question, it is important to outline the optimization process of the B*-tree. After each module is inserted, the B*-tree adjusts the entire layout to ensure seamless placement of modules. This adjustment is performed in a depth-first traversal of the tree, during which each node is updated sequentially, and the adjustment order is recorded.\n    \nConsequently, when converting a layout back into a decision sequence, we must model it according to this recorded order. The reason is that any deviation from the original traversal order can lead to inconsistencies between the reconstructed and original layouts. For example, adjusting the left or right child first may result in entirely different outcomes. This process is inherently tied to the structure of the B*-tree, which relies on depth-first traversal for both module placement and reordering. Therefore, reconstructing the sequence must also follow the same depth-first traversal logic.\n    \n**5. [Re: Do the roles of the EA and RL shift over the course of training? For example, one might hypothesize that the EA's guidance is more critical in the early stages to help the RL agent find promising regions of the search space, while the RL agent's injections are more impactful later for fine-tuning the population. Do your observations support this, or is the collaborative balance relatively stable?]**\n    \nWe observe a three-phase phenomenon: (1) **Initial Phase**: RL) dominates. (2) **Middle Phase**: The effect of EA gradually becomes more prominent, while the impact of RL diminishes. (3) **Final Phase**: RL regains influence and starts contributing effectively again.\n    \nThe reason behind Phase 1 is that, in our experiments, layouts generated by randomly initialized policy networks often outperform those constructed by random layout initialization by EA. As a result, the RL injection success rate reaches nearly 100% in the early stages.\n    \nAs EA proceeds through several iterations, it begins to discover solutions that outperform those generated by RL, leading to a drop in the RL injection success rate.\n    \nAfter a period of training, we observe that the quality of EA-generated solutions relative to RL begins to decline, and the RL injection success rate starts to rise again.\n    \nExcept for the first phase, our findings are consistent with the reviewer’s observations in the second and third phases. The first phase is a notable exception, which we attribute to the fundamental difference between the two paradigms: EA focuses on solution optimization, whereas RL optimizes the policy. This structural difference leads to a performance gap in initial solution quality.\n    \n**6. [Re: Did you consider other compact representations like Sequence-Pair or Corner Block Lists? What are the specific advantages of B\\*-Tree for this hybrid EA+RL framework that made it the preferred choice?]**\n    \nWe choose the **B*-tree representation** for three main reasons:\n    \n1. **Efficient evaluation and update.** Since the floorplanner is embedded as the environment of an MDP, each state–action transition must be evaluated quickly. Unlike other representations, the B*-tree does not require the construction of horizontal/vertical constraint graphs or shortest-path computations to evaluate performance, which significantly reduces computational overhead. Moreover, it supports incremental evaluation, making it well-suited for sequential decision-making settings.\n2. **Consise search space.** A B*-tree encodes fewer permutations compared to most alternative representations, resulting in a more concise search space that facilitates efficient exploration and learning.\n3. **Modifiable open-source infrastructure.** Effective heuristics such as SA have already been developed for B*-trees, and robust open-source implementations are readily available, making it easier to integrate and extend.\n    \nWe opt for the B*-tree representation, but our approach is not tied to it. Any other encoding could be substituted, since our work concentrates on the higher-level framework; the core contribution is the development of a cooperative optimization scheme.\n    \n**7. **[Re: Limitations]****\n    \nWe thank the reviewer for the valuable suggestion. Below, we provide a discussion of some limitations of CORE.\n    \n1. **Time overhead**: Learning-based methods typically incur additional training costs. Although we mitigate this through a parallel framework or potential framework modifications to reduce training time, the process remains relatively time-consuming.\n2. **Theoretical support**: Our method currently lacks theoretical guarantees and relies more on empirical validation.\n\n---\n\nWe hope our replies have addressed the concerns the reviewer posed and shown the improved quality of the paper. **We are always willing to answer any of the reviewer's concerns about our work** and we are looking forward to more inspiring discussions."}, {"review_id": "qDBVjO7FKC", "reviewer": "Reviewer_ayg2", "summary": "In this paper the authors propose an approach combining reinforcement learning and evolutionary algorithms for floor planning in chip design, all in a B*-Tree representation. They describe the approach in terms of the two parts (RL and EA) separately and then how they can influence one another. They then present their results compared to several other floor planning baselines. They are able to consistently outperform these baselines in terms of wire length. The authors also include an ablation and some additional analysis, which supports the importance of both the EA and RL components of their approach.", "questions": "1. Can the authors provide some clarity on their EA approach related to my concerns above?\n2. Is there a reason the authors don't mention MAP-Elites in the paper?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# Quality \n\nThis represents a very well conducted research project. I appreciate the concept of combining RL and EAs for floorplanning. The evaluation is well-argued and supports the authors' claims and the ablation and additional graphs support the authors' choices behind their approach. I have no major concerns with the quality of the work. I do have some concerns with the quality of the paper which I'll get to next. \n\n# Clarity\n\nI think this paper could be improved substantially. At present, the authors do not include their algorithm pseudocode in the main text of the paper. This is very confusing and essentially means that readers must read the appendices to understand their approach, which is against the formatting guide. \n\nOn a more minor note, the authors repeatedly refer to their approach as an evolutionary algorithm, but it's unclear if it has a crossover function. The concept of selecting parents is mentioned once, but it seems this may only have been meant in terms of which members of a population are mutated. Similarly I'm unclear why EAs are referred to as black box optimization methods when their process is transparent. Clarity around these points would be appreciated. I have another more substantial concern around the EA but I'll leave that for the originality section below. \n\n# Significance\n\nThis work would be fairly significant within the Electronic Design Automation community. It would also be significant to some extent within the broader AI community due to the clever combination of EAs and RL. But I think this may be the weakest attribute of the paper for this venue, since the contributions are not really to neural information processing systems. \n\n# Originality\n\nThis is not the first work to combine RL and EAs but it may be the first work to combine these approaches to floorplanning and is certainly the first work to combine these with B*-Trees. That would lead to a fairly minor originality all considered. However, I have one major concern which is that the authors appear to have potentially reinvented MAP-Elites. The authors make use of quality and diversity search based on binning solutions, which is very close to MAP-Elites. Despite this, the authors do not mention MAP-Elites or even directly identify their work as falling within the Quality-Diversity (QD) paradigm. If the authors are unfamiliar with this area of EA research I'd recommend looking into it.", "was_revised": true, "final_justification": "Based on the discussion and the other reviews I have chosen to maintain my score. I acknowledge the value of the approach but I'm not sure it is sufficient for me to move beyond a borderline accept.", "rebuttal": "We appreciate the reviewer's valuable review and constructive comments, and we would like you to know that your questions provide considerably helpful guidance to improve the quality of our paper. \n\nWe will try our best to address each of the concerns and questions raised by the reviewer below:\n\n**1. [Re: The authors do not include their algorithm pseudocode in the main text of the paper.]**\n    \nWe thank the reviewer for the valuable suggestion. We will incorporate the pseudocode into the main text to improve readability in the revised version.\n    \n**2. [Re: the authors repeatedly refer to their approach as an evolutionary algorithm, but it's unclear if it has a crossover function. The concept of selecting parents is mentioned once, but it seems this may only have been meant in terms of which members of a population are mutated.]**\n    \nWe thank the reviewer for the valuable suggestion. We would like to clarify that our algorithm does not employ crossover operations. The primary reason is that crossover is difficult to apply to tree-based representations. For instance, in two parent trees, each node corresponds to a different module. Directly exchanging subtrees may lead to module duplication.\n    \n**3. [Re: I'm unclear why EAs are referred to as black box optimization methods when their process is transparent. Clarity around these points would be appreciated.]**\n    \nA black-box optimization algorithm refers to an approach that only requires access to the objective function (fitness function), without needing any explicit gradients, constraint formulations, or internal structural information. EAs fall under this category: although their optimization process is transparent, they rely solely on the evaluation of solution scores f(x), without analytically exploiting the structure of f(x) to guide the optimization of x. For this reason, EAs are commonly referred to as black-box optimization methods.\n    \n**4. [Re: I have one major concern which is that the authors appear to have potentially reinvented MAP-Elites. The authors make use of quality and diversity search based on binning solutions, which is very close to MAP-Elites. Despite this, the authors do not mention MAP-Elites or even directly identify their work as falling within the Quality-Diversity (QD) paradigm.]**\n    \nWe thank the reviewer for pointing out this omission. MAP-Elites typically requires the explicit construction of an archive to store the best-performing solutions across different behavioral descriptors. In contrast, our method employs clustering to identify diverse and high-quality solutions, thereby mitigating the risk of convergence to suboptimal regions.\n    \nOur results confirm that incorporating quality diversity leads to more efficient optimization and yields higher-quality solutions compared to a standard EA baseline. (See the table below for details.)\n    \n|  | n30 | n50 | ami33 | ami49 |\n| --- | --- | --- | --- | --- |\n| CORE w/ EA | 101252 | 127929 | 52604 | 698285 |\n| CORE w/ CDES | 98713 | 123195 | 46849 | 627620 |\n    \nThank you for the valuable suggestions. We will include this discussion about MAP-Elites in the revised version. We would greatly appreciate it if the reviewer could point out any relevant references we may have overlooked.\n    \n\n---\n\nWe hope our replies have addressed the concerns the reviewer posed and shown the improved quality of the paper. **We are always willing to answer any of the reviewer's concerns about our work** and we are looking forward to more inspiring discussions."}, {"review_id": "ABmmvcP7yu", "reviewer": "Reviewer_y8c9", "summary": "This paper proposes CORE, an Evolutionary Algorithms (EA) and RL-based floorplanning framework for EDA. B*-Tree and clustering-based diversified evolutionary search are introduced for floorplanning optimization, as well as a PPO-based RL+EA framework. Experimental results show that compared with existing methods, CORE achieves a 12.9% improvement in HPWL.", "questions": "1. Could you provide additional experimental evidence demonstrating CORE's superiority over existing EA+RL methods?\n2. Could you provide further discussion on the positioning of the B*-tree root? Could you explain why not use methods based on orthogonal constraint graph and shortest path spanning tree?\n3. Could you provide additional formal proof or experimental evidence that B*-trees are superior to standard ordered search trees?\n4. While App. B addresses scalability concerns, could you provide further discussion on the algorithm's scalability and complexity?\n5. Could you provide further comparison on CORE with recent commercial tools, including commercial tools (e.g., Cadence Innovus) or macro-placement methods (e.g., DREAMPlace)?\n\nIf the authors address my concerns, I may raise my score accordingly.", "limitations": "While App. B addresses scalability concerns, a more rigorous complexity analysis would strengthen the methodological claims. I think this paper has no potential negative societal impact.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Related work will be open-sourced upon acceptance, providing the community with an effective approach for floorplanning using novel RL+EA-based methods.\n2. Combining RL+EA is an established methodology. It's happy to see the authors experimentally validate this approach to address floorplanning challenges in EDA.\n3. Experimental results demonstrate effective improvements in HPWL by 12.9%.\n\nWeaknesses:\n1. The bottom-left positioning of the B*-tree root (Fig. 1) lacks clear justification. What justifies this over central placement or placement at other positions?\n2. The authors provide inadequate coverage of related works, particularly regarding EA+RL approaches such as [1][2] and [3]. Additionally, there is a lack of comparative analysis between the proposed method and classic EA+RL techniques.\n* [1] Moriarty D E, Schultz A C, Grefenstette J J. Evolutionary algorithms for reinforcement learning[J]. Journal of Artificial Intelligence Research, 1999, 11: 241-276.\n* [2] Khadka S, Majumdar S, Nassar T, et al. Collaborative evolutionary reinforcement learning[C]//International conference on machine learning. PMLR, 2019: 3341-3350.\n* [3] Bai H, Cheng R, Jin Y. Evolutionary reinforcement learning: A survey[J]. Intelligent Computing, 2023, 2: 0025.\n3. No convergence guarantees for EA-RL collaboration. I noticed the authors mentioned other methods may be \"trapped in local optima\". How do the proposed CORE avoid local optima? \n4. I understand that B*-trees can constrain the search space within approximate bounds, but why are they superior to standard ordered search trees for floorplanning? This requires formal proof or experimental analysis in the context of floorplanning problems.\n5. No comparison with commercial tools (e.g., Cadence Innovus) or macro-placement methods (e.g., DREAMPlace).\n6. Minor formatting issues: \n* typos such as \"​​Transfermer\" and \"Concanated\" in Figure 3.\n* Figures 2-3 display rendering artifacts ($?_?$) in MS Edge, though they appear correctly in other PDF readers (e.g., Adobe Acrobat). I recommend migrating to cross-platform fonts.", "was_revised": true, "final_justification": "The authors have addressed most of my concerns in their rebuttal.", "rebuttal": "We appreciate the reviewer's valuable review and constructive comments, and we would like you to know that your questions provide considerably helpful guidance to improve the quality of our paper. \n\nWe will try our best to address each of the concerns and questions raised by the reviewer below:\n\n**1. [Re: The bottom-left positioning of the B\\*-tree root (Fig. 1) lacks clear justification. What justifies this over central placement or placement at other positions? Could you provide further discussion on the positioning of the B\\*-tree root? ]**\n    \nB*-tree is an ordered binary tree, where the root node must be placed first and is defined using a bottom-left positioning scheme. Newly added modules are inserted as left or right children, causing the overall tree to grow toward the right and upward.\n    \nWe follow the original definition of the B*-tree, as stated in the literature: “A B-tree is an ordered binary tree whose root corresponds to the module on the bottom-left corner.”\n    \n[1]. B*-Trees: A New Representation for Non-Slicing Floorplans\n    \n**2. [Re: Could you explain why not use methods based on orthogonal constraint graph and shortest path spanning tree?]**\n    \nWe choose the **B*-tree representation** for three main reasons:\n    \n1. **Efficient evaluation and update.** Since the floorplanner is embedded as the environment of an MDP, each state–action transition must be evaluated quickly. Unlike other representations, the B*-tree does not require the construction of horizontal/vertical constraint graphs or shortest-path computations to evaluate performance, which significantly reduces computational overhead. Moreover, it supports incremental evaluation, making it well-suited for sequential decision-making settings.\n2. **Consise search space.** A B*-tree encodes fewer permutations compared to most alternative representations, resulting in a more concise search space that facilitates efficient exploration and learning.\n3. **Modifiable open-source infrastructure.** Effective heuristics such as SA have already been developed for B*-trees, and robust open-source implementations are readily available, making it easier to integrate and extend.\n    \nWe opt for the B*-tree representation, but our approach is not tied to it. Any other encoding could be substituted, since our work concentrates on the higher-level framework; the core contribution is the development of a cooperative optimization scheme.\n    \n**3. [Re: The authors provide inadequate coverage of related works, particularly regarding EA+RL approaches such as [1][2] and [3]. Additionally, there is a lack of comparative analysis between the proposed method and classic EA+RL techniques. [1] Moriarty D E, Schultz A C, Grefenstette J J. Evolutionary algorithms for reinforcement learning[J]. Journal of Artificial Intelligence Research, 1999, 11: 241-276.  Khadka S, Majumdar S, Nassar T, et al. Collaborative evolutionary reinforcement learning[C]//International conference on machine learning. PMLR, 2019: 3341-3350. Bai H, Cheng R, Jin Y. Evolutionary reinforcement learning: A survey[J]. Intelligent Computing, 2023, 2: 0025.]**\n    \nWe sincerely thank the reviewer for the insightful comment. The algorithms highlighted by the reviewer primarily focus on optimizing policy parameters using EA or RL. In contrast, CORE adopts a direct solution-space optimization approach via EA, which operates in a significantly smaller and more tractable search space. This design choice enhances optimization efficiency compared to methods that rely on parameter-space exploration.\n    \nTo address the reviewer’s concern, we additionally implemented a parameter-centric ERL framework. It is important to note that such frameworks typically rely on off-policy algorithms, such as TD3 or SAC, which face inherent challenges when applied to discrete action spaces. To ensure a fair and consistent comparison, we adapt the baseline methods to use B* tree-based PPO, which is better suited for discrete structural representations. The corresponding experimental results are presented below:\n    \n|  | n10 | n30 | n50 | n100 |\n| --- | --- | --- | --- | --- |\n| ERL | 34645 | 99911 | 124418 | 190343 |\n| CORE | 34296 | 98713 | 123195 | 188376 |\n    \nWe observe that CORE consistently outperforms the ERL-based methods across all cases. To enhance the completeness of the manuscript, we will include this additional discussion in the revised version. If we have overlooked any relevant work, we would greatly appreciate it if the reviewer could kindly point it out. \n    \n**4. [Re: No convergence guarantees for EA-RL collaboration. I noticed the authors mentioned other methods may be \"trapped in local optima\". How do the proposed CORE avoid local optima?]**\n    \nEA excels at global exploration and is capable of rapidly identifying promising solutions, but it often struggles with fine-tuning due to its inherent randomness. In contrast, RL is well-suited for local refinement and policy improvement. To avoid suboptimal solutions and fully leverage the strengths of both paradigms, we design a collaborative mechanism: EA guides RL by providing high-quality samples that help shape its learning direction, while RL in turn injects refined policies into the EA population to steer evolution more effectively. This bidirectional synergy enables efficient cooperation between the two components.\n    \nWe present the results of our ablation study below, which clearly demonstrate that the integration of both mechanisms leads to improved performance.\n    \n|  | n30 | n50 | ami33 | ami49 |\n| --- | --- | --- | --- | --- |\n| CORE w/ Injection | 98888 | 123938 | 49061 | 641670 |\n| CORE w/ IL | 99302 | 124172 | 48403 | 635198 |\n| CORE w/o both | 99561 | 124683 | 49903 | 642811 |\n| CORE | **98713** | **123195** | **46849** | **627620** |\n**5. [Re: why are they superior to standard **ordered search trees** for floorplanning? This requires formal proof or experimental analysis in the context of floorplanning problems.Could you provide additional formal proof or experimental evidence that B\\*-trees are superior to **standard ordered search trees**?]**\n    \nThe original B*-tree paper [1] provides theoretical justifications and empirical results demonstrating several advantages over other tree-based representations:\n    \n1. **Fixed binary structure enables fast operations.** Due to its fixed binary nature, insertion and lookup require only simple pointer manipulations in **O(1) to O(log n)** time, making it significantly lighter than representations like the O-tree, which involve variable branching. This allows for **O(n)** evaluation per layout, making B*-tree well-suited for high-frequency calls in search or reinforcement learning environments.\n2. **No need for constraint graph construction.** Unlike sequence pair or constraint-graph-based methods, the B*-tree allows direct area computation without constructing horizontal or vertical constraint graphs, thus avoiding shortest-path computations and reducing overhead.\n3. **Efficient incremental updates.** After module swaps, only the affected nodes along the DFS traversal path require local incremental updates, keeping computational costs low.\n    \nThese advantages have been empirically validated on MCNC benchmarks, where B*-tree implementations demonstrate faster runtime, lower memory usage, and superior solution quality. \n    \n**6. [Re: No comparison with commercial tools (e.g., Cadence Innovus) or macro-placement methods.]**\n    \nWe thank the reviewer for the suggestion. Cadence Innovus relies on human expert intervention, while DREAMPlace is a macro placement algorithm that typically struggles to produce high-quality results on floorplanning problems. In our work, we provide comparisons with two state-of-the-art macro placement algorithms.\n    \n|  | n10 | n30 | n50 | n100 | n200 | n300 | ami33 | ami49 |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| MaskPlace | 54105 | 150706 | 201743 | 355918 | 666599 | 960487 | 126360 | 2410415 |\n| ChipFormer | 54954 | 122035 | 165600 | 264180 | 500447 | 693501 | 88129 | 1377496 |\n| CORE | 35170 |  99964 | 124890 | 191349 | 348956 | 469242 | 52672 |  656068 |\n    \nWe can observe that CORE significantly outperforms both MaskPlace and ChipFormer, further demonstrating the effectiveness of methods specifically designed for floorplanning.\n    \n**7. [Re: While App. B addresses scalability concerns, could you provide further discussion on the algorithm's scalability and complexity? ]**\n    \nCORE introduces RL, which results in additional time overhead compared to heuristic methods. Thus we design a parallel architecture to improve efficiency. Unfortunately, existing learning-based methods for floorplanning have not released open-source implementations. Therefore, we compare the runtime and achieved HPWL with the available baselines.\n    \n|  | EA | Maskplace | CORE |\n| --- | --- | --- | --- |\n| n30 | 1.65 \\|  104045 | 4.15 \\| 150706 | 3.78 \\|  99964 |\n| n50 | 3.54 \\|  126047 | 6.37 \\|  201743 |  8.3 \\| 124890 |\n    \nCORE achieves better HPWL compared to baselines, and requires more time than EA. However, it outperforms other learning-based method MaskPlace.\n    \nFurthermore, CORE can be further accelerated by switching to a statically compiled framework such as JAX. That said, we acknowledge that CORE still has limitations in terms of runtime efficiency. Nevertheless, we believe that learning-based methods represent a promising direction with the potential to deliver competitive solutions.\n\n---\n\nWe hope our replies have addressed the concerns the reviewer posed and shown the improved quality of the paper. **We are always willing to answer any of the reviewer's concerns about our work** and we are looking forward to more inspiring discussions."}]

Author rebuttal:

Meta-review: The paper introduces a method for floorplanning which combines evolutionary strategies and reinforcement learning over B* trees.
The method shows significant gains in wirelength with respect to other methods in the literature and the paper is well written. 
While the method has a somewhat limited novelty, introduces several hyperparameters and has longer runtime, the authors in the rebuttal have addressed concerned about the flexibility and ablations, providing further empirical evidence.
Decision: Accept (poster)