FlexPlanner: Flexible 3D Floorplanning via Deep Reinforcement Learning in Hybrid Action Space with Multi-Modality Representation
Authors: Ruizhe Zhong
Xingbo Du
Shixiong Kai
Zhentao Tang
Siyuan Xu
Jianye HAO
Mingxuan Yuan
Junchi Yan
Venue: NeurIPS 2024 poster (2024)
Keywords: Floorplanning
3D Floorplanning
EDA
Reinforcement Learning
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: In the Integrated Circuit (IC) design flow, floorplanning (FP) determines the position and shape of each block. Serving as a prototype for downstream tasks, it is critical and establishes the upper bound of the final PPA (Power, Performance, Area). However, with the emergence of 3D IC with stacked layers, existing methods are not flexible enough to handle the versatile constraints. Besides, they typically face difficulties in aligning the cross-die modules in 3D ICs due to their heuristic representations, which could potentially result in severe data transfer failures. To address these issues, we propose FlexPlanner, a flexible learning-based method in hybrid action space with multi-modality representation to simultaneously handle position, aspect ratio, and alignment of blocks. To our best knowledge, FlexPlanner is the first learning-based approach to discard heuristic-based search in the 3D FP task. Thus, the solution space is not limited by the heuristic floorplanning representation, allowing for significant improvements in both wirelength and alignment scores. Specifically, FlexPlanner models 3D FP based on multi-modalities, including vision, graph, and sequence. To address the non-trivial heuristic-dependent issue, we design a sophisticated policy network with hybrid action space and asynchronous layer decision mechanism that allow for determining the versatile properties of each block. Experiments on public benchmarks MCNC and GSRC show the effectiveness. We significantly improve the alignment score from 0.474 to 0.940 and achieve an average reduction of 16% in wirelength. Moreover, our method also demonstrates zero-shot transferability on unseen circuits.

FlexPlanner: Flexible 3D Floorplanning via Deep Reinforcement Learning in Hybrid Action Space with Multi-Modality Representation
Ruizhe Zhong', Xingbo Du', Shixiong Kai?, Zhentao Tang", Siyuan Xu²,
Jianye Hao Mingxuan Yuan", Junchi Yan
Dept. of CSE & School of AI & MoE Key Lab of AI, Shanghai Jiao Tong University Noah's s Ark Lab, Huawei
*College of Intelligence and Computing, Tianjin University
(2erzer2er22718288 duxingbo, yanjunchi//ssji edu. (kaishixiong, tangzhentaol, xusiyuan520, yuan. mingxuann/huaaaaeeeeee Chhaawii com jianye.haolt hao@tju. edu.
Abstract
In the Integrated Circuit (IC) design flow, floorplanning (FP) determines the po- sition and shape of each block. Serving as a prototype for downstream tasks, it is critical and establishes the upper bound of the final PPA (Power, Performance, Area). However, with the emergence of 3D IC with stacked layers, existing methods are not flexible enough to handle the versatile constraints. Besides, they typically face difficulties in aligning the cross-die modules in 3D ICs due to their heuristic representations, which could potentially result in severe data transfer failures. To address these issues, we propose Flex Planner, a flexible learning-bass based method in hybrid action space with multi-modality representation simultaneously handle position, aspect ratio, and aligmment of blocks. To our best knowledge, FlexPlanner is the first learning- -based approach to discard heuristic-based search in the 3D FP task. Thus, the solution space is not limited by the heuristic floorplanning repre- sentation, allowing for significant improvements both wirelength and alignment scores. Specifically, FlexPlanner models 3D FP based on multi multi-modalities, includ- mouristic-depenennt
ing vision, graph, and sequence. To address the non-trivial issue, we design sophisticated policy network with hybrid action space and asynchronous layer decision mechanism that allow for determining the versatile properties of each block. Experiments on public benchmarks MCNC and GSRC show the effectiveness. We significantly improve the alignment score from 0.474 0.940 and achieve average reduction of 16% in wirelength. Moreover, method also demonstrates zero-shot transferability on unseen circuits. Code is publicly available at: https://gi com/ www.hikaab-..tu/aa...
Introduction
In the very beginning stage of physical design in Electronic Design Automation (EDA), floorplanning (FP) plays critical role. As a subsequent stage of hardware design [1 and logic synthesis (2]. floorplanning provides prototype for downstream tasks 3 ranging from power delivery network (PDN) design (4) placement (5) 6 & routing of final PPA (Power, Performance, Area). Recognized as an NP-hard problem [9) FP establishes
(P&R), hence determining the upper bound
"Corresponding Author. This work was partly supported by NSFC (62222607, 92370201) and Shanghai Municipal Science and Technology Major Project under Grant 202ISHZDZ2.002..
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

the chip's physical layout by optimizing the position and shape of the major blocks to minimize interconnect lengths and ensure efficient silicon area utilization. As floorplanning technologies evolve, 3D FP with stacked layers emerges with more challenges. In particular, the cross-die module alignment becomes another pivotal factor in 31D FP [0
12 For instance, vertical buses [[]]
for cross-die communication connect the aligned blocks spread among multiple dies [ Another
example is the Memory-on-LLoge technology [12] [3 [4], partitioning the processor [15] into two
tiers: memory tier and logic tier. The memory tier consists of memory blocks and customized intellectual property cores (IP cores) [16., while the logic tier contains other components, such as logic blocks. Blocks different dies should be aligned together, enabling the communication established by bonding bumps or pads [1].
Existing works can be categorized into heuristic- based methods, analytical approaches, and learning- based methods. Heuristics- -based methods [17 18 19 20 21] model the FP with a certain heuristic representation. To refine the current FP, they modify the heuristic representation and convert it to the corresponding FP through a decoding scheme. However, this implementation limits the flexibility to directly adjust the position of blocks. After single modification. the entire FP result needs to be regenerated, incapable of making fine-grained adjustments. Moreover, the alignment constraints cannot be satisfied by simply incorporating alignment metrics into its heuristics. Analytical approaches (9 [22] compute the gradient of objectives w.r.t. block position and utilize gradient descent technique to optimize FP. However, the calculation of alignment is non- non--ifferentiable, making them inapplicable in 3D FP. Recently, with the emergence of Reinforcement Learning (RL), baarning-basee methods are promising in FP 23 24 25
26
27 28 29]. Among these approaches,
23 24 25
still retain traditional FP representation, thereby leading to the same issues encountered by heuristic 28 29 focus determining block positions in 2D scenarios. However, when directly implemented in 3D scenarios, these methods could result in 1) overlooking alignment requirements and 2) multi-die property. Specifically, in 2D FP, blocks are arranged exclusively on a single die and organized in a queue that represents the placing order of all blocks. However, in 3D scenarios, multiple queues exist due to the multi-die property, necessitating a layer decision mechanism to merge these queues and determine the comprehensive placing order. This is a critical issue, yet it remains insufficiently explored in current research. Additionally. current learning-baat based methods are 3) incapable of addressing the variable aspect ratio of modules. Characteristies of typical approaches are summarized in Table T
methods. Conversely, (26 27
Table I: Characteristics of typical methods.
To address the aforementioned challenges, we propose FlexPlanner, flexible deep-learning- based approach in hybrid action space with multi-modality representation for 3D FP. Flex- Planner directly outputs the final FP result, with- out the reliance on any heuristic representation. Under the Actor-Critic framework, the policy network consists of three sub-modules, respon- sible for determining the position, layer, and aspect ratio of blocks, spanning hybrid action space. Empirical results demonstrate the effectiveness and significance of FlexPlanner. The main contributions are highlighted as follows:
Method
Type
AR
Aln
3D
Mod
Pef
Analytical Heuristics
X
NIA
3D-B*-SA
11z 21
X
/
H
Wiremask-
30
Heuristics
X
RL-CBL
24
Heuristics, RL
X
X
X
H.G
GraphPlace
20
RL.
X
G
DeepPlace 22
RL
X
V.G
MaskPlacc 8
RL
X
V
Ours
RL
/
G.
aRpost ratio blucks. Alt: cruss-dio block aligument. Mod: Modility. "H - Heuristies, V Vision, G- Claph, S Sequence.
First learning-based method to discard heuristic-based search in the 3D FP task. We propose a novel learning-bas based method with flexible hybrid action space for 3D FP, simultancously handling the position, aspect ratio, and cross-die die alignment of blocks. Without relying on the heuristic- based search, FlexPlanner allows the position and aspect ratio of each block to be explored across comprehensive spectrum, rather than be limited by the constraints of heuristic FP representation, thereby breaking through the upper bound of performance. And propose innovative strategy for more effectively addressing the alignment issue.
Tackle the non-trivial issue of dependency on heuristics by incorporating hybrid action space and multi-modality representation. It is non-trivial avoid the dependency on hauristics-based search in 3D FP due to the difficulty of modeling the complex solution space. Heuristics can only represent a subset of the entire solution space, resulting in limitations on the upper bound performance. To address this issue, we initially introduce three modalities, including vision, graph, and seduuence sequence, to comprehensively represent the state space. Additionally, we design sophisticated
policy network with hybrid action space and asynchronous layer decision mechanisms, enabling learning versatile properties such as position, aspect ratio, layer for each block in a 3D FP setting.
2

---

Zero-shot transferability. Leveraging the advantage of the learning scheme and FlexPlanner demonstrates the ability to exhibit zero-shot transferability on previously unseen circuits. This capability shows strengths in its efficiency, as it conserves substantial training resources when confronted with new IC cases.
SOTA experimental results with significant alignment improvement. Within the learning framework, FlexPlanner achieves state-of-the-ee performance wirelength and alignment in 3D FP. Specifically, the average reduction in wirelength arrives at 16,, compared to previous works. Moreover, by effectively incorporating the alignment constraint, we achieve 0.940 on the alignment score, significantly surpassing the previous SOTA score of 0.474.
2 Preliminary and Formulation
Floorplan. The 2D floorplan task aims to determine the position and shape of each block given the block list, I/O port list, and netlist. Based on this, the 3D floorplan task is further required to place all blocks across multiple dies/layers. Each die/layer d E 'D is rectangular region with width W and height H, and all dies are of the same shape. Specifically, the block list is denoted as B = (b), b2. ... bn) with n blocks, where each block b; is a rectangle with width wi height h4, and area a, = w;- hi. The bottom-left coordinate of block b, is denoted as (1), yi), and the layer where the b, is located is denoted 24. There are two types of blocks: hard block and seft block. For a hard block, its aspect ratio AR, = wi is fixed. For a soft block, AR, can vary between ht
[ARmin, A.max]. while the area must always satisfy a, = wy h4, Additionally, we denote the [ Each port ij is viewed determined position (1g. zj); where (Iz. yj) is the coordinate, and 2j is the layer index. Netlist defined set consisting all nets, where each net is a set of blocks and ports, representing the interconnections. Since the layer 2; of block b, is pre-assigned, we only need to determine the coordinate (zi, 3:) and the aspect ratio AR, for all blocks optimize the following objectives:
I/O port list as T (4,,4, ......... bm} with m ports
point with a pre-
1) Alignment. Given two blocks b,, b, on different layers, alignment evaluates the overlap/intersection area between them on the common projected 2D plane. We define the alignment score ain(
(i.j) max (0, min (z+ + w4, I. + w,) - max(a;, 2z))) aln, = max (0, min y4 + h4s Uj + h2) max(yi, U;)),
aln(i.j) min (1, aln,, () aln,
alnm (i.
where aln, (i:j) is the required minimum alignment area between block b, and by. The total alignment score should be maximized to satisfy the alignment requirement. (b;, by) forms an alignment pair if aln, (i, > and b,, by are mutual aligument partners.
Cross-dic block alignment is common in 3D FP I 14 Taking two blocks in different dies as an example, vertical buses [1] or bonding bumps/pads [14] are employed for communication, which requires capabilities for cross-die block aligmment. That is, considering their projection onto a 2D) plane, the related blocks must exhibit some minimum intersecting region, denoted as aln, (i,
2) HPWL and 3) Overlap. Half Perimeter Wire Length (HPWL) is an approximate metric of wirelength. It can be computed much more efficiently, as accurate wirelength can be accessed only after the time-consuming routing stage. The summation of HPWL should be minimized:
Σ
max
J. min Enet
of max of min y/ Euenet
E.enet
mr Enet
DecEnetllat
where m, is either a block or port in net and xf is the center x-coordinate. For a block, x/ x4 + 24 and for port, 2τद xx. Given two blocks b;, by on the same die, the overlap area between them should be minimized (detailed calculation is given in Alg. in Appendix G.I.. Besides, all blocks should be placed within the fixed outline.
It is also commonly referred to teminna.
3

---

Multi-Modal State
Policy Network
0-0-0-0-0
Layer Decider
0-0-0-0-0
Hybrid Action Space
Sequence
layer prob
Aspeet Ratio Decider
pos mask
constraints
prob
Vision
Position
Decider
pos prob
masked
Shared Encoder
Graph
pos prob
Critic
V
ain mask
Figure 1: Pipeline of FlexPlanner Under the Actor-Critic framework, taking the multi- multi-modality representation as input, the policy network consists of three sub-modules, responsible for determining the position, layer, and aspect ratio of blocks. Alignment mask and position mask incorporated to filter out invalid positions where constraints (alignment, non-overlap, etc.) are not satisfied.
Canvas
Alignment
Alignment
Alignment
Valid Position
Axis Y
20
500
10
10
250
(a)
(b)
(c)
(d)
(e)
Figure 2: Demonstration of alignment. In (a). the light blue region will be occupied by the block to place, and the dark blue region is occupied by its alignment partner block which has been placed. By sliding block place across the plane, we obtain the alignment values each position along the X and Y dimensions, element- wise matrix multiplication, as illustrated in (d). Only (æ, y) satisfying aln, aln, 2 aln,, are valid positions shown in (c), and this binary mask can be incorporated to filter out invalid positions.
shown in (b) and (c). Final alignment can then be calculated through
3 Methodology
Overview. The floorplanning task can formulated as an episodic Markov Decision Process. As shown in Fig- D pipeline of our approach mainly includes state, hybrid action space, policy network and critic network. State 8, consists of three modalities, including vision, graph, and sequence. Action is represented as (I., y. AR), where position (x, 2) are discrete variables and aspect ratio AR is a continuous variable. These properties form hybrid action space. Under the Actor-Critic framework, critic (5(5z) evaluates the current state. Policy network πe Ta(a()))) determines the 2D coordinate (x,, 31) current block b, the layer žt41 to access next block bt+1, and the aspect ratio AR,-1 of bl+1. To explicitly impose constraints the probability matrix of block positions. This process filters out invalid coordinates that violate constraints, such non-alignment, overlap, out-of-boundl locations. We respectively introduce
the action space, masks are applied to
the multi-modalities, layer decision, and reward function design in Sec. 3.1 Sec. 3.2 and Sec. 3.3
3.1
Multi-Modality Representation of 3D Floorplanning
The state space contains three modalities: FP vision, netlist graph, and block placing sequence.
3.1.1
Vision Modality
represents current floorplanning through images. We utilize four vision masks to depict chip layout.
Alignment Mask. fa NW x (N is the field of natural number) is matrix to evaluate the alignment area between blocks b; and its alignment partner by. by has already been placed, fa [xr, y] the intersection area 2D projected plane if b, is placed (x, y). If by has not been placed yet, the

---

alignment mask of b; will be set to a matrix full-filled with aln, (i.j). The native approach has the complexity O( H). However, when W or H is large, has low efficiency. We design an efficient alignment mask generation algorithm with meshgrid operation, which only iterates in regions causing projection overlap between these blocks and harnessing the power of parallel computing, thus reducing the complexity to (wh) namely (1). Demonstration of alignment and details of the Alignment mask is also utilized to filter out the positions that do not satisfy the alignment constraint.
algorithm are shown in Fig. 2 and Alg. [
Canvas Mask. Canvas mask Se E NiP|xW'H
Algorithm 1: Alignment mask generation.
is global observation of current chip layout. The canvas mask is initialized to all zeros. After placing a block b each step, we modify the canvas mask with fe[2, x : x+ y y+ += 1. Based on the canvas mask, we can implement a fast calculation of the total overlap of chip layout, shown in Alg. in Appendix G.I
Input: Current block by, alignment partner by. chip width and height W, H
Output: Alignment mask fa" for block b;
F, max z; W)),
min(ar; + w/y W) Ys max(0, y, h.). ye = min(y, + hyg H)
x, = arange(Is, Te), Y1 = arangelys Ue)
X, Y; = mesttgrid(xx..1
U: = X,, V. = X; ws
Wire Mask & Position Mask (28). Wire mask
fm NW* IT is matrix for how HPWL will
U; Ij. V, , 2; + wy // broadcast a matrix
U where UU, > U,, U. U,)
increase if a block is placed at the position. It
= where( (. < V,, V., V))
records the increase of HPWI. by placing the current block to all candidate positions. Position mask fp E {0, 1) [ indicates available posi- tions for current block to place without overlap or out-of-boundary. implies this position is feasible to place the block.
f0 = max (0, UU) I/ we omit y-dimension due to page limitation
f! = zeros(W, H), fL [X., Y.
( C f
Overall, in step i, given current block by, we concatenate f(() , f(()), f s10). We also incorporate
masks Ju. ((++1) 1 ((++1) of header block in FIFO queue q (introduced in sequence modality in Sec. 3.1.2
in each die, which represent all possible choices for next block in step + 1, providing policy with future horizon. All of these input masks constitute the floorplanning vision modality.
3.1.2 Graph and Sequence Modality
Graph Modality. Given netlist, we convert it to graph G(V, F.,, where V is the set of vertices and E is the set of edges. For two blocks 6;, by within net, we add edges ej,, between vertex pair (y), For the vertex feature, We select (bid, I, y/, 2, w, h, a, p).. where bid is the block index and p indicates whether this block has already been placed or not. Considering each block has a placing order showing the property as a sequence, it is natural to utilize positional encoding [I]] to model this feature. Furthermore, we employ Graph Attention Network [32 to produce embeddings of graph and nodes, representing the logical connection among blocks.
Sequence Modality. Given a die d;, blocks in d; are sorted by their area in descending order, forming FIFO (first in, first out) queue gi- Thus, the block placing order for each die is pre-determined, and each queue can be viewed as a sequence. Combined with block features, we enhance and re-organize the multi-die sequence S E R xXX) where L is the maximum number of blocks in a single die, C' is the number of features. We select (bid, x. y, 2, w. h, a) as sequence features. The sequence modality provides the model with global observation of entire block placing order, and is employed in the asynchronous layer decision discussed in Sec. 3.2
3.2 Asynchronous Layer Decision
In 2D FP, blocks are arranged on single die and are organized in a FIFO queue representing the placing order. However, in 3D scenarios, multiple queues exist due to the multi-dic property, necessitating a layer decision mechanism to merge these queues and determine the comprehensive placing order. A native approach is synchronous block placing, in which we initially place all blocks on the first die, followed by blocks on the second die, and continue in this manner until the last die. In 26 27 28 30) with synchronous placing, the entire placing order keeps fixed. However, this synchronous die-by-die placing order neglects the cross-die connections and alignment requirements, leading to relatively poor FP layout. To address this issue, we propose an asynchromous layer decision mechanism, which determines the layer for accessing next block, If the policy selects die d, for next step, the header block of the FIFO queue qs will be popped as the next block to place.
5

---

During the training process under asynchronous block placing, on one hand, the entire block placing order could vary hugely and become unstable due to the sparsity of the reward, making non non-trivial for position decider to determine the positions of blocks. On the other hand, the layer decision module could converge too fast to mode collapse (a fixed placing order leading to poor quality), empirically degenerating to synchronous die-by-die block placing. In our analysis, it is the short forward horizon leading to this problem. Since policy is only able to sense current and next states, lacks the global receptive field of entire placing order.
To address these issues, we enhance the representation with the sequence modality, and employ Transformer [31] to extract global placing order feature. Given the multi-die sequence feature E RIP LxC self-attention is computed. S is viewed as the source sequence input with length I.. The memory M E g|P|XLXL, output of self-attention of S is given by [31]:
M LN(MSA(S) + S), M LN(PFN(MM) + M),
where MSA is the muati-head self-attention, LN layer normalization, and FFN is the ferdaforward network. Next, cross-attention is applied on M and current block feature by. The single block feature by E 1xC is treated as the target sequence with length serving as the query vector. The output of
cross-attention O E RP x1xC is [3I]:
O = LN(MHA(bb M, M) + by), O = LN(FFN(O) + +0),
(4)
where MHA K, V) denotes multi-head attention. Finally, the next layer decision probability vector can be further computed through linear projection and soft operation.
3.3 Reward Function with Local Advantage and Global Baseline
In 3D floorplanning task, final wirelength, over- lap and alignment can only be accessed at the end of each episode, leading to a sparse re- ward. GraphPlace [26] uses this sparse reward design, where rewards at intermediate steps are all zeros. DeepPlace (27) adopts it with ad- ditional intrinsic reward via Random Network Distillation 33). MaskPlace (2 introduces
Algorithm 2: Reward function with local ad- vantage and global baseline.
Input: (Partial) Alignment score aln, (partial) overlap (partial) HPWL, and corresponding weight Way Wo; wi Output: Reward for each step for from len (en(sisode) to do
if is the end of an episode then
dense reward scheme based on partial HPWL, which is calculated only on the currently placed blocks at each step. However, in two former methods (26 227, they fail to accurately sense the intermediate quality through reward, and in MaskPlace 288. only difference between local
Ti = Wa aln, Wo O, wt HPWL, b b=r.
else
(aln, aln.- wo (o. - (HPWL. - 11 +
0:-1) wi
adjacent steps is involved, lacking the global view of the whole episode. As result, all these reward designs demonstrate relatively poor performances, especially with the complicated hybrid action space consisting of position, layer and aspect ratio.
To alleviate this problem, we design a novel reward function with local advantage and global baseline. We define local advantage as the difference of metric between two adjacent steps, and global baseline as the overall metric at the end of an episode. With local advantage, our model has the ability to acquire current state is whether better worse than the previous. Global baseline depicts the overall quality of the final floorplanning result. It is also essential for asynchronous layer decision module to avoid early convergence and degeneration to poor die-by synchronous block placing order, shown in Sec. 4.4 and Fig. 6b Details of reward design is shown in Alg. 2
3.4 Flexible RL with Hybrid Action Space for 3D Floorplanning
We employ RL with a hybrid action space to address the 3D FP task. The state space consists of three modalities, including floorplanning vision, netlist graph and sequence of block placing order. The hybrid action space is formulated as X' x y x Z x R, with X', V. Z as discrete sets and R as a continuous set. In each step I, the policy outputs three distributions: 1) a discrete probability distribution for 2D position (xt, V/) of current block b. 2) a discrete probability distribution to determine the layer 2e41 for accessing next block b4+1, and finally 3) the mean value and standard deviation of contiguous Gaussian distribution to determine the aspect ratio AR for bi-1. The

---

Table Alignment score comparison among baselines and our method. The higher the alignment
score, the better, and the optimal results are shown in bold, C/M means Circuit/Method.
CM
3D-B*-SA
21
RI-CBI, 24
Wiremask-Bie Beermask-BBOBBo (30)
GrapPPlace 26
DeepPlace 27]
MaskPlace 28
Ours
ami33
0.550++0058
0.132:-0.038
0.179:00.09
0.207 +00067
0.286=0.005
0.300+0.0117
0.905 00.017
ami49
0.438.10.099
0.107
0.222 082
0.265:10.066
0.18010.056
0.218.00.022
0.955
n10
0.38342.118
0.241=0:007
00.01
0.197:0.049
0.235=0.080
0.35420.066
0.917=0.012
n30
0,537400. 159
0. 0.10=00.04 040
0,288=000551
0,233=000039
0.287=0.074
0.511=0.067
0.920=0.024
550
0.626++0.1 1588
0.048-00016
0.290-0.053
0.378+00. 120
0.343-0.005 064
0.764+0.002
A.970-0.004
n100
0.131100.051
0.016.00.08
0.195.10.034
0.279 0..000
0.332.10.073
0.575.10.046
0,961 0.017
n200
0.033:20.22
0.013=00009
0.182400031
0.36020.060
0.387=00039
0,534:200441
0.923=00020
a300
0.009::00009
0.005=0.005
0.205=0.038
0.383+0.023
0.399=0.031
0.553=00003
0.965=0.010
Avg.
0.338
0.084
0.222
0.288
0.306
0.474
0,940
Table 3: HPWL (the lower the better) comparison. The optimal results are shown in bold.
CMM
3D-B*-SA M1
RL-CBL X1
Wiremas-BBO X0
CriphPiace 20
DecpyPace bti
MaskPlace e2
Ours
am333
85 55.16225,563
85,305:4.147
66 3,3333
82,685::6,271
79.457:16.885 6.8
62,125::899
58,339
2m1449
08,0
1,338,119
1.100,91::95, 467
155588724880.00
1.366,083136,48
1,128.1 102.9100
762,712::12.575
n10
55, 115
34, 14,805-1,112
53 36
86,520+880
34,371+817
35.648-1.096
29,781
101 67662.6665
796 1844
87.198= 1.62
98,457=1
97.293#+3,192
N6,291=764
83,962:=1,030
132.421 3.133
111.8784333371
136, 2,2299
126.910::2.886
113,145::407
105,8899 :916
223,38 +9,
275,082- 10.948
181.572+11.66
209, (10,990--44100
223.3500--5.500
201100-2.133
176.875-960
n200
422.06019: 9.10 101
572.619 37,831
325.45331..088
102.650.6.117
418.338815.134
375,250 2.939
316,199 111.99911..0 1,80)
8500
633,344=5,5155
990,466-:377719
467.9006:-5562
396,615:448883
635,1655996636
552,087=:22144
459,221-+5,95
Avg.
379,257
444,917
296,791
377,462
371,388
$14,969
249,053
reason for action a containing the layer and aspect ratio for next step t + 1 instead of t is that: after the execution of dz, next state 9t41 can be generated only after block b place at step t + 1 and its shape have already been determined, since b++1 and its shape (Wi+1, h++1) are involved in the calculation of alignment mask and wire mask. To guarantee adherence to the specified non-overlap and alignment constraints, position mask fp and alignment mask fa are incorporated in 2D position decision. Only (z, y) satisfying [z, yl =1 and fa(æ, y) 2 aln, are considered as valid positions, where aln,. is the required minimum alignment area between current block and its alignment partner. Finally, without the reliance on conventional heuristic FP representation, our approach exhibits more flexibility to directly solve position, aspect ratio and cross-die alignment for blocks.
We select the Actor-Critic (34) framework and Hybrid Proximal Policy Optimization algorithm. The objective function of our hybrid policy Te ra(a:(s)) can be formulated as:
35
36
Ak E2 min rr (0) clip (1: (1/ -,, c,,+ A.)]
I(0)= =
(5)
where k = 3, represents position, layer and aspect ratio decision. Ae is the weight for each clip
loss. (8)) the probability ratio = (sc) A, denotes the generalized advantage estimation
Teota (a;") (sx)"
(GAE) 377, and G, = A, + V, is the cumulative discounted reward [37 38). V, is the estimated state value from critic network (s(s)), and critic network is updated with minimizing Mean Squared Error (MSE) L(Ø)= A E, [(G, (s.))) Entropy of each action distribution is also added as a regularization term for exploration encouragement. Training algorithm is shown in Appendix G.2 and details of model architecture are shown in Appendix D
4 Experiment and Analysis
4.1 Evaluation Protocol and Benchmark
We evaluate the performance of FlexPlanner and other typical methods on public benchmark MCNC and G shown in Table 7 in Appendix B The floorplanning region is set to square region. I/O ports are projected to the fixed outline, and their positions are kept unchanged. Aspect ratio of each block can vary in range [ 2]. Each experiment is run for five times with different seeds. More implementation details and hyper-parameter settings can be found in Appendix C
http://vleicad eecs. umich. edu//K//MCCChhhcch/
http://vlsicad.c eecs. umich.
7

---

wio ain partner
w/o ain partner
11
11
10
11
14
14
13
16
13
16
21
15
15
28
24
20
26
15
28
24
20
23
26
Die
9
30
30
12
10
22
(a) Floorplan of circuit n50
(b) Floorplan circuit n100
Figure 3: Our 3D floorplan result. Two blocks with the same index and the color different dies form an alignment pair, which roughly locate on the same positions and share 2D common projected area. Gray blocks mean they do not have alignment partners.
4.2 Comparison with Baselines
Methods including heuristic-based and learning-based approaches are selected as baselines, and the implementation details are shown in Appendix E Average alignment score and total HPWL are employed evaluation metrics. Results are shown Table and 3 For alignment score, our approach achieves 0.940, significantly surpassing the second-best best method which scores 0.474. For HPWL, our approach achieves average reduction 16%. Our method reduces HPWL 249,053, and the second-bor method (Wiremask-Bl (tremaskk-BBO) reduces it to 296,791. However, it only achieves 0.222 for alignment score, failing to effectively tackle the alignment constraint, empirically showing that approach is capable of address alignment and HPWL simultaneously. Demonstration of floorplanning results by our method are also shown in Fig. 3 and Fig.
also compare Capability and flexibility
performances on overlap, out-of-bound and runtime, shown in Appendix F of our approach to address pre- pre-placee placed modules (PPMs) are also demonstrated in Appendix F.5
Aigred
Not Aligred
without ain parther
Not Alinndd
Aligned
without ain pattner
Die
8
10
10
(a) GraphPlace, alignment score = 0.40
(b) DeepPlace, alignment score 0.31
Aligned
Alignee
without partner
Aligned
Without ain partner
B
Dic
7
Die
10
(c) Wiremask-BBol alignment 0.35
(d) Ours, alignment
0.98
Figure 4: Visualization of cross-die block alignment circuit n50. Two blocks with the same index forms an alignment pair. For pair with block j. we calculate individual alignment score aln(i, j) according to Eq. 1. Green means these two blocks are aligned (aln(i,j) 2 0.5) while red means not aligned (aln(i,) J) 0.5). Total alignment score is calculated according to Alg i Appendix G.1 demonstrates that our method achieves much better alignment score than other baselines.

---

165
Finetuna From scratch
5.75
Rinetune
3.8
0.9
5.50
From scratch
Algrmert
Afgnmert
0.8
3.6
5.25
0.7
3.4
5.00
0.6
Finelune
From scratch
3.2
0.5
From scratch
500
1000
500
1000
200
400
600
200
400
600
Epoch
Epoch
Epoch
Epoch
(a) Training curve on circuit n200.
(h) Training curve on circuit n3300.
Figure 5: Training curve between fine-tune (based on circuit n n100) and training from scratch.
Table 4: Zero-shot transferability evaluation (training on circuit n100).
Mctric/Circuit
ami33
ami49
n10
n50
n200
n300
value
0.859
0.894
0.875
0,947
0,970
0.877
0,908
Alignment (t)
ratio
0.949
0.936
0.954
1.029
1.000
0,950
0,941
value
59,923
835,170
30,720
87,784
111,039
322,242
462,780
HPWL (U)
ratio
1.027
1.095
1.032
1.046
1.049
1.019
1.008
Table 5: Ablation study on n100. sync: synchronous die-by-die placing. w/o aln: remove alignment
mask in vision modality. w/o seq: remove sequence modality. w/o graph: remove graph modality.
sparse rew: the same reward as GraphPlace [26). diff rew: the same reward as MaskPlace (28).
MetrieMMehod
syne
w/o als
w/o graph
wlo sog
sparse rew
rew
Ours
Aligament HPWL
0.85000.042
0.349:0.0026
0.87410.027
0.860 186,005:1846
0.840=00118 189,21941.164
0,731#00054 190.184-2,192
0.917
185.624
187,61743,096
1.8,079
176,639.11000
4.3 Transferable 3D Floorplanning
The zero-shot transferability of our method is also evaluated. We firstly train the model on circuit n000, and directly test the performance without any fine-tuning. In Table 4 ratio is calculated between running inference and training on corresponding circuit. It demonstrates that our model exhibits a good zero-shot transferability, demonstrated through the training curves between fine-tuning (based on pre- pre-trained weights of circuit nl00) and training from scratch. In Fig. []] through fine-tune technique, better or similar performance can be achieved, conserving substantial training resources.
either smaller larger cases. The transferability is also
4.4 Ablation Studies
We ablate effectiveness of each component in our approach, including asynchronous layer decision, multi-modality representation and reward function. Experiments on circuit n n100 are shown in Table Removing the alignment mask leads a huge drop of alignment score, from 0.961 0.349. We also evaluate the effectiveness of incorporating the alignment mask as an input feature and a constraint. In Fig. [aa] 'input" refers the scenario where we feed the alignment mask as input to the model, while 'constr. indicates that the alignment mask is utilized to filter out invalid positions that do not satisfy the alignment constraint. As the input, alignment mask plays a eritical role to effectively capture the alignment information. As the constraint, it reduces the action space and accelerates training process.
Shown in the column 'sync' in Table fixed synchronous die-by-die block placing order performs worse than asynchronous layer decision, since in the latter, more flexibility is provided to plan the entire placing order. Our reward design, incorporating the local advantage and global baseline, is crucial for fully leveraging the asynchronous layer decision mechanism. In Fig. 6b with the guidance of our reward, the layer decision module keeps active to learn an optimal placing order. However, with other reward design schemes, is faced with either rapid convergence to the degeneration of synchronous bie-by--de decision-making or unstable oscillation. Under reward function 'diff' in MaskPlace (28), only the difference between two consecutive steps focused, while the global reward is disregarded, leading to rapid convergence to the degeneration of synchronous die-by die-by-die layer decision. Under reward function 'sparse' in GraphPlace (26), rewards for intermediate steps are all zeros, and only the reward of the final step is non-zero. The policy lacks the modeling of local information, resulting in its inability to accurately assess the influence of current action on the entire decision-male making process.

---

input constr.
constr. only both N/A
diff
sparse
ours
uniform
input only
1.0
Sum of first half seq (2)
40
0.8
Alignment
0.6
30
0.4
20
0.2
10
0.0
200
400
600
800
1000
500
2000
Epoch
Epoch
(a) Effectiveness of the alignment mask.
(b) Effectiveness of rewards on layer decision.
Figure 6: (a) Effectiveness of the alignment mask. As the input feature, plays a critical role in capturing the alignment information. As the constraint (constr.). it reduces the action space and accelerates the training process. (b) Effectiveness of rewards on layer decision, shown in circuit n00 with episode length L = 100 and [D]
dies. 2ς is the determined layer index at step and note Z = D[/ 2t. + 0 or Z - L/2 means degeneration to die-by-die synchronous layer decision (almost all die 0 or in the first half episode).
5 Related Works
Classical methods in FP can roughly categorized into heuristics and analytical approaches. The former typically models FP with certain representation, such B*-tree (17), Corner Block List [18]
and Sequence Pair (39, based on which heuristic algorithm (especially Simulated Annealing [40) is
utilized to search for optimal solution. Finally, the representation is converted to corresponding FP
result via a decoding scheme. Apart from the heuristics, analytical approaches [9] 22) 41 regard the
FP problem as an electrostatic system 442 43). They compute the gradient of objective functions w.r.t. block coordinates, and utilize gradient descent- -based algorithm to optimize the solution.
RL-based floorplanning approaches. Recently, floorplanning also attracted attention from the rein- forcement learning communities. For instance, [23] 24 (25) incorporate traditional FP representation and RL. Among these methods. at each step, the policy network either 1) decides whether to accept the randomly perturbed state or not [23 25 or 2) determines how to perturb the current state [24).
26 17 28
primarily focus on making decisions about block position in 2D scenarios with RI..
GraphPlace [26] and DeepPlace (77 incorporate graph and vision as representation. MaskPlace 28
employs visual representation, and designs a dense reward function based on partial HPWL.
RL with hybrid action space. Common RL can be categorized into discrete and continuous action spaces. However, in certain scenarios, besides the discrete action, we need to make decisions regarding its parameters, typically residing in continuous spaces [36. Consequently. it gives rise to hybrid action space. [44] proposes to discretize the continuous portion, which creates large 47 48) convert the discrete action selection into continuous space with actor network and employ DQN-based
discrete set and sacrifices aen-igrined control. Alternatively, [5]
44) 50)
algorithm for training, but found to be unstable and inefficient. (36 suggests to use multiple policy heads consisting of one for discrete actions and the others for corresponding continuous parameters separately. [51) proposes to encode the hybrid action space to a continuous space with GAE [37]
Conclusion
In this paper, we propose FlexPlanner, flexible learning-based based method in hybrid action space with multi-modality representation for 3D floorplanning task. It involves no reliance on heuristic- based search, thus achieves better flexibility to tackle position, aspect ratio and cross-die alignment for blocks under complex constraints. Flex outperforms baselines in alignment score and wirelength, and it also demonstrates zero-shot transferability on unseen circuits. This paper also has some limitations for future work: further optimization in 3D FP could be involved such as thermal optimization. Our method has no potential harm to the public society at the moment.
10

---

References
[1] Yao Lai, Sungyoung Lee, Guojin Chen, Souradip Poddar, Mengkang Hu, David Z Pan, and Ping Luio. Analogcoder: Analog circuit design via training-free code generation. arXiv preprint wwXiv:2405.1491 2024.
[2] Yao Lai, Jinxin Liu, David Pan, and Ping Luo. Scalable and effective arithmetic tree generation for adder and multiplier designs. arXiv preprint Sakii::20050667555 2024.
[3] Lei Chen, Yiqi Chen, Zhufei Chu, Wenji Fang. Tsung- Yi Ho, Ru Huang, Yu Huang, Sadaf Khan, Min Li, Xingquan Li, et al. Large circuit models: opportunities and challenges. Science China Information Sciences, 2024.
[4] Chen Wang. Jingkun Mao, Giuseppe Selli, Shaofeng Luan, Lin Zhang, Jun Fan, David J Pommerenke, Richard E. DuBroff, and James I. Drewniak. An efficient approach for power delivery network design with closed- expressions for parasitic interconnect inductances. IEEE Transactions on Advanced Puckaging, 2006.
[5] Chung- Kuan Cheng, Andrew B Kahng, Ilgweon Kang, and Lutong Wang. Replace: Advancing solution quality and routability validation in global placement. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018.
[6] Ruizhe Zhong, Junjie Ye, Zhentao Tang, Shixiong Kai, Mingxuan Yuan, Jianye Hao, and Junchi Yan. Preroutgnn for timing prediction with order preserving partition: Global circuit pre-training, local delay learning and attentional cell modeling. In Proceedings of the AAAI Conference on Artificial Intelligence. 2024.
[7] Jinwei Liu, Chak-Wa Wa Pui, Fangzhou Wang, and Evangeline FY Young. Cugr: Detailed--outabiiit-tiivee 3d glohal routing with prohabilistic resource model. In 2020 57th ACM/IEEE Design Automation Conference (DAC), 2020.
[8] Xingbo Du, Chonghua Wang, Ruizhe Zhong, and Junchi Yan. Hubrouter: Learning global routing via hub generation and pin-hub connection. Advances in Neural Information Processing Systems, 36, 2023.
[9] Ximeng Li, Keyu Peng, Fuxing Huang, and Wenxing 7hu. Pef: Poisson's equation based large-scale fixed-outiine floorplanning. IEFE Transactions Computer-Aided Design of Integrated Circuits and Systems, 2022.
[10] Jill HY Law, Evangeline FY Young, and Royce LS Ching. Block alignment 3d floorplan using layered teg. In Proceedings of the 16th ACM Great Lakes symposium on VLSI, 2006.
[11] Johann Knechtel, Evangeline FY Young, and Jens Lienig. Planning massive interconnects in 3-d chips. TCAD, 2015.
[12] Anthony Agnesina, Moritz Brunion, Jinwoo Kim, Alberto Garcia-Orte Dragomir Milojevic, Francky Catthoor, Gioele Mirabelli, Manu Komalan, and Sung Kyu Lim. Power, performance. area, and cost analysis of face-to-face face bonded 3d ies. IEEE Transactions Components, Puckaging and Manufacturing Technology, 2023.
[13] Lennart Bamberg, Alberto Gareía-Ortiz, Lingjun Linguuu Zhu, Sai Pentapati, Sung Kyu Lim, al. Macro-3d: physical design methodology for face-to-face- heterogeneous ics. In 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2020.
[14] Sai Pentapati, Anthony Agnesina, Moritz Brunion, Yen- Ven-Hsiang Huang, and Sung Kyu Lim. On legalization of die honding bumps and pads for 3d ics. In Proceedings of the 2023 International Symposium on Physical Design 2023.
[15] Sankatali Venkateswarlu, Subrat Mishra, Herman Oprins, Bjorn Vermeersch, Moritz Brunion, Jun-Han Han. Mircea R Stan, Dwaipayan Biswas, Pieter Weekx, and Francky Catthoor. Impact of 3-d integration on thermal performance risc-v mempool multicore soc, IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2023.
[16] David Koblah, Rabin Acharya, Daniel Capecci, Olivia PrroonParaddi, Shahin Tajik, Fatemeh Ganji, Damon Woodard, and Domenic Forte. A survey and perspective on artificial intelligence for security-aware electronic design automation. ACM Transactions on Design Automation of Electronic Systems, 2023.
[17] Yun- Chi-Chih Chang. Yao Wen Chang, Guang-Min Ming Wu, and Shu- Wei Wu. Bretress: A new representation for non- non-slicing floorplans. In Design Automation Conference (DAC), 2000.
11

---

[18] Jai- Jai-Ming Lin, Yao-Wen Chang, and Shih-Ping Lin. Corner sequence-a p-admissible floorplan representation with worst case linear-time packing scheme. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2003.
[19] Xianlong Hong, Sheqin Dong, Gang Huang, Yici Cai, Chung-Kun Kuan Cheng, and Jun Gu. Comer block list representation and its application floorplan optimization. IEFF Transactions on Circuits and Systems II: Express Briefs, 2004.
[20] Hai Zhou and Jia Wang. Acg-adjacent constraint graph for general floorplans. In IEEE International Conference on Computer Design: VLSI Computers and Processors, 2004.
[21] Paul Falkenstern, Yuan Xie, Yao-Wen Chang, and Yu Wang. Three-dimensional integrated circuits (3d ic) floorplan and power/ground network co-synthesis. In Asia and South Pacific Design Automation Conference (ASP-DAC), 2010.
[22] Fuxing Huang, Duanxiang Liu, Xingquan Li, Bei Yu, and Wenxing Zhu. Handling orientation and aspect ratio of modules in electrostatics bassod based large scale fixed-outline floorplanning. In IEEE/ACM International Conference on Computer Aided Design (ICCAD), 2023.
[23] Qi Xu, Hao Geng. Song Chen, Bo Yuan. Cheng Zhuo, Yi Kang. and Xiaoqing Wen. Goodfloorplan: Graph convolutional network and reinforcement lcarning-based floorplanning. IEFE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2021.
[24] Mohammad Amini, Zhanguang Zhang, Surya Penmetsa, Yingxue Zhang, Jianye Hao, and Wulong Liu. Generalizable floorplanner through corner block list representation and hypergraph embedding. In SIGKDD, 2022.
[25] Wenbo Guan, Xiaoyan Tang, Hongliang Lu, Yuming Zhang, and Yimen Zhang. Thermal-aware fixed- outline 3-d floorplanning: An end-to-end learning-based approach. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2023.
[26] Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan, Joc Wenjie Jiang, Ebrahim Songhori, Shen Wang, Young-Joon Lee, Eric Johnson, Omkar Pathak, Azade Nazi, et al. A graph placement methodology for fast chip design. Nature, 2021.
[27] Ruoyu Cheng and Junchi Yan. On joint learning for solving placement and routing in chip design. NeurIPS, 2021.
[28] Yao Lai, Yao Mu, and Ping Luo. Maskplace: Fast chip placement via reinforced visual representation learning. NeurlPS, 2022.
[29] Yao Lai, Jinxin Liu, Zhentao Tang, Bin Wang, Jianye Hao, and Ping Luo. Chipformer: Transferable chip placement via offline decision transformer. JCML, 2023.
[30] Yunqi Shi, Ke Xue, Song Lei, and Chao Qian. Macro placement by tee--mask-guided black-box optimiza- tion. NeurlPS, 2023.
[31] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurlPS, 2017.
[32] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.
[33] Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov, Exploration by random network distillation. arXiv preprint 1810v: 12894, 2018.
[34] Vijay Konda and John Tsitsiklis. Actor-critic algorithms. NeurIPS. 1999.
[35] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint 170. 2017.
[36] Zhou Fan, Rui Su, Weinan Zhang, and Yong Yu. Hybrid actor-critic reinforcement learning in parameterized action space. In IJCAI, 2019.
[37] John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abhecl. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv: 150.0038, 2015.
[38] Jiayi Weng. Huayu Chen, Dong Yan. Kaichao You, Alexis Duburcq, Minghao Zhang, Yi Su, Hang Su, and Jun Zhu. Tianshou: A highly modularized deep reinforeement learning library. Journal of Machine Learning Research, 2022.
12

---

[39] Hiroshi Murata, Kunihiro Fujiyoshi, Shigetoshi Nakatake, and Yoji Kajitani. Visi module placement based rectangle-packing by the sequence-pair. IEEE Transactions on Computer-Aided Design Integrated Circuits and Systems, 1996.
[40]
Dimitris Bertsimas and John Tsitsiklis. Simulated annealing. Statistical science, 1993.
[41] Xingbo Du, Ruizhe Zhong, Shixiong Kai, Zhentao Tang, Siyuan Xu, Jianye Hao, Mingxuan Yuan, and Junchi Yan. Jigsawplanner: Jigsaw-like floorplanner for eliminating whitespace and overlap among complex rectilinear modules. In 2024 IEEE/ACM International Conference on Computer Aided Design (ICCAD), pages 1 9. ACM, 2024.
[42] Jingwei Lu, Pengwen Chen, Chin- Chang, Lu Sha, Dennis Jen-Hsin Huang. Chin-Chi Teng, and Chung-Kuar Kuang--Kunnna Cheng. eplace: Electrostatics-base placement using fast fourier transform and nesterov's method. ACM Transactions on Design Automation of Electronic Systems (TODAES), 2015.
[43] Yibo Lin, Shounak Dhar, Wuxi Li, Haoxing Ren, Brucek Khailany, and David Z Pan. Dreamplace: Deep learning toolkit-enabled gpu acceleration for modern vlsi placement. In Proceedings of the 56th Annual Design Automation Conference 2019, 2019.
[44] Alexander A Sherstov and Peter Stone. Function approximation via tile coding: Automating parameter choice. In International symposium abstraction, reformulation, and approximation. 2005.
[45] Matthew Hausknecht and Peter Stone. Deep reinforcement learning in parameterized action space. arXiv preprint 15
2015.
[46] Warwick Masson, Pravesh Ranchod, and George Konidaris. Reinforcement learning with parameterized actions. In Proceedings of the AAAI conference on artificial intelligence, 2016.
[47] Jiechao Xiong, Qing Wang, Zhuoran Yang, Peng Sun, Lei Han, Yang Zheng, Haobo Fu, Tong Zhang, Ji Liu, and Han Liu. Parametrized deep gentwwrrks learning: Reinforcement learning with discrete-continuout hybrid action space. arXiv preprint arXiv: 1810.0639, 2018.
[48] Craig Bester, Steven D James, and George D Konidaris. Multi-pass g-networks for deep reinforcement learning with parameterised action spaces. arXiv preprint arXiv:I905. 003884 2019.
[49] Volodymyr Mnih, Koray Kavukcuoglu. David Silver, Alex Graves, loannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602
2013.
[50] Timothy Lillicrap, Jonathan Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. In ICLR, 2016.
[51] Boyan Li. Hongyao Tang, YAN ZHENG, HAO Jianye. Pengyi Li, Zhen Wang, Zhaopeng Meng, and LI Wang. Hyar: Addressing discrete-continuous action reinforcement learning via hybrid action represen- tation. In ICLR, 2021.
[52] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. NeurIPS, 2019.
[53] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
[54] Xi Chen, Yan Duan. Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. NeurIPS. 2016.
[55] Bing Xu. Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.0088: 2015.
13

---

A
Notation
All notations this paper are shown Table
Table 6: Notation.
Notation
Meaning
block
I/O port (terminal) die/layer
d
D
the set of all dies/layers width, height of a block area of block aspect ratio of a block
w,
AR
2,y
2D coordinate of a block/port the layer/die index of block/port
W, H
width, height of die/layer overlap area between block by bj alignment score between block b; bj required minimum alignment area between block b; bj the block placing order FIFO queue of die d,
Oij
aln(i.
alum (i,j)
fa
alignment mask canvas mask
fe
fro
wire mask position mask
fp
B Statistics of Benchmark
We evaluate the performance of FlexPlanner and other typical methods on public benchmark MCNC and GSRC shown in Table The 'alignment" means the number of blocks with alignment partner.
Table 7: MCNC and GSRC benchmark
circuit
block
I/O port
net
alignment
ami33
33
40
121
20
ami49
49
22
396
20
n10
10
69
118
10
n30
30
212
349
20
n50
50
209
485
30
n000
100
334
885
60
n200
200
564
1585
60
n300
300
569
1893
60
C Implementation Details
C.1
Computational Resources
We use PyTorch deep learning framework and tianshou 68 Reinforcement Learning framework. We optimizer with learning rate 0.0001. We train and test our model on Linux server with one NVIDIA GeForce RTX 3090 GPU with 24 GB CUDA memory, two AMD Ryzen Thrcadripper 3970X 32-Core Processors at 3.70 GHz and 128 GB RAM.
select Adam 53
http://vleicad. eecs. umich http://vlsicad. eecs umich, edu/ Bad//B///// GSRCbench/
14

---

C.2 Minimum Alignment Requirement Configuration
For the alignment setting, given alignment pair (b;, by), the minimum alignment requirement alt (1, is alum (i.j) = dig min min(a,, az}. where djs aj are the of block by bj. The coefficient orj controls minimum alignment requirement between block b,, by. and can be adjusted according cireumstances for each alignment pair. In our experiments, we set all cxy = 1.0, which is the most challenging scenario to evaluate the effectiveness of our approach.
specitic
C.3 Hyper-Parameters
Other hyper-paramette are shown Table g
Table 8: Hyper-parameters.
Argument
Value
learning rate
0.0001
parallel environments ne buffer size
8
ne x len (enepsoode)
reward weight for alignment w.
0.5
reward weight for HPWL wy
1.0
reward weight for overlap w.
0.5
area utilization
85%
clip loss weight for position decision Å
1.0
clip loss weight for layer decision X2
1.0
clip loss weight for ratio decision \3
0.5
range of block aspect ratio
[
batch size die width W die height H
128
128
128
number of dies [D]
2
value loss weight A number of epochs
0.5
1,000
number of update epochs
10
clip
0.2
reward discount factor
0.99
GAE
0.95
D Model Architecture
We introduce the networks in our pipeline shown in Fig. 1 follows:
Shared Encoder.
The shared encoder Ea mainly consists of two parts: 1) a CNN-based backbone for floorplanning vision modality input, and 2) a GNN (Graph Neural Network)-based backbone Graph Attention Network 332 with two layers for graph modality.
In step f, given current block b,, we concatenate the alignment mask fl canvas mask f wire mask f and position mask f( together. We also incorporate f ((t+1) ff(++) of header block in FIFO queue q of each die. These masks of queue header blocks represent all possible choices in step + 1. providing policy with future horizon. All of these input masks constitute the floorplanning vision modality.
Taking the netlist graph input, the GNN backbone outputs nodes embeddings. representing the local receptive field information. Besides, global average pooling layer applied on these node embeddings calculate the graph embedding, which captures the glohal view of the whole netlist graph.
Critic Network. The critic network V& mainly consists of three parts: 1) the shared encoder F.g, 2) a sequence Transformer (3I) with two encoder layers and two decoder layers for block placing order sequence, and 3) die/layer embedding model to represent each die/layer with learnable feature vector. The shared encoder Ep is responsible for processing the floorplanning vision input, and the Transformer is for the input sequence modality. Serving as the source sequence in Transformer, the multi-die sequence feature S E E |DXXXXC is employed in the calculation of memory, which is the output of self-attentin And the single block feature
15

---

by E B is treated as the target sequence with length 1, serving the query in cross-attention. To Stabilize training. the shared encoder Eg is only updated during the updating process of critic network. As result, the critic network is responsible to take input as the state 8; and output corresponding state value vx.
Policy Network.
(1) mainly consists of generator network. It takes input as Position Decider. The position decider network T6 the output of the shared encoder Fig, and outputs a 2D feature map as the probability matrix determine 2D position for current block. We borrow the generator architecture from InfoGAN (54), which utilizes up-sampling layer instead of transposed convolutional layer to realize scaling-up. It contains three blocks and
each block is with convolutional layer, batch normalization, leaky ReLU [55] activation and up-sampling layer. Finally, it outputs a prohability matrix M E R x to determine the position (Tr, ye) for current block by.
Layer Decider. The layer decider network Fq mainly consists of three parts: 1) a CNN-based model for FP vision modality, 2) a die embedding model represent current input die, and 3) Transformer network to process the input block placing order sequence for cach die, The CNN model consists of three blocks, and cach block has one convolutional layer, one max pooling layer, with ReLU as activation function. The Transformer model consists of two encoder layers and two decoder layers, and is designed process the input multi-die block placing order sequence. The layer decider network takes input as the state 8, and outputs probability vector p
R (PI
to determine the layer 2++11 for accessing next block bz+1.
Aspect Ratio Decider. The aspect ratio decider network (3) mainly has a CNN model, consisting of three 2D convolutional layers with ReLU activation function and max pooling layer. It takes input as the floorplanning vision modality in state s and the feature of next block 61+1 determine the aspect ratio AR241 for b Since the aspect ratio an action in contiguous space, we select Gaussian distribution to depict it, and the network outputs corresponding mean value and standard deviation.
E Baselines
The baselines referred in Sec. 4 are introduced as follows:
3D-B*-SA 21 is heuristic-basoe approach, which represents 3D floorplanning by B*etree [17. Simulated Annealing (SA) 40 is selected to search an optimal result. At each iteration, current Btetree is randomly
perturbed to new state, which will be accepted based on a certain probability. Finally, the B is converted to corresponding floorplannning result via decoding scheme. In previous scenarios, only HPWL. and out-of-hound penalty are incorporated into heuristies (cost/energy function). To optimize the alignment in 3D FP, we also combine the alignment score into its heuristics to guide the solution searching.
RL-CBL 24 combines mouristcc-bssed search Comer Block List (CBL) 18 and Reinforcement Learning together, utilizing policy network to determine how to perturb current FP at each step. It utilizes intermediate out-of-bound penalty and final HPWL reward function. In 3D FP with multiple stacked layers, we extend CBL to 3D scenario. In order to rcalize the optimization of cross-dic block alignment, also incorporate alignment score into reward function. However, it incapable of addressing the variable aspect ratio of soft blocks.
Wiremask- BBO 30 is black-box optimization (BBO) framework, by using a fire-mask-guider 28 greedy procedure for objective evaluation. At each step, it utilize the position mask to filter out invalid positions causing overlap or out-of-boumt selected to place current block. Considering the alignment requirement in 3D FP, we incorporate the alignment mask to further filter invalid positions which do satisfy the alignment constraint. However, is incapable of addressing variable aspect ratio of soft blocks.
Within the valid positions, the location with the minimum increment of HPWL is
GraphPlace 26 DeepPlace 277. Mask 28 are three learning- based approaches, utilizing Reinforce- ment Learning to decide the position for each block on 2D chip layout. Variable aspect ratios of soft blocks are not taken into consideration. In this task, we extend them to 3D scenarios, and incorporate the alignment mask to filter out invalid positions where alignment constraint is not satisfied. Besides, alignment score is also involved into the calculation of reward function, with the purpose of assisting them realize the optimization of cross-die alignment.
F
Addtional Experiments
F1
Runtime Comparison
Comparison of runtime among our approach and other baselines are shown in Table The runtime of our method
is either lower or comparable to other learn- methods (26] 28). 1 and is better than the heuristic-basse
method 21.
where tens of thousands of iterations are required. Pipeline of Wiremask-BBi (30 is the same
16

---

MaskPlace (2)] while neural network is not involved in it, contributing to the shortest inference time. For the approach RL-CBL 24), is faster than other learning-basee based methods since its environment is relatively simple, only responsible for decoding the heuristic representation Corner Block List (CBL) [8 19 into corresponding FP result. However, the performance is also limited by CBL, and the final FP result is further worse than other methods without the reliance of heuristic representation.
Table 9: Runtime (second) comparison among baselines and our method. The lower the runtime, the better. The unit of runtime in seconds. C/M means Circuit/Method.
CM
3D-B*-SA 2
RL-CBL
Wiremask-BBO 30
GraphPlace 2
DeepPlace
MaskPlace 28
Ours
ami33
18.374
1.131
0.228
3.155
1,819
4.679
2.419
ami49
31.917
1,308
0.426
2.782
2.736
3.593
2.219
nl0
6.679
0,965
0.094
1.813
1.330
2.682
2.026
n30
21.944
1.122
0,262
1.867
2.456
2.256
2.709
35.385
1.323
0,427
4.778
3.599
2.342
2.533
n100
65.971
2.575
1,027
4,408
4,259
4.593
3.975
127.140
2,583
2.477
5.915
6.187
5,475
7,991
n300
176.222
3,599
4.492
10.225
9.521
10.128
10.261
Avg.
60.454
1,826
1.179
4.368
3.989
4.469
4,266
F.2
Out-of-Round
the hcuristic representation will be converted to corresponding floorplan result through decoding scheme. Although non-overlap can be guaranteed, blocks the final FP could be out the fixed outline, leading to the low area utilization chip die and out-of-bound. In our method FlexPlanner, each block can be naturally placed within the boundary due to the alignment mask and position mask. Consequently, the occurrence of out-of-hound floorplan is effectively prevented. We compare the out-of-bound in Table TO and the outbound is calculated follows:
For heuristics-basse methods, such as 3D-B*-SA 21 and RL-CBL 24
mAX ( w/ b:EB
Vo
HIRX [y: + 14)
(6)
max (0, Tm. W}
max {0, Vm H
outbound
2W
2H
Table 10: Comparison of out-of-bounde the lower the better.
Circuit/Outounnd
3D-B*-SA
21
RL-CBL. (4
Ours
ami33
0.000:10.000
0.087:+0.043
0.000=0.000
ami49
0.000.10.000
0.213.10.059
0.00010.000
n10
0.006110.011
0.026110.31
0.000110.000
n30
0.000++0.000
0.116+00.025
0.000+0.000
n50
0.000.10.000
0.156110.052
0.00010.000
n n000
0.000++0.000
0.223++0.081
0.000=00000
n200
0.008:10.014
0.359410.177
0.00010.000
n300
0.014:10.009
0,549:+00097
0.00010.000
Average
0.004
0.216
0.000
F33
Overlap
We also compare performance overlap among our method and baselines, shown in Table Our method reduces the overlap to 0,001, which comparable with 3D-B*-SA [21] RL-CBL (24], and better than other methods. In 3D-B*-SA (D and RL-CBL (24), although non-overlap
guaranteed, blocks in corresponding
FP could out the fixed outline, leading to the low area utilization of chip and out-of-bound as discussed in Appendix F2
F,4 Inffuence of Grid Size
Due to the different size circuits, we discretize the sizes into uniform region. Given a chip die with original shape W. x H.. the region is projected into W H. For each block with shape w its shape also We also empirically investigate the
projected into max round(w W.
max round(h H
H. }
W.
17

---

Table 11: Overlap comparison among baselines and our method. The lower the overlap, the better.
C/M
3D-B"-SA
21
RL-CBI 24
Wiremask-BBO 30
GraphPlace 2n
DeepPlace 2
MaskPlace
25
Ours
ami33
0.000200000
0.000
0.0501000227
0.037:20025
0.02420.016
0.012:40009
0.000=000000
ami49
0.000+0.000
0.000=0.000
0.000=0.001
0.002=00.022
0.001=0.001
0.009+00008
0.000=00000
n10
0.000:10.000
0.00020.000
0.20810.005
0.229:000447
0.112::0028
0.085=20.065
0.009=0.000
n30
0.000+0.000
0.000=0.000
0.007=0.007
0,042=0.0228
0.037=0.022
0.016+0.000 005
0.000=0.000
n50
0.000++0.000
0.000=0.000
0.000+000000
0.009+0.014
0.029=00.020
0.004++.007
0.000=0.000
n100
0,00010.000
0.000.0.000
0.00010.000
0.000.0.0000
0.002.10.001
0.000.10.000
0.000.10.000
2200
0.000110.000
0.0002000000
0.0002000000
0.0001000000
0.0001:00000
0.000:::0000
0.000:000000
n300
0.000#0.000
0.000=0.000
0.000=0.000
0.00000.000
0,000=00000
0,000=0.000
0.000=0.000
Avg.
0.000
0,000
0.033
0.040
0.026
0.016
0.001
influence of different grid size W, H, shown in Table projected region and the original region follows:
12
Metric 'Error' evaluates the error between the
1
We W
x h H
x ho!
Error
18
Wo ho
BEB
where is of blocks. Similar performances terms of alignment, HPWL, and overlap are achieved across different grid number settings. Lower grid numbers offer faster execution speeds but may result in relatively higher area crrors that may not meet precision requirements. Higher grid numbers produce more fine-grained floorplanning results but lead larger action space and require additional computational resources. As trade-off between runtime and precision, grid size of 128 is selected.
Table 12: The influence of different grid size on circuit n100.
Grid/Metric
Alignment
HPWL
Overlap
Runtime (s)
Error
32
0.903
179,371
0.000
3.981
0.106
64
0.936
172,159
0.000
4.122
0.052
128
0.941
176,320
0.000
4.417
0.024
256
0.961
176,375
0.000
4.572
0.013
512
0.937
178,395
0.000
5.948
0.007
F,5 Floorplanning with Pre-Placed Modules
Our approach also capable of addressing the circuits with pre-placed modules (PPMs). PPMs are blocks whose position and aspect ratio are pre-determined and fixed during the FP process. Hcuristics-based methods [21] 24 are incapable of address the existence of PPMs. In these heuristics approaches, after the perturbation is applied current FP representation, the entire FP layout result may undergo changes. During this process, it cannot be guaranteed that the positions and shapes of pre-placed modules will remain unchanged. Consequently. they lack the flexibility to address circuits with PPMs. Our method FlexPlanner is capable to solve this issue. At the beginning of FP process (the beginning of an episode), all PPMs are placed on their pre-determined positions, and their aspect ratios keep unchanged. After that, modify the canvas mask and position mask, indicating that these positions have been occupied by PPMs. In each step place movable block bt, the overlap among b, other blocks (PPMs and other placed blocks) can avoided. The floorplanning result with PPMs shown in Fig.
G Algorithms
G.1 Algorithm for Calculation of Alignment Rate and Overlap
calculation of alignment score alm, and overlap of step I are shown Alg. and Alg These values are involved in the calculation of reward function with local advantage and global baseline shown in Alg.
G.2 Training Algorithm
The overall training pipeline shown in Alg. based on PPO 335 algorithm with RL framework tianshou (38 During a training epoch, the policy network collects training data into replay buffer by interacting with the envi- ronment. To accelerate the training process, the policy network sinultancously interacts with environments
18

---

Pre-Places Module (PPM)
w/o aln partner
3
4
4
1
5
5
Die
7
9
14
Die
9
14
8
13
10
2
8
13
10
11
15
12
11
15
12
(a) Circuit with 6 PPMs.
Pre-Placed Module (PPM) w/o aln partner
4
9
2
9
2
27
28
20
28
20
13
15
10
21
29
13
15
10
21
29
O
18
23
8
18
23
Die
11
Die
11
14
19
26
5
14
19
26
16
25
30
16
25
30
12
24
12
22
24
17
7
6
1
17
7
(b) Circuit n100 with 12 PPMs.
Figure 7: Our 3D floorplan result. Blue blocks are PPMs whose position and aspect ratio are fixed during FP. Two blocks with the same index on different dies form an alignment pair, which roughly locate the same positions and share 2D common projected area.
Algorithm 3: Alignment score calculation.
Input: Current block b, alignment score aln,-1 at step - 1, number of alignment pairs Output: Alignment score aln, at step aln, = aln,-1
b; has alignment parnter b, and both b, by have already been placed then aln, += aln(f.)) // according to Eq.
end
Algorithm 4: Overlap caleulation.
Input: Current block b, canvas mask fer chip die width W and height H Output: Overlap o, step t
fo fe T; Iz + Wi, y/ + hi] calculate overlap o follows:
+ 1
Σ Ee,o.r max (0, fa(2, v) 1}
0
WH
19

---

in parallel. We set the buffer size to satisfy
Louf 0 mod (ne len (episode)),
(8)
ensuring that each episode in the replay buffer is completed and terminated with its final step, where T is the size of replay buffer. After data collection, we further compute reward, cumulative discounted reward and advantage. Since each episode is completed in the replay buffer, each step has the corresponding terminated step (end step) within the same episode to calculate the reward with local advantage and global baseline. Finally, both critic network and policy network are updated.
20

---

Algorithm 5: Training Algorithm.
for epoch from 1 to n_epochs do Il data collection
while replay buffer is not full do
// action probability calculation
calculate action probability for position of current block by: To 1ss
sample (Ix, Wr)
(1 (sc)
(1)
calculate action probability for layer next block b441: no (se
sample 1 πT
d
(2)
(8t)
calculate action probability for aspect ratio of next block b1+1: no aa (3)
1st
sample AR To ar |8t
// action execution and next state observation exccute the action ay:
1. place block b, at position (Iz, V4)
2. access next block b from the FIFO queue le44 of layer #++1
3. set the aspect ratio of b4+1 to A
observe next state 8141 from environment get alignment score aln, wirelength HPWL,, overlap or from environment
add sample (s, (r, y. AR), (aln, HPWL, o), (Tas""" "Bola" ア (all)) into buffer
(1) (2)
(2)
end
// data processing for each sample in buffer do re-compute reward r with local advantage and global baseline according Alg. calculate state value for current state s: (8) calculate state value for next state s': v' = V (s')
modify current sample with adding (r, (7
end
for each sample in buffer do compute cumulative discounted reward G and advantage À for current sample according
to generalized advantage estimation (GAE) 37 35)
A. - (x1)' where 6, = Tt + y 1/
G; A.+ modify current sample with adding (Á, G) end
!| network update
for update_epoed from to pr_uddate_ppocke do for minibatch in buffer do
// policy/actor network update
compute action probability Th n T
(1)
(2) (3)
compute action distribution entropy hig hg hg
(1) (2) (3)
update policy/actor network according to Eq. // critic network update compute state value = (s))) update critic network according to Sec. 3.4
end end
end
21

---

NeurIPS Paper Checklist
1. Claims
Question: the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We make clear claims about contributions and scope of our paper.
Guidelines:
The answer NA means that abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match thcorctical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the liminations in Sec. i
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed the paper.
The authors are encouraged create separate "Limitations" scction their paper.
The paper should point out any strong assumptions and how robust the results these assumptions (e.g.. independence assumptions, noiseless settings, model well well-specifiennt specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
violations of
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should he articulated.
The authors should reflect the factors that inffuence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
22

---

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have provided detailed hyper-parameters settings in Appendix and the full training pipeline in Alg. Alg-5lin Appendix G.T
Guidelines:
The answer NA means that the paper does not include experiments. the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example. if the contribution is novel architecture, describing the architecture fully might suffice, if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of a large language model), releasing of model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g.. a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g.- with open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, which case authors are welcome to describe the particular way they provide for reprodueibility. In the case of closed-source source models, it may be that access to the model is limited in some way (c.g.. registered users), but it should be possible for other rescarchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code. with sufficient instructions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
We have provided detailed prrra-parrmeters settings in Appendix C and the full training pipeline in Alg. 5 in Appendix G.1 Code and data will be public available once our paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (httpe::/nips.cc/ cct.bppicccppbbbccc/ccccc
Criidea/CodeSule htttee//oooeuubiiiiiiooooo...... for more details.
While we encourage release of code and data, we understand that this might not possible,
so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g.. for a new open-souree benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: : //nnis/ cc/public/ oul.dess/Cooeeuubmiiissooooooo. for more details.
23

---

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from script and why.
At submission time, preserve anonymity, authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommeneed, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have provided detailed hyper parameters settings in Appendix Cand the full training pipeline in Alg. in -
Appendix G.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [Yes]
Justification: Experiments in Sec. are run five times with different sceds. Both mean value and standard deviation are provided.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for caleulating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report 96% CI. if the hypothesis of Normality of errors
a 2-sigma error than state that they not verified.
asymmetric distributions, authors should be careful not show in tables or figures symmetric error bars that would yicld results that are out of range (c.g. negative error rates). error bars are reported tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification:
We provide the information about compute resources in Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
24

---

The paper should provide the amount of compute required for each of the individual experimental well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, every respect, with NeurIPS Code of Ethics hattp:://eeurti
Answer:
[Yes]
Justification: The research conducted in the paper conform, in every respect, is with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure preserve anonymity (e.g., if there is special consideration due laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts the work performed?
Answer: [Yes]
Justification: Broader impacts including both positive and negative societal impacts are discussed Sec.
G
Guidelines:
The answer NA means that there is no societal impact of the work performed.
the authors answer NA or No, they should explain why their work has no socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploy- ment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it not needed point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions access the model implementing safety filters.
25

---

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly eredited and are the license and terms of usc explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The license for both datasets GSRC and MCNC MIT which can found http: //vlsicad. eecs. umich edu B/t/////ooopiiiie html
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paperewi thcode. cwm/ca////aa//oc daaa/dattstt has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations,
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing research with human subjects.
Guidelincs:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least minimum wage in country of data collector.
15. Institutional Review Board (IRB) Approvals Equivalent for Research with Human Subjects
26

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country which research is conducted, approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such the institution conducting the review.
27


Reviews:
[{"review_id": "27FwEPf6Lb", "reviewer": "Reviewer_5TLE", "summary": "The paper proposes a learning-based method called FlexPlanner  in hybrid action space with multi-modality representation to simultaneously handle position, aspect ratio, and alignment of blocks.  FlexPlanner models 3D FP based on multi-modalities, includ15 ing vision, graph, and sequence.  The work designs a policy network with hybrid action space and asynchronous layer decision mechanisms that allow for determining the versatile properties of each block.  Experiments on public benchmarks MCNC and GSRC  show the effectiveness.", "questions": "Unfair comparisons: There are unfair comparisons with other work. For instance, the methods  GraphPlace [35] and DeepPlace [13] originally incorporated graph and vision as representations.  MaskPlace [25] only employed visual representation, whereas the authors’ method used graph, vision and sequence. Three multimodalities provide more information.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a new learning-based method in hybrid action space with multi-modality representation for 3D floorplanning task.  \n2. The paper explains the rationale behind each proposed module/component of the model with experimental verifications of the module's effectiveness.", "weaknesses": "Lack of Clarity: Certain sentences in the paper are unclear and difficult to understand, hindering the comprehension of the proposed methodology.  For example, the elaborated form of ablation experiment section is similar to an experiment report.\n\nInsufficient Experimental: The paper may lack some ablation experiments about the important hyperparameters.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer 5TLE (5: Borderline accept)\nThank you for your time and valuable feedback. Our replies to the concerns and questions are as follows.\n\n> **W1: Clarity**\n\nIn the final version, we will conduct more ablations studies in terms of different hyperparameters and the impact of different modalities, and integrate more detailed corresponding introduction. Additionally, we further polish the paper and clarify the following aspects:\n\n- We refresh the ablations study table and add more analysis including trying different combinations of modalities in Table 1 in the rebuttal PDF.\n\n- Additionally, we detail the introduction, analysis, and experimental protocals, and **polish the content** in our final version, including:\n\n  - **Additional Explanations on the Background of Integrated Circuit (IC) Design.** We add more background about ICs and introduce the vivid visualization of cross-die alignment, as shown in Fig. 2 in our rebuttal PDF, to assist readers to understand its definition and significance.\n  - **Design Details.** We will revise the paper to include critical design details, such as the critic network, ensuring that readers have a comprehensive understanding without needing to refer to the appendix.\n  - **Details of Benchmarks and Baselines.** In the final version, we will expand this section to provide a thorough analysis, highlighting the relevance and implications of our choices.\n\n\n> **W2: Ablation on Hyperparameters**\n\nWe appreciate your valuable comments. According to your suggestions, we have made the following supplements:\n\n- We provide **additional experiments to evaluate the sensitivity** of RL with respect to hyperparameters, such as learning rate, mini-batch size. Experiments are shown in Figure 1 in the rebuttal pdf. It demonstrates that our approach achieves good stability on different hyperparams settings (stable training curve across different hyperparameters).\n- Moreover, our method is capable of employing the **same hyperparameter settings for all circuits**, rather than adjusting hyperparameters individually for each circuit. It demonstrates the robustness, generalizability of our method.\n\n\n> **Q1: Comparisons With Same Multimodalities**\n\nWe greatly appreciate your constructive concern. \n- First, we believe that **incorporating multimodalities is one of our main contributions** (the second contribution in 'Introduction' part) and to our best knowledge, it is the first trial to simultaneously incorporate vision, graph, and sequence.\n- Second, it is **non-trivial to directly apply multimodalities** to existing baselines considering the model convergence and the more complicated setting in our paper.\n- Last, we fully acknowledge and understand the latent unfairness. To address this, in addition to the existing ablation study in Table 5, we conducted **further ablation studies**, as presented in Table 1 in the rebuttal PDF (or see below for clarification). Specifically, we display the alignment and HPWL for our FlexPlanner using the same modalities as DeepPlace, GraphPlace, and MaskPlace. As shown in the table, FlexPlanner surpasses all other baselines in both alignment and HPWL with the same modalities.\n\n| |Method|MaskPlace|Ours|GraphPlace|DeepPlace|Ours|Ours|Ours |Ours|Ours|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Modality| Vision | ✔ | ✔ | ✔ | ✔ | ✔ | | ✔ |✔ (no canvas mask) |✔ |\n| | Graph | | | ✔ | ✔ | ✔ |✔ | |✔ |✔ |\n| | Sequence| | | | | |✔ |✔ |✔ |✔ |\n| Metric |Alignment| 0.575 |**0.847** | 0.279 | 0.332 |**0.860** |0.301 |0.874 | 0.744 | **0.961** |\n| | HPWL | 189,100 |**187,026**| 209,940 | 223,359 |**186,005**|221,602|185,079|186,732 |**176,639**|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vCcE74lBSr", "reviewer": "Reviewer_pH9C", "summary": "This paper presents a new learning-based method for IC design that simultaneously handles the position, aspect ratios, and alignment of blocks.  The method achieves significant improvements compared to baselines by leveraging reinforcement learning with a hybrid action space and multi-modality representation to optimize block positions, aspect ratios, and cross-die alignments.", "questions": "see above.", "limitations": "Yes, the limitations are briefly addressed in the conclusion.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "* The application of RL with a novel reward function in a hybrid action space to 3D IC FP is very interesting and novel, moving away from traditional heuristic-based methods. \n* The paper demonstrates notable improvements compared to existing baselines on the evaluated benchmarks.\n* The paper clearly defines the used metrics.", "weaknesses": "Clarity and Writing:\n* While the paper is overall well-organized, some sections could benefit from additional explanations or simplifications to make the content more accessible to readers without an IC design background.\n* Important design details  (e.g., the critic network) are deferred to the appendix and not introduced in the main text.\n* Although the benchmarks and baselines used are relevant, discussing them in more detail could significantly enhance the presentation.\n\n\nEvaluation Pipeline:\n* Providing more details on the exact setup of training and test data could improve transparency.\n* RL methods are known to suffer from stability issues and are often sensitive to hyperparams. How sensitive is the method to the hyperparams used? This should be evaluated and discussed to comment on the reproducibility of the results.\n* How well does the method generalize to larger circuits beyond those in the benchmarks? A discussion in this direction would be highly appreciated.\n\nDesign Choices:\n* There are many design choices in the paper. For instance, the significance of the multi-modality input is not entirely clear. The model takes in \"vision,\" \"graph,\" and \"sequence\" modalities, but it is not specified how much each of these contributes to the final performance. While the ablation studies show that the alignment mask is a good design choice, they do not provide much intuition about the importance of other design choices, such as the canvas mask or input modalities (e.g., sequence and graph). Are these necessary?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer pH9C (5: Borderline accept)\nThank you for your time and valuable feedback. Our replies to the concerns and questions are as follows.\n> **W1: Clarity and Writing.**\n\nAccording to your suggestions, we have polished the paper in the following aspects:\n- We will provide **more explanations for readers without an IC background**. For example, in Fig.2 in rebuttal PDF, we provide more vivid visualizations of cross-die alignment to help readers understand its definition and significance.\n- We will revise the paper to include **key design details** (e.g., critic network), ensuring that readers have a comprehensive understanding without refering to appendix.\n- In the final version, we will give more detailed analysis and discussion on **benchmarks and baselines.**\n\n**All revisions will be updated in the final version.**\n\n> **W2.1: More details about training and test data**\n\n- For **experiments of training from scratch**, RL is trained **case by case** by PPO via interaction with floorplan environment. After training, we evaluate its performance on this circuit.\n- For **experiments of fine-tuning and zero-shot inference**, we train RL on circuit n100 and test it on other circuits.\n\nWe will clarify the above details in the final version.\n\n> **W2.2: Sensitivity/Ablation on Hyperparams**\n\nAccording to your suggestions, we make the following supplements:\n- **Additional experiments to evaluate the sensitivity** to hyperparameters, such as learning rate, mini-batch size. Experiments are shown in Figure 1 in the rebuttal PDF. Stable training curves show that our approach achieves good stability on different hyperparams settings.\n- Moreover, our method is capable of employing the **same hyperparameter settings for all circuits**, rather than adjusting hyperparameters for each circuit. It shows the robustness and generalizability of our method.\n\n> **W2.3: Generalization on Larger Circuits**\n\nWe sincerely appreciate your suggestion. MCNC and GSRC benchmarks are indeed widely used datasets for 2D and 3D FP tasks. The largest case in these benchmarks is the circuit n300, with 300 blocks, 569 I/O ports, and 1893 nets. It is sufficiently large, even by industry standards. Recently, Intel, a leading CPU manufacturer, released FloorSet [1] (accepted by ICCAD 2024 but not published), a VLSI floorplanning dataset derived from real-world System on Chip (SoC). In FloorSet, the largest case contains 120 blocks, which is **significantly smaller** than n300 in our evaluation pipeline. Given that FloorSet is based on actual SoCs, we believe that current scale of our benchmarks is adequate to ensure robust performance in **real-world industrial applications**.\n\nTo further demonstrate the capability of our approach, we design additional experiments on larger circuits shown in Table 2 in rebuttal pdf (or see below).\n|Circuit|adaptec2|adaptec2|adaptec2|adaptec3|adaptec3| adaptec3 | n300_dup3 | n300_dup3 | n300_dup3 |\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n||#Block|#Net|#I/O Port|#Block|#Net|#I/O Port|#Block|#Net| #I/O Port|\n||566|860|0|723|1,154|0|900|5,679|1,707|\n|Method/Metric|Alignment|HPWL|Overlap|Alignment|HPWL|Overlap|Alignment|HPWL|Overlap|\n|Wiremask-BBO|0.285|2,956,546|0.000|0.365|5,000,467 |0.000|0.391|2,389,902|0.000|\n|Ours|**0.839**|**2,911,438**|**0.000**|**0.817**| **4,758,283**|**0.000**|**0.928**|**2,305,070**|**0.000**|\n\n**1. Training from scratch**\nISPD 2005 benchmark [2] is a standard for 2D global placement (GP), a task performed after floorplanning. In this benchmark, each circuit consists of more than 500 macros (large functional blocks) and millions of cells (tiny logic gates). The primary objective of GP is to optimize locations of cells to minimize wire length.\nFor our purposes, we remove cells from the chip canvas and netlist, constructing new circuits (with more blocks than n300) that consists solely of macros. Macros are then assigned to two dies, and alignment pairs are constructed. We evaluate our method on these modified circuits. Experiments show that our approach achieves an alignment score with 0.839 and 0.817, significantly surpassing 0.285 and 0.365 by Wiremask-BBO.\n\n**2. Zero-shot inference**\nWe construct a synthetic circuit, n300_dup3, by duplicating all components of circuit n300 three times. It results in a larger circuit with 900 blocks, 1707 I/O ports, and 5679 nets.\nFor this circuit, we perform inference directly using the pre-trained checkpoint obtained from training on circuit n100. It achieves an alignment score of 0.928, significantly surpassing 0.391 by Wiremask-BBO. It shows the capability of zero-shot inference, even when the testing circuit is nine times larger than the training circuit.\n\n**References**\n\n[1] FloorSet-a VLSI Floorplanning Dataset with Design Constraints of Real-World SoCs. ICCAD. 2024.\n\n[2] The ISPD2005 placement contest and benchmark suite. ISPD. 2005.\n\n> **W3: Ablation on Design Choices**\n\nThank you for your suggestions. Actually, in Table 5 in main paper, we have already compared our FlexPlanner **without sequence (w/o seq)** or **without graph (w/o graph)**. They are inferior to FlexPlanner with full modalities, so the choices of graph and sequence are necessary.\n\nFor more sufficient and clearer comparisons, we conducte **further ablation studies** in Table 1 in rebuttal PDF (or see below). Vision includes alignment mask, wire mask, position mask and canvas mask. Specifically, we compare more combinations of three modalities. As shown in the Table 1, vision is the most important but graph and sequence can also promote the effectiveness.\n||Method|MaskPlace|Ours|GraphPlace|DeepPlace|Ours|Ours|Ours |Ours|Ours|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Modality|Vision|✔|✔|✔|✔|✔||✔|✔(no canvas mask)|✔|\n||Graph|||✔|✔|✔|✔ ||✔|✔|\n||Sequence||||||✔|✔|✔|✔|\n|Metric|Alignment|0.575|**0.847**|0.279|0.332|**0.860**|0.301|0.874|0.744|**0.961**|\n||HPWL|189,100|**187,026**|209,940|223,359|**186,005**|221,602|185,079|186,732|**176,639**|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XyI11lyj81", "reviewer": "Reviewer_3CLd", "summary": "This paper proposes the FlexPlanner, a reinforcement learning-based method utilizing multi-modality representation, including vision, graph, and sequence, to handle different challenging scenarios.\nAdditionally, the design of the action space to uniformly handle constraints represents a new and innovative method. \nThis approach, being largely free from heuristics, is more aligned with machine learning principles and advances the field of learning for floorplanning, particularly in 3D scenarios. \nThe performance improvement from 0.474 to 0.940 on public benchmarks is significant, and the method also demonstrates strong transfer learning capabilities.", "questions": "As RL could be a controversial approach for placement and routing, how do the authors defend their RL-based methodology to the floorplanning problem?", "limitations": "No specific limitations.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well written, with a clear background and motivation, and Table 1 is informative and categorizes the features of existing methods, which helps me understand the approach. \n\n2. The use of vision, graph, and sequence modalities provides a rich representation of the planning problem, expanding the action space and enabling the learning of multiple properties, such as position, aspect ratio, and layer for each block.\n\n3. The experimental results are much better than previous methods on benchmarks in terms of wirelength and alignment. \n\n4. The appendix part is also informative and useful for better understanding the approach and the results", "weaknesses": "There is no weakness in this paper, to be honest.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer 3CLd (7: Accept)\nThank you for your time and valuable feedback. Our replies to the questions are as follows.\n\n> **Q1: Defend RL on Floorplanning Task**\n\nThanks for your insightful comment. Compared to other methods, our RL-based approach offers the following advantages in 3D floorplan task:\n\n- **Better to handle hard constraints compared to analytical methods**: Analytical methods often require smooth approximations of optimization objectives to make them differentiable, which reduces precision and accuracy. Moreover, many optimization objectives, such as alignment score, are difficult to approximate. As a result, it is hard for analytical algorithms to optimize these objectives. In contrast, RL models optimization objectives through reward designs, where differentiability of reward calculation is not a prerequisite. As a consequence, approximations are unnecessary, resulting in greater precision. Additionally, analytical methods struggle to meet hard constraints (e.g., cross-die module alignment constraint, non-overlap constraint), typically relaxing them to soft constraints via penalty terms. On the contrary, our FlexPlanner introduces masks (e.g., alignment mask and position mask) to directly handle hard constraints in the action space by filtering out invalid positions.\n\n- **Higher performance upper bound and more flexibility compared to heuristic algorithms**: The solution space modeled by heuristics representation is limited and cannot encompass all solutions, making it challenging to find the optimal solution. Besides, they lack flexibility, making it difficult to precisely fine-tune the position of each block. On the contrary, our RL approach, FlexPlanner, leverages a hybrid action space with more flexibility. It directly determines the coordinate, aspect ratio and layer for each block. Moreover, Heuristic methods can only guide the searching process through cost functions and fail to satisfy specific hard constraints. In FlexPlanner, masks are incorporated to enforce hard constraints in the action space.\n\n- **Generalization performance**: Our method benefits from a general multimodal representation and a unified reward function design, enabling fine-tuning and zero-shot inference capabilities across different circuits. Moreover, our method employs **the same hyperparameter settings for all circuits**, demonstrating its robustness and generalizability.\n\n**We will integrate corresponding discussions in the final version.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2V4nLgWNb1", "reviewer": "Reviewer_jFwz", "summary": "In this paper, the authors have proposed FlexPlanner, a flexible 3D floorplanning method with deep reinforcement learning. Existing learning methods mainly focus on the 2D scenarios. However, it suffers from overlooking alignment requirements and multi-die property. To address these, FlexPlanner learns a hybrid action space with multi-modality representation. It contains three modules to estimate position, layer and aspect ratio of blocks. Experiments demonstrate the effectiveness of the proposed method.", "questions": "Please see the weakness.", "limitations": "N/A", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The problem it addresses is clearly stated. While most existing learning methods target at 2D FP task, this paper address 3D FP task, and also show the difficulties when directly applying 2D methods to 3D.\n\n2. This work introduces three modalities to represent the state space, which show better representative ability than heuristics-based methods.\n\n3. The writing of the paper is clear.\n\n4. Significant improvements on alignment scores according to Table 2.", "weaknesses": "1. It would be better to include a teaser that better shows the problems when directly applying 2D FP method to this 3D scenario.\n\n2. The novelty of the paper is a concern. From my point of view, the idea of using three modalities to address the issue is quite straightforward, which lacks of novelty. In the rebuttal, the authors should further highlight the novelty of the technical designs of this work.\n\n3. Please further explain the significant improvements in Table 2, which is very impressive. It would be better to include some visualization comparisons as well as more explanations on why existing methods obtain such low alignment scores.\n\n4. I have some concerns about the baselines in the table. It should include more recent baselines within 1 year. Also, I notice that [14] was published in 2010 -- why do this method still achieves comparable results with other baselines?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer jFwz (5: Borderline accept)\nThank you for your time and valuable feedback. Our replies to the concerns and questions are as follows.\n\n> **W1: Better to include a teaser showing the problems when directly applying 2D FP method to 3D scenario**\n\nWe sincerely appreciate your constructive suggestion. Compared to 2D FP, which involves only a single die, the 3D scenario requires placing blocks across multiple dies. The positions and shapes of the blocks on different dies should be optimized simultaneously to achieve a better result, rather than optimizing each die sequentially as in 2D FP. Furthermore, cross-die alignment becomes a critical optimization objective, and it is insufficient to rely solely on incorporating the alignment score into the heuristic cost or RL reward.\n**We will integrate corresponding discussions in the final version.**\n\n> **W2: The idea of using three modalities to address the issue is quite straightforward**\n\nWe greatly appreciate your constructive concern. \n- First, **incorporating three modalities is only part of our main contributions**, in addition to other important contributions such as addressing the 3D alignment problems and zero-shot transferability, and most importantly, our approach is the first learning-based method to discard heuristic-based search in the 3D FP task and it achieves new state-of-the-art. Additionally, we propose a novel mechanism, asynchronous layer decision to determine the layer for accessing the next block to place. This approach offers greater flexibility in fine-tuning the placing order, instead of adhering to a fixed sequence.\n- Second, it is **non-trivial to directly apply multimodalities** to our approach. It should be noted that our approach is not incremental to previous works but is devised from scratch. The choices of multimodalities and other modules like alignment mask are based on large amounts of experiments and observations, and they are empirically proved to be effective in the complicated 3D FP setting with a comprehensive consideration of module position, module aspect ratio, overlap area, cross-die alignment, and the fixed outline.\n- Last, according to your suggestions, we will polish the contributions in the 'Introduction' part in our final version.\n\n> **W3: Explanation and Visualization of Improvement on Alignment Score**\n\nWe provide both additional visualization comparisons and explanations on alignment score.\n- The **visualization comparisons on alignment score** are shown in Fig. 2 in the rebuttal PDF. It demonstrates that our method performs much better than other baselines on alignment score.\n- Why our method performes better\n    - **Alignment mask as representation.** The alignment mask provides RL with a more refined modeling of cross-die alignment, rather than merely perceiving alignment through reward function. This detailed guidance is crucial for the alignment task, which is inherently challenging and requires more precision.\n    - **Alignment mask as constraint.** By utilizing the alignment mask to filter out invalid positions, we place blocks in locations where the alignment constraint is satisfied. This approach reduces the action space without sacrificing quality and enhances efficiency.\n    - **Asynchronous layer decision.** This approach provides greater flexibility, allowing the method to autonomously decide the planning order, rather than completing one layer before starting the next. For instance, after placing a block on the first die, an intuitive next step is to immediately place its alignment partner on the second die to prevent other blocks from occupying the alignment region.\n\n- Why other baselines performes worse\n    - **Heuristic Methods** (e.g., 3D-B\\*-SA) cannot directly constrain the action space to ensure the cross-die alignment. Instead, they calculate the alignment score and incorporate it into the heuristic cost to guide the search. While effective for smaller cases, the performance of this approach significantly deteriorates as the circuit scale increases, shown in Table 2 in main paper.\n    - **Wiremask-BBO** employs a greedy algorithm to select the highest-scoring position from the current legal positions for the next block. This greedy nature makes it prone to local optima, preventing it from achieving global optimization. Additionally, its synchronous decision-making mechanism, which completes one layer before moving to the next, is less effective.\n    - **Other RL Baselines** lack representation of the cross-die alignment, relying solely on rewards to guide optimization, which is quite limited. Ablation study in Fig. 5(a) in main text further verifies this point: removing the alignment mask either as input or as an action space constraint significantly impacts the alignment score. It demonstrates the effectiveness of our proposed alignment mask in both input representation and action space constraint.\n\n> **W4: Comparable Results of 3D-B\\*-SA**\n\nWe fully understand your suggestion of incorporating more recent baselines, however, wiremask-BBO is the current state-of-the-art approach and, to our best knowledge, it is the merely one within one year that can be compared due to the sophisticated experimental settings and the common closed source community of Electronic Design Automation.\n\nOn the other hand, though 3D-B\\*-SA was published in 2010, as a classical simulated annealing (SA) method, it is still formidable especially on small circuits, and thus, SA-based approaches are also common baselines in recent works [1,2]. However, due to the exponential increment of searching space, the performance of SA rapidly degrades with the increasement of the circuit scale and is gradually overpassed by RL in recent years.\n\n**References**\n\n[1] Generalizable floorplanner through corner block list representation and hypergraph embedding. SIGKDD. 2022.\n\n[2] Macro placement by wire-mask-guided black-box optimization. NeurIPS. 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## Global Response
Dear Area Chairs and Reviewers,

We appreciate your time, valuable comments, and constructive suggestions. From an overall perspective, we are happy to see that **all reviews are positive** and the reviewers approve of the **novelty** (`3CLd`, `pH9C`, `5TLE`), **notable improvements** (`jFwz`, `3CLd`, `pH9C`), and **strong transfer learning capabilities** (`3CLd`). Additionally, we are grateful for the acknowledgment that this paper is **well-motivated** (`jFwz`,  `3CLd`) and **well-organized** (`pH9C`, `jFwz`, `3CLd`).

According to the reviewers' suggestions, we have made the following revisions:

- **More Ablation Studies.** In addition to the existing ablation study in Table 5 in main text, we conduct further ablation studies, as presented in Table 1 in the rebuttal PDF (or see below for clarification). Specifically, we display the alignment and HPWL for our FlexPlanner using the same modalities as DeepPlace, GraphPlace, and MaskPlace. As shown in the table, FlexPlanner surpasses all other baselines in both alignment and HPWL with the same modalities.

| | Method |MaskPlace| Ours |GraphPlace|DeepPlace| Ours | Ours |Ours |Ours |Ours |
|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|
|Modality| Vision | ✔ | ✔ | ✔ | ✔ | ✔ | | ✔ |✔ (no canvas mask) |✔ |
| | Graph | | | ✔ | ✔ | ✔ |✔ | |✔ |✔ |
| | Sequence| | | | | |✔ |✔ |✔ |✔ |
| Metric |Alignment| 0.575 |**0.847** | 0.279 | 0.332 |**0.860** |0.301 |0.874 | 0.744 | **0.961** |
| | HPWL | 189,100 |**187,026**| 209,940 | 223,359 |**186,005**|221,602|185,079|186,732 |**176,639**|

- **More Clarity.** We detail the introduction, analysis, and experimental protocals, and polish the content in our final version, including:
    - **Additional Explanations on the background of Integrated Circuit (IC) design.** We add more background about ICs and introduce the vivid visualization of cross-die alignment, as shown in our rebuttal pdf, to assist readers to understand the definition and significance of it.
    - **Design Details.** We will revise the paper to include critical design details, such as the critic network, ensuring that readers have a comprehensive understanding without needing to refer to the appendix.
    - **Details of Benchmarks and Baselines.** In the final version, we will expand this section to provide a thorough analysis, highlighting the relevance and implications of our choices.

**All the above revisions will be updated in the final version.**

**A one-page PDF is uploaded that contains corresponding tables and figures in the response.**

In the following, we provide detailed answers. We are glad to further response for informed evaluation.

Meta-review: The submitted paper proposed a method for "floor planning", ie determining the positions and shapes of blocks in integrated circuit design, with the difficulty being the emergence of 3D circuits. Five expert reviewers have provided a favorable assessment. They appreciated novelty (an RL formulation with a hybrid action space, based on Hybrid PPO), motivation, performance gains, 

Some weaknesses were raised, the paper somewhat lacking in clarity in some aspects, doubts on comparisons with prior work, information on stability in RL training, generalization, abundane of design choices.

A consensus was reached that the paper is a worthy contribution for the NeurIPS community, with the strengths 
The AC concurs.
Decision: Accept (poster)