MaskPlace: Fast Chip Placement via Reinforced Visual Representation Learning
Authors: Yao Lai
Yao Mu
Ping Luo
Venue: NeurIPS 2022 Accept (2022)
Keywords: Applications
Reinforcement Learning
Electronics Design Automation
Combinatorial Optimization

Abstract: Placement is an essential task in modern chip design, aiming at placing millions of circuit modules on a 2D chip canvas. Unlike the human-centric solution, which requires months of intense effort by hardware engineers to produce a layout to minimize delay and energy consumption, deep reinforcement learning has become an emerging autonomous tool. However, the learning-centric method is still in its early stage, impeded by a massive design space of size ten to the order of a few thousand. This work presents MaskPlace to automatically generate a valid chip layout design within a few hours, whose performance can be superior or comparable to recent advanced approaches. It has several appealing benefits that prior arts do not have. Firstly, MaskPlace recasts placement as a problem of learning pixel-level visual representation to comprehensively describe millions of modules on a chip,  enabling placement in a high-resolution canvas and a large action space. It outperforms recent methods that represent a chip as a hypergraph. Secondly, it enables training the policy network by an intuitive reward function with dense reward, rather than a complicated reward function with sparse reward from previous methods. Thirdly, extensive experiments on many public benchmarks show that MaskPlace outperforms existing RL approaches in all key performance metrics, including wirelength, congestion, and density. For example, it achieves 60%-90% wirelength reduction and guarantees zero overlaps. We believe MaskPlace can improve AI-assisted chip layout design. The deliverables are released at https://laiyao1.github.io/maskplace.

MaskPlace: Fast Chip Placement via Reinforced
Visual Representation Learning
Yao Mu Ping Luo" Department of Computer Science The University of Hong Kong {ylai, ymu ymu,
Yao Lai
pluu)ecs hku. hk
Abstract
Placement is an essential task modern chip design, aiming at placing millions circuit modules 2D chip canvas. Unlike the human-centric solution, which requires months of intense effort by hardware engineers to produce a layout to minimize delay and energy consumption, deep reinforcement learning has become an emerging autonomous tool. However, the learning-centrie method is still in its early stage, impeded by a massive design space of size ten the order of few thousand. This work presents MaskPlace to automatically generate a valid chip layout design within a few hours, whose performance can be superior or comparable to recent advanced approaches. It has several appealing benefits that prior arts do not have. Firstly, MaskPlace recasts placement as problem of learning pixel- level visual representation to comprehensively describe millions of modules on chip, enabling placement in high-resolution canvas and large action space. It outperforms recent methods that represent chip hypergraph. Secondly, it enables training the policy network by an intuitive reward function with dense reward, rather than a complicated reward function with sparse reward from previous methods. Thirdly, extensive experiments on many public benchmarks show that MaskPlace outperforms existing RL approaches in all key performance metrics, including wirelength, congestion, and density. For example, it achieves 60%- 90% wirelength reduction and guarantees zero overlaps. We believe MaskPlace can improve Al-assisted chip layout design. The deliverables are released at laiyao I. wtt.raoo.gitubbooooee github.o/mmaskpace.
Introduction
The scalability and efficiency are two significant factors of autonomous chip layout design. Placement one of the most challenging and time-consuning problems in the design flow, aiming to determine the locations of millions of circuit modules on a 2D chip canvas represented by a two-dimensional grid. A netlist can describe these modules, that is, a large-scale hypergraph consisting of massive macros (functional blocks such as memory) and standard cells (logic gates), where each macro and each standard cell can contain several or even hundreds of pins connected by wires, as shown Fig. I.
Placing a large number of circuit modules onto the chip canvas is challenging because many perfor- mance metrics such as power consumption, timing, area, and wirelength should be minimized while satisfying some hard constraints such as placement density and routing congestion. For example, the wirelength (the length of wires that connect all modules) determines the delay and the power consumption of chip [1]. Shorter wires often indicate less delay and less power consumption [2]. However, wirelength cannot be reduced by overlapping modules because the module density is hard constraint to ensure that valid and manufacturable chip layout has non-overlapping modules. More
"Corresponding author is Ping Luo
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

G
(a) DREAMPlace [9] (b) Graph Placement [3] (c) DeepPR [22]
(d) MaskPlace (ours)
Visualizing different placements of a circuit benchmark bigblue3, where the modules visualized by blue rectangles and wires are shown in brown lines to connect massive pins on modules. For clarity, we only show %% wires. The proposed MaskPlace is compared with three representative approaches, including (a) DREAMPlace [9] (HPWL- = 1.04 x 10", WL 1.08 x 10", OL 8.06%), (b) Graph Placement [3] (HPWL 3.45 10", WL = 3.73 10", OL 0.80%), (c) DeepPR [3] (HPWL- 4.39 x 10?, WL 5.18 x 10 OL 83.,3)) and (d) MaskPlace (HPWL 0.83 10", WL 0.88 10 OL 0%). where HPWL, WL, and OL represent half-perimeter wirclength", wirelength, and overlap area ratio, respectively. All the metric values are smaller the better. The best performances are underlined (d). We see that MaskPlace surpasses the recent popular placement approaches in all key metrics, and it can satisfy the 0% hard density constraint. Better zoom in 400%. %.
Figure 1:
examples of the performance metrics are given in 8 and Fig.9 in Appendix. As pointed out in [3], the design space of placement is larger than 102.500 when there are just 1, 000 circuit modules, whereas neural architecture search (NAS) typically has space of 1030 and the Go game has a state space of 103600
Methods of chip placement can be generally divided into two categories, classic optimization- based approaches [4-21] and learning based approaches [3, 22, 23]. In the first category, hardware scientists often formulate placement as an optimization problem and relax the hard constraints. For example, let a pair of vectors (xr, y) denote the (z, y)-coordinaat value of all circuit modules on a 2D canvas, the objective function of placement can be formulated as minimizing (Z, y). subject to D(2, y) S a, where :T. and .) are the estimation functions of wirelength and density respectively, and D(æ;, y) I is a hard constraint with a very small density value a, which ensures that all modules do not overlap. For instance, DREAMPlace [9] is a recent advanced method that minimizes WL(x, y) + AD(x, y). which relaxes the hard density constraint. However, it cannot directly produce a valid and manufacturable layout because the non-overlapping constraint is not satisfied after relaxation. These approaches often need post-processing step, such as manual refinement and legalization (LG), to remove the overlapping in placement, resulting in two issues, (1) the wirelength may increase substantially after LG, and (2) no feasible solution can be found if the available chip area is insufficient hefore post-processing.
In the second category, reinforcement learning (RI.) is employed to solve placement as a sequential decision-making problem, placing each circuit module time. Although the learning- approaches are still in their early stage, they can produce promising results to automate the chip design flow end-to-end significantly without human effort. For instance, Graph Placement [3] and DeepPR [22] represent a netlist as hypergraph, denoted as G == (V, E), where V represents a set of nodes, and each node is a module, and E is a set of edges, which are the wires connecting all modules. They train RL agents to place one module
time by maximizing the metric values rewards. However, the hypergraph is not scalable to comprehensively encode information of a netlist. For example, the relative positions (offsets) of pins are discarded in [3, 22]. The wirelength estimation is inaccurate without the pin information, but encoding this rich information would make the hypergraph large
complicated because each module can have hundreds of pins. Furthermore, placement hypergraph requires heavy computations. Mirhoseini et al. [3] reduced computations by placing 15% of the modules using reinforcement learning (the remaining modules are placed by classic method), and Cheng and Yan [22] decreased the size (resolution) of module and chip canvas as shown in Table 1. Both of them sacrificed their placement performance.
'HPWL (Half Perimeter Wire Length) is a common approximation metric of the wirelength and can be computed much more efficiently than wirelength.
2

---

Table 1: Comparisons of representative placement methods different aspects, including method types ("Family"), canvas size ("Resolution"), state space, "0% overlap" (if the method can produce a layout without overlapping placement), training/inferene, speed ("Efficiency") " and the performance metries to be optimized. We see that MaskPlace of 224 (much larger than prior works) and producing a valid placement with 0% overlap (which cannot be achieved by previous methods). Maskelace can also be trained and tested efficiently.
outperform recent advanced methods by performing placement full canvas size
Family
Resolution
State Space
0% Overlap
Roward
Efficiency
Metrics
DRFAMPlace [9]
Nonlinear
Continuous
-NHigh
H.
Graph Placement [3]
RL+Nonlinear
128
(1282)aV
X
Sparse
Med/MMed.
H. C.
DeepPR [22)
RL
32
(322)*
X
Dense
High/Med.
H.C
MaskPlace (ours)
RL
224
(224")
Dense
High/High
H. C,
DrsamPlace seeds post processing step, such legalization (LG) may fail.
HIPLL C= Congestion, D Density
cirouit
in Graph Placement.
To address the issues of prior arts, we propose a novel RL method, named MaskPlace, which can automatically generate high-quality and valid layout (non-overlapping modules) within a few hours, unlike previous methods that need manual refinement to modify invalid placement, which may wait up to 72 hours for commercial electronic design automation (EDA) tools evaluate the placement. MaskPlace casts placement as a problem of pixel-level visual representation learning for circuit modules using convolutional neural networks. This representation can comprehensively capture the configurations of thousands of pins, enabling fast placement in a full action space on a large canvas size e.g., x24x224. As shown in Fig. and Table I, MaskPlace has many attractive benefits that existing works do not have. MaskPlace is mainly for macro placement due to the problem size.
This paper has three main contributions. Firstly, we recast chip placement problem of learning visual representation to describe millions of circuit modules on chip comprehensively. It opens up new perspective for Al-assist chip placement. Secondly, we carefully design a new policy network that can capture and aggregate both the global and subtle information on a chip canvas, maximizing the reward of wirelength and ensuring non-overlapping placement efficiently. Thirdly, extensive experiments demonstrate that MaskPlace outperforms recent advanced methods on 24 public chip benchmarks. For example, MaskPlace can always produce a layout with 0% overlap while reducing wirelength up to 5x and 9x compared to Graph Placement [3] and DeepPR [22] respectively.
2 Preliminary and Notation
The placement quality can be measured by the HPWL (half perimeter wirelength), which estimates the wirelength with marginal computational cost [24]. Intuitively, Fig.2(e) illustrates a 2D chip canvas. Let M' and p(i.) denote the i-th module and its j-th pin, respectively. A net contains a set of pins connecting modules by wires. For example, "Net 1" (in red) connects all four modules (i.e., M M .3, M using wires through pins P(1.2), p(2.2) p(3.2), and P(4.1), while "Net 2" (in green) connects three modules (i.e., M _ M using wires through pins p(1.1), p(2.1), and p(3.1), HPWI. estimates the wirelength by summing up the half perimeters of bounding boxes of all the nets, as shown by the red and green boxes in Fig.2(e). Intuitively, the half perimeter of a net bounding box equals the sum of its height and width. For example, HPWI. in Fig.2(e) is h + +w + h22 + W2-
Given a netlist containing set of nets, minimizing the wirelength can be treated as minimizing HPWL by placing modules to the optimal positions on 2D chip canvas. To achieve valid and manufacturable chip layout, we need to satisfy two hard constraints: (1) congestion constraint: the wire congestion should be lower than a desired small threshold to reduce chip cost, and (2) overlap constraint: the density should be minimized to achieve non- non-overlapping placement.
min Σ
(max max p(i.) (P Enet
min p(ia) max guet
p(id)
min p(fJ))
pli.i Enet
p(..) Enet
y
Vaeteneeliet
S.L.
Congestion My. M.., Mr) Ctn and Overlap( My, M... Mn) =0,
where Pz and Py represent the (x. y) coordinaae value of a pin respectively, Congestion()) is the con- gestion function, Cth is a desired threshold, Overlap(-) is the overlap function, and M,. My, M., Mr.
3

---

Macio a Standar: Call
Grid
M
Pin Net
Net bbox
Net bbox
MF pi13)
Net 1 [petas, peaa) poan, p(km)) Not 2 (10a)) pizz), peaaly (e) example placement
offset info lost
(a) placement (b) position mask (c) wire mask
(d) fusion
① graph representation
Figure 2: Mask Visualization, placement example, and hypergraph representation prior work. visualize different masks in MaskPlace (a-d) and illustrate an example of placement in (e). In the position mask (b), the green color means feasible positions to place while the gray color represents the placed modules. In the wire mask (c), lighter color indicates shorter wirelength if module is placed fusion mask in (d) is an example of output after the mask fusion model using x convolutions, where A denotes position with a high probability place (i.e, no overlap and shorter wirelength). (f) is result when converting the circuit in (e) into a hypergraph in prior works, where the critical information of pin locations is lost.
specific position. The
represent the position, width, and height of modules respectively. Firstly, lower congestion often indicates shorter wirelength, which is crucial to reduce chip cost because the wire resources are limited real chip. Inspired by prior arts [3, 22], employ the RUDY estimator [25] to estimate wire congestion. Details of RUDY can be found the Appendix A.2. Secondly, the placement density calculates the overlapping region between every pair of circuit modules. It is time-consuming since its computational complexity is (VV2) where V is the number of modules [1]. The proposed approach can ensure non-overlapping placement avoid calculating this density metric explicitly in training, thus reducing computations while producing a valid layout.
Our Approach
Model Architecture Overview. Chip placement can be formulated as Markov Decision Process (MDP) [26] by placing each module time. Fig.4 illustrates the overall architecture of MaskPlace, which trains a policy T0(a,)8) (52) represented by a convolutional encoder-decoder network with pa- rameter set 0, and value function (s.):) represented by embedding model with parameter set 6. The policy network receives previous observations and actions as input 8, and selects an action a, output. Specifically, is set of pixel-level feature maps that comprehensively capture the net and pin configurations in M1 M and M where M/1:-1 denotes the modules that have been placed in the previous time steps from placed at the current step and the next step / + 1, respectively. Intuitively, MaskPlace looks one step forward achieve better placement.
1, while M and M denote the modules be
Although prior arts [3, 22] represented a netlist as a hypergraph as shown in F(f. where each node is module, and each edge is wire between two modules, they lost the information of pin offsets fully
unfeesiible
feasible
for each module. Unlike previous works, MaskPlace represent massive net and pin configurations using three types of pixel-level feature maps, shown in Fig.2(a-d), including position mask, wire mask, and view mask, as discussed below. Different masks are fused by convolutions learn the state representation.
Slide on chip to find feasible positions
Position Mask. The position mask, denoted by fp E (0, 1) 224x224 is a binary matrix of canvas grid with size 224 224 as shown in Fig.3, where value "1" means feasi- ble position to place a module. The purpose the position mask is guarantee no overlaps between modules (i.e., satisfy the over- lap constraint) and learn the relationship between placement and wirelength. Specifically, slide module M' (for example,
M (5.1)
p(5.) Net Net 1
Figure 3: Position Mask Example.

---

St next two modules
Local Mask Fusion
Mask unforsible action
Position Mask
My+1 placed moduless
fut
Probability
Wire Mask
Merge
matrix of actions
Mask
(Oe (5p)
M
fa, fit
Decoder
Wire Mask
Global Mask Encoder
Can
Congestion Satisfaction
fd
Vo(ss)
Congestion threshold
View Mask
FC
Value
P.
Action a.
Pixel-level Module Representation
cmabedding
Value Network
Policy Network
Figure 4: Overview of MaskPlace, which contains three main parts: pixel mask generation model, a policy network, and value network. The pixel mask generation model converts the current placement state into pixel-level masks. The policy and value networks convert these masks actions and values based global and local features. The congestion satisfaction block is to satisfy the congestion constraint and give the final action.
t= 5) on the entire chip canvas. The trajectory of the feasible positions (in green) can be labeled with "1". Intuitively, we can check each position for each module using the cumulative sum array [27]. This naive approach has the computational complexity of ((N2) when 2D canvas grid is divided into N x N cells. However, this simple approach is not efficient when N is large. Therefore, since all modules are rectangles, we design an efficient generation algorithm, which iterates through all placed modules (in blue) and excludes positions that will cause overlap. In this case, all remaining positions are available for placement. The new algorithm is summarized in Appendix A.3, which costs (V) for each module, where V is the number of modules.
Wire Mask. The wire mask, denoted as fu [0, 2|244x224 is a continuous matrix for representing how HPWL increases if we place module M in a specific position. Fig.5 shows sample of wire mask, where each value means the increase of HPWL. The wire mask aims at finding the best position with the minimum increase of the wirelength. Intuitively, we can calculate the HPWL each canvas position, leading
30
ID
10
30
13
22
13
25
14
10
complexity of O(N2 where P is the total number of pins. However, a fast algorithm can be designed by considering the relationships between the pin offset, the net bounding box, and the linear property of the HPWL metric. For example, Fig.3 illustrates that the next module M has two pins, p(5.1) and p(5,2), belonging to "Net 1" and "Net 2" respectively ((e). 5 illustrates the increase of wirelength when placing .M at each canvas location. For instance, if M is at the bottom-left corner, its Manhattan distance to the two net bounding boxes (in red and green) 2 + 4. To calculate the Manhattan distance more accurately, move the net bounding box compared to the pin location. For example, since ((5.2) is located
Figure 5: Wire Mask Example.
at (2, 1) move the bounding box of Net 2 (in green) in the direction -1 (5.2) (-2, -1) to encode the information of pin offset. The time complexity can be reduced to O(NP). The algorithm
can be found in Appendix A.3.
View Mask. The view mask, denoted as f. E {0, 1| 2224x224 is a global observation of the current chip layout, where the value "I" means module has occupied this grid cell. Different from DeepPR [22] that assumed all modules have unit size, we consider real sizes of modules. For instance, if a module has size x h. it covers W x [hN/H] cells in the canvas, where W and H represent the canvas size and [-1 denotes the ceiling function.
Learning Algorithm. We train different blocks in Fig.4 as whole using reinforcement learning. The detailed network architectures are provided in Appendix A.4. Firstly, we apply the above masks to represent the entire circuits and feed them to downstream networks. Secondly, global feature encoder embeds the view mask of current placement and the wire masks of the following two steps into an embedding vector. Then we combine it with the positional embedding of the t-th circuit module in the value network to generate a scalar to evaluate the current state by fully-connected layers.
We index the bottom- left corner the origin (0, 0) in two-dimensional coordinate.
5

---

Thirdly, global mask decoder recovers feature map of size N2, which is fused with different position masks and wire masks in the policy network using signal diffusion. The policy network predicts probability action matrix of size N x N, indicating where to put the next module. Before sampling actions, we can remove unfeasible actions using the position mask. Finally, the congestion satisfaction block applies the congestion threshold on the probability matrix to select a final action.
convolutions to avoid the local
Reinforcement Learning. We borrow the representative actor-critic diagram [28] and PPO2 frame- work [29] to train the policy No (o(((s))) where the state representation 8, is listed in Table 11 in Appendix. The action is the canvas position (cell) to place the circuit module. Specifically, we treat the chip canvas as a grid and divide into N x N cells, leading to N possible actions. The objective function of the policy network can be formulated as
-,, Lpolicy (0) = E min (re(O) -ip.::(0).11..1+)A4)) (.11))))))
where the ratio rf)) To ne Is:) and A, G. Vi denotes the advantage function. We employ ma (ar
G. = >k= " r+44+1 that is the cumulative discounted reward and V. is the estimated value produced by the value network. We update the the value network by optimizing its objective,
Livaue (Ø) E8 [[. - Vj
Reward rt. We treat HPWL as the reward because wirelength the main optimization target in different performance metrics. This is different from prior arts [3, 22] that weighted combines HPWL and congestion as the reward, which introduces the weighting coefficient as an extra hyper- parameter tune. Specifically, computes the currently placed pins. For example, the partial HPWL for / modules can be defined as HPWL,. In other words, we compute HPWL, after taking action de The reward for the step I is HPWL, - HPWT,, i.e., the opposite number of the increase of HPWI. Furthermore, instead of computing HPWL at each step, we can maintain the ranges of all net bounding boxes in one episode and update the changes of their sizes with a cost of ((P), where P is the number of pins. Thus we can generate dense rewards while maintaining efficiency.
achieve dense reward by defining a partial HPWL, which only
Training and Testing. Before training, we follow previous work [3] to sort the circuit modules according to the number of nets, areas, and the number of connected modules that have been placed to determine the place order. In training, we update the policy and value networks at each epoch by ignoring the congestion satisfaction block. When updating the value network, we stop the gradient back-propagated in the global mask encoder to avoid influence on the policy network. The detailed training setup is provided in Appendix A.S.
In the testing stage, for each step t, we obtain a probability matrix from the policy network and sample one place action ag. Then, the congestion satisfaction block will check whether the congestion exceeds a threshold Csh, after applying this action. If it exceeds, we uniformly sample a few actions, look up the corresponding values from the wire mask fh.. and estimate the congestion before taking these actions. We choose the action with the minimal value in f and the congestion lower than Cin- If all actions cannot satisfy congestion Cn. select the action with the minimal congestion and move to the next step. Detailed congestion satisfaction algorithm can be seen in Appendix A.3.
4 Experiments
We extensively evaluate MaskPlace and compare it with several recent advanced placement methods, including NTUPlace3 [6]. RePlAce [8]. DREAMPlace [9], Graph Placement [3], and DeepPR [22], where NTUPlace3, RePlAce and DREAMPlace are optimization- based methods, whilst Graph Placement and DeepPR are learning-basse approaches. All of them are evaluated on different public circuit benchmarks. All previous works are evaluated by following their experimental settings.
Benchmark. We evaluate MaskPlace in 24 circuit benchmarks selected from public datasets including the widely-used ISPD2005 [30], IBM benchmark suite [31], and Ariane RISC-V V CPU design [32]. The number of evaluated benchmarks is three times more than previous work [9, 22, 3). The statistics of benchmarks are given in Table in Appendix A.6, where the largest circuit contains 1,293 macros, 22,802 pins, and more than a million standard cells, leading to vast state space aforementioned.

---

Main Results. Table 2 compares the HPWL results between all the above methods to place all macros. To enable fair comparison, we evaluate all approaches" by using five random seeds and report the means and variances. Since the original DeepPR method did not capture macro size (thus does not avoid overlap between adjacent macros because all macros have unit size), we extend DeepPR to model macro size to reduce the overlapping ratio. We name it "DDepPR-nno-ovelay)
Similar to prior works, we use the minimum spanning tree algorithm [33] to estimate routing wirelength [34]. From Table 2, we can see that MaskPlace achieves the lowest wirelength in six out of seven benchmarks (except "adaptec4" where still outperforms all learning- conventional optimization- based approaches may fail when the circuit benchmark has high chip area usage, such as "bigblue3 and "ariane". : Also, MaskPlace gets the lowest wirelength compared with Graph Placement and simulated annealing [35] in the IBM benchmark, which is shown in Appendix A.7. This project website" visualizes and compares different placements.
methods). We also see that the
Table 2: Comparisons of HPWL (x10º). HPWL is the smaller the better. We see that MaskPlace outperforms other methods by large margins in six out of seven benchmarks. The traditional optimization such as NTUPlace3 and DREAMPlace may fail in few benchmarks such as "ariane".
Method
adaptec
adaplec2
adaptec3
adapiec4
highluel
highlue3
ariane
Random
61,0043.85
483.12+13.65
576.25416.03
600.07:14.17
36.6743.18
918.05243.49
$2.20%00.0
NTUPlace3 [6]
26.62
321.17
328.44
462.93
22.85
455.53
LG fail
RePlAce [8]
16.19+2.10
153,26#29 01
111.21=11.99
37.64+1.05
2.45=0.06
119. 119.84#34,43
LG fail
DREAMPlace [9]
15.81+1.64
140.79+26.77 73'
121.94425.05
37.41
2.44+0.06
107.19+29.91-
1.G fail
Graph Placement [3]
30. 30.10+22.8
351.71 20
358. 188118413.99
4249.722
10.58:1.29
357.48847.83
16.8910.60
DecpP'R (22)
19.9142.13
203.51:26.7
347.1624.32
311.86256.74
23.3343.65
430.48::2.18
52.2010.89
DeepP'R ov-orlr [22]
47.39:4.00 02
425, 425.86219.59
545.40+16.40
125.51
26.29:1.48
$15.10240.36
62. 62.8240082
MaskPlace
6.3820.35
73.7526.35
84.4423.60
79.21
2.3920.05
91.11=7.83
14.63+0.20
2 (of 5) seeds fail in logalization (L.G).
1 (ot's 5) seed fails in legalization (LG).
Compare to Graph Representation. Since Graph Placement [3] is the recent advanced learning- based approach that employs hypergraph for placement, we compare MaskPlace with it in all four performance metries, including HPWL, congestion, density, and overlap area ratio. Table and report the results. The overlap area ratio describes the ratio of the overlapping area between macros divided by the chip area. In Table 3, MaskPlace (soft constraint) means that the round function rather than the ceiling function is used to calculate the number of grid cells occupied by the placed macros. MaskPlace with soft constraints may produce better HPWL and congestion than its counterpart with hard constraints, but the overlap area ratio could not be zero because the constraints have been relaxed. From Table and 4, we see that MaskPlace outperforms graph placement by large margins, especially in the ISPD benchmark, where the former reduces HPWL. compared to the latter one by up to 80% while ensuring zero overlaps in all benchmarks. More results in the IBM benchmark can be found in Appendix Table 15.
Table 3: Comparisons between GraphPlace [3] and the proposed MaskPlace using different performance metries (normalized to [0, 1)) in the "arianc" benchmark, including HPWL, congestion, density, and overlap area ratio. All values are smaller the better. We see that MaskPlace surpasses other methods significantly while ensuring zero overlaps, which is essential for a valid and manufacturable chip layout.
Method
HPWL
Congestion
Density
Overlap (5)
Graph Placement (juunal) [3]
0.1 198::0000
0.9718=0.0346
0.5729=0.0086
5,13
Graph Placement (github) [3]
0.1013+0.0036
0.917400.0447
0.5502=0 0568
4.29+ :25
MaskPlace (hard constraint)
1..025=0.0015
1.0137=0.0451
0.5000+0.0000
0.00:20.00
MaskPlace (soft constraint)
0.087920.0012
0.904920.0115
0.526220.0015
3.3340.79
Routing Wirelength. Table 5 compares the routing wirelength between MaskPlace and DeepPR [22]. We show that using the true wirelength the reward would lower the efficiency and produce sparse reward. We see that MaskPlace, which employs HPWL as the reward, can achieve 60% to 90% shorter routing wirelength than DeepPR, which directly used the true wirelength as the reward.
Standard Cells. Table 6 compares the HPWL of both the macros and the standard cells by using MaskPlace, DeepPR [22], and DREAMPlace [9], where DREAMPlace is employed to place the standard cells following the experimental setup in [22]. We can see that the proposed method
"The random seed does not apply in classic method NTUPlace3.
$ laiyaol github. iombsspio/maskpacceeee...
7

---

Table 4: Comparisons between GraphPlace [3] and MaskPlace in four performance metrics (normalized to [0. 1) in the ISPD benchmark. All values are smaller the better. We see that MaskPlace can reduce the HPWL up to 80% compared to its counterpart while cnsuring the modules' overlaps are zeros in all benchmarks.
benchmark
Graph Placement [3]
MaskPlace
HPWL
Congestion
Density
Overlap(%)
HPWL
Congestion
Density
Overlop
adapteel
0.7370
0.5340
1.89
0.0384
0.6961
0.5000
adaptec2
0.2814
0.7387
0.5147
1.54
0,0549
0.6990
0,5000
0,00
adaptee3
0,2248
0.7431
0.5226
1.24
0.0540
0.7130
0,5000
0.00
adaptec4
0.1107
0.7369
0.7472
7.59
0,0560
0.7078
0.5000
0.00
bigbluel
0.0958
0.7346
0,5181
1.98
0.0255
0.6953
0.4876
0.00
higblue3
0.5174
0.0430
Table 5: Compare routing wirelength (x10²) between DeepPR [22] and MaskPlace.
method
adaptec1
adapces22
aduptec:
adaptee4
bighlucl
highlue3
ariane
DeepPR [22]
23.25=3.03
212,9765.88 84
377,80+5.49
367.57664,44
28. 28.51
507.39+14.22
56.77=0.87
DeepPR noopellno-vverrap [22]
52.46:3.97
451.22+19.00
583.32+15.92
128.22210.02
31.02+1.41
945.60=43.24
68.89=0.81
MaskPlace
7.12:0.34
77.7016.77
90.4013882
92.5120.38
281:0.51
10324410444
15.61:0.19
surpasses the other approaches by up to 50% in the large benchmark "bigblue3", which has more than million standard cells.
Table 6: Comparisons of HPWL (x103) for macro and standard cell placement.
Method
adaptec1
aduptec2
adaptec1
adaptec4
higbluel
highlue3
DREAMPlace [9]
11.01+1.37
16. 19+2.60
21 24:54:1119
35.47+4.97
10.2841.11
70.02446.06
DoePRR [22] + DREAMPlace [9]
8.01
12.32
24.11
23.64
14.04
45.06
MaskPlace DREAMPlace 7.93:00.20
9.9520.29
21.4920.00
22.97:0.92
9.4320.13
37.2920.67
Placement w/o real size Considering that DeepPR ignored the module size, we implement MaskPlace in the same setting, and the result can be found in Table 7. The result shows that method has significant advantages.
Table 7: Routing wirelength for macro placement w/o real size
Method
adaptecl
aduplec2
adaptec3
adaptec4
bighluel
bighlue3
DeepPR [22]
5298
22256
32839
63560
8602
94083
MaskPlace
2331
27403
Transferability Test the performance of the model trained on adaptecl on other benchmarks as Table 8. The results show that our method also has good transfcrability.
Table 8: HPWL (x10)) results for transfcrability. HPWL the smaller the better. The model has been trained on adaptec/ benchmark and just took the inference other benchmarks.
adaptec?
adaptec3
adaptec4
bighluel
highlue3
ariane
HPWL
85.56+9.41
89.77a6.72
87.32+1.93
2.87+0.31
160.63=10.41
19.32+2.02
ratio'
1.16
1.06
1.11
1.20
1.76
1.32
Compared with result from the model trained on corresponding benchmark.
Efficiency. Table 9 compares the inference efficiency of different approaches. All of them are evaluated on one GeForee RTX 3090 GPU, and the CPU version of DREAMPlace is allocated with 16 threads in a 16 CPU cores environment. We see that the careful design of MaskPlace makes faster than two other learning-based approaches.
Ablation Study. We compare different components MaskPlace as shown Fig.6. Each curve produced by five seeds using the benchmark "adaptec1". For example, MaskPlace w/ CL means using 1/3 of the circuit macros to pretrain the model for 30 epochs like curriculum learning. MaskPlace w/o M means only considering 2 as input without looking one step forward. Moreover, MaskPlace
8

---

Table 9: Comparisons wall-clock runtime (second) of different placement methods inference.
Method
adaptee1
adaptee2
adaptec3
adaptec4
bighluel
bigblue3
DREAMPlace (CPU) [9]
4.47
11.50
11.52
15.55
9.32
27.36
DREAMPlace (GPU) [9]
12.25
Graph Placement [3]
6.32
16.97
20.05
13:40
4.54
15.65
DeepPR [22)
10.25
10.46
22.82
42.24
9.86
32.53
MaskPlace
4.26
6.98
7.63
13.36
4.32
13.87
w/o number of nets means this feature is not considered when determining the placement order. MaskPlace w/o Ixl means that 7x7 kernels are used replace the 1x1 kernels in the local feature fusion block. Also, MaskPlace with sparse reward means compute HPWL reward only when all macros have been placed, and the rewards for other steps are set zeros. MaskPlace w/o view mask and w/o wire mask means that the corresponding mask is not inputted into the model. We can that MaskPlace (standard) with curriculum learning has the best performance.
Congestion Satisfaction. To evaluate our congestion satisfaction block, we implement a placement without any congestion threshold (i.e., oc) as shown in Fig.7. We evaluate the "adaptec3" benchmark, where MaskPlace outperforms DeepPR. We gradually lower the threshold Ct., from 60 to 10. We find that lower congestion leads to increase in the HPWL. Our method can always satisfy the congestion constraint in five seeds in a suitable range (above 40 in this benchmark). If we continue reduce the congestion threshold after specific value (say 40 in Fig.7), 7), we found that it hardly satisfies the threshold because nets must take up a certain amount of wire resources.
20
10a
standard
Reward HPWL
w/o N
HPWL. p2000)
view
w/o rum of nets
1s1 sparse reward w/o wire mask
20
50
15
-5x10
DeepPR MankHHace (Expecied
500m
50
100
150
2000
250
Congestion
Figure 6: Compare reward of different compo- nents in MaskPlace.
Figure 7: Study of congestion satisfaction.
Conclusion
This paper proposes MaskPlace, an RL-based placement method based rich visual representation by learning position, wirelength, and view information. It helps the model take action effectively and efficiently without reducing the search space. We design direct reward function based on practical scenarios and get satisfactory results on all key metrics. This work can facilitate the placement process and avoid undesired overlaps between modules. In the future, will explore the standard cell placement by designing a suitable representation, which is an open problem for RI. due to its vast space.
Limitation and Potential Negative Societal Impact. Chip design flow contains many stages, and method shows its potential in single stage. Similar previous RL methods, also requires an optimization method when placing millions of standard cells because RL's state space is too large. Our method does not have potential harm to the public society the moment.
Acknowledgments and Disclosure of Funding
We thank Xibo Sun from Huawei answering questions about EDA. We also thank Runjian Chen for participating in our discussion. Ping Luo supported by the General Research Fund HK No.27208720, No.17212120 and No. 17200622.

---

References
[1] L.-T. Wang. Y.-W. Chang, and K.-T. T. Cheng, Electromic design automation: synthesis, verification. and rest. Morgan Kaufmann, 2009.
[2] J. M. Rabaey, A. P. Chandrakasan, and B. Nikolic, Digital integrated circuits. Cliffs, 2002, vol. 2.
Prentice hall Englewood
[3] A. Mirhoseini, A. Goldie, M. Yazgan, J. W. Jiang, E. Songhori, S. Wang, Lee, E. Johnson, O. Pathak,
A. Nazi et al., "A graph placement methodology for fast chip design." Nature, vol. 594, no. 7862, pp. 207-212, 2021.
[4] J. A. Roy, N. Adya, A. Papa, and Markov, "Min-cut floorplacement" IEEE Transactions on Computer-Aiin Design of Integrated Circuits and Systems, vol. 25, no. 7. pp. 1313-1326, 2006.
[5] A. Khatkhate, C. Li, A. R. Agnihotri, M. C. Yildiz, S. Ono, C.-K. Koh, and H. Madden, "Recursive bisection based mixed block placement," Proceedings of the 2004 international symposium on Physical design, 2004, pp. 94-89.
[6] T.-C. Chen, Z.-W. Jiang, T.-C. Hsu, H.-C. Chen, and Y.-W. Chang, "Nuplace3: An analytical placer for large-scale mixed-size designs with preplaced blocks and density constraints," JEEE Transactions on Computer-Aider Design of Integrated Circuits and Systems, vol. 27, no. 7, pp. 1228 1240, 2008
[7] J. Lu, P. Chen, C.-C. Chang, L. Sha, J. Dennis, H. Huang, C.-C. Teng, and C.-K. Cheng, "eplace: Elec- trostatics based placement using nesterov's 's method," in 2014 51st ACM/EDAC//EE Design Automation Conference (DAC). TEEE, 2014, pp. 1-6.
[8] C.-K. Cheng, A. B. Kahng. I. Kang. and L. Wang, "Replace: Advancing solution quality and routability validation in global placement." IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 38, no. 9, pp. 1717-1730, 2018.
[9] Y. Lin, Z. Jiang, J. Gu, W. Li, S. Dhar, H. Ren, B. Khailany, and D. Z. Pan, "Dreamplace: Deep learning toolkit-enabled gpu acceleration for modern vlsi placement," JEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 40, no. 4, pp. 748-761, 2020.
[10] X. Yang, M. Sarrafzadeh et al., "Dragon2000: Standard-cell placement tool for large industry circuits." " in IEEE/ACM International Conference on Computer Aided Design. ICCAD-2000. IEEE/ACM Digest of Technical Pupers (Cat. No. 00CH37140). IEEE, 2000, pp. 260-263.
[11] D. Vashisht, H. Rampal, H. Liao, Y. Lu, D. Shanbhag, F. Fallon, and L., B. Kara, "Placement in integrated circuits using cyclic reinforcement learning and simulated annealing," arXiv preprint arXiv:2011.0757 2020.
[12] N. Viswanathan, G.-J. Nam, C. J. Alpert, P. Villarrubia, H. Ren, and C. Chu, "Rql: Global placement via relaxed quadratic spreading and lincarization," in Proceedings of the 44th annual Design Automation Conference, 2007, pp. 453-458.
[13] N. Viswanathan, M. Pan, and C, Chu, "Fastplace 3.0: A fast multilevel quadratic placement algorithm with placement congestion control," in 2007 Asia and South Pacific Design Automation Conference. IEEE, 2007, pp. 135 140.
[14] M.-C. Kim, N. Viswanathan, C. J. Alpert, I, L.. Markov, and S. Ramji, "Maple: Multilevel adaptive place- ment for mixed-size designs," in Proceedings of the 2012 ACM international symposiun on International Symposium on Physical Design, 2012, pp. 193-200.
[15] M.-C. Kim and I. L. Markov, "Complx: A competitive primal-dual lagrange optimization for global placement," Proceedings of the 49th Annual Design Automation Conference, 2012, pp. 747-752.
[16] U. Brenner, A. Hermann, N. Hoppmann, and P. Ochsendorf, "Bonnplace: A self-stabilizing placement framework," in Proceedings of the 2015 Symposium on International Symposium on Physical Design, 2015, pp. 9-16.
[17] T. Lin, C. Chu, J. R. Shinnerl, I. Bustany, and I. Nedelchev, "Polar: Placement based on novel rough legalization and refinement," in 2013 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). IEEE, 2013, pp. 357-362.
[18] P. Spindler, U. Schlichtmann, and F. M. Johannes, "Krafteer22 a fast force- force-directed quadratic placement approach using an accurate net model," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 27, no. 8, pp.
2008.
10

---

[19] T. F. Chan, J. Cong. J. R. Shinnerl, K. Sze, and M. Xie. "mpl6: Enhanced multilevel mixed-size placement," in Proceedings of the 2006 international symposium Physical dexign, 2006, pp. 212-214.
[20] A. B. Kahng, S. Reda, and Q. Wang, "Aplace: A general analytic placement framework," in Proceedings of the 2005 international symposium Physical design, 2005, pp. 233-235.
[21] J. Gu, Z. Jiang, Y. Lin, and D. Z. Pan, "Dreamplace 3.0: Multi-electrostaties based robust vlsi placement with region constraints," " in 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 2020, pp. 1-9.
[22] R. Cheng and J. Yan. "On joint learning for solving placement and routing in chip design." Advances in Neural Information Processing Systems, vol. 34, 2021.
[23] Z. Jiang, E. Songhori, S. Wang, A. Goldie, A. Mirhoseini. J. Jiang. Y.J. Lee, and D. Z. Pan, "Delving into macro placement with reinforcement learning," in 202/ ACM/IEEE 3rd Workshop on Machine Learning for CAD (MLCAD). IEEE, 2021, pp. 1-3.
[24] T.-C. Chen, Z.-W. Jiang, T.-C. Hsu, H.-C. Chen, and Y.-W. Chang, high-quality mixed-size analytical placer considering preplaced blocks and density constraints," in Proceedings of the 2006 IEEF/ACM International Conference on Computer-Aided Design, 2006, pp. 187-192.
[25] P. Spindler and F. M. Johannes, "Fast and accurate routing demand estimation for efficient routability placement," in 2007 Design, Automation & Test in Europe Conference & Exhibition. IEEE, 2007, pp.
1-6.
[26] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction.
MIT press, 2018.
[27] Z. Guo, J. Mai, and Y. Lin, "Ultrafast cpu/gpu kemels for density accumulation in placement," in 202/ 58th ACM/IEEE Design Automation Conference (DAC). IECE, 2021, pp. 1123-1128.
[28] V. Konda and J. Tsitsiklis, "Actor-critic algorithms," Advances in newral information processing systems, vol. 12, 1999.
[29] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal policy optimization algorithms," arXiv preprint 130iv:7777006333 2017.
[30] G.-. Nam, C. J. Alport, P. Villarrubia, B. Winter, and M. Yildiz, "The ispd2005 placement contest and benchmark suite," in Proceedings of the 2005 international symposium Physical design, 2005, pp. 216-220.
[31] S. Adya, S. Chaturvedi, and I. Markov, "Iccad '04 mixed-size placement benchmarks," GSRC Bookshelf. 2009.
[32] F. Zaruba and L. Benini, "The cost of application-clas. processing: Energy and performance analysis of linux- Anux-ready 1.7-ghz 64-bit risc-v core in 22 nm fdsoi technology," IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 27, no. 11, pp. 2629-2640, 2019.
[33] T. H. Cormen, C. E. Leiserson, R. L.. Rivest, and C. Stein, Introduction to algorithms. MIT press, 2022,
[34] H. Liao, Q. Dong, X. Dong, W. Zhang, W. Zhang, W. Qi, E. Fallon, and L. B. Kara, "Attention routing: track- assigneett detailed routing using attention based reinforcement learning"" in International Design Engineering Technical Conferences and Computers and Information Engineering Conference, vol. 84003. American Society of Mechanical Engineers, 2020, p. VI1ATI1
[35] S. Kirkpatrick, C. D. Gelatt Jr, and M. P. Vecchi, "Optimization by simulated annealing," science, vol. 220, no. 4598, pp. 671-680, 1983.
[36] J. Yan. X. Lyu, R. Cheng. and Y. Lin, "Towards machine learning for placement and routing in chip design: methodological overview," arXiv preprint arXiv:2202. 13564, 2022.
[37] Y. Huang, Z. Xie, G.-Q. Fang, T.-C. Yu, H. Ren, S.-Y. Fang, Y. Chen, and J. Hu, "Routability-dii driven macro placement with embedded enn- cnn-based prediction model," in 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 2019, pp. 180-185.
[38] R. Kirby, K. Nottingham, R. Roy, S. Godil, and B. Catanzaro, "Guiding global placement with reinforce- ment learning," arXiv preprint ar arXiv:2109.022
2021.
[39] A. Agnesina, K. Chang, and S. K. Lim, "VIsi placement parameter optimization using deep reinforcement learning"" in Proceedings of the 39th International Conference on Computer-Aided Design, 2020, pp. 1-9.
11

---

[40] F.-C. Chang, Y.-W. Tseng, Y.-W. Yu, S.-R. Lee. A. Cioba, I.-L. Tseng, D.-s. Shiu, J.-W. Hsu, C.-Y. Wang. C.-Y. Yang al., "Flexible mulliple-objective reinforcement learning for chip placement," arXiv preprint arXiv:2204. 04407, 2022.
[41] T. N. Kipf and M. Welling. "Semi-suuprrvsed classification with graph convolutional networks," arXiv preprint wXXiv:600.0290000 2016.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(C) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
12


Reviews:
[{"review_id": "t-VkVI1p2te", "reviewer": "Reviewer_MdDg", "summary": "Placement is one critical step to ensure circuit performance including power consumption, delay, and chip area in physical design. This paper presents a reinforcement-learning-based approach to optimize the half perimeter wirelength (HPWL), placing circuit modules sequentially. This proposed method adopts multi-view (mask) visual representation from the placement to ensure the non-overlapping property among modules. Each mask clearly captures a certain aspect (allowable positioning, metric change, global placement information) at the current placement snapshot, resulting in efficient policy learning. The paper performed an extensive experimental study on multiple benchmarks and demonstrated the effectiveness of individual components through the ablation study.", "questions": "1. For congestion satisfaction, the reviewer wonders what is the current reasoning behind this inconsistency in the training and inference phases, and if there is any way to mitigate this effect.\n1. Detailed components clarification 1: How did the position embedding at time $t$ in the value network works? In addition, what is the purpose of this component?\n1. Detailed components clarification 2: It seems that the ordering of the module for placement matters a lot according to Fig. 6. The reviewer wonders how the current sorting is done exactly. The authors said \"*we follow previous works to sort the circuit modules ...*\" in Training and Testing of Section 3. However, no corresponding literature is given. In addition, it would be appreciated if the authors can give more comments on the reasoning behind the sorting heuristics, and what is the consequence of bad ordering.\n1. Detailed components clarification 3: The reviewer understands the power of looking more steps forward to the RL agent, and wonder how the step are generated here. In addition, it would be great if the authors can comment on the tradeoff between the computation efficiency and learning quality w.r.t. the number of steps looking ahead.\n1. Detailed components clarification 4: It is unclear how the algorithm works if the module size doesn't fit the grid cell. For example, if a module occupies one grid cell without covering the entire grid, is it still possible to place another module in the same grid?\n1. To the reviewer's understanding, the view mask contains only two values ${0,1}$. However, the view mask in Fig.3 contains three different colors. Is it a simple illustration error?\n1. Table 7 gives the inference time for various methods. The reviewer is curious about how long the training takes. Moreover, do the training examples come from only a single placement task which is also used in inference, or multiple placement tasks which can be generalized to different tasks during the inference time?", "limitations": "The proposed methods can be hardly applied to standard cell placement due to the large state and action space. However, the authors show in Table 6 that the placement tool which can handle standard cell placement (like DREAMPlace) may also benefit from the proposed method of handling module placement. This is beneficial for the EDA tool development in general.\nThere is no identified negative societal impact.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. This paper transforms the geometric placement problem into multiple visual representations using three masks, which opens the possibility of using mature convolution networks to extract the global layout information. In addition, each mask has its distinct purpose, benefiting the overall placement task from different angles. The position mask guarantees the non-overlapping property of the proposed method; the wire mask gives a good estimation of the metric (HPWL) change for the action; and the view mask provides a global view of the current placement results.\n1. The paper provides multiple components which can be beneficial for future research in this area. The authors designed multiple masks for the placement and their corresponding efficient generation algorithms, which can be used as individual components to inspire other related placement and routing tasks.\n1. This paper conducts an extensive experimental study of the proposed methods over various benchmarks and multiple metrics against both state-of-the-art optimization-based and RL-based approaches, demonstrating the effectiveness of their method. In addition, the ablation study also shows the power of each individual component of the proposed method.\n\nWeaknesses:\n1. There exist certain inconsistencies between the training and inference phase due to congestion satisfaction. The congestion satisfaction is not considered during the RL training phase. However, it is used during the inference phase. During the inference phase, the probability matrix of the action doesn't anticipate the effect of the congestion satisfaction, which may result in lower quality placement compared to the training phase (ignoring the congestion constraint).\n1. Some detailed components of the design RL frameworks are not fully explained, as listed in the questions below.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "jSmF3aIHjL_", "reviewer": "Reviewer_iBtL", "summary": "This paper proposes MaskPlace, which can automatically generate a high-quality and valid layout within a few hours. Unlike previous methods, that requires manual refinement to modify invalid placement, MaskPlace casts placement as a problem of pixel-level visual representation learning. More particularly, 1) Visual representation learning enables describing millions of circuit modules on a chip comprehensively. 2) MaskPlace suggests a new policy network that can capture and aggregate both the global and subtle information on a chip canvas, ensuring non-overlapping placement efficiently. 2) The demonstrated performance on 24 public chip benchmarks by outperforming Graph Placement and DeepPR 5x and 9x in reducing wirelength with 0% overlap in layout. ", "questions": "1. The reviewer likes the paper for  reasons listed in above session. However, the reviewer is not quite sure about the adds-on values of thsi work compared to the original chip placement paper from Google [1] and why it is so important to the NeurIPS community. I would think the content in the paper makes it more suited for a design automation conference. The contributions in representation learning or algorithm design are small. \n\n2. Can the authors provide some concrete comparisons on search time or optimization time with baseline algorithms? A great thing to have is to create a figure like Figure 6 but with different curves being the baselines. Make sure to plot wall clock time in the x-axis, instead of epochs as you might create a very complex network which can be 10x slower in step time, compared to the very simple baselines. \n\n[1] \"Chip Placement with Deep Reinforcement Learning\", https://arxiv.org/abs/2004.10746", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper suggested a more accurate representation of chip layout: the three types of pixel-level feature maps can fully represent massive net and pin configurations. Different from DeepPR, that assumed all modules have unit size, this work considers real sizes of modules. \n\n- The dense reward is an interesting idea . It is an inexpensive way to create an advantage value at each reinforce step, instead of generating one reward at only the end of the episode after the placement is done. This can be both good and bad those, as the decisions can be short-sighted such that the algorithm tries to incrementally improve wire length, losing performance opportunity from sacrificing short-term objectives. However, both the main results and ablation show that dense reward positively improves learning. \n\n- The paper has thorough comparisons with SoTA related work and demonstrated significant gains on a set of benchmark. Metrics like wire length, density, and overlap are improved. \n\n- The reviewer is happy to see Ablation Figure 6 of a decomposition of the gains. All the design decisions add up to some of the performance. \n\nWeaknesses:\n- The results look very strong, however, are a bit counter intuitive. The reward only considers wire length by continuously measuring an advantage function at each reinforce step. The reviewer is not so sure how it could improve density, which is demonstrated to be improved in all the results. Wire length and density are non-orthogonal metrics and are usually in conflict with each other. \n\n- The paper lacks more details on how the baseline methods are implemented and whether the comparisons are fair and all the promising numbers are trustable. \n\n- There is little formulations. Local mask fusion, global mask encoder and decoder, and other components in Figure 3 should be formulated mathematically for others to reproduce the experiments. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ckQ37N5LyCF", "reviewer": "Reviewer_TA3J", "summary": "The paper uses reinforcement learning to solve the chip placement problem. It views the placement canvas as a 2D image during the representation learning.\n", "questions": "1. Lines 192 - 201. The authors design the reward as the change of HPWL. How is the HPWL_t calculated if there are nodes not placed? For example, in a net with 3 pins on 3 nodes, what is the HPWL_t when we only place the first node? Since the other 2 nodes are not placed, it does not make sense to compute the HPWL for this net.\n2. The authors use CNNs in representation learning. What about other neural architectures, such as Transformers?\n3. What is the generation ability of the proposed method? Can it be used on unseen circuit benchmark? ", "limitations": "The authors discuss the limitation at the end of the paper. The proposed method can only handle a small number of macros. It can not tackle a large number of movable instances, such as millions of standard cells. I think the authors should stress this limitation throughout the paper. Otherwise, readers may think that the paper proposes a method to place all instances. In the first three section, the authors use the term \"placement\" instead of \"macro placement\".", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. The paper is well written and organized. The appendix discusses related details.\n2. The method is sound and well defined.\n3. The results are charming and convincing.\n\nWeaknesses\n1. DREAMPlace is used as a baseline, which is not a good implementation for macro placement. The current DREAMPlace is majorly optimized for standard cell placement. It cannot handle macro placement very well. The authors may consider RePlAce [1] as the baseline.\n[1] C. Cheng, A. B. Kahng, I. Kang and L. Wang, \"RePlAce: Advancing Solution Quality and Routability Validation in Global Placement,\" in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 38, no. 9, pp. 1717-1730, Sept. 2019, doi: 10.1109/TCAD.2018.2859220.\n2. The authors use ISPD 2005 benchmarks, which are for standard cell placement. Macros are fixed in these benchmarks. Authors should describe how they edit the benchmarks. The authors should also consider using MMS benchmarks [2].\n[2] J. Z. Yan, N. Viswanathan and C. Chu, \"Handling complexities in modern large-scale mixed-size placement,\" 2009 46th ACM/IEEE Design Automation Conference, 2009, pp. 436-441, doi: 10.1145/1629911.1630028.\n3. Several benchmarks are missing, such as bigblue2, bigblue4. The authors should report the results of these benchmarks if they select the benchmark suites. Otherwise, readers may suspect that these results are not ideal.\n4. The ablation study is not thorough. For instance, there are many masks in the proposed method. What is the contribution of each mask? Is every single mask necessary? What is the impact of the combination of these masks? Specifically, if the method is really great, it does not need the position mask that tells the agent the availability since the agent should have this information.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "RhVNVCugcAg", "reviewer": "Reviewer_f5N4", "summary": "This paper presents a RL-based chip placement approach, MaskPlace, to automatically generate a valid chip layout design. Compared with the former approaches that apply hypergraph to represent the chip layout, MaskPlace adopts the pixel level graphical representation to represent the layout and pin offset, which allows for MaskPlace to produce a better performance. The results show that MaskPlace can achieve 60%-90% wirelength reduction with zero overlaps.", "questions": "I have the following questions for the paper:\n1. Since MaskPlace uses the complicated 2D image as the input of the RL agents, I believe this will harm the convergence behavior of the MaskPlace compared with the other hypergraph-based approaches, as their input states will be much simpler. What's the convergence behavior of MaskPlace? What makes MaskPlace converge within a few hours?\n\n2. How is the simulation is performed? Is it based on the real RTL synthesize netlist as [3]? If not, is it simulator based? Please describe the experiment settings in more detail.\n\n3. MaskPlace eliminates the decision which violates the congestion constraint, why not include this congestion constraint in the reward function, so that a decision that violates the congestion constraint will produce a large negative reward?", "limitations": "NA.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths of the paper:\n1. Chip placement with RL has been recently studied by multiple literature, this work proposes a better solution for chip placement. The comprehensive 2D pixelwise representation of the chip layout and wire length enables MaskPlace to involve the pin offset information when making the placement decision. This results in a better performance than the previous work which adopts hypergraph to represent the chip layout.\n\n2. To generate the input state for the RL agents, heuristic algorithms are proposed to produce the input masks for the RL agents with relatively low complexity.\n\n3. The presentation of the paper is clear, the experimental results are comprehensive and promising.\n\nWeakness of the paper:\n\nSee the question section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept