MaskPlace: Fast Chip Placement via Reinforced Visual Representation Learning
Abstract
Placement is an essential task in modern chip design, aiming at placing millions of circuit modules on a 2D chip canvas.
Unlike the human-centric solution, which requires months of intense effort by hardware engineers to produce a layout to minimize delay and energy consumption, deep reinforcement learning has become an emerging autonomous tool.
However, the learning-centric method is still in its early stage, impeded by a massive design space of size ten to the order of a few thousand.
This work presents MaskPlace to automatically generate a valid chip layout design within a few hours, whose performance can be superior or comparable to recent advanced approaches.
It has several appealing benefits that prior arts do not have.
Firstly, MaskPlace recasts placement as a problem of learning pixellevel visual representation to comprehensively describe millions of modules on a chip, enabling placement in a high-resolution canvas and a large action space.
It outperforms recent methods that represent a chip as a hypergraph.
Secondly, it enables training the policy network by an intuitive reward function with dense reward, rather than a complicated reward function with sparse reward from previous methods.
Thirdly, extensive experiments on many public benchmarks show that
MaskPlace outperforms existing RL approaches in all key performance metrics, including wirelength, congestion, and density.
For example, it achieves 60%- 90% wirelength reduction and guarantees zero overlaps.
We believe MaskPlace can improve AI-assisted chip layout design.
The deliverables are released at laiyao1.github.io/maskplace.
Introduction
The scalability and efficiency are two significant factors of autonomous chip layout design.
Placement is one of the most challenging and time-consuming problems in the design flow, aiming to determine the locations of millions of circuit modules on a 2D chip canvas represented by a two-dimensional grid.
A netlist can describe these modules, that is, a large-scale hypergraph consisting of massive macros (functional blocks such as memory) and standard cells (logic gates) where each macro and each standard cell can contain several or even hundreds of pins connected by wires, as shown in Fig.1.
Placing a large number of circuit modules onto the chip canvas is challenging because many performance metrics such as power consumption, timing, area, and wirelength should be minimized while satisfying some hard constraints such as placement density and routing congestion.
For example, the wirelength (the length of wires that connect all modules) determines the delay and the power consumption of a chip [1].
Shorter wires often indicate less delay and less power consumption [2].
However, wirelength cannot be reduced by overlapping modules because the module density is a hard constraint to ensure that a valid and manufacturable chip layout has non-overlapping modules.
More
[Figure]
examples of the performance metrics are given in Fig.8 and Fig.9 in Appendix.
As pointed out in [3], the design space of placement is larger than 10 2 , 500 when there are just 1 , 000 circuit modules, whereas neural architecture search (NAS) typically has a space of 10 30 and the Go game has a state space of 10 360 .
Methods of chip placement can be generally divided into two categories, classic optimizationbased approaches [4–21] and learning-based approaches [3, 22, 23].
In the first category, hardware scientists often formulate placement as an optimization problem and relax the hard constraints.
For example, let a pair of vectors ( x , y ) denote the ( x, y )-coordinate value of all circuit modules on a 2D canvas, the objective function of placement can be formulated as minimizing WL( x , y ) , subject to D( x , y ) ≤ α , where WL( · , · ) and D( · , · ) are the estimation functions of wirelength and density respectively, and D( x , y ) ≤ α is a hard constraint with a very small density value α , which ensures that all modules do not overlap.
For instance, DREAMPlace [9] is a recent advanced method that minimizes WL( x , y )
+ λ D( x , y ) , which relaxes the hard density constraint.
However, it cannot directly produce a valid and manufacturable layout because the non-overlapping constraint is not satisfied after relaxation.
These approaches often need a post-processing step, such as manual refinement and legalization (LG), to remove the overlapping in placement, resulting in two issues, (1) the wirelength may increase substantially after LG, and (2) no feasible solution can be found if the available chip area is insufficient before post-processing.
In the second category, reinforcement learning (RL) is employed to solve placement as a sequential decision-making problem, placing each circuit module at a time.
Although the learning-based approaches are still in their early stage, they can produce promising results to automate the chip design flow end-to-end significantly without human effort.
For instance, Graph Placement [3] and
DeepPR [22] represent a netlist as a hypergraph, denoted as G =
( V, E ) , where V represents a set of nodes, and each node is a module, and E is a set of edges, which are the wires connecting all modules.
They train RL agents to place one module at a time by maximizing the metric values as rewards.
However, the hypergraph is not scalable to comprehensively encode information of a netlist.
For example, the relative positions (offsets) of pins are discarded in [3, 22].
The wirelength estimation is inaccurate without the pin information, but encoding this rich information would make the hypergraph too complicated because each module can have hundreds of pins.
Furthermore, placement on a large hypergraph requires heavy computations.
Mirhoseini et al.
[3] reduced computations by placing 15% of the modules using reinforcement learning (the remaining modules are placed by classic method), and Cheng and Yan [22] decreased the size (resolution) of module and chip canvas as shown in Table 1.
Both of them sacrificed their placement performance.
We see that MaskPlace can outperform recent advanced methods by performing placement on a full canvas size of 224 × 224 (much larger than prior works) and producing a valid placement with 0% overlap (which cannot be achieved by previous methods).
MaskPlace can also be trained and tested efficiently
[Table]
To address the issues of prior arts
[Table]
we propose a novel RL method, named MaskPlace, which can automatically generate a high-quality and valid layout (non-overlapping modules) within a few hours, unlike previous methods that need manual refinement to modify invalid placement
[Table]
which may wait up to 72 hours for commercial electronic design automation (EDA) tools to evaluate the placement.
MaskPlace casts placement as a problem of pixel-level visual representation learning for circuit modules using convolutional neural networks.
This representation can comprehensively capture the configurations of thousands of pins, enabling fast placement in a full action space on a large canvas size e.g., 224 × 224.
As shown in Fig.1 and Table 1, MaskPlace has many attractive benefits that existing works do not have.
MaskPlace is mainly for macro placement due to the problem size.
This paper has three main contributions .
Firstly, we recast chip placement as a problem of learning visual representation to describe millions of circuit modules on a chip comprehensively.
It opens up a new perspective for AI-assist chip placement.
Secondly, we carefully design a new policy network that can capture and aggregate both the global and subtle information on a chip canvas, maximizing the reward of wirelength and ensuring non-overlapping placement efficiently.
Thirdly, extensive experiments demonstrate that MaskPlace outperforms recent advanced methods on 24 public chip benchmarks.
For example, MaskPlace can always produce a layout with 0% overlap while reducing wirelength up to 5 × and 9 × compared to Graph Placement [3] and DeepPR [22] respectively.
Preliminary and Notation
The placement quality can be measured by the HPWL (half perimeter wirelength), which estimates the wirelength with marginal computational cost [24].
Intuitively, Fig.2(e) illustrates a 2D chip canvas.
Let M i and P ( i,j ) denote the i -th module and its j -th pin, respectively.
A net contains a set of pins connecting modules by wires.
For example, “Net 1” (in red) connects all four modules ( i.e., M 1 , M 2 , M 3 , M 4 ) using wires through pins P (1 , 2) , P (2 , 2) , P (3 , 2) , and P (4 , 1) , while “Net 2” (in green) connects three modules ( i.e., M 1 , M 2 , M 3 ) using wires through pins P (1 , 1) , P (2 , 1) , and
P (3 , 1) .
HPWL estimates the wirelength by summing up the half perimeters of bounding boxes of all the nets, as shown by the red and green boxes in Fig.2(e).
Intuitively, the half perimeter of a net bounding box equals the sum of its height and width.
For example, HPWL in Fig.2(e) is h 1 + w 1 + h 2 + w 2 .
Given a netlist containing a set of nets, minimizing the wirelength can be treated as minimizing
HPWL by placing modules to the optimal positions on a 2D chip canvas.
To achieve a valid and manufacturable chip layout, we need to satisfy two hard constraints: (1) congestion constraint : the wire congestion should be lower than a desired small threshold to reduce chip cost, and (2) overlap constraint : the density should be minimized to achieve non-overlapping placement.
[Equation]
.t
[Equation]
where P x and P y represent the ( x, y ) -coordinate value of a pin respectively, Congestion( · ) is the congestion function, C th is a desired threshold, Overlap( · ) is the overlap function, and M x , M y , M w , M h
[Figure]
represent the position, width and height of modules respectively.
Firstly, lower congestion often indicates shorter wirelength, which is crucial to reduce chip cost because the wire resources are limited on a real chip.
Inspired by prior arts [3, 22], we employ the RUDY estimator [25] to estimate wire congestion.
Details of RUDY can be found in the Appendix A.2.
Secondly, the placement density calculates the overlapping region between every pair of circuit modules.
It is time-consuming since its computational complexity is O ( V 2 ) where V is the number of modules [1].
The proposed approach can ensure non-overlapping placement to avoid calculating this density metric explicitly in training, thus reducing computations while producing a valid layout.
Our Approach
Model Architecture Overview.
Chip placement can be formulated as a Markov Decision Process (MDP) [26] by placing each module at a time.
Fig.4 illustrates the overall architecture of MaskPlace, which trains a policy π θ ( a t | s t ) represented by a convolutional encoder-decoder network with parameter set θ , and a value function V ϕ ( s t ) represented by an embedding model with parameter set ϕ .
The policy network receives previous observations and actions as input s t and selects an action a t as output.
Specifically, s t is a set of pixel-level feature maps that comprehensively capture the net and pin configurations in M 1: t − 1 , M t , and M t +1 , where M 1: t − 1 denotes the modules that have been placed in the previous time steps from 1 to t − 1 , while M t and M t +1 denote the modules to be placed at the current step t and the next step t + 1 , respectively.
Intuitively, MaskPlace looks one step forward to achieve better placement.
[Figure]
Although prior arts [3, 22] represented a netlist as a hypergraph as shown in Fig.2(f) where each node is a module, and each edge is a wire between two modules, they lost the information of pin offsets for each module.
Unlike previous works, MaskPlace can fully represent massive net and pin configurations using three types of pixel-level feature maps, as shown in Fig.2(a-d), including position mask, wire mask, and view mask, as discussed below.
Different masks are fused by convolutions to learn the state representation.
Position Mask.
The position mask, denoted by f p ∈ { 0 , 1 } 224 × 224 , is a binary matrix of a canvas grid with size 224 × 224 as shown in Fig.3, where value “ 1 ” means a feasible position to place a module.
The purpose of the position mask is to guarantee no overlaps between modules ( i.e., satisfy the overlap constraint) and to learn the relationship between placement and wirelength.
Specifically, we slide a module M t (for example,
[Figure]
t = 5 ) on the entire chip canvas.
The trajectory of the feasible positions (in green) can be labeled with “1”.
Intuitively, we can check each position for each module using the cumulative sum array [27].
This naive approach has the computational complexity of O ( N 2 ) when a 2D canvas grid is divided into N × N cells.
However, this simple approach is not efficient when N is large.
Therefore, since all modules are rectangles, we design an efficient generation algorithm, which iterates through all placed modules (in blue) and excludes positions that will cause overlap.
In this case, all remaining positions are available for placement.
The new algorithm is summarized in Appendix A.3, which costs O ( V ) for each module, where V is the number of modules.
[Figure]
Wire Mask.
The wire mask, denoted as f w ∈ [0 , 1] 224 × 224 , is a continuous matrix for representing how HPWL increases if we place a module M t in a specific position.
Fig.5 shows a sample of wire mask, where each value means the increase of HPWL.
The wire mask aims at finding the best position with the minimum increase of the wirelength.
Intuitively, we can calculate the HPWL at each canvas position, leading to a complexity of O ( N 2 P ) , where P is the total number of pins.
However, a fast algorithm can be designed by considering the relationships between the pin offset, the net bounding box, and the linear property of the HPWL metric.
For example, Fig.3 illustrates that the next module Mhas two pins, P (5 , 1) and P (5 , 2) , belonging to “Net 1” and “Net 2” respectively (Fig.2(e)).
Fig.5 illustrates the increase of wirelength when placing M 5 at each canvas location.
For instance, if Mis at the bottom-left corner, its Manhattan distance to the two net bounding boxes (in red and green) is 2 + 2 = 4 .
To calculate the Manhattan distance more accurately, we move the net bounding box compared to the pin location.
For example, since P (5 , 2) is located at (2 , 1) 3 , we move the bounding box of Net 2 (in green) in the direction − ∆ (5 , 2) =
( − 2 , − 1) to encode the information of pin offset.
The time complexity can be reduced to O ( NP ) .
The algorithm can be found in Appendix A.3.
View Mask.
The view mask, denoted as f v ∈ { 0 , 1 } 224 × 224 , is a global observation of the current chip layout, where the value “ 1 ” means a module has occupied this grid cell.
Different from DeepPR [22] that assumed all modules have unit size, we consider real sizes of modules.
For instance, if a module has size w × h , it covers ⌈ wN/W ⌉×⌈ hN/H ⌉ cells in the canvas, where W and H represent the canvas size and ⌈·⌉ denotes the ceiling function.
Learning Algorithm.
We train different blocks in Fig.4 as a whole using reinforcement learning.
The detailed network architectures are provided in Appendix A.4.
Firstly, we apply the above masks to represent the entire circuits and feed them to downstream networks.
Secondly, a global feature encoder embeds the view mask of current placement and the wire masks of the following two steps into an embedding vector.
Then we combine it with the positional embedding of the t -th circuit module in the value network to generate a scalar to evaluate the current state by fully-connected layers.
Thirdly, a global mask decoder recovers a feature map of size N 2 , which is fused with different position masks and wire masks in the policy network using 1 × 1 convolutions to avoid the local signal diffusion.
The policy network predicts a probability action matrix of size N × N , indicating where to put the next module.
Before sampling actions, we can remove unfeasible actions using the position mask.
Finally, the congestion satisfaction block applies the congestion threshold on the probability matrix to select a final action.
Reinforcement Learning.
We borrow the representative actor-critic diagram [28] and PPO2 framework [29] to train the policy π θ ( a t | s t ) , where the state representation s t is listed in Table 11 in
Appendix.
The action a t is the canvas position (cell) to place the circuit module.
Specifically, we treat the chip canvas as a grid and divide it into N × N cells, leading to N 2 possible actions.
The objective function of the policy network can be formulated as
[Equation]
where the ratio r t ( θ ) =
π θ ( a t | s t )
π θ old ( a t | s t ) and ˆ A t =
G t − ˆ V t denotes the advantage function.
We employ
G t = (cid:80) V − t − 1 k = γ k r t + k +1 that is the cumulative discounted reward and ˆ V t is the estimated value produced by the value network.
We update the the value network by optimizing its objective,
L value ( ϕ ) =
ˆ E (cid:2) ( G t − ˆ V t ) 2 (cid:3) .
Reward r t .
We treat HPWL as the reward because wirelength is the main optimization target in different performance metrics.
This is different from prior arts [3, 22] that weighted combines
HPWL and congestion as the reward, which introduces the weighting coefficient as an extra hyperparameter to tune.
Specifically, we achieve a dense reward by defining a partial HPWL, which only computes the currently placed pins.
For example, the partial HPWL for t modules can be defined as HPWL t .
In other words, we compute HPWL t after taking action a t .
The reward for the step t is r t =
HPWL t − 1 − HPWL t , i.e., the opposite number of the increase of HPWL.
Furthermore, instead of computing HPWL at each step, we can maintain the ranges of all net bounding boxes in one episode and update the changes of their sizes with a cost of O ( P ) , where P is the number of pins.
Thus we can generate dense rewards while maintaining efficiency.
Training and Testing.
Before training, we follow previous work [3] to sort the circuit modules according to the number of nets, areas, and the number of connected modules that have been placed to determine the place order.
In training, we update the policy and value networks at each epoch by ignoring the congestion satisfaction block.
When updating the value network, we stop the gradient back-propagated in the global mask encoder to avoid influence on the policy network.
The detailed training setup is provided in Appendix A.5.
In the testing stage, for each step t , we obtain a probability matrix from the policy network and sample one place action a t .
Then, the congestion satisfaction block will check whether the congestion exceeds a threshold C th after applying this action.
If it exceeds, we uniformly sample a few actions, look up the corresponding values from the wire mask f tw , and estimate the congestion before taking these actions.
We choose the action with the minimal value in f tw and the congestion lower than C th .
If all actions cannot satisfy congestion C th , we select the action with the minimal congestion and move to the next step.
Detailed congestion satisfaction algorithm can be seen in Appendix A.3.
Experiments
We extensively evaluate MaskPlace and compare it with several recent advanced placement methods, including NTUPlace3 [6], RePlAce [8], DREAMPlace [9], Graph Placement [3], and DeepPR [22], where NTUPlace3, RePlAce and DREAMPlace are optimization-based methods, whilst Graph
Placement and DeepPR are learning-based approaches.
All of them are evaluated on different public circuit benchmarks.
All previous works are evaluated by following their experimental settings.
Benchmark.
We evaluate MaskPlace in 24 circuit benchmarks selected from public datasets including the widely-used ISPD2005 [30], IBM benchmark suite [31], and Ariane RISC-V CPU design [32].
The number of evaluated benchmarks is three times more than previous work [9, 22, 3].
The statistics of benchmarks are given in Table 14 in Appendix A.6, where the largest circuit contains 1,293 macros, 22,802 pins, and more than a million standard cells, leading to a vast state space as aforementioned.
Main Results.
Table 2 compares the HPWL results between all the above methods to place all macros.
To enable a fair comparison, we evaluate all approaches 4 by using five random seeds and report the means and variances.
Since the original DeepPR method did not capture macro size (thus does not avoid overlap between adjacent macros because all macros have unit size), we extend DeepPR to model macro size to reduce the overlapping ratio.
We name it “DeepPR-no-overlap”.
Similar to prior works, we use the minimum spanning tree algorithm
[33] to estimate routing wirelength [34].
From
Table 2, we can see that MaskPlace achieves the lowest wirelength in six out of seven benchmarks (except “adaptec4” where it still outperforms all learning-centric methods).
We also see that the conventional optimization-based approaches may fail when the circuit benchmark has high chip area usage, such as “bigblue3 ” and “ariane”.
Also, MaskPlace gets the lowest wirelength compared with
Graph Placement and simulated annealing [35] in the IBM benchmark, which is shown in Appendix
A.7.
This project website 5 visualizes and compares different placements.
[Table]
Compare to Graph Representation.
Since Graph Placement [3] is the recent advanced learningbased approach that employs hypergraph for placement, we compare MaskPlace with it in all four performance metrics, including HPWL, congestion, density
[Table]
and overlap area ratio.
Table 3 andreport the results.
The overlap area ratio describes the ratio of the overlapping area between macros divided by the chip area.
In Table
[Table]
MaskPlace (soft constraint) means that the round function rather than the ceiling function is used to calculate the number of grid cells occupied by the placed macros.
MaskPlace with soft constraints may produce better HPWL and congestion than its counterpart with hard constraints, but the overlap area ratio could not be zero because the constraints have been relaxed.
From Table 3 and
[Table]
we see that MaskPlace outperforms graph placement by large margins, especially in the ISPD benchmark, where the former reduces HPWL compared to the latter one by up to 80% while ensuring zero overlaps in all benchmarks.
More results in the IBM benchmark can be found in
Appendix Table
[Table]
[Table]
Routing Wirelength.
Table 5 compares the routing wirelength between MaskPlace and DeepPR [22].
We show that using the true wirelength as the reward would lower the efficiency and produce a sparse reward.
We see that MaskPlace, which employs HPWL as the reward, can achieve 60% to 90% shorter routing wirelength than DeepPR, which directly used the true wirelength as the reward.
Standard Cells.
Table 6 compares the HPWL of both the macros and the standard cells by using
MaskPlace, DeepPR [22]
[Table]
and DREAMPlace [9], where DREAMPlace is employed to place the standard cells following the experimental setup in [22].
We can see that the proposed method
[Table] [Table]
[Table]
surpasses the other approaches by up to 50% in the large benchmark “bigblue3”, which has more than a million standard cells
[Table] [Table]
Placement w/o real size Considering that DeepPR ignored the module size, we implement
MaskPlace in the same setting, and the result can be found in Table 7.
The result shows that our method has significant advantages
[Table] [Table]
Transferability Test the performance of the model trained on adaptec1 on other benchmarks as
Table 8.
The results show that our method also has a good transferability.
[Table]
Efficiency.
Table 9 compares the inference efficiency of different approaches.
All of them are evaluated on one GeForce RTX 3090 GPU, and the CPU version of DREAMPlace is allocated with 16 threads in a 16 CPU cores environment.
We see that the careful design of MaskPlace makes it faster than two other learning-based approaches.
Ablation Study.
We compare different components in MaskPlace as shown in Fig.6.
Each curve is produced by five seeds using the benchmark “adaptec1”.
For example, MaskPlace w/ CL means using 1/3 of the circuit macros to pretrain the model for 30 epochs like curriculum learning.
MaskPlace w/o
M t +1 means only considering M t as input without looking one step forward.
Moreover, MaskPlace
[Table]
w/o number of nets means this feature is not considered when determining the placement order.
MaskPlace w/o 1x1 conv means that 7x7 kernels are used to replace the 1x1 kernels in the local feature fusion block.
Also, MaskPlace with sparse reward means compute HPWL reward only when all macros have been placed, and the rewards for other steps are set to zeros.
MaskPlace w/o view mask and w/o wire mask means that the corresponding mask is not inputted into the model.
We can see that MaskPlace (standard) with curriculum learning has the best performance.
Congestion Satisfaction.
To evaluate our congestion satisfaction block, we implement a placement without any congestion threshold ( i.e., ∞ ) as shown in Fig.7.
We evaluate the “adaptec3” benchmark, where MaskPlace outperforms DeepPR.
We gradually lower the threshold C th from 60 to 10.
We find that lower congestion leads to an increase in the HPWL.
Our method can always satisfy the congestion constraint in five seeds in a suitable range (above 40 in this benchmark).
If we continue to reduce the congestion threshold after a specific value (say 40 in Fig.7), we found that it hardly satisfies the threshold because nets must take up a certain amount of wire resources.
[Figure] [Figure]
Conclusion
This paper proposes MaskPlace, an RL-based placement method based on rich visual representation by learning position, wirelength, and view information.
It helps the model take action effectively and efficiently without reducing the search space.
We design a direct reward function based on practical scenarios and get satisfactory results on all key metrics.
This work can facilitate the placement process and avoid undesired overlaps between modules.
In the future, we will explore the standard cell placement by designing a suitable representation, which is an open problem for RL due to its vast space.
Limitation and Potential Negative Societal Impact.
Chip design flow contains many stages, and our method shows its potential in a single stage.
Similar to previous RL methods, it also requires an optimization method when placing millions of standard cells because RL’s state space is too large.
Our method does not have potential harm to the public society at the moment.
Acknowledgments and Disclosure of Funding
We thank Xibo Sun from Huawei answering questions about EDA.
We also thank Runjian Chen for participating in our discussion.
Ping Luo is supported by the General Research Fund of HK
No.27208720, No.17212120, and No.17200622.
[-For example, as Fig.8 (c), the maximum stackable coverage area ratio is 2 . 
-]