MaskPlace: Fast Chip Placement via Reinforced Visual Representation Learning
Abstract
Placement is an essential task in modern chip design, aiming at placing millions of circuit modules on a 2D chip canvas.
Unlike the human-centric solution, which requires months of intense effort by hardware engineers to produce a layout to minimize delay and energy consumption, deep reinforcement learning has become an emerging autonomous tool.
However, the learning-centric method is still in its early stage, impeded by a massive design space of size ten to the order of a few thousand.
This work presents MaskPlace to automatically generate a valid chip layout design within a few hours, whose performance can be superior or comparable to recent advanced approaches.
It has several appealing benefits that prior arts do not have.
Firstly, MaskPlace recasts placement as a problem of learning pixellevel visual representation to comprehensively describe millions of modules on a chip, enabling placement in a high-resolution canvas and a large action space.
It outperforms recent methods that represent a chip as a hypergraph.
Secondly, it enables training the policy network by an intuitive reward function with dense reward, rather than a complicated reward function with sparse reward from previous methods.
Thirdly, extensive experiments on many public benchmarks show that
MaskPlace outperforms existing RL approaches in all key performance metrics, including wirelength, congestion, and density.
For example, it achieves 60%- 90% wirelength reduction and guarantees zero overlaps.
We believe MaskPlace can improve AI-assisted chip layout design.
The deliverables are released at laiyao1.github.io/maskplace.
Introduction
The scalability and efficiency are two significant factors of autonomous chip layout design.
Placement is one of the most challenging and time-consuming problems in the design flow, aiming to determine the locations of millions of circuit modules on a 2D chip canvas represented by a two-dimensional grid.
A netlist can describe these modules, that is, a large-scale hypergraph consisting of massive macros (functional blocks such as memory) and standard cells (logic gates) where each macro and each standard cell can contain several or even hundreds of pins connected by wires, as shown in Fig.1.
Placing a large number of circuit modules onto the chip canvas is challenging because many performance metrics such as power consumption, timing, area, and wirelength should be minimized while satisfying some hard constraints such as placement density and routing congestion.
For example, the wirelength (the length of wires that connect all modules) determines the delay and the power consumption of a chip [1].
Shorter wires often indicate less delay and less power consumption [2].
However, wirelength cannot be reduced by overlapping modules because the module density is a hard constraint to ensure that a valid and manufacturable chip layout has non-overlapping modules.
More [Figure] examples of the performance metrics are given in Fig.8 and Fig.9 in Appendix.
As pointed out in [3], the design space of placement is larger than 10 2 , 500 when there are just 1 , 000 circuit modules, whereas neural architecture search (NAS) typically has a space of 10 30 and the Go game has a state space of 10 360 .
Methods of chip placement can be generally divided into two categories, classic optimizationbased approaches [4–21] and learning-based approaches [3, 22, 23].
In the first category, hardware scientists often formulate placement as an optimization problem and relax the hard constraints.
For example, let a pair of vectors ( x , y ) denote the ( x, y )-coordinate value of all circuit modules on a 2D canvas, the objective function of placement can be formulated as minimizing WL( x , y ) , subject to D( x , y ) ≤ α , where WL( · , · ) and D( · , · ) are the estimation functions of wirelength and density respectively, and D( x , y ) ≤ α is a hard constraint with a very small density value α , which ensures that all modules do not overlap.
For instance, DREAMPlace [9] is a recent advanced method that minimizes WL( x , y ) + λ D( x , y ) , which relaxes the hard density constraint.
However, it cannot directly produce a valid and manufacturable layout because the non-overlapping constraint is not satisfied after relaxation.
These approaches often need a post-processing step, such as manual refinement and legalization (LG), to remove the overlapping in placement, resulting in two issues, (1) the wirelength may increase substantially after LG, and (2) no feasible solution can be found if the available chip area is insufficient before post-processing.
In the second category, reinforcement learning (RL) is employed to solve placement as a sequential decision-making problem, placing each circuit module at a time.
Although the learning-based approaches are still in their early stage, they can produce promising results to automate the chip design flow end-to-end significantly without human effort.
For instance, Graph Placement [3] and
DeepPR [22] represent a netlist as a hypergraph, denoted as G =
( V, E ) , where V represents a set of nodes, and each node is a module, and E is a set of edges, which are the wires connecting all modules.
They train RL agents to place one module at a time by maximizing the metric values as rewards.
However, the hypergraph is not scalable to comprehensively encode information of a netlist.
For example, the relative positions (offsets) of pins are discarded in [3, 22].
The wirelength estimation is inaccurate without the pin information, but encoding this rich information would make the hypergraph too complicated because each module can have hundreds of pins.
Furthermore, placement on a large hypergraph requires heavy computations.
Mirhoseini et al.
[3] reduced computations by placing 15% of the modules using reinforcement learning (the remaining modules are placed by classic method), and Cheng and Yan [22] decreased the size (resolution) of module and chip canvas as shown in Table 1.
Both of them sacrificed their placement performance.
We see that MaskPlace can outperform recent advanced methods by performing placement on a full canvas size of 224 × 224 (much larger than prior works) and producing a valid placement with 0% overlap (which cannot be achieved by previous methods).
MaskPlace can also be trained and tested efficiently
[Table]
To address the issues of prior arts
[Table]
we propose a novel RL method, named MaskPlace, which can automatically generate a high-quality and valid layout (non-overlapping modules) within a few hours, unlike previous methods that need manual refinement to modify invalid placement [Table] which may wait up to 72 hours for commercial electronic design automation (EDA) tools to evaluate the placement.
MaskPlace casts placement as a problem of pixel-level visual representation learning for circuit modules using convolutional neural networks.
This representation can comprehensively capture the configurations of thousands of pins, enabling fast placement in a full action space on a large canvas size e.g., 224 × 224.
As shown in Fig.1 and Table 1, MaskPlace has many attractive benefits that existing works do not have.
MaskPlace is mainly for macro placement due to the problem size.
This paper has three main contributions .
Firstly, we recast chip placement as a problem of learning visual representation to describe millions of circuit modules on a chip comprehensively.
It opens up a new perspective for AI-assist chip placement.
Secondly, we carefully design a new policy network that can capture and aggregate both the global and subtle information on a chip canvas, maximizing the reward of wirelength and ensuring non-overlapping placement efficiently.
Thirdly, extensive experiments demonstrate that MaskPlace outperforms recent advanced methods on 24 public chip benchmarks.
For example, MaskPlace can always produce a layout with 0% overlap while reducing wirelength up to 5 × and 9 × compared to Graph Placement [3] and DeepPR [22] respectively.
Preliminary and Notation
The placement quality can be measured by the HPWL (half perimeter wirelength), which estimates the wirelength with marginal computational cost [24].
Intuitively, Fig.2(e) illustrates a 2D chip canvas.
Let M i and P ( i,j ) denote the i -th module and its j -th pin, respectively.
A net contains a set of pins connecting modules by wires.
For example, “Net 1” (in red) connects all four modules ( i.e., M 1 , M 2 , M 3 , M 4 ) using wires through pins P (1 , 2) , P (2 , 2) , P (3 , 2) , and P (4 , 1) , while “Net 2” (in green) connects three modules ( i.e., M 1 , M 2 , M 3 ) using wires through pins P (1 , 1) , P (2 , 1) , and
P (3 , 1) .
HPWL estimates the wirelength by summing up the half perimeters of bounding boxes of all the nets, as shown by the red and green boxes in Fig.2(e).
Intuitively, the half perimeter of a net bounding box equals the sum of its height and width.
For example, HPWL in Fig.2(e) is h 1 + w 1 + h 2 + w 2 .
Given a netlist containing a set of nets, minimizing the wirelength can be treated as minimizing
HPWL by placing modules to the optimal positions on a 2D chip canvas.
To achieve a valid and manufacturable chip layout, we need to satisfy two hard constraints: (1) congestion constraint : the wire congestion should be lower than a desired small threshold to reduce chip cost, and (2) overlap constraint : the density should be minimized to achieve non-overlapping placement.
[Equation]
.t
[Equation]
where P x and P y represent the ( x, y ) -coordinate value of a pin respectively, Congestion( · ) is the congestion function, C th is a desired threshold, Overlap( · ) is the overlap function, and M x , M y , M w , M h
[Figure]
represent the position, width and height of modules respectively.
Firstly, lower congestion often indicates shorter wirelength, which is crucial to reduce chip cost because the wire resources are limited on a real chip.
Inspired by prior arts [3, 22], we employ the RUDY estimator [25] to estimate wire congestion.
Details of RUDY can be found in the Appendix A.2.
Secondly, the placement density calculates the overlapping region between every pair of circuit modules.
It is time-consuming since its computational complexity is O ( V 2 ) where V is the number of modules [1].
The proposed approach can ensure non-overlapping placement to avoid calculating this density metric explicitly in training, thus reducing computations while producing a valid layout.
Our Approach
Model Architecture Overview.
Chip placement can be formulated as a Markov Decision Process (MDP) [26] by placing each module at a time.
Fig.4 illustrates the overall architecture of MaskPlace, which trains a policy π θ ( a t | s t ) represented by a convolutional encoder-decoder network with parameter set θ , and a value function V ϕ ( s t ) represented by an embedding model with parameter set ϕ .
The policy network receives previous observations and actions as input s t and selects an action a t as output.
Specifically, s t is a set of pixel-level feature maps that comprehensively capture the net and pin configurations in M 1: t − 1 , M t , and M t +1 , where M 1: t − 1 denotes the modules that have been placed in the previous time steps from 1 to t − 1 , while M t and M t +1 denote the modules to be placed at the current step t and the next step t + 1 , respectively.
Intuitively, MaskPlace looks one step forward to achieve better placement.
[Figure] Although prior arts [3, 22] represented a netlist as a hypergraph as shown in Fig.2(f) where each node is a module, and each edge is a wire between two modules, they lost the information of pin offsets for each module.
Unlike previous works, MaskPlace can fully represent massive net and pin configurations using three types of pixel-level feature maps, as shown in Fig.2(a-d), including position mask, wire mask, and view mask, as discussed below.
Different masks are fused by convolutions to learn the state representation.
Position Mask. The position mask, denoted by f p ∈ { 0 , 1 } 224 × 224 , is a binary matrix of a canvas grid with size 224 × 224 as shown in Fig.3, where value “ 1 ” means a feasible position to place a module.
The purpose of the position mask is to guarantee no overlaps between modules ( i.e., satisfy the overlap constraint) and to learn the relationship between placement and wirelength.
Specifically, we slide a module M t (for example, [Figure] t = 5 ) on the entire chip canvas.
The trajectory of the feasible positions (in green) can be labeled with “1”.
Intuitively, we can check each position for each module using the cumulative sum array [27].
This naive approach has the computational complexity of O ( N 2 ) when a 2D canvas grid is divided into N × N cells.
However, this simple approach is not efficient when N is large.
Therefore, since all modules are rectangles, we design an efficient generation algorithm, which iterates through all placed modules (in blue) and excludes positions that will cause overlap.
In this case, all remaining positions are available for placement.
The new algorithm is summarized in Appendix A.3, which costs O ( V ) for each module, where V is the number of modules.
[Figure]
Wire Mask.
The wire mask, denoted as f w ∈ [0 , 1] 224 × 224 , is a continuous matrix for representing how HPWL increases if we place a module M t in a specific position.
Fig.5 shows a sample of wire mask, where each value means the increase of HPWL.
The wire mask aims at finding the best position with the minimum increase of the wirelength.
Intuitively, we can calculate the HPWL at each canvas position, leading to a complexity of O ( N 2 P ) , where P is the total number of pins.
However, a fast algorithm can be designed by considering the relationships between the pin offset, the net bounding box, and the linear property of the HPWL metric.
For example, Fig.3 illustrates that the next module Mhas two pins, P (5 , 1) and P (5 , 2) , belonging to “Net 1” and “Net 2” respectively (Fig.2(e)).
Fig.5 illustrates the increase of wirelength when placing M 5 at each canvas location.
For instance, if Mis at the bottom-left corner, its Manhattan distance to the two net bounding boxes (in red and green) is 2 + 2 = 4 .
To calculate the Manhattan distance more accurately, we move the net bounding box compared to the pin location.
For example, since P (5 , 2) is located at (2 , 1) 3 , we move the bounding box of Net 2 (in green) in the direction − ∆ (5 , 2) =
( − 2 , − 1) to encode the information of pin offset.
The time complexity can be reduced to O ( NP ) .
The algorithm can be found in Appendix A.3.
View Mask.
The view mask, denoted as f v ∈ { 0 , 1 } 224 × 224 , is a global observation of the current chip layout, where the value “ 1 ” means a module has occupied this grid cell.
Different from DeepPR [22] that assumed all modules have unit size, we consider real sizes of modules.
For instance, if a module has size w × h , it covers ⌈ wN/W ⌉×⌈ hN/H ⌉ cells in the canvas, where W and H represent the canvas size and ⌈·⌉ denotes the ceiling function.
Learning Algorithm.
We train different blocks in Fig.4 as a whole using reinforcement learning.
The detailed network architectures are provided in Appendix A.4.
Firstly, we apply the above masks to represent the entire circuits and feed them to downstream networks.
Secondly, a global feature encoder embeds the view mask of current placement and the wire masks of the following two steps into an embedding vector.
Then we combine it with the positional embedding of the t -th circuit module in the value network to generate a scalar to evaluate the current state by fully-connected layers.
Thirdly, a global mask decoder recovers a feature map of size N 2 , which is fused with different position masks and wire masks in the policy network using 1 × 1 convolutions to avoid the local signal diffusion.
The policy network predicts a probability action matrix of size N × N , indicating where to put the next module.
Before sampling actions, we can remove unfeasible actions using the position mask.
Finally, the congestion satisfaction block applies the congestion threshold on the probability matrix to select a final action.
Reinforcement Learning.
We borrow the representative actor-critic diagram [28] and PPO2 framework [29] to train the policy π θ ( a t | s t ) , where the state representation s t is listed in Table 11 in
Appendix.
The action a t is the canvas position (cell) to place the circuit module.
Specifically, we treat the chip canvas as a grid and divide it into N × N cells, leading to N 2 possible actions.
The objective function of the policy network can be formulated as
[Equation]
where the ratio r t ( θ ) =
π θ ( a t | s t )
π θ old ( a t | s t ) and ˆ A t =
G t − ˆ V t denotes the advantage function.
We employ
G t = (cid:80) V − t − 1 k = γ k r t + k +1 that is the cumulative discounted reward and ˆ V t is the estimated value produced by the value network.
We update the the value network by optimizing its objective,
L value ( ϕ ) =
ˆ E (cid:2) ( G t − ˆ V t ) 2 (cid:3) .
Reward r t .
We treat HPWL as the reward because wirelength is the main optimization target in different performance metrics.
This is different from prior arts [3, 22] that weighted combines
HPWL and congestion as the reward, which introduces the weighting coefficient as an extra hyperparameter to tune.
Specifically, we achieve a dense reward by defining a partial HPWL, which only computes the currently placed pins.
For example, the partial HPWL for t modules can be defined as HPWL t .
In other words, we compute HPWL t after taking action a t .
The reward for the step t is r t =
HPWL t − 1 − HPWL t , i.e., the opposite number of the increase of HPWL.
Furthermore, instead of computing HPWL at each step, we can maintain the ranges of all net bounding boxes in one episode and update the changes of their sizes with a cost of O ( P ) , where P is the number of pins.
Thus we can generate dense rewards while maintaining efficiency.
Training and Testing.
Before training, we follow previous work [3] to sort the circuit modules according to the number of nets, areas, and the number of connected modules that have been placed to determine the place order.
In training, we update the policy and value networks at each epoch by ignoring the congestion satisfaction block.
When updating the value network, we stop the gradient back-propagated in the global mask encoder to avoid influence on the policy network.
The detailed training setup is provided in Appendix A.5.
In the testing stage, for each step t , we obtain a probability matrix from the policy network and sample one place action a t .
Then, the congestion satisfaction block will check whether the congestion exceeds a threshold C th after applying this action.
If it exceeds, we uniformly sample a few actions, look up the corresponding values from the wire mask f tw , and estimate the congestion before taking these actions.
We choose the action with the minimal value in f tw and the congestion lower than C th .
If all actions cannot satisfy congestion C th , we select the action with the minimal congestion and move to the next step.
Detailed congestion satisfaction algorithm can be seen in Appendix A.3.
Experiments
We extensively evaluate MaskPlace and compare it with several recent advanced placement methods, including NTUPlace3 [6], RePlAce [8], DREAMPlace [9], Graph Placement [3], and DeepPR [22], where NTUPlace3, RePlAce and DREAMPlace are optimization-based methods, whilst Graph
Placement and DeepPR are learning-based approaches.
All of them are evaluated on different public circuit benchmarks.
All previous works are evaluated by following their experimental settings.
Benchmark.
We evaluate MaskPlace in 24 circuit benchmarks selected from public datasets including the widely-used ISPD2005 [30], IBM benchmark suite [31], and Ariane RISC-V CPU design [32].
The number of evaluated benchmarks is three times more than previous work [9, 22, 3].
The statistics of benchmarks are given in Table 14 in Appendix A.6, where the largest circuit contains 1,293 macros, 22,802 pins, and more than a million standard cells, leading to a vast state space as aforementioned.
Main Results.
Table 2 compares the HPWL results between all the above methods to place all macros.
To enable a fair comparison, we evaluate all approaches 4 by using five random seeds and report the means and variances.
Since the original DeepPR method did not capture macro size (thus does not avoid overlap between adjacent macros because all macros have unit size), we extend DeepPR to model macro size to reduce the overlapping ratio.
We name it “DeepPR-no-overlap”.
Similar to prior works, we use the minimum spanning tree algorithm
[33] to estimate routing wirelength [34].
From
Table 2, we can see that MaskPlace achieves the lowest wirelength in six out of seven benchmarks (except “adaptec4” where it still outperforms all learning-centric methods).
We also see that the conventional optimization-based approaches may fail when the circuit benchmark has high chip area usage, such as “bigblue3 ” and “ariane”.
Also, MaskPlace gets the lowest wirelength compared with
Graph Placement and simulated annealing [35] in the IBM benchmark, which is shown in Appendix
A.7.
This project website 5 visualizes and compares different placements.
[Table]
Compare to Graph Representation.
Since Graph Placement [3] is the recent advanced learningbased approach that employs hypergraph for placement, we compare MaskPlace with it in all four performance metrics, including HPWL, congestion, density [Table] and overlap area ratio.
Table 3 andreport the results.
The overlap area ratio describes the ratio of the overlapping area between macros divided by the chip area.
In Table [Table] MaskPlace (soft constraint) means that the round function rather than the ceiling function is used to calculate the number of grid cells occupied by the placed macros.
MaskPlace with soft constraints may produce better HPWL and congestion than its counterpart with hard constraints, but the overlap area ratio could not be zero because the constraints have been relaxed.
From Table 3 and [Table] we see that MaskPlace outperforms graph placement by large margins, especially in the ISPD benchmark, where the former reduces HPWL compared to the latter one by up to 80% while ensuring zero overlaps in all benchmarks.
More results in the IBM benchmark can be found in
Appendix Table
[Table]
[Table]
Routing Wirelength.
Table 5 compares the routing wirelength between MaskPlace and DeepPR [22].
We show that using the true wirelength as the reward would lower the efficiency and produce a sparse reward.
We see that MaskPlace, which employs HPWL as the reward, can achieve 60% to 90% shorter routing wirelength than DeepPR, which directly used the true wirelength as the reward.
Standard Cells.
Table 6 compares the HPWL of both the macros and the standard cells by using
MaskPlace, DeepPR [22] [Table] and DREAMPlace [9], where DREAMPlace is employed to place the standard cells following the experimental setup in [22].
We can see that the proposed method
[Table]
[Table] surpasses the other approaches by up to 50% in the large benchmark “bigblue3”, which has more than a million standard cells [Table] [Table]
Placement w/o real size Considering that DeepPR ignored the module size, we implement
MaskPlace in the same setting
[Table]
and the result can be found in Table 7.
The result shows that our method has significant advantages
[Table]
[Table]
Transferability Test the performance of the model trained on adaptec1 on other benchmarks as
Table 8.
The results show that our method also has a good transferability.
[Table]
Efficiency.
Table 9 compares the inference efficiency of different approaches.
All of them are evaluated on one GeForce RTX 3090 GPU, and the CPU version of DREAMPlace is allocated with 16 threads in a 16 CPU cores environment.
We see that the careful design of MaskPlace makes it faster than two other learning-based approaches.
Ablation Study.
We compare different components in MaskPlace as shown in Fig.6.
Each curve is produced by five seeds using the benchmark “adaptec1”.
For example, MaskPlace w/ CL means using 1/3 of the circuit macros to pretrain the model for 30 epochs like curriculum learning.
MaskPlace w/o
M t +1 means only considering M t as input without looking one step forward.
Moreover, MaskPlace [Table] w/o number of nets means this feature is not considered when determining the placement order.
MaskPlace w/o 1x1 conv means that 7x7 kernels are used to replace the 1x1 kernels in the local feature fusion block.
Also, MaskPlace with sparse reward means compute HPWL reward only when all macros have been placed, and the rewards for other steps are set to zeros.
MaskPlace w/o view mask and w/o wire mask means that the corresponding mask is not inputted into the model.
We can see that MaskPlace (standard) with curriculum learning has the best performance.
Congestion Satisfaction.
To evaluate our congestion satisfaction block, we implement a placement without any congestion threshold ( i.e., ∞ ) as shown in Fig.7.
We evaluate the “adaptec3” benchmark, where MaskPlace outperforms DeepPR.
We gradually lower the threshold C th from 60 to 10.
We find that lower congestion leads to an increase in the HPWL.
Our method can always satisfy the congestion constraint in five seeds in a suitable range (above 40 in this benchmark).
If we continue to reduce the congestion threshold after a specific value (say 40 in Fig.7), we found that it hardly satisfies the threshold because nets must take up a certain amount of wire resources.
[Figure] [Figure]
Conclusion
This paper proposes MaskPlace, an RL-based placement method based on rich visual representation by learning position, wirelength, and view information.
It helps the model take action effectively and efficiently without reducing the search space.
We design a direct reward function based on practical scenarios and get satisfactory results on all key metrics.
This work can facilitate the placement process and avoid undesired overlaps between modules.
In the future, we will explore the standard cell placement by designing a suitable representation, which is an open problem for RL due to its vast space.
Limitation and Potential Negative Societal Impact.
Chip design flow contains many stages, and our method shows its potential in a single stage.
Similar to previous RL methods, it also requires an optimization method when placing millions of standard cells because RL’s state space is too large.
Our method does not have potential harm to the public society at the moment.
Acknowledgments and Disclosure of Funding
We thank Xibo Sun from Huawei answering questions about EDA.
Ping Luo is supported by the
General Research Fund of HK No.27208720, No.17212120, and No.17200622.
A Appendix A.1 Module, Net and Pin
Module.
A chip is a combination of numerous modules, and there are two types of them: macros and standard cells.
Macros are relatively large, including DRAMs, caches, and IO interfaces.
Standard cells are mainly logical gates, much smaller than macros, and the size can be ignored.
As in Fig. (a), there are four macros and several standard cells.
Placement methods usually place macros first and then the standard cells to ensure there is enough space for macros [36].
Due to the considerable number of standard cells, we currently use our MaskPlace method on macro placement.
Pin.
Pins are input/output interfaces for modules and are connected by wires directly, which have fixed relative positions on modules.
We define the relative position of the pin P ( i,j ) from the leftbottom corner of the module it belongs to as ∆ ( i,j ) =
(∆ ( i,j ) x , ∆ ( i,j ) y ) .
For example, there are five pins and three macros in Fig.9 (a), and the pin offset information is also shown at the bottom.
In the placement task, we should not ignore the positions of pins because it determines the wirelength.
However, graph neural network-based models [3, 22] ignored them when converting circuits into a graph, which may lead to sub-optimal results.
Net.
A net contains a set of pins connected by the same wires.
Thus the pins have the same information (0/1 in digital circuits).
For example, four pins belong to Net 1, and the other three pins belong to Net 2 in Fig.8 (a).
Usually, one pin belongs to only one net, and one net has more than two pins (one input and several outputs).
Pins from the same net can form a net bounding box as Fig. (a)(b). [Equation] [Figure]
[Table]
[Figure]
[Table]
[Equation]
[Table] [Equation] [Table] [Equation]
[Table]
[Figure]
A.2 Metric
HPWL.
HPWL (Half Perimeter Wire Length) is widely used to estimate wirelength by small computation cost [24].
It is the sum of half perimeter of net bounding boxes as Fig.8 (a)(b), where the bounding box is the minimal rectangle including all pins belonging to this net.
Congestion.
The congestion metric is used to avoid routing congestion, resulting in an increase in the actual wirelength because the resources for wires are limited in a real chip.
There are many ways to estimate congestion, one is to compute a rough routing result [3], but it is very computationally intensive.
We use RUDY [25] as the estimation of congestion, which is a common way to evaluate.
In RUDY, each grid cell needs to accumulate the inverse of the height and width (1 /h + 1 /w ) of all the net bounding boxes covering itself and take out the maximum value (or the average of the first k maximums) of all grid cells as Fig.8 (a)(b).
Density.
Density is a metric to reduce overlaps and avoid time-consuming computation for O ( V 2 ) constraints [1].
So, it is an approximate calculation essentially.
It is defined as the maximum stackable coverage area ratio for each grid cell on a chip canvas.
For example, as Fig.8 (c), the maximum stackable coverage area ratio is 2 .
in grid cell g 7 , 5 because two modules fully occupy it.
However, density less than a small value is not a sufficient condition for the absence of overlap.
Because our method can ensure no overlaps, we only consider it in evaluation.
In the practical application scenario of chip design, HPWL is an optimization item.
Conversely, congestion and density are constraint items.
Examples.
We give a set of placement results to explain the metrics in Fig.8.
We can see that HPWL is the sum of width and height of net bounding boxes.
Congestion (RUDY) is the max congestion value of grid cell g i,j , and the value in each grid cell is cumulative from the reciprocal of the width and height of the net bounding box containing that grid cell.
(a) and (b) are from the same circuit, but (b) is a better placement because (b) has lower HPWL and congestion.
Density is the max density value of grid cell
g i,j , and the value in each grid cell is stackable coverage area ratio of the grid cell.
The density of Fig.8 (c) is 2.0 because g 7 , 5 completely covered by two modules.
Relationship between pin offset and HPWL.
The pin offset can affect the HPWL.
In the graphbased method, the input features for module include size ( M w , M h ) , position ( M x , M y ) and type.
So, the network can hardly infer the real position of pins and tend to use the center positions of modules to predict the positions of pins.
In this way, the agent will align the centers of the two modules horizontally, and the result of placement is like Fig.9 (b) to get the wirelength 6.
However, when considering the pins are near the bottom of the modules, it is better to align the bottom of the two modules as Fig.9 (c), and thus wirelength can be reduced to 2 if we consider the pin offset. [Figure] [Equation] [Figure]
[Equation]
[Figure]
[Equation]
[Figure] [Equation] [Figure] When we remove the pin offset information, the model tends to align the centers of the two modules horizontally like (b) because it uses the center position of modules to estimate pin location.
However, have a better design are
A.3 Algorithms
Reward Computation.
The dense reward generation algorithm is shown in Algorithm 1.
It can generate dense rewards without an efficiency decrease.
For simplicity, we omit the calculation of the y dimension, which is the same as the x dimension.
Position Mask Generation. The efficient position mask generation algorithm is in Algorithm 2. Wire Mask Generation. The efficient wire mask generation algorithm is shown in Algorithm 3.
For simplicity, we omit the calculation of the y dimension, which is the same as the x dimension.
Data:
( M tx , M ty ) , max/min x/y nets
MaxMinCoord , pin offsets (∆ ( t,j ) x , ∆ ( t,j ) y ) , pin to net connection P ( t,j ) n ;
Result:
Incremental HPWL Reward reward ; reward ← 0 ; foreach ∆ ( t,j ) x , P ( t,j ) n of all pins P ( t,j ) from M t do x ← M tx + ∆ ( t,j ) x ; // calculate pin coordinates if P ( t,j ) n not in MaxMinCoord then //
The net for the first time has a definite location of the pin
MaxMinCoord [ P ( t,j ) n ] .x.max
← x ;
MaxMinCoord [ P ( t,j ) n ] .x.min
← x ; else // Update the bounding box range if MaxMinCoord [ P ( t,j ) n ] .x.max <
x then reward ← reward + ( x − MaxMinCoord
[ P ( t,j ) n ] .x.max ) ;
MaxMinCoord [ P ( t,j ) n ] .x.max
=
x ; else if MaxMinCoord [ P ( t,j ) n ] .x.min
> x then reward ← reward + ( MaxMinCoord [ P ( t,j ) n ] .x.min
− x ) ;
MaxMinCoord [ P ( t,j ) n ] .x.min
= x ; end end end
Data t
M t −( M 1: t − 1 w , M 1: t − 1 h , M 1: t − 1 x , M 1: t − 1 y ) Result:
Position Mask f tp Module M t f tp i do tmp ← ones ( N, N )
; // find positions that will cause M t and M i to overlap tmp [ M ix − M tw + 1 : M ix + M iw − 1 , M iy − M th + 1 : M iy + M ih − 1] ← 0 ; // exclude infeasible positions f
tp ← tmp ⊙ f tp ; // ⊙ is element-wise product end
Data: Hash offsets (∆ ( t,j ) x , ∆ ( t,j ) y ) ,
P ( t,j ) n
Result: Wire Mask f tw for module M t f tw ← zeros ( N, N ) ; // Accumulate the wirelength increase for each net foreach ∆ ( t,j ) x , P ( t,j ) n of all pins P ( t,j ) from M t do //
If the pin is to the left of the net bounding box for i ← 0 to MaxMinCoord [ P ( t,j ) n ] .x.min
+ ∆ ( t,j ) x − 1 do f tw [ i, :] ← f tw [ i, :]
+ MaxMinCoord
[ P ( t,j ) n ] .x.min
+ ∆ ( t,j ) x − i ; end //
If the pin is to the right of the net bounding box for i ← MaxMinCoord [ P ( t,j ) n ] .x.max + ∆ ( t,j ) x
+ 1 to N − 1 do f tw [ i, :] ← f tw [ i, :]
+ i − ( MaxMinCoord [ P ( t,j ) n ] .x.max
+ ∆ ( t,j ) x ) ; end end
Congestion Satisfaction. The algorithm implemented in the congestion satisfaction block can be seen in Algorithm 4. Algorithm Data
Result:
A placement [ a , a a the congestion requirement for i ← 1 to V do
Compute
Randomly
Compute congestion congestion Get
1: N i from wire masks; Sort the N actions according to w 1: N i (as
1: N i (as
; flag ← False ; for j ← 1 to N do if c ji ≤ C th then flag ← True ; a i ← a ji ; break; end end //
If all sampled actions cannot satisfy congestion threshold, we choose the one with minimal congestion increase.
if flag is False then a i ← the action a ji with minimum c ji ; end
Take action a i as the final action; end
A.4 Details of Model Architecture
The parameters of layers in model architecture are in Table 10.
Also, the features used by pixel-level mask generation are in Table 11.
The comparison of features for the placement order in different methods can be seen in Table 12.
[Table] [Table]
[Table]
A.5 Training Configuration
The detailed configuration and hyperparameter settings of our model is in Table 13.
[Table]
Also, we implement DREAMPlace 6 [9] [Table] Graph Placement 7 [3]
[Table] and DeepPR 8 [22] by their open source codes with their default settings
[Table]
A.6 Details of Benchmark
The detailed statistics of benchmarks are in Table 14.
Hard macros are macros placed by the RL method in Graph Placement [3] [Table] and the remaining macros, also named soft macros, are placed by the classic optimization-based method.
However, this distinction does not apply to the process of our method
[Table]
which means we place all macros by RL. The statistical range of nets, pins, and area utilization are macros.
Ports are terminals connecting to an external circuit, seen as fixed and no-size modules.
Our method is also applicable to circuits with ports without additional modifications.
[Table]
A.7 Supplementary Experiment
More benchmarks We also conducted experiments in the IBM benchmark suite (ICCAD 2004)
[Table]
which has been used to evaluate placement for more than a decade.
We remove the “ibm05” because it does not contain any macros.
We use our MaskPlace to place large macros and DREAMPlace [9] to place standard cells.
We compared our method with graph placement [3] and the simulated annealing method used in [3].
The results are in Table 15 and our method can achieve the lowest
HPWL in all benchmarks.
[Table]
Simulated
[Table]
[Table]
[Table]
MaskPlace+DREAMPlace [9]
[Table]
Simulated
[Table]
MaskPlace+DREAMPlace [9]
[Table]
Simulated
[Table] [Table]
MaskPlace+DREAMPlace [9]
[Table]
For the larger circuit bigblue4 in ISPD 2005 benchmark, the result of our method and baselines can been seen as Table 16.
MaskPlace still achieved the best performance.
[Table]
RePlAce[8]
DREAMPlace [9] bigblue 128.06±3.
11.80±0.
12.29±1.
Benchmark
Graph Placement [3]
DeepPR [22]
DeepPR-no-overlap [8]
MaskPlace bigblue 53.35±4.
68.30±4. 115.08±2. 11.07±0. Search time We compared the search time of our method, Graph Placement [3] and DeepPR [22].
We tested all methods in the same environment and took the HPWL as the metric in benchmark adaptec1 .
The result is in Fig.
We can see that our approach can achieve the best performance in a few hours.
[Figure]
A.8 Detailed equation description of the model
We describe our model architecture in Fig.
4 in the form of equation.
With current state s t , we first calculate the position masks f tp , f t +1 p , wire masks f tw , f t +1 w and view mask f tv via the mask generation function m ( · ) .
[Equation]
Then we extract the local feature z lt via local mask fusion g ω ( · ) and the global features z gt via the global mask encoder enc η ( · ) .
[Equation]
where g ω ( · ) is a 1×1 convolutional neural network with parameter ω .
[Equation]
where enc η ( · ) is a convolutional neural network with ResNet-18 architecture with parameter η .
With local features z lt and global features z gt , the state value ˆ V t is derived by
[Equation]
where v ϕ is an MLP-like neural network with parameter ϕ and pos ( t ) is an embedding vector which is related to step t .
We decode the global features z gt into the dimension as same as the action space by the global mask decoder
[Equation]
where dec δ ( · ) is a transpose convolutional neural network with parameter δ .
Finally, we concatenate the local features z lt and global features z ′ gt in the channel dimension and merge them by another 1×1 convolutional neural network ψ ξ ( · ) .
We further combine it with the position mask f tp to generate action a t via the policy network π θ ( · )
[Equation]
[Equation]
where π θ ( · ) is an MLP-like neural network with parameter θ and ψ ξ ( · ) is a 1×1 convolutional neural network with parameter ξ .
B Related Work
Classic optimization-based methods.
Optimization has been the dominant method in placement for decades.
They can be divide into three categories: partitioning-based methods [4, 5], simulated annealing methods [10, 11] and analytical methods [6–9, 12–21].
Partitioning-based methods [4, 5] cluster the whole circuits into several parts to minimize the connections between parts.
These methods first solve the placement problems within the same part and then place these parts to suitable positions on the chip based on the divide-and-conquer idea.
However, optimizing modules within one part is an isolated problem, and sometimes it is hard to divide the circuits into relatively independent parts, which is highly related to the topology of the circuits.
Simulated Annealing (SA) methods [10, 11] are also known as hill-climbing methods, a widely used iterative heuristic algorithm for solving combinatorial optimization problems.
They initialize a random status and then search for the following status by moving from the current status to a neighbor status.
If the metrics of the neighbor status are better than that of the current status, they move to the neighbor status.
Otherwise, the move may still be taken with a decreasing probability over time.
The advantage is that they can be implemented when metrics do not have the analysis formula or cannot be differentiable.
However, it is not efficient enough, and the placement results are highly dependent on the random initial state.
Analytical methods gradually replace the above two methods because of the best performance.
They can be divided into quadratic methods [12–18] and nonlinear (non-quadratic) methods [6–9, 19–21].
Quadratic methods [12–18] transform the placement problem into a sequence of convex quadratic problems, and there are well-established solvers for such problems.
However, it is a very rough approximation.
Nonlinear methods [6–9, 19–21] design a single differentiable objective function and optimize it.
The advantage is that it can handle large-scale modules.
However, the objective function is still approximated, and they cannot avoid overlaps when combining multiple metrics in one objective function.
Methods in this category achieved the highest placement quality among all classic methods [9].
Learning-based methods.
With the development of deep learning, some learning-based approaches [11, 37–39] have been proposed to assist classic methods.
Huang et al.
[37] uses convolutional neural networks to estimate the congestion for SA placement.
Vashisht et al.
[11] uses the reinforcement learning models to generate the initial placement of SA.
Kirby et al.
[38], Agnesina et al.
[39] help classic placement tools choose the most suitable hyperparameters with reinforcement learning methods.
However, these methods do not implement end-to-end placement by deep learning, so the placement results depend heavily on the classic methods.
Pure reinforcement learning methods [3, 22, 23, 40] view placement as a process of placing modules sequentially.
Mirhoseini et al.
[3] uses reinforcement learning to place hard macros, and the forcedirected method [18] to place remaining soft macros.
Jiang et al.
[23] replaces the force-directed method with DREAMPlace [9] to place soft macros based on Graph Placement [3].
Cheng and Yan [22] proposes a reinforcement learning method by using wirelength as the reward.
Moreover, Chang et al.
[40] puts all metrics in the RL reward.
They have in common that they convert the circuit as a graph structure and input them to the graph neural networks [41].
However, the pin information has been lost, leading to sub-optimal placement.
Also, they cannot avoid overlaps because of the reduction in search space.
These methods still have room for improvement in terms of realistic chip placement.
For instance, DeepPR [22] ignores the realistic size of the module.
However, the size of the modules varies widely in most circuits.
Although it proposes to use routing wirelength instead of
HPWL as the reward, it will affect the efficiency and lead to sparse reward, making models hard to train.
In contrast, HPWL is a high-quality wirelength estimation, and we do not need to discard this inherent dense reward.