MaskPlace: Fast Chip Placement via Reinforced Visual Representation Learning
Anonymous Author(s)
AffiliationAddress email
Abstract
Placement is an essential task in modern chip design, aiming at placing millions ofcircuit modules on a 2D chip canvas.
Unlike the human-centric solution, whichrequires months of intense effort by hardware engineers to produce a layout tominimize delay and energy consumption, deep reinforcement learning has becomean emerging autonomous tool.
However, the learning-centric method is still in itsearly stage, impeded by a massive design space of size ten to the order of a fewthousand.
This work presents MaskPlace to automatically generate a valid chiplayout design within a few hours, whose performance can be superior or comparableto recent advanced approaches.
It has several appealing benefits that prior artsdo not have.
Firstly, MaskPlace recasts placement as a problem of learning pixellevel visual representation to comprehensively describe millions of modules ona chip, enabling placement in a high-resolution canvas and a large action space.
It outperforms recent methods that represent a chip as a hypergraph.
Secondly,it enables training the policy network by an intuitive reward function with densereward, rather than a complicated reward function with sparse reward from previousmethods.
Thirdly, extensive experiments on many public benchmarks show that
MaskPlace outperforms existing RL approaches in all key performance metrics,including wirelength, congestion, and density.
For example, it achieves 60%-90% wirelength reduction and guarantees zero overlaps.
We believe MaskPlacecan improve AI-assisted chip layout design.
The deliverables are released atanonymous-reposi.github.io.
Introduction
The scalability and efficiency are two significant factors of autonomous chip layout design.
Placementis one of the most challenging and time-consuming problems in the design flow, aiming to determinethe locations of millions of circuit modules on a 2D chip canvas represented by a two-dimensionalgrid.
A netlist can describe these modules, that is, a large-scale hypergraph consisting of massivemacros (functional blocks such as memory) and standard cells (logic gates) where each macro andeach standard cell can contain several or even hundreds of pins connected by wires, as shown in Fig.1.
Placing a large number of circuit modules onto the chip canvas is challenging because many perfor- mance metrics such as power consumption, timing, area, and wirelength should be minimized while satisfying some hard constraints such as placement density and routing congestion.
For example, the wirelength (the length of wires that connect all modules) determines the delay and the power consumption of a chip [1].
Shorter wires often indicate less delay and less power consumption [2]
[Figure]
However, wirelength cannot be reduced by overlapping modules because the module density is a hardconstraint to ensure that a valid and manufacturable chip layout has non-overlapping modules.
Moreexamples of the performance metrics are given in Fig.8 and Fig.9 in Appendix.
As pointed out in[3], the design space of placement is larger than 10 2 , 500 when there are just 1 , 000 circuit modules,whereas neural architecture search (NAS) typically has a space of 10 30 and the Go game has a statespace of 10 360 .
Methods of chip placement can be generally divided into two categories, classic optimizationbased approaches [4–21] and learning-based approaches [3, 22, 23].
In the first category, hardwarescientists often formulate placement as an optimization problem and relax the hard constraints.
Forexample, let a pair of vectors ( x , y ) denote the ( x, y )-coordinate value of all circuit modules on a2D canvas, the objective function of placement can be formulated as minimizing WL( x , y ) , subjectto D( x , y ) ≤ α , where WL( · , · ) and D( · , · ) are the estimation functions of wirelength and densityrespectively, and D( x , y ) ≤ α is a hard constraint with a very small density value α , which ensuresthat all modules do not overlap.
For instance, DREAMPlace [9] is a recent advanced method thatminimizes WL( x , y ) + λ D( x , y ) , which relaxes the hard density constraint.
However, it cannotdirectly produce a valid and manufacturable layout because the non-overlapping constraint is notsatisfied after relaxation.
These approaches often need a post-processing step, such as manualrefinement and legalization (LG), to remove the overlapping in placement, resulting in two issues, (1)the wirelength may increase substantially after LG, and (2) no feasible solution can be found if theavailable chip area is insufficient before post-processing.
In the second category, reinforcement learning (RL) is employed to solve placement as a sequentialdecision-making problem, placing each circuit module at a time.
Although the learning-basedapproaches are still in their early stage, they can produce promising results to automate the chipdesign flow end-to-end significantly without human effort.
For instance, Graph Placement [3] and
DeepPR [22] represent a netlist as a hypergraph, denoted as G =
( V, E ) , where V represents a set ofnodes, and each node is a module, and E is a set of edges, which are the wires connecting all modules.
They train RL agents to place one module at a time by maximizing the metric values as rewards.
However, the hypergraph is not scalable to comprehensively encode information of a netlist.
Forexample, the relative positions (offsets) of pins are discarded in [3, 22].
The wirelength estimation isinaccurate without the pin information, but encoding this rich information would make the hypergraph
We see that MaskPlace can outperform recent advanced methods by performing placement on a full canvas size of 224 × 224 (much larger than prior works) and producing a valid placement with 0% overlap (which cannot be achieved by previous methods).
MaskPlace can also be trained and tested efficiently
[Table]
too complicated because each module can have hundreds of pins.
Furthermore, placement on a largehypergraph requires heavy computations.
Mirhoseini et al.
[3] reduced computations by placing 15%of the modules using reinforcement learning (the remaining modules are placed by classic method),and Cheng and Yan [22] decreased the size (resolution) of module and chip canvas as shown in Table1.
Both of them sacrificed their placement performance
[Table]
To address the issues of prior arts
[Table]
we propose a novel RL method, named MaskPlace, which canautomatically generate a high-quality and valid layout (non-overlapping modules) within a few hours,unlike previous methods that need manual refinement to modify invalid placement, which may waitup to 72 hours for commercial electronic design automation (EDA) tools to evaluate the placement.
MaskPlace casts placement as a problem of pixel-level visual representation learning for circuitmodules using convolutional neural networks.
This representation can comprehensively capture theconfigurations of thousands of pins, enabling fast placement in a full action space on a large canvassize e.g., 224 × 224.
As shown in Fig.1 and Table 1, MaskPlace has many attractive benefits thatexisting works do not have. MaskPlace is mainly for macro placement due to the problem size.
This paper has three main contributions .
Firstly, we recast chip placement as a problem of learningvisual representation to describe millions of circuit modules on a chip comprehensively.
It opens up anew perspective for AI-assist chip placement.
Secondly, we carefully design a new policy networkthat can capture and aggregate both the global and subtle information on a chip canvas, maximizingthe reward of wirelength and ensuring non-overlapping placement efficiently.
Thirdly, extensiveexperiments demonstrate that MaskPlace outperforms recent advanced methods on 24 public chipbenchmarks.
For example, MaskPlace can always produce a layout with 0% overlap while reducingwirelength up to 5 × and 9 × compared to Graph Placement [3] and DeepPR [22] respectively.
Preliminary and Notation
The placement quality can be measured by the HPWL (half perimeter wirelength), which estimatesthe wirelength with marginal computational cost [24].
Intuitively, Fig.2(e) illustrates a 2D chipcanvas.
Let M i and P ( i,j ) denote the i -th module and its j -th pin, respectively.
A net contains aset of pins connecting modules by wires.
For example, “Net 1” (in red) connects all four modules( i.e., M 1 , M 2 , M 3 , M 4 ) using wires through pins P (1 , 2) , P (2 , 2) , P (3 , 2) , and P (4 , 1) , while “Net 2”(in green) connects three modules ( i.e., M 1 , M 2 , M 3 ) using wires through pins P (1 , 1) , P (2 , 1) , and
P (3 , 1) .
HPWL estimates the wirelength by summing up the half perimeters of bounding boxesof all the nets, as shown by the red and green boxes in Fig.2(e).
Intuitively, the half perimeter ofa net bounding box equals the sum of its height and width.
For example, HPWL in Fig.2(e) ish 1 + w 1 + h 2 + w 2 .
Given a netlist containing a set of nets, minimizing the wirelength can be treated as minimizing
HPWL by placing modules to the optimal positions on a 2D chip canvas.
To achieve a valid and
[Figure]
manufacturable chip layout, we need to satisfy two hard constraints: (1) congestion constraint : thewire congestion should be lower than a desired small threshold to reduce chip cost and (2) overlapconstraint : the density should be minimized to achieve non-overlapping placement.
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
) ∈ net P ( i
[Equation]
j ) y − min P ( i
[Equation]
j ) ∈ net
[Equation]
i
[Equation]
j ) y (cid:1) s.t.
Congestion( M x , M y , M w , M h ) ≤ C th and Overlap( M x , M y , M w , M h )
= 0 , (1) where P x and P y represent the ( x, y ) -coordinate value of a pin respectively, Congestion( · ) is the congestion function, C th is a desired threshold, Overlap( · ) is the overlap function, and M x , M y , M w , M hrepresent the position, width, and height of modules respectively.
Firstly, lower congestion oftenindicates shorter wirelength, which is crucial to reduce chip cost because the wire resources arelimited on a real chip.
Inspired by prior arts [3, 22], we employ the RUDY estimator [25] to estimatewire congestion.
Details of RUDY can be found in the Appendix A.2.
Secondly, the placementdensity calculates the overlapping region between every pair of circuit modules.
It is time-consumingsince its computational complexity is O
( V 2 ) where V is the number of modules [1].
The proposedapproach can ensure non-overlapping placement to avoid calculating this density metric explicitly intraining, thus reducing computations while producing a valid layout.
Our Approach
Model Architecture Overview.
Chip placement can be formulated as a Markov Decision Process(MDP)
[26] by placing each module at a time.
Fig.3 illustrates the overall architecture of MaskPlace,which trains a policy π θ ( a t | s t ) represented by a convolutional encoder-decoder network with parameter set θ , and a value function V ϕ ( s t ) represented by an embedding model with parameter setϕ .
The policy network receives previous observations and actions as input s t and selects an actiona t as output.
Specifically, s t is a set of pixel-level feature maps that comprehensively capture thenet and pin configurations in M 1: t − 1 , M t , and M t +1 , where M 1: t − 1 denotes the modules that havebeen placed in the previous time steps from 1 to t − 1 , while M t and M t +1 denote the modules to beplaced at the current step t and the next step t + 1 , respectively.
Intuitively, MaskPlace looks one stepforward to achieve better placement.
Although prior arts [3, 22] represented a netlist as a hypergraph as shown in Fig.2(f) where eachnode is a module, and each edge is a wire between two modules, they lost the information of pinoffsets for each module.
Unlike previous works, MaskPlace can fully represent massive net and pinconfigurations using three types of pixel-level feature maps, as shown in Fig.2(a-d), including position
[Figure]
mask, wire mask, and view mask, as discussed below.
Different masks are fused by convolutions tolearn the state representation.
[Figure]
The position mask, denoted by f p ∈{ 0 , 1 } 224 × 224 , is a binary matrix of a canvas grid with size224 × 224 as shown in Fig.4, where value “ 1 ” means a feasible position to place a module.
The purpose of the position maskis to guarantee no overlaps between modules ( i.e., satisfy the overlap constraint) and to learn the relationship between placementand wirelength.
Specifically, we slide a module M t (for example,t = 5 ) on the entire chip canvas.
The trajectory of the feasiblepositions (in green) can be labeled with “1”.
Intuitively, we cancheck each position for each module using the cumulative sumarray [27].
This naive approach has the computational complexityof O ( N 2 ) when a 2D canvas grid is divided into N × N cells.
However, this simple approach is not efficient when N is large.
Therefore, since all modules are rectangles, we design an efficientgeneration algorithm, which iterates through all placed modules(in blue) and excludes positions that will cause overlap.
In this case, all remaining positions areavailable for placement.
The new algorithm is summarized in Appendix A.3, which costs O ( V ) foreach module, where V is the number of modules.
[Figure]
Wire Mask.
The wire mask, denoted as f w ∈ [0 , 1] 224 × 224 , isa continuous matrix for representing how HPWL increases if weplace a module M t in a specific position.
Fig.5 shows a sample ofwire mask, where each value means the increase of HPWL.
Thewire mask aims at finding the best position with the minimumincrease of the wirelength.
Intuitively, we can calculate the HPWLat each canvas position, leading to a complexity of O ( N 2 P ) ,where P is the total number of pins.
However, a fast algorithmcan be designed by considering the relationships between the pinoffset, the net bounding box, and the linear property of the HPWLmetric.
For example, Fig.4 illustrates that the next module M 5has two pins, P (5 , 1) and P (5 , 2) , belonging to “Net 1” and “Net 2”respectively (Fig.2(e)).
Fig.5 illustrates the increase of wirelengthwhen placing M 5 at each canvas location.
For instance, if M 5 isat the bottom-left corner, its Manhattan distance to the two net bounding boxes (in red and green)is 2 + 2 = 4 .
To calculate the Manhattan distance more accurately, we move the net bounding boxcompared to the pin location.
For example, since P (5 , 2) is located at (2 , 1) 2 , we move the bounding box of Net 2 (in green) in the direction − ∆ (5 , 2) =
( − 2 , − 1) to encode the information of pin offset.
The time complexity can be reduced to O ( NP ) .
The algorithm can be found in Appendix A.3.
View Mask.
The view mask, denoted as f v ∈ { 0 , 1 } 224 × 224 , is a global observation of the currentchip layout, where the value “ 1 ” means a module has occupied this grid cell.
Different from DeepPR[22] that assumed all modules have unit size, we consider real sizes of modules.
For instance, if amodule has size w × h , it covers ⌈ wN/W ⌉×⌈ hN/H ⌉ cells in the canvas, where W and H representthe canvas size and ⌈·⌉ denotes the ceiling function.
Learning Algorithm.
We train different blocks in Fig.3 as a whole using reinforcement learning.
The detailed network architectures are provided in Appendix A.4.
Firstly, we apply the above masksto represent the entire circuits and feed them to downstream networks.
Secondly, a global featureencoder embeds the view mask of current placement and the wire masks of the following two stepsinto an embedding vector.
Then we combine it with the positional embedding of the t -th circuitmodule in the value network to generate a scalar to evaluate the current state by fully-connected layers.
Thirdly, a global mask decoder recovers a feature map of size N 2 , which is fused with differentposition masks and wire masks in the policy network using 1 × 1 convolutions to avoid the localsignal diffusion.
The policy network predicts a probability action matrix of size N × N , indicatingwhere to put the next module.
Before sampling actions, we can remove unfeasible actions usingthe position mask.
Finally, the congestion satisfaction block applies the congestion threshold on theprobability matrix to select a final action.
Reinforcement Learning.
We borrow the representative actor-critic diagram [28] and PPO2 framework [29] to train the policy π θ ( a t | s t ) , where the state representation s t is listed in Table 9 in
Appendix.
The action a t is the canvas position (cell) to place the circuit module.
Specifically, wetreat the chip canvas as a grid and divide it into N × N cells, leading to N 2 possible actions.
Theobjective function of the policy network can be formulated as
[Equation]
where the ratio r t ( θ ) =
π θ ( a t | s t )
π θ old ( a t | s t ) and ˆ A t =
G t − ˆ V t denotes the advantage function.
We employ
G t = (cid:80) V − t − 1 k = γ k r t + k +1 that is the cumulative discounted reward and ˆ V t is the estimated valueproduced by the value network.
We update the the value network by optimizing its objective,
L value ( ϕ ) =
ˆ E (cid:2) ( G t − ˆ V t ) 2 (cid:3) .
Reward r t .
We treat HPWL as the reward because wirelength is the main optimization target indifferent performance metrics.
This is different from prior arts [3, 22] that weighted combines
HPWL and congestion as the reward, which introduces the weighting coefficient as an extra hyperparameter to tune.
Specifically, we achieve a dense reward by defining a partial HPWL, which onlycomputes the currently placed pins.
For example, the partial HPWL for t modules can be definedas HPWL t .
In other words, we compute HPWL t after taking action a t .
The reward for the stept is r t =
HPWL t − 1 − HPWL t , i.e., the opposite number of the increase of HPWL.
Furthermore,instead of computing HPWL at each step, we can maintain the ranges of all net bounding boxes inone episode and update the changes of their sizes with a cost of O ( P ) , where P is the number ofpins.
Thus we can generate dense rewards while maintaining efficiency.
Training and Testing.
Before training, we follow previous work [3] to sort the circuit modulesaccording to the number of nets, areas, and the number of connected modules that have been placedto determine the place order.
In training, we update the policy and value networks at each epoch byignoring the congestion satisfaction block.
When updating the value network, we stop the gradientback-propagated in the global mask encoder to avoid influence on the policy network.
The detailedtraining setup is provided in Appendix A.5.
In the testing stage, for each step t , we obtain a probability matrix from the policy network andsample one place action a t .
Then, the congestion satisfaction block will check whether the congestionexceeds a threshold
C th after applying this action.
If it exceeds, we uniformly sample a few actions,look up the corresponding values from the wire mask f tw , and estimate the congestion before takingthese actions.
We choose the action with the minimal value in f tw and the congestion lower than C th .
If all actions cannot satisfy congestion C th , we select the action with the minimal congestion andmove to the next step.
Detailed congestion satisfaction algorithm can be seen in Appendix A.3.
Experiments
We extensively evaluate MaskPlace and compare it with several recent advanced placement methods,including NTUPlace3 [6], RePlAce [8], DREAMPlace [9], Graph Placement [3], and DeepPR[22], where NTUPlace3, RePlAce and DREAMPlace are optimization-based methods, whilst Graph Placement and DeepPR are learning-based approaches.
All of them are evaluated on different publiccircuit benchmarks.
All previous works are evaluated by following their experimental settings.
Benchmark.
We evaluate MaskPlace in 24 circuit benchmarks selected from public datasets includingthe widely-used ISPD2005 [30], IBM benchmark suite [31], and Ariane RISC-V CPU design [32].
The number of evaluated benchmarks is three times more than previous work [9, 22, 3].
The statisticsof benchmarks are given in Table 12 in Appendix A.6, where the largest circuit contains 1,293 macros,22,802 pins, and more than a million standard cells, leading to a vast state space as aforementioned.
Main Results.
Table 2 compares the HPWL results between all the above methods to place all macros.
To enable a fair comparison, we evaluate all approaches 3 by using five random seeds and report themeans and variances.
Since the original DeepPR method did not capture macro size (thus does notavoid overlap between adjacent macros because all macros have unit size), we extend DeepPR tomodel macro size to reduce the overlapping ratio.
We name it “DeepPR-no-overlap”.
Similar to priorworks, we use the minimum spanning tree algorithm
[33] to estimate routing wirelength [34].
From
Table 2, we can see that MaskPlace achieves the lowest wirelength in six out of seven benchmarks(except “adaptec4” where it still outperforms all learning-centric methods).
We also see that theconventional optimization-based approaches may fail when the circuit benchmark has high chip areausage, such as “bigblue3 ” and “ariane”.
Also, MaskPlace gets the lowest wirelength compared with
Graph Placement and simulated annealing [35] in the IBM benchmark, which is shown in Appendix
A.7.
This anonymous website 4 visualizes and compares different placements.
[Table]
Compare to Graph Representation.
Since Graph Placement [3] is the recent advanced learningbased approach that employs hypergraph for placement, we compare MaskPlace with it in all fourperformance metrics, including HPWL, congestion, density, and overlap area ratio.
Table 3 and 4report the results.
The overlap area ratio describes the ratio of the overlapping area between macrosdivided by the chip area.
In Table 3, MaskPlace (soft constraint) means that the round function ratherthan the ceiling function is used to calculate the number of grid cells occupied by the placed macros.
MaskPlace with soft constraints may produce better HPWL and congestion than its counterpart withhard constraints, but the overlap area ratio could not be zero because the constraints have been relaxed.
From Table 3 and 4, we see that MaskPlace outperforms graph placement by large margins, especiallyin the ISPD benchmark, where the former reduces HPWL compared to the latter one by up to 80% while ensuring zero overlaps in all benchmarks.
More results in the IBM benchmark can be found in
Appendix Table 14.
[Table]
[Table]
Routing Wirelength.
Table 5 compares the routing wirelength between MaskPlace and DeepPR[22].
We show that using the true wirelength as the reward would lower the efficiency and produce asparse reward.
We see that MaskPlace, which employs HPWL as the reward
[Table]
can achieve 60% to90% shorter routing wirelength than DeepPR, which directly used the true wirelength as the reward
[Table] [Table]
Standard Cells.
Table 6 compares the HPWL of both the macros and the standard cells by using
MaskPlace, DeepPR [22], and DREAMPlace [9], where DREAMPlace is employed to place thestandard cells following the experimental setup in [22].
We can see that the proposed methodsurpasses the other approaches by up to 50% in the large benchmark “bigblue3”, which has morethan a million standard cells.
[Table]
Efficiency.
Table 7 compares the inference efficiency of different approaches.
All of them areevaluated on one GeForce RTX 3090 GPU, and the CPU version of DREAMPlace is allocated with16 threads in a 16 CPU cores environment.
We see that the careful design of MaskPlace makes itfaster than two other learning-based approaches.
Ablation Study.
We compare different components in MaskPlace as shown in Fig.6.
Each curve isproduced by five seeds using the benchmark “adaptec1”.
For example, MaskPlace w/ CL means using1/3 of the circuit macros to pretrain the model for 30 epochs like curriculum learning.
MaskPlace w/o
[Table]
M t +1 means only considering M t as input without looking one step forward.
Moreover, MaskPlacew/o number of nets means this feature is not considered when determining the placement order.
MaskPlace w/o 1x1 conv means that 7x7 kernels are used to replace the 1x1 kernels in the localfeature fusion block.
Also, MaskPlace with sparse reward means compute HPWL reward only whenall macros have been placed, and the rewards for other steps are set to zeros. MaskPlace w/o viewmask and w/o wire mask means that the corresponding mask is not inputted into the model.
We cansee that MaskPlace (standard) with curriculum learning has the best performance.
Congestion Satisfaction.
To evaluate our congestion satisfaction block, we implement a placementwithout any congestion threshold ( i.e., ∞ ) as shown in Fig.7.
We evaluate the “adaptec3” benchmark,where MaskPlace outperforms DeepPR.
We gradually lower the threshold C th from 60 to 10.
Wefind that lower congestion leads to an increase in the HPWL.
Our method can always satisfy thecongestion constraint in five seeds in a suitable range (above 40 in this benchmark).
If we continueto reduce the congestion threshold after a specific value (say 40 in Fig.7), we found that it hardlysatisfies the threshold because nets must take up a certain amount of wire resources.
[Figure] [Figure]
Conclusion
This paper proposes MaskPlace, an RL-based placement method based on rich visual representationby learning position, wirelength, and view information.
It helps the model take action effectively andefficiently without reducing the search space.
We design a direct reward function based on practicalscenarios and get satisfactory results on all key metrics.
This work can facilitate the placementprocess and avoid undesired overlaps between modules.
In the future, we will explore the standardcell placement by designing a suitable representation, which is an open problem for RL due to itsvast space.
Limitation and Potential Negative Societal Impact.
Chip design flow contains many stages, andour method shows its potential in a single stage.
Similar to previous RL methods, it also requires anoptimization method when placing millions of standard cells because RL’s state space is too large.
Our method does not have potential harm to the public society at the moment.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](
b) Did you describe the limitations of your work
[Yes]
c) Did you
discuss any potential negative societal impacts of your work
[Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes](d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation? [N/A] A AppendixA.1 Module, Net and Pin Module.
A chip is a combination of numerous modules, and there are two types of them: macrosand standard cells.
Macros are relatively large, including DRAMs, caches, and IO interfaces.
Standardcells are mainly logical gates, much smaller than macros, and the size can be ignored.
As in Fig.8(a), there are four macros and several standard cells.
Placement methods usually place macros
firstand then the standard cells to ensure there is enough space for macros [36].
Due to the considerablenumber of standard cells, we currently use our MaskPlace method on macro placement.
Pin.
Pins are input/output interfaces for modules and are connected by wires directly, which havefixed relative positions on modules.
We define the relative position of the pin P ( i,j ) from the leftbottom corner of the module it belongs to as ∆ ( i,j ) =
(∆ ( i,j ) x , ∆ ( i,j ) y ) .
For example, there are fivepins and three macros in Fig.9 (a), and the pin offset information is also shown at the bottom.
Inthe placement task, we should not ignore the positions of pins because it determines the wirelength.
However, graph neural network-based models [3, 22] ignored them when converting circuits into agraph, which may lead to sub-optimal results.
Net.
A net contains a set of pins connected by the same wires.
Thus the pins have the sameinformation (0/1 in digital circuits).
For example, four pins belong to Net 1, and the other three pinsbelong to Net 2 in Fig.8 (a).
Usually, one pin belongs to only one net, and one net has more than twopins (one input and several outputs).
Pins from the same net can form a net bounding box as Fig.8(a)(b).(𝑥 1 ,𝑦 1 ) (𝑥 2 ,𝑦 2 ) 𝑤𝑤 2 ℎ
[Table]
[Table]
[Figure]
A.2 Metric
HPWL.
HPWL (Half Perimeter Wire Length) is widely used to estimate wirelength by smallcomputation cost [24].
It is the sum of half perimeter of net bounding boxes as Fig.8 (a)(b), wherethe bounding box is the minimal rectangle including all pins belonging to this net.
Congestion.
The congestion metric is used to avoid routing congestion, resulting in an increase inthe actual wirelength because the resources for wires are limited in a real chip.
There are many waysto estimate congestion, one is to compute a rough routing result [3], but it is very computationally intensive.
We use RUDY [25] as the estimation of congestion, which is a common way to evaluate.
In RUDY, each grid cell needs to accumulate the inverse of the height and width (1 /h + 1 /w ) of allthe net bounding boxes covering itself and take out the maximum value (or the average of the first kmaximums) of all grid cells as Fig.8 (a)(b).
Density.
Density is a metric to reduce overlaps and avoid time-consuming computation for O ( V 2 )constraints [1].
So, it is an approximate calculation essentially.
It is defined as the maximum stackablecoverage area ratio for each grid cell on a chip canvas.
{+For example, as Fig.8 (c), the maximumstackable coverage area ratio is 2 . 
+}
in grid cell g 7 , 5 because two modules fully occupy it.
However,density less than a small value is not a sufficient condition for the absence of overlap.
Because ourmethod can ensure no overlaps, we only consider it in evaluation.
In the practical application scenarioof chip design, HPWL is an optimization item.
Conversely, congestion and density are constraintitems.
Examples.
We give a set of placement results to explain the metrics in Fig.8.
We can see that HPWLis the sum of width and height of net bounding boxes.
Congestion (RUDY) is the max congestionvalue of grid cell g i,j , and the value in each grid cell is cumulative from the reciprocal of the widthand height of the net bounding box containing that grid cell.
(a) and (b) are from the same circuit, but(b) is a better placement because (b) has lower HPWL and congestion.
Density is the max densityvalue of grid cell
g i,j , and the value in each grid cell is stackable coverage area ratio of the grid cell.
The density of Fig.8 (c) is 2.0 because g 7 , 5 completely covered by two modules.
Relationship between pin offset and HPWL.
The pin offset can affect the HPWL.
In the graphbased method, the input features for module include size ( M w , M h ) , position ( M x , M y ) and type.
So, the network can hardly infer the real position of pins and tend to use the center positions ofmodules to predict the positions of pins.
In this way, the agent will align the centers of the twomodules horizontally, and the result of placement is like Fig.9 (b) to get the wirelength 6.
However,when considering the pins are near the bottom of the modules, it is better to align the bottom of thetwo modules as Fig.9 (c), and thus wirelength can be reduced to 2 if we consider the pin offset.
[Figure]
When we remove the pin offset information, the model tends to align the centers of the two modules horizontally like (b) because it uses the center position of modules to estimate pin location.
However, we have a better design as (c) when considering pins are located on the bottom of the modules.
A.3 Algorithms
Reward Computation.
The dense reward generation algorithm is shown in Algorithm 1.
It cangenerate dense rewards without an efficiency decrease.
For simplicity, we omit the calculation of they dimension, which is the same as the x dimension.
MaxMinCoord
Result
Incremental
; reward ← 0 ; foreach ∆ ( t,j ) x , P ( t,j ) n of all pins P ( t,j ) from M t do x ← M tx + ∆ ( t,j ) x ; // calculate pin coordinates if P ( t,j ) n not in MaxMinCoord then //
The net for the first time has a definite location of the pin
MaxMinCoord [ P ( t,j ) n ] .x.max
← x ;
MaxMinCoord [ P ( t,j ) n ] .x.min
← x ; else // Update the bounding box range if MaxMinCoord [ P ( t,j ) n ] .x.max <
x then reward ← reward + ( x − MaxMinCoord
[ P ( t,j ) n ] .x.max ) ;
MaxMinCoord [ P ( t,j ) n ] .x.max
=
x ; else if MaxMinCoord [ P ( t,j ) n ] .x.min
> x then reward ← reward + ( MaxMinCoord [ P ( t,j ) n ] .x.min − x ) ; MaxMinCoord [ P ( t,j ) n ] .x.min = x ; end end end
Position Mask Generation.
The efficient position mask generation algorithm is in Algorithm 2.
Data
( M 1: t − 1 w , M 1: t − 1 h , M 1: t
1 x , M 1: t − 1 y )
N × N matrix for i ← 1 to t − 1 do tmp ← ones ( N, N ) ; // find positions that will cause M t and M i to overlap tmp
[Equation]
; // exclude infeasible positions f
tp ← tmp ⊙ f tp ; // ⊙ is element-wise product end
Wire Mask Generation.
The efficient wire mask generation algorithm is shown in Algorithm 3.
For simplicity, we omit the calculation of the y dimension, which is the same as the x dimension.
Congestion Satisfaction.
The algorithm implemented in the congestion satisfaction block can beseen in Algorithm 4.
Algorithm
Data: Hash
Max
Min offsets (∆ ( t,j ) x , ∆ ( t,j ) y ) , pin to connection P ( t,j ) n
Result: Wire Mask f tw for module M t f tw ← zeros ( N, N ) ; // Accumulate the wirelength increase for each net foreach ∆ ( t,j ) x , P ( t,j ) n of all pins P ( t,j ) from M t do //
If the pin is to the left of the net bounding box for i ← 0 to MaxMinCoord [ P ( t,j ) n ] .x.min
+ ∆ ( t,j ) x − 1 do f tw [ i, :] ← f tw [ i, :]
+ MaxMinCoord
[ P ( t,j ) n ] .x.min
+ ∆ ( t,j ) x − i ; end //
If the pin is to the right of the net bounding box for i ← MaxMinCoord [ P ( t,j ) n ] .x.max + ∆ ( t,j ) x
+ 1 to N − 1 do f tw [ i, :] ← f tw [ i, :]
+ i − ( MaxMinCoord [ P ( t,j ) n ] .x.max
+ ∆ ( t,j ) x ) ; end end
Data: expected threshold C
Result:
A placement plan [ a 1 , a 2 , ..., a V ] that meet the congestion requirement i ←
V do
Choose a i from the probability matrix generated by policy network agent ;
Cong ← congestion matrix from the state after taking a i ;
Compute congestion value c from Cong ; if c > C th then
Randomly sample N different actions a 1: N i from action space;
Compute N congestion values c 1: N i from congestion metrics;
Get N wirelength values w 1: N i from wire masks;
Sort the N actions according to w 1: N flag ← False ; for j ← 1 to N do if c ji ≤ C th then flag ←
; a
; break; end end //
If all sampled actions cannot satisfy congestion threshold, we choose the one with minimal congestion increase.
if flag is False then a i ← the action a ji with minimum c ji ; end
Take action a i as the final action; end
A.4 Details of Model Architecture
The parameters of layers in model architecture are in Table 8.
Also, the features used by pixel-levelmask generation are in Table 9.
The comparison of features for the placement order in differentmethods can be seen in Table 10.
[Table]
[Table]
[Table]
A.5 Training Configuration
The detailed configuration and hyperparameter settings of our model is in Table 11.
[Table]
Also, we implement DREAMPlace 5 [9], Graph Placement 6 [3] ,and DeepPR 7 [22] by their opensource codes with their default settings.
A.6 Details of Benchmark
[Table]
The detailed statistics of benchmarks are in Table 12.
Hard macros are macros placed by the RLmethod in Graph Placement [3], and the remaining macros, also named soft macros, are placed bythe classic optimization-based method.
However, this distinction does not apply to the process ofour method, which means we place all macros by RL.
The statistical range of nets, pins
[Table]
and areautilization are macros.
Ports are terminals connecting to an external circuit, seen as fixed and no-size
[Table]
modules.
Our method is also applicable to circuits with ports without additional modifications.
[Table]
[Table]
[Table]
[Table]
[Table]
A.7 Supplementary Experiment
Placement w/o real size Considering that DeepPR ignored the module size, we implement
MaskPlace in the same setting, and the result can be found in Table 13.
The result shows thatour method has significant advantages.
[Table]
More benchmarks We also conducted experiments in the IBM benchmark suite (ICCAD 2004)[31], which has been used to evaluate placement for more than a decade.
We remove the “ibm05” because it does not contain any macros.
We use our MaskPlace to place large macros and DREAMPlace[9] to place standard cells.
We compared our method with graph placement [3] and the simulatedannealing method used in [3].
The results are in Table 14 and our method can achieve the lowest
HPWL in all benchmarks.
[Table]
MaskPlace+DREAMPlace [9]
[Table]
Graph Placement [3]
[Table]
Simulated Annealing [3]
[Table]
Graph
[Table]
[Table]
MaskPlace+DREAMPlace [9]
[Table]
For the larger circuit bigblue4 in ISPD 2005 benchmark, the result of our method and baselines canbeen seen as Table 15.
MaskPlace still achieved the best performance.
[Table]
Search time We compared the search time of our method, Graph Placement [3] and DeepPR [22].
We tested all methods in the same environment and took the HPWL as the metric in benchmarkadaptec1 .
The result is in Fig.
We can see that our approach can achieve the best performance ina few hours.
Transferability Test the performance of the model trained on adaptec1 on other benchmarks as
Table 16.
The results show that our method also has a good transferability.
[Table]
[Figure]
A.8 Detailed equation description of the model
We describe our model architecture in Fig.
3 in the form of equation.
With current state s t , we first calculate the position masks f tp , f t +1 p , wire masks f tw , f t +1 w and viewmask f tv via the mask generation function m ( · ) .
[Equation]
Then we extract the local feature z lt via local mask fusion g ω ( · ) and the global features z gt via theglobal mask encoder enc η ( · ) .
[Equation]
(4) where g ω ( · ) is a 1×1 convolutional neural network with parameter ω .
[Equation]
(5) where enc η ( · ) is a convolutional neural network with ResNet-18 architecture with parameter η .
With local features z lt and global features z gt , the state value ˆ V t is derived by
[Equation]
where v ϕ is an MLP-like neural network with parameter ϕ and pos ( t ) is an embedding vector whichis related to step t .
We decode the global features z gt into the dimension as same as the action space by the global maskdecoder
[Equation]
where dec δ ( · ) is a transpose convolutional neural network with parameter δ .
Finally, we concatenate the local features z lt and global features z ′ gt in the channel dimension andmerge them by another 1×1 convolutional neural network ψ ξ ( · ) .
We further combine it with theposition mask f tp to generate action a t via the policy network π θ ( · )
[Equation]
where π θ ( · ) is an MLP-like neural network with parameter θ and ψ ξ ( · ) is a 1×1 convolutional neuralnetwork with parameter ξ .
B Related Work
Classic optimization-based methods.
Optimization has been the dominant method in placementfor decades.
They can be divide into three categories: partitioning-based methods [4, 5], simulatedannealing methods [10, 11] and analytical methods [6–9, 12–21].
Partitioning-based methods [4, 5] cluster the whole circuits into several parts to minimize theconnections between parts.
These methods first solve the placement problems within the same partand then place these parts to suitable positions on the chip based on the divide-and-conquer idea.
However, optimizing modules within one part is an isolated problem, and sometimes it is hard todivide the circuits into relatively independent parts, which is highly related to the topology of thecircuits.
Simulated Annealing (SA) methods [10, 11] are also known as hill-climbing methods, a widelyused iterative heuristic algorithm for solving combinatorial optimization problems.
They initialize arandom status and then search for the following status by moving from the current status to a neighborstatus.
If the metrics of the neighbor status are better than that of the current status, they move to theneighbor status.
Otherwise, the move may still be taken with a decreasing probability over time.
Theadvantage is that they can be implemented when metrics do not have the analysis formula or cannotbe differentiable.
However, it is not efficient enough, and the placement results are highly dependenton the random initial state.
Analytical methods gradually replace the above two methods because of the best performance.
Theycan be divided into quadratic methods [12–18] and nonlinear (non-quadratic) methods [6–9, 19–21].
Quadratic methods [12–18] transform the placement problem into a sequence of convex quadraticproblems, and there are well-established solvers for such problems.
However, it is a very roughapproximation.
Nonlinear methods [6–9, 19–21] design a single differentiable objective functionand optimize it.
The advantage is that it can handle large-scale modules.
However, the objectivefunction is still approximated, and they cannot avoid overlaps when combining multiple metrics inone objective function.
Methods in this category achieved the highest placement quality among allclassic methods [9].
Learning-based methods.
With the development of deep learning, some learning-based approaches[11, 37–39] have been proposed to assist classic methods.
Huang et al.
[37] uses convolutional neuralnetworks to estimate the congestion for SA placement.
Vashisht et al.
[11] uses the reinforcementlearning models to generate the initial placement of SA.
Kirby et al.
[38], Agnesina et al.
[39]help classic placement tools choose the most suitable hyperparameters with reinforcement learningmethods.
However, these methods do not implement end-to-end placement by deep learning, so theplacement results depend heavily on the classic methods.
Pure reinforcement learning methods [3, 22, 23, 40] view placement as a process of placing modulessequentially.
Mirhoseini et al.
[3] uses reinforcement learning to place hard macros, and the forcedirected method [18] to place remaining soft macros.
Jiang et al.
[23] replaces the force-directedmethod with DREAMPlace [9] to place soft macros based on Graph Placement [3].
Cheng and Yan[22] proposes a reinforcement learning method by using wirelength as the reward.
Moreover, Changet al.
[40] puts all metrics in the RL reward.
They have in common that they convert the circuit asa graph structure and input them to the graph neural networks [41].
However, the pin informationhas been lost, leading to sub-optimal placement.
Also, they cannot avoid overlaps because of thereduction in search space.
These methods still have room for improvement in terms of realistic chipplacement.
For instance, DeepPR [22] ignores the realistic size of the module.
However, the size ofthe modules varies widely in most circuits.
Although it proposes to use routing wirelength instead of
HPWL as the reward, it will affect the efficiency and lead to sparse reward, making models hard totrain.
In contrast, HPWL is a high-quality wirelength estimation, and we do not need to discard thisinherent dense reward.