Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: Placement is one critical step to ensure circuit performance including power consumption, delay, and chip area in physical design. This paper presents a reinforcement-learning-based approach to optimize the half perimeter wirelength (HPWL), placing circuit modules sequentially. This proposed method adopts multi-view (mask) visual representation from the placement to ensure the non-overlapping property among modules. Each mask clearly captures a certain aspect (allowable positioning, metric change, global placement information) at the current placement snapshot, resulting in efficient policy learning. The paper performed an extensive experimental study on multiple benchmarks and demonstrated the effectiveness of individual components through the ablation study.
strengths_and_weaknesses: Strengths:
1. This paper transforms the geometric placement problem into multiple visual representations using three masks, which opens the possibility of using mature convolution networks to extract the global layout information. In addition, each mask has its distinct purpose, benefiting the overall placement task from different angles. The position mask guarantees the non-overlapping property of the proposed method; the wire mask gives a good estimation of the metric (HPWL) change for the action; and the view mask provides a global view of the current placement results.
1. The paper provides multiple components which can be beneficial for future research in this area. The authors designed multiple masks for the placement and their corresponding efficient generation algorithms, which can be used as individual components to inspire other related placement and routing tasks.
1. This paper conducts an extensive experimental study of the proposed methods over various benchmarks and multiple metrics against both state-of-the-art optimization-based and RL-based approaches, demonstrating the effectiveness of their method. In addition, the ablation study also shows the power of each individual component of the proposed method.

Weaknesses:
1. There exist certain inconsistencies between the training and inference phase due to congestion satisfaction. The congestion satisfaction is not considered during the RL training phase. However, it is used during the inference phase. During the inference phase, the probability matrix of the action doesn't anticipate the effect of the congestion satisfaction, which may result in lower quality placement compared to the training phase (ignoring the congestion constraint).
1. Some detailed components of the design RL frameworks are not fully explained, as listed in the questions below.
questions: 1. For congestion satisfaction, the reviewer wonders what is the current reasoning behind this inconsistency in the training and inference phases, and if there is any way to mitigate this effect.
1. Detailed components clarification 1: How did the position embedding at time $t$ in the value network works? In addition, what is the purpose of this component?
1. Detailed components clarification 2: It seems that the ordering of the module for placement matters a lot according to Fig. 6. The reviewer wonders how the current sorting is done exactly. The authors said "*we follow previous works to sort the circuit modules ...*" in Training and Testing of Section 3. However, no corresponding literature is given. In addition, it would be appreciated if the authors can give more comments on the reasoning behind the sorting heuristics, and what is the consequence of bad ordering.
1. Detailed components clarification 3: The reviewer understands the power of looking more steps forward to the RL agent, and wonder how the step are generated here. In addition, it would be great if the authors can comment on the tradeoff between the computation efficiency and learning quality w.r.t. the number of steps looking ahead.
1. Detailed components clarification 4: It is unclear how the algorithm works if the module size doesn't fit the grid cell. For example, if a module occupies one grid cell without covering the entire grid, is it still possible to place another module in the same grid?
1. To the reviewer's understanding, the view mask contains only two values ${0,1}$. However, the view mask in Fig.3 contains three different colors. Is it a simple illustration error?
1. Table 7 gives the inference time for various methods. The reviewer is curious about how long the training takes. Moreover, do the training examples come from only a single placement task which is also used in inference, or multiple placement tasks which can be generalized to different tasks during the inference time?
limitations: The proposed methods can be hardly applied to standard cell placement due to the large state and action space. However, the authors show in Table 6 that the placement tool which can handle standard cell placement (like DREAMPlace) may also benefit from the proposed method of handling module placement. This is beneficial for the EDA tool development in general.
There is no identified negative societal impact.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents a RL-based chip placement approach, MaskPlace, to automatically generate a valid chip layout design. Compared with the former approaches that apply hypergraph to represent the chip layout, MaskPlace adopts the pixel level graphical representation to represent the layout and pin offset, which allows for MaskPlace to produce a better performance. The results show that MaskPlace can achieve 60%-90% wirelength reduction with zero overlaps.
strengths_and_weaknesses: Strengths of the paper:
1. Chip placement with RL has been recently studied by multiple literature, this work proposes a better solution for chip placement. The comprehensive 2D pixelwise representation of the chip layout and wire length enables MaskPlace to involve the pin offset information when making the placement decision. This results in a better performance than the previous work which adopts hypergraph to represent the chip layout.

2. To generate the input state for the RL agents, heuristic algorithms are proposed to produce the input masks for the RL agents with relatively low complexity.

3. The presentation of the paper is clear, the experimental results are comprehensive and promising.

Weakness of the paper:

See the question section.
questions: I have the following questions for the paper:
1. Since MaskPlace uses the complicated 2D image as the input of the RL agents, I believe this will harm the convergence behavior of the MaskPlace compared with the other hypergraph-based approaches, as their input states will be much simpler. What's the convergence behavior of MaskPlace? What makes MaskPlace converge within a few hours?

2. How is the simulation is performed? Is it based on the real RTL synthesize netlist as [3]? If not, is it simulator based? Please describe the experiment settings in more detail.

3. MaskPlace eliminates the decision which violates the congestion constraint, why not include this congestion constraint in the reward function, so that a decision that violates the congestion constraint will produce a large negative reward?
limitations: NA.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper uses reinforcement learning to solve the chip placement problem. It views the placement canvas as a 2D image during the representation learning.

strengths_and_weaknesses: Strengths
1. The paper is well written and organized. The appendix discusses related details.
2. The method is sound and well defined.
3. The results are charming and convincing.

Weaknesses
1. DREAMPlace is used as a baseline, which is not a good implementation for macro placement. The current DREAMPlace is majorly optimized for standard cell placement. It cannot handle macro placement very well. The authors may consider RePlAce [1] as the baseline.
[1] C. Cheng, A. B. Kahng, I. Kang and L. Wang, "RePlAce: Advancing Solution Quality and Routability Validation in Global Placement," in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 38, no. 9, pp. 1717-1730, Sept. 2019, doi: 10.1109/TCAD.2018.2859220.
2. The authors use ISPD 2005 benchmarks, which are for standard cell placement. Macros are fixed in these benchmarks. Authors should describe how they edit the benchmarks. The authors should also consider using MMS benchmarks [2].
[2] J. Z. Yan, N. Viswanathan and C. Chu, "Handling complexities in modern large-scale mixed-size placement," 2009 46th ACM/IEEE Design Automation Conference, 2009, pp. 436-441, doi: 10.1145/1629911.1630028.
3. Several benchmarks are missing, such as bigblue2, bigblue4. The authors should report the results of these benchmarks if they select the benchmark suites. Otherwise, readers may suspect that these results are not ideal.
4. The ablation study is not thorough. For instance, there are many masks in the proposed method. What is the contribution of each mask? Is every single mask necessary? What is the impact of the combination of these masks? Specifically, if the method is really great, it does not need the position mask that tells the agent the availability since the agent should have this information.
questions: 1. Lines 192 - 201. The authors design the reward as the change of HPWL. How is the HPWL_t calculated if there are nodes not placed? For example, in a net with 3 pins on 3 nodes, what is the HPWL_t when we only place the first node? Since the other 2 nodes are not placed, it does not make sense to compute the HPWL for this net.
2. The authors use CNNs in representation learning. What about other neural architectures, such as Transformers?
3. What is the generation ability of the proposed method? Can it be used on unseen circuit benchmark? 
limitations: The authors discuss the limitation at the end of the paper. The proposed method can only handle a small number of macros. It can not tackle a large number of movable instances, such as millions of standard cells. I think the authors should stress this limitation throughout the paper. Otherwise, readers may think that the paper proposes a method to place all instances. In the first three section, the authors use the term "placement" instead of "macro placement".
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes MaskPlace, which can automatically generate a high-quality and valid layout within a few hours. Unlike previous methods, that requires manual refinement to modify invalid placement, MaskPlace casts placement as a problem of pixel-level visual representation learning. More particularly, 1) Visual representation learning enables describing millions of circuit modules on a chip comprehensively. 2) MaskPlace suggests a new policy network that can capture and aggregate both the global and subtle information on a chip canvas, ensuring non-overlapping placement efficiently. 2) The demonstrated performance on 24 public chip benchmarks by outperforming Graph Placement and DeepPR 5x and 9x in reducing wirelength with 0% overlap in layout. 
strengths_and_weaknesses: Strengths:
- The paper suggested a more accurate representation of chip layout: the three types of pixel-level feature maps can fully represent massive net and pin configurations. Different from DeepPR, that assumed all modules have unit size, this work considers real sizes of modules. 

- The dense reward is an interesting idea . It is an inexpensive way to create an advantage value at each reinforce step, instead of generating one reward at only the end of the episode after the placement is done. This can be both good and bad those, as the decisions can be short-sighted such that the algorithm tries to incrementally improve wire length, losing performance opportunity from sacrificing short-term objectives. However, both the main results and ablation show that dense reward positively improves learning. 

- The paper has thorough comparisons with SoTA related work and demonstrated significant gains on a set of benchmark. Metrics like wire length, density, and overlap are improved. 

- The reviewer is happy to see Ablation Figure 6 of a decomposition of the gains. All the design decisions add up to some of the performance. 

Weaknesses:
- The results look very strong, however, are a bit counter intuitive. The reward only considers wire length by continuously measuring an advantage function at each reinforce step. The reviewer is not so sure how it could improve density, which is demonstrated to be improved in all the results. Wire length and density are non-orthogonal metrics and are usually in conflict with each other. 

- The paper lacks more details on how the baseline methods are implemented and whether the comparisons are fair and all the promising numbers are trustable. 

- There is little formulations. Local mask fusion, global mask encoder and decoder, and other components in Figure 3 should be formulated mathematically for others to reproduce the experiments. 
questions: 1. The reviewer likes the paper for  reasons listed in above session. However, the reviewer is not quite sure about the adds-on values of thsi work compared to the original chip placement paper from Google [1] and why it is so important to the NeurIPS community. I would think the content in the paper makes it more suited for a design automation conference. The contributions in representation learning or algorithm design are small. 

2. Can the authors provide some concrete comparisons on search time or optimization time with baseline algorithms? A great thing to have is to create a figure like Figure 6 but with different curves being the baselines. Make sure to plot wall clock time in the x-axis, instead of epochs as you might create a very complex network which can be 10x slower in step time, compared to the very simple baselines. 

[1] "Chip Placement with Deep Reinforcement Learning", https://arxiv.org/abs/2004.10746
limitations: Yes.
review_title: null

Meta-review: metareview: The reviewers are enthusiastic about the work, and all recommended for the acceptance of the paper. The reviewers think the work is solid and novel, and potentially impactful. For example, Reviewer MdDg noted "This paper transforms the geometric placement problem into multiple visual representations using three masks, which opens the possibility of using mature convolution networks to extract the global layout information." Thanks to the authors for the detail rebuttal and the thorough discussion with the reviewers. Incorporating these points raised in the communication will further improve the paper.
confidence: Certain
Decision: Accept