# A Deep Reinforcement Learning Floorplanning Algorithm Based on Sequence Pairs +
###### Abstract  
In integrated circuit (IC) design, floorplanning is an important stage in obtaining the floorplan of the circuit to be designed. Floorplanning determines the performance, size, yield, and reliability of very large-scale integration circuit (VLSI) ICs. The results obtained in this step are necessary for the subsequent continuous processes of chip design. From a computational perspective, VLSI floorplanning is an NP-hard problem, making it difficult to be efficiently solved by classical optimization techniques. In this paper, we propose a deep reinforcement learning floorplanning algorithm based on sequence pairs (SP) to address the placement problem. Reinforcement learning utilizes an agent to explore the search space in sequence pairs to find the optimal solution. Experimental results on the international standard test circuit benchmarks, MCNC and GSRC, demonstrate that the proposed deep reinforcement learning floorplanning algorithm based on sequence pairs can produce a superior solution. VLSI; floorplanning; sequence pair; deep reinforcement learning; MCNC; GSRC 2024 14 2  
be divided into two categories. One category is based on planar graph representations. For floorplan graphs with slicing structures [8], binary trees are widely used, where leaves correspond to blocks and internal nodes define the vertical or horizontal merge operations of their respective descendants. For more general non-slicing floorplan representations, several effective forms have been developed, including sequence pairs (SP) [9], the bounded slicing grid (BSG) [10], O-trees [11], transitive closure graphs with packed sequences (TCG-S) [12], and B*-trees [13]. Among these, the representation of block placement with sequence pairs, which uses positive and negative sequences to represent the geometric relationships between any two modules, has been extended in subsequent work to handle obstacles [14], soft modules, rectilinear blocks, and analog floorplans [15; 16; 17; 18]. The decoding time complexity of the sequence pair representation is O(N\({}^{2}\)). In order to reduce the decoding complexity, [PERSON] et al. [19]. utilized the longest common subsequence algorithm to decrease the decoding complexity to O(NlogN). Subsequently, [PERSON] and [PERSON] [20] proposed an enhanced Fast Sequence Pair (FSP) algorithm, further reducing the decoding time complexity to O(NloglogN). Another category involves the study of floorplanning algorithms. By employing suitable planar graph representations and/or efficient perturbation methods, high-quality floorplans can be achieved through linear programming [21] or some metaheuristic methods such as simulated annealing (SA) [22; 23], genetic algorithms (GA) [24; 25], memetic algorithms (MA) [26], and ant colony optimization [27]. Despite decades of research on VLSI floorplanning problems, the existing studies indicate that current EDA floorplan tools still struggle to achieve a floorplan close to optimal. These tools continue to face numerous limitations, making it challenging to obtain satisfactory design outcomes. Existing floorplan tools generally require long runtimes and experienced experts to spend weeks designing integrated circuit floorplans. Furthermore, these tools have a limited scalability and often require a time-consuming redesign when faced with new problems or different constraints. Reinforcement learning (RL) [28] provides a promising direction to address these challenges. Reinforcement learning possesses autonomy and generalization capabilities, allowing the agent in reinforcement learning, through interactions with the environment, to automatically extract knowledge about the space it operates in. In addition to breakthroughs in gaming [29] and robot control [30], reinforcement learning has been applied to solve combinatorial optimization problems. [PERSON]. [31] proposed deep reinforcement learning (DRL) for solving the Traveling Salesman Problem (TSP).