Ranking Cost: One-Stage Circuit Routing by Directly Optimizing Global Objective Function

1 INTRODUCTION . As described in Moore ’ s Law ( Schaller , 1997 ) , the number of transistors in a dense integrated circuit ( IC ) increases exponentially over time and the complexity of chips and printed circuit boards ( PCBs ) becomes higher and higher . Such high complexity makes the IC design a time-consuming and errorprone work . Thus more capable automatic design systems , such as electronic design automation ( EDA ) tools , are needed to improve the performance . In the flow of IC designs , we need to find proper paths to place wires which connect electronic components on ICs , and these wires need to achieve expected connectivity under certain constraints . One of the most important constraints is that wires on the same layout should not intersect . In addition , to reduce the signal propagation delay , the wire-length should be minimized . This is a critical and challenging stage in the IC design flow ( Hu & Sapatnekar , 2001 ) , known as circuit routing , which has been studied by lots of researchers ( Kramer , 1984 ; Zhang & Chu , 2012 ; He & Bao , 2020 ) . Circuit routing involves a large number of nets ( a net is a set of vertices with the same electrical property ) to be routed , which is computationally expensive and makes manual design extremely time-consuming ( Kong et al. , 2009 ; Coombs & Holden , 2001 ) . Even under the simplest setting , where only two pairs of pins need to be routed , it is an NP-complete problem ( Kramer , 1984 ) . Although lots of circuit routing algorithms have been proposed ( Zhang , 2016 ) , there still remain three major challenges : ( 1 ) Early solutions ( Hu & Sapatnekar , 2001 ) are typically designed with hard-coded heuristics , which suffer from problems of non-optimal solutions ( Zhang , 2016 ) and lack of flexibility over new design needs . Therefore , a more powerful routing method that does not depend on domain knowledge is highly desired . ( 2 ) To reduce the difficulty of complex routing problems , traditional routing algorithms often adopt a two-stage procedure — first global routing and then detailed routing ( Chen & Chang , 2009 ; Kahng et al. , 2011 ) . The problem is that these two stages do not always coordinate well ( Zhang & Chu , 2012 ; Shi & Davoodi , 2017 ) . Sometimes a low-congested global routing result may lead to downstream detailed router un-routable . Hence , an end-to-end algorithm is preferred which can optimize the final global objective ( e.g. , the total wire-length ) directly . ( 3 ) Although a few learning-based methods have been proposed ( Liao et al. , 2020 ; He & Bao , 2020 ) recently , their methods are hard to extend to large-scale applications . In real settings , there are lots of components and nets on a single chip , which shows greater demand for the scalability of routing algorithms . To relieve the problems mentioned above , we propose a new algorithm , denoted as Ranking Cost ( RC ) , for circuit routing . In this paper , we innovatively combine search-based methods ( i.e. , A * algorithm ) and learning-based methods ( i.e. , Evolution Strategies ( Salimans et al. , 2017 ) ) to form an trainable router with proper parametrization . Our method is flexible for integrating new constraints and rules so long as they can be merged into the global objective function . Moreover , our method is an one-stage algorithm , which optimizes the global objective function directly . In our method , we introduce a new set of variables called cost maps , which can help the A * routers to find out proper paths to achieve the global objective . We also train a ranking parameter , which can produce the ranking order and further improve the performance of our method . In the experiments , we compare our method with the commonly used A * method and a canonical reinforcement learning approach , and results show that our method outperforms these baselines with higher connectivity rates . Our ablation study also shows that trained cost maps can capture the global information and guide the routing solution to approach global optimal . Experiments also show that our method is scalable to larger applications . 2 RELATED WORK . In this section , we summarize the related work on circuit routing . 2.1 TWO-STAGE ROUTING ALGORITHMS . The routing problem can be heuristically separated into two stages , the first being the global routing step , followed by detailed routing . On the one hand , there are multiple heuristic-based approaches for global routing including regionwise routing ( Hu & Sapatnekar , 2001 ) , force-directed routing ( Mo et al. , 2001 ) , and rip-up and reroute ( Cho et al. , 2007 ) . On the other hand , the most commonly used detailed routing algorithms are channel routing and its variants ( Ho et al. , 1991 ; Mandal et al. , 2020 ) , which decompose the routing region into routing channels and generate wires in these channels ( Chen & Chang , 2009 ) . One main issue of the two-stage methods is that these two stages do not always coordinate well ( Shi & Davoodi , 2017 ) , which results in enormous difficulty in joint optimization . Instead , our method is an one-stage algorithm and new design constraints can be simply involved into the objective function without changing the algorithm itself . 2.2 SEQUENTIAL ROUTING ALGORITHMS . A more straightforward strategy for circuit routing is to select a specific order and then route nets sequentially , e.g. , sequential A * algorithm and Lees algorithm ( Huang et al. , 2014 ; Malavasi & Sangiovanni-Vincentelli , 1993 ) . The major advantage of this type of approaches is that the congestion information for previously routed nets can be taken into consideration while routing the current one . However , the drawback of these sequential approaches is that the quality of the solution is very sensitive to the orders ( Zhang , 2016 ) . Moreover , earlier routed paths only focus on finding their own best solutions and are impossible to take into account the situation of subsequent paths . Such greedy strategies may make a solvable circuit routing problem insolvable . Figure 1 shows an example that the sequential A * algorithm will fail to handle . In this example , there are two pairs of points to be routed , i.e. , we should connect start vertices Si and end vertices Ei , i ∈ { 1 , 2 } , respectively . If we connect the pair ( S1 , E1 ) first in its shortest path , it will make ( S2 , E2 ) disconnected as showed in Figure 1 ( d ) , and vice versa ( Figure 1 ( e ) ) . But this case can be solved easily by our algorithm as showed in Figure 1 ( f ) , which means that our algorithm can take into account the global information when generating each path . 2.3 LEARNING-BASED ALGORITHMS FOR CIRCUIT ROUTING . There are some learning-based algorithms for circuit routing ( Qi et al. , 2014 ; Liao et al. , 2020 ) . Early methods are more focusing on supervised learning , such as learning models to predict routing congestion ( Qi et al. , 2014 ) or predicting the routability ( Zhou et al. , 2015 ; Xie et al. , 2018 ) . But the performances of these supervised models are constrained by the limits of human knowledge . Recently , Liao et al . ( 2020 ) trained a deep Q-network ( DQN ) to solve the global routing problem and their method still suffers from the miscoupling problem of two-stage routing algorithms . In addition , He & Bao ( 2020 ) reform circuit routing to a tree search problem and utilize a deep neural network as the rollout policy . However , their rollout policy is trained in a supervised manner with artificial data . Instead , our method optimizes the global objective function directly and does not require any artificial data or human demonstration . 3 BACKGROUND AND NOTATIONS . In this section , we will formalize the circuit routing problem first and then give a brief introduction to the OpenAI-ES algorithm ( Salimans et al. , 2017 ) . 3.1 CIRCUIT ROUTING . Circuit routing is a path search problem , where the goal is to find non-intersecting paths that connect an arbitrary number of pairs of vertices . It can be formalized as a grid graph G = ( V , E ) , where each vertex vi ∈ V represents an intersection on the grid , and each edge eij ∈ E represents the path between vi and its 1-hop neighbors vj ∈ V . And the non-routable obstructive vertices form a obstacle set , denoted as O ⊂ V . In circuit routing , a net N = { vn1 , vn2 , ... } ⊂ V is a set of vertices that need to be connected and a multi-vertex net can be decomposed into multiple two-vertex nets via a minimum spanning tree ( MST ) or a rectilinear Steiner tree ( RST ) ( de Vincente et al. , 1998 ; Hu & Sapatnekar , 2001 ) . Following He & Bao ( 2020 ) , we simplify this problem by letting each net only contain two vertices , i.e , a two-vertex net is defined as N = { vs , ve } with a start vertex vs and an end vertex ve . The net should be connected by a path P = [ v1 , v2 , ... , vn ] with vs = v1 , ve = vn and P ∩ O = ∅ . We use |P | to represent the length of the path P . Given a set N of nets { Ni } , we need to find a set P of paths { Pi } that connect these nets . A reasonable routing plan for a given set of nets is that all the paths do not share any vertex . In some cases , the routing problem do not have a solution because such a non-intersecting set P does not exist . In circuit routing , the routed paths should satisfy some constraints or achieve some goals . In this paper , we choose the most commonly used goal , i.e. , minimize the total length of paths . Given a solution P , the total length of paths is defined as L = ∑ P∈P |P | . 3.2 EVOLUTION STRATEGIES . Evolution Strategies ( ES ) is a class of black box optimization methods ( Rechenberg , 1994 ; Hansen & Ostermeier , 2001 ) . Recently , an evolution strategy variant , referred as the OpenAI-ES ( Salimans et al. , 2017 ) , has attracted attention because it could rival the performance of modern deep reinforcement learning methods on multiple control tasks . In the OpenAI-ES , there is a reward function , denoted as F ( ψ ) , where ψ is a solution vector , which is sampled from a probability distribution function ψ ∼ pθ ( ψ ) . The goal is to maximize the expected value J ( θ ) , which is defined as : max θ J ( θ ) = max θ Eψ∼pθ [ F ( ψ ) ] = max θ ∫ F ( ψ ) pθ ( ψ ) dψ . ( 1 ) A straightforward way to achieve this is to use the gradient ascent : θ ← θ + α∇θJ ( θ ) , ( 2 ) where α is the learning rate . In the OpenAI-ES , a score function estimator is used to calculate the gradient , which is similar to REINFORCE ( Williams , 1992 ) : ∇θJ ( θ ) = ∇θEψ∼pθ [ F ( ψ ) ] = Eψ∼pθ [ F ( ψ ) ∇θ log pθ ( ψ ) ] . ( 3 ) Usually , pθ is an isotropic multivariate Gaussian with mean θ and fixed covariance σ2I . And the expected value can be rewritten as Eψ∼pθ [ F ( ψ ) ] = E ∼N ( 0 , I ) [ F ( θ+σ ) ] . Thus the gradient estimator changes to : ∇θJ ( θ ) = ∇θE ∼N ( 0 , I ) [ F ( θ + σ ) ] = 1 σ E ∼N ( 0 , I ) [ F ( θ + σ ) ] , ( 4 ) where is sampled from a standard normal distribution . Once we form an objective function J ( θ ) , the gradient ∇θJ ( θ ) can be approximated via Equation 4 and parameters θ can be updated via Equation 2 .

Summary: In this paper the authors attempt to solve the circuit routing problem through a novel approach of combining search-based routing techniques (e.g. A* search) and evolutionary strategies (e.g. OpenAI-ES). The authors define new parameters e.g. cost maps and a ranking parameter which improve the efficiency and effectiveness of their circuit routing solution. As opposed to heuristics based (2-step) routing solutions, the authors develop a solution which optimizes a single global cost function. This makes the solution easily scalable to new constraints and designs while avoiding sub-optimal solutions (common problem for 2-stage routing). The authors show that their approach doesn’t require human demonstrations or training data of any kind thereby giving their solution an edge over learning-based techniques requiring training data. Lastly, through experiments the authors establish that their approach is scalable to larger applications.