SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $b_{\zeta}(V_i)$  is a predicted length of RSMT by the critic network and  $b(V_i, r_i)$  is the actual length evaluated in linear time.  $p_{\xi}(r_i)$  is the learnable probability of generating a specific RES with regard to  $\xi$ . This objective is learned with an RL-algorithm [52]. Apart from the actor network, the critic one is learned simply by mean square error (MSE) with gradient descent training:
$$
\min  _ {\zeta} \mathbb {E} _ {V _ {i} \sim \bar {V}, r _ {i} \sim R (V _ {i})} \| b _ {\zeta} (V _ {i}) - b (V _ {i}, r _ {i}) \| ^ {2}. \tag {3}
$$
At test time, in line with [33], we apply 8 transformations that rotate the point set by [0, 90, 180, 270] degrees with/without swapping  $x$  and  $y$  coordinates and then choose the best solution. This trick is relatively more effective when hubs are not well generated.
Table 2: Experiments on ISPD-07 with 4 kinds of cases (detailedly introduced in Appendix D). We compare HubRouter with PRNet [6] on three metrics. Optimal results are in **bold**.
Metric	Case	PRNet (GAN)	HubRouter (VAE)	HubRouter (DPM)	HubRouter (GAN)
Correctness Rate (%)	Route-small-4	0.806	1.000±0.000	1.000±0.000	1.000±0.000
Route-small	0.334	1.000±0.000	1.000±0.000	1.000±0.000
Route-large-4	0.196	1.000±0.000	1.000±0.000	1.000±0.000
Route-large	0.040	1.000±0.000	1.000±0.000	1.000±0.000
Wirelength Rate (%)	Route-small-4	1.001	1.099±0.020	1.060±0.011	1.011±0.003
Route-small	1.009	1.042±0.006	1.174±0.009	1.002±0.001
Route-large-4*	-	1.122±0.039	1.100±0.021	1.005±0.002
Route-large*	-	1.041±0.014	1.242±0.021	1.001±0.000
Generation Time (GPU Sec)	Route-small-4	14.99	7.14±0.19	673.21±5.08	7.16±0.05
Route-small	18.51	7.70±0.20	670.23±2.45	8.36±0.35
Route-large-4	19.47	7.24±0.30	673.01±5.18	7.53±0.16
Route-large	19.22	10.65±0.09	672.86±4.44	9.75±0.35
* The correctness rate is too low for PRNet to reasonably evaluate the wirelength rate.
4 Experiment and Analysis
We introduce datasets and setups in Sec. 4.1. To show the performance of HubRouter, in Sec. 4.3, we respectively perform global routing on three kinds of datasets with both classical and learning baselines. Ablation study is performed to evaluate key modules in HubRouter. In Sec. 4.4, we introduce two applications, including RSMT construction and interactive path replanning. Finally, we analyze the time overhead and show the scalability of HubRouter. Each experiment in this section is run on a machine with i9-10920X CPU, NVIDIA RTX 3090 GPU and 128 GB RAM, and is repeated 3 times under different seeds with mean and standard deviation values in line with [6].
4.1 Datasets and Setups
Datasets. For training, we construct global routing instances by adopting NCTU-GR [34] to route on ISPD-07 routing benchmarks [38], which is in line with [6]. For test, we split the samples outside the training set into 4 types of routes according to the number of pins and the distance of pins. In detail, 'Route-small-4' and 'Route-small' respectively represent cases with no more than and more than 4 pins and the Half-perimeter wirelength (HPWL) of pins is less than 16. Here, HPWL =  $\left[\max(x_i) - \min(x_i)\right] + \left[\max(y_i) - \min(y_i)\right]$  for a group of pin positions  $\{(x_i, y_i)\}_{i=1}^{n_p}$  in a net with  $n_p$  pins. 'Route-large-4' and 'Route-large' are similar, whose HPWL is more than 16. Moreover, we introduce ISPD-98 [1] routing benchmarks and some simulated small-scale cases used by a deep reinforcement learning (DRL) method [31], named DRL-8 and DRL-16.
Metrics. On ISPD-07 routing benchmarks, we choose correctness (all pins are connected within one route) rate and wirelength ratio (WLR, the ratio of the generated route length to the ground truth route length) introduced in [6] as our metrics. In the experiments of ISPD-98 and DRL cases, wirelength (WL), overflow (OF), and routing runtime are adopted. We also investigate some common metrics for generative models like Fréchet Inception Distance (FID) [15] and Inception Score (IS) [2, 42], but they are not suitable for taking on the real performance on global routing.
Other Implementation Details. Details of training/test datasets and other protocols, including introduction of baselines and model structures, are presented in Appendix D.
4.2 Unconnected Cases in Previous Generative Global Routing Approaches
We investigate the state-of-the-art generative global routing approach [6] and show that its correctness rate (CrrtR) is extremely low for cases with complex connection of pins. As declared in Table 2, the CrrtR is only 0.04 for the Route-large case. This is reasonable because connectivity is assured only when almost each pixel of routed path is correctly generated in the routing image, which is a very strict condition for complex cases. However, under our proposed two-phase learning scheme, HubRouter guarantees the CrrtR on all nets with the intermediately generated hubs. Moreover, despite
Table 3: Experiments on ISPD-98 (IBM01-06) and GRL cases. Wirelength (WL) and running time are compared among 4 baselines and HubRouter (brown) with generative structures (HR-VAE, HR-DPM, HR-GAN). Optimal results of WL and time are in bold. The results of DQN [31] on ISPD-98 are out of time (OOT) within 2 weeks, so only GRL cases are displayed.
Metric	Model	IBM01	IBM02	IBM03	IBM04	IBM05	IBM06	GRL-8	GRL-16
WL	Labyrinth [24]	75909	201286	187345	195856	420581	341618	2376	8204
Boxrouter [8]	63687	172304	147463	169033	410614	280477	2328	7991
DQN [31]	OOT	OOT	OOT	OOT	OOT	OOT	2434	8356
PRNet [6]	61950	172802	152037	170493	420274	287777	2497	8172
HR-VAE	64703±1498	176492±6830	159968±3281	179895±5274	434942±2916	300448±5560	2415±33	8584±244
HR-DPM	66446±1586	190588±2337	168454±2486	183696±1736	475820±5516	316700±2843	2285±7	7746±29
HR-GAN	61056±151	167545±236	147050±208	164298±326	411857±472	278198±423	2306±8	7768±30
Time (Sec)	Labyrinth [24]	6.47	10.14	12.07	36.81	7.54	18.43	<1 second	<1 second
Boxrouter [8]	7.01	12.91	11.74	41.76	13.45	29.17	<1 second	<1 second
DQN [31]	OOT	OOT	OOT	OOT	OOT	OOT	>1 day	>1 day
PRNet [6]	254.31	585.23	523.34	573.19	1606.00	1227.31	11.54	28.25
HR-VAE	9.66±0.08	9.69±0.04	10.19±0.06	12.93±0.07	14.58±0.00	17.28±0.16	5.83±0.01	5.88±0.02
HR-DPM	1796.09±38.68	2772.29±16.83	2936.52±21.23	3865.21±25.07	4369.47±22.56	4965.08±121.46	37.00±2.58	53.90±2.85
HR-GAN	41.02±0.51	46.58±0.56	52.04±2.35	67.31±3.51	72.28±3.72	88.02±4.45	6.31±0.20	6.38±0.04
Figure 5: Initialized overflow on ISPD-98 (IBM01-06) and the DRL cases.
that PRNet only counts the wirelength of the connected routes, HubRouter (GAN) is competitive with it in WLR. Note that we only evaluate the inference time of generation in Table 2 (total time is analyzed in the next part), where PRNet is efficient with its one-shot superiority, though HubRouter (VAE/GAN) are still slightly better than it. An exception comes that HubRouter (DPM) suffers from time-consuming sampling for its multiple sampling steps.
4.3 Benchmarking Global Routing and Ablation Study
We test the WL, OF, and inference time on IBM01-06 from ISPD-98 as well as DRL cases. The detailed inference time that is divided into generation time and connection time is shown in Appendix E.4. As shown in Table 3, the WL of HubRouter (GAN) outperforms others on ISPD-98 cases while HubRouter (DPM) hits the optimum on DRL cases. This implies that HubRouter (DPM) is more capable on small-scale cases as the model complexity is not very high that easily incurs overfitting; however, due to its large inference time, we do not decide to increase its model complexity, but we believe it has high potential with more support of computing power. In addition, though HubRouter (VAE) fails to reach the top in WL, it is the fastest among all ML-based algorithms and is even competitive with classical algorithms in some cases. Since there exists the theoretical lower bound of this task [8], we also compare the relative error on ISPD-98 cases in Appendix E.3 to better demonstrate the performance of HubRouter. As these generative models empirically show different behaviors, we argue that when given a new dataset, the choice of generative algorithms is decided by the industrial needs. Specifically, if one needs high quality, GAN is the best. If speed is highly required, VAE is the best. If one has abundant resources, DPM can be adopted. Note that PRNet is significantly more time-consuming in this experiment compared with the results in Table 2 due to its post-processing. In line with PRNet [6], as the overflow is decreased by rerouting, we compare the initialized overflow among ML-based algorithms. As shown in Fig. 5, HubRouter-GAN has a minimal overflow in all test cases.
Ablation study is conducted in both generation and connection phases to show the influence of the introduced modules. As depicted in Table 4, in the hub-generation phase, we respectively denote the HubRouter with inputs of hubs, hubs + routers, and hubs + routers + stripe masks as HubRouter-h,
Table 4: Ablation Study on ISPD-07 with HubRouter (using GAN as generative embodiment).
Generation-phase	Model	HubRouter-h	HubRouter-hr	HubRouter-hrm
WLR	1.031	1.022	1.005
Connection-phase	Model	HubRouter-RMST	HubRouter (T=1)	HubRouter (T=8)
WLR	1.099	1.087	1.005
Figure 6: Samples of interactive path replanning. The first line is routes with some added points while the second line is the replanned paths.
HubRouter-hr, and HubRouter-hrm. Among all, HubRouter-hrm reaches the optimal wirelength rate and HubRouter-hr also outperforms HubRouter-h, which indicates that the multi-task learning with route and stripe mask is effective. In the pin-hub-connection phase, HubRouter-RMST is denoted as the one using R-MST as the connection model. In addition, HubRouter  $(\mathrm{T} = 1) / (\mathrm{T} = 8)$  respectively represent the HubRouter with/without 8 transformations introduced in Sec. 3.2. As expected, HubRouter  $(\mathrm{T} = 8)$  performs the best. Details on hyperparameter are in Appendix E.
4.4 Further Studies and Discussion
We envision that HubRouter can do more than just global routing. Here, we respectively perform RSMT construction and interactive path replanning to reflect the generality of HubRouter. The HubRouter in this section uses GAN as the generative embodiment due to its better performance.
RSMT Construction. It is intuitive that global routing for each net without overflow constraint is to minimize the wirelength, and thus is close to the problem of RSMT construction. A key question is whether the generated hubs can enhance the construction of the pure RSMT during the connection phase. To evaluate it, we test the average percent errors compared with GeoSteiner [51] on the random point data given by REST [33], together with running time, in Table 5. We compare baselines including R-MST<sup>1</sup>, BGA [23], FLUTE [53] and REST [33]. When degree is smaller than 25 REST performs the best, but HubRouter outperforms others when degree is larger than 30. In particular, an unintuitive phenomenon shows that HubRouter performs even better with a higher degree. This is due to the characteristic of hub generation since a noise point in a dense case has much less impact on the error of wirelength than in a sparse case. Apart from the error, REST has the shortest running time depending on large batch size. However, HubRouter has to run sequentially in this task because degrees are different after generating some new hubs.
Interactive Path Replanning. Path replanning [19] refers to the process of revising a given path when the environment changes. It is a technique used in human-machine interaction. HubRouter is capable of deciding a new path with user-defined hubs. Fig. 6 shows samples of interactive path replanning results, where HubRouter connects new points and maintains most of the previous paths.
Analysis of Scalability and Time Overhead. We claim that HubRouter is scalable in both phases. In the generation phase, for a fixed scale of routing graph, similar to PRNet [6], the hub generation is one-shot. This implies that its generation speed is unrelated to the number of pins and is linear to the number of nets, as respectively shown in Fig. 7 (left/middle). For connection, we show in Sec. 3.2 that the RSMT construction can degrade to a  $O(n \log n)$  R-MST problem under some ideal assumptions. In addition, as depicted in Fig. 7 (right), equipped with an actor-critic network, the time overhead of HubRouter increases more slowly than pure RSMT construction.
Non End-to-End framework. HubRouter is a two-phase framework that is designed with careful analysis. It differs from PRNet [6], which is an end-to-end attempt to produce the routes directly, but
<https://github.com/shininglion/rectilinear_spanning_graph
Figure 7: Generation time of HubRouter is constant to number of pins (left) and is linear to number of nets (middle). The RSMT construction time of HubRouter rises slower than GeoSteiner [51] (right).
Table 5: Experiments on RSMT construction. HubRouter and four baselines are compared to the RSMT solver GeoSteiner (GeoSt) and the best results on errors are in bold. Time overhead is also evaluated on GeoSt, REST, and HubRouter. Mean and variance are given with three trials.
Degree	Average Percent Errors (%)	Time (Sec)
GeoSt	R-MST	BGA	FLUTE	REST	HubRouter	GeoSt	REST	HubRouter
5	0.00	10.91	0.23	0.00	0.00	0.24±0.07	0.31±0.02	11.89±0.24	13.44±2.41
10	0.00	11.96	0.48	0.04	0.01	0.18±0.02	4.05±0.03	12.16±0.10	16.43±0.30
15	0.00	12.19	0.53	0.06	0.03	0.15±0.01	9.26±0.08	13.80±0.43	16.92±2.43
20	0.00	12.41	0.57	0.11	0.07	0.13±0.01	17.35±0.19	14.79±0.17	19.85±1.32
25	0.00	12.47	0.58	0.18	0.12	0.12±0.01	29.01±0.07	16.68±0.54	23.01±1.99
30	0.00	12.56	0.60	0.23	0.16	0.10±0.01	43.00±0.17	18.53±0.29	27.02±2.30
35	0.00	12.63	0.62	0.26	0.21	0.09±0.01	60.32±0.20	19.67±0.85	33.11±2.12
40	0.00	12.65	0.63	0.29	0.25	0.08±0.01	78.80±0.06	20.91±0.51	39.42±3.26
45	0.00	12.67	0.63	0.30	0.32	0.07±0.01	102.26±0.21	22.36±0.42	49.57±3.05
50	0.00	12.72	0.64	0.29	0.36	0.06±0.01	125.26±0.41	23.79±0.20	61.55±3.07
faces the challenge of non-connectivity and inefficient post-processing. HubRouter addresses this by generating hubs in the middle stage and connecting them with pins to ensure connectivity. However, this two-stage framework involves a discretization process, which prevents differentiable learning and simultaneous optimization. We also pursuit the cost-effective end-to-end solver for further work.
5 Conclusion and Outlook
We have proposed HubRouter, a generative global router in VLSI with a two-phase learning scheme: the hub-generation phase and the pin-hub-connection phase. In the generation phase, HubRouter generates hubs instead of routes while in the connection phase, it connects the hubs with RSMT construction. Experiments on real-world and simulated datasets show its effectiveness as well as scalability. When adopting HubRouter, the potential negative impacts should be taken care: 1) The incorrect generation might lead to poor routing results; 2) Generative models are computationally intensive and might consume a lot of resources.
This paper also has some limitations for future work: 1) The two-phase scheme is not end-to-end under a joint training scheme; 2) The supervised module for hub generation depends on large amount of training data; 3) Like SOTA generative global routing algorithms [6], the further decrease on overflow needs a reroute process; 4) Finally, it would also be compelling to integrate the learning-based methods of logic synthesis [55], with routing methodologies, including those presented in this work. This would allow for a data-driven approach to the chip design pipeline.
Global Routing. Traditional global routing algorithms typically start with decomposing multi-pin nets into two-pin nets by minimum spanning tree (MST) or rectilinear Steiner tree (RST) [9, 18]. Then, two-pin nets are routed by heuristic-based methods such as pattern routing [24] and maze routing. Finally, a rip-up and reroute [43] process is adopted to refine the final result. Negotiation-based rip-up and reroute algorithms [36, 8] reveal their superiority by considering rerouting history. In addition to classical solutions, recent methods start to resort to learning techniques. RL-based routers with a DQN agent [31] sequentially decide every route direction in a net and aim to generate a better result than  $\mathbf{A}^*$  router, but they are time-consuming and currently often only applicable to small grids, rather than realistic settings. Generative models are recently introduced to generate routes in a net without decomposing it into two-pin nets. [48] uses generative models to speed up global routing at the cost of reduced routability. In PRNet [6], global routing is solved by conditional generative models (specifically CGAN [37]), but it still heavily depends on unscalable post-processing due to some unconnected routes. This paper achieves connectivity by decomposing global routing into a two-phase learning scheme.
Generative Model. GAN [11], VAE [26], and DPM [16] are three widely-used deep generative models applied in the generation of different forms of data. Moreover, some works [37, 44, 17] incorporate conditions to make the generation under more fine-grained control, which is important in applications like layout design [5, 56], combinatorial optimization [29], and chip design [6]. Generation tasks also attract the research on the distributions of real-world data [30], which show high potentials. In this paper, we present a novel and more scalable hub generation scheme to address global routing problems.
RSMT Construction. Rectilinear Steiner minimum tree (RSMT) gives the shortest possible solution to interconnect pins in a net. This problem belongs to the class of combinatorial problems [12, 57] that are NP-Complete [10] and challenging for machine learning researchers. Since it is computationally intractable to find an optimal solution in general, rectilinear minimum spanning tree (R-MST) [21] is widely used to approximate RSMT with time complexity of  $O(n \log n)$  [21], but its length is at most  $1.5 \times$  of the optimal RSMT [20]. FLUTE [53] designs a look-up table for small nets and decomposes larger nets into smaller nets that can be solved by the table. REST [33] introduces rectilinear edge sequence (RES) to encode RSMT and train an actor-critic neural network by reinforcement learning. Compared to existing RSMT solvers, this paper performs RSMT construction after a hub generation phase, and as such, global routing is completed by connecting the pins and generated hubs.
B.1 Deep Generative Approaches
We incorporate three prevailing generative models into our generation framework. These approaches give us different perspectives of hub generation:
GAN [11]. Conditional GAN (CGAN) [37] applies a discriminator  $D(\cdot)$  to classify whether the input is real or fake (generated), which minimizes the logistic loss function  $-\log D_{\phi}(p(\mathbf{x}|\mathbf{c})) - \log(1 - D_{\phi}(p_{\theta}(\mathbf{x}|\mathbf{c},\mathbf{z})))$  with regard to  $\phi$ . It equals maximizing its negative value. Meanwhile, the generated distribution  $p_{\theta}(\mathbf{x}|\mathbf{c},\mathbf{z})$  is called a generator in GAN, which approximates to  $p(\mathbf{x}|\mathbf{c})$  by deceiving the discriminator with regard to  $\theta$ . Together optimize the above objectives, we have a min-max strategy:
$$
\min  _ {\theta} \max  _ {\phi} \sum_ {i = 1} ^ {N} \left[ \log D _ {\phi} (\mathbf {x} _ {i}) + \log \left(1 - D _ {\phi} \left(p _ {\theta} \left(\mathbf {x} \mid \mathbf {c} = \mathbf {c} _ {i}, \mathbf {z} \sim p _ {\mathbf {z}} (\mathbf {z})\right)\right)\right) \right]. \tag {4}
$$
We use neural networks to learn the discriminator  $D_{\phi}(\cdot)$  and the generator  $p_{\theta}(\mathbf{x}|\mathbf{c},\mathbf{z})$
VAE [26]. Conditional VAE (CVAE) [44] minimizes Kullback-Leibler (KL) divergence to approximate a simulated inference network  $q_{\phi}(\mathbf{z}|\mathbf{x},\mathbf{c})$  to the intractable posterior distribution  $p_{\theta}(\mathbf{z}|\mathbf{x},\mathbf{c})$ , such that we can apply the stochastic gradient variational Bayes (SGVB) framework to optimize the variational lower bound
$$
\log p _ {\theta} (\mathbf {x} | \mathbf {c}) \geq \underbrace {- \operatorname {K L} \left[ q _ {\phi} (\mathbf {z} | \mathbf {x} , \mathbf {c}) \| p _ {\theta} (\mathbf {z} | \mathbf {x} , \mathbf {c}) \right]} _ {①} + \underbrace {\mathbb {E} _ {q _ {\phi} (\mathbf {z} | \mathbf {x} , \mathbf {c})} \log p _ {\theta} (\mathbf {x} | \mathbf {c} , \mathbf {z})} _ {②}. \tag {5}
$$
Figure 8: The simplified 2-pin problem, namely, routing the source and target node (pin) under constraints while searching for the shortest wirelength for a single net. The constraints represent the obstacles derived from capacity.
Note that the first term in Eq. 5 is easy to integrate since the KL divergence for Gaussian distribution has closed form, and the second term is often simplified by Monte-Carlo sampling  $\mathbb{E}_{q_{\phi}(\mathbf{z}|\mathbf{x},\mathbf{c})}\log p_{\theta}(\mathbf{x}|\mathbf{c},\mathbf{z})\approx \frac{1}{L}\sum_{l = 1}^{L}\log p_{\theta}(\mathbf{x}|\mathbf{c},\mathbf{z}^{(l)})$ . So, the objective of minimizing the empirical lower bound can be written as:
$$
\min  _ {\theta , \phi} \left[ - \operatorname {K L} \left(q _ {\phi} (\mathbf {z} | \mathbf {x}, \mathbf {c}) \| p _ {\theta} (\mathbf {z} | \mathbf {c})\right) + \frac {1}{L} \sum_ {l = 1} ^ {L} \log p _ {\theta} \left(\mathbf {x} | \mathbf {c}, \mathbf {z} ^ {(l)}\right) \right], \tag {6}
$$
where  $\mathbf{z}^{(l)} = g_{\phi}(\mathbf{x},\mathbf{c},\epsilon^{(l)})$ $\epsilon^{(l)}\sim \mathcal{N}(\mathbf{0},\mathbf{I})$  is a reparameterization trick [26] and  $L$  is the number of samples. Neural networks are used to learn the encoder  $q_{\phi}(\mathbf{z}|\mathbf{x},\mathbf{c})$ , the decoder  $p_{\theta}(\mathbf{x}|\mathbf{c},\mathbf{z})$ , and the conditional prior network  $p_{\theta}(\mathbf{z}|\mathbf{c})$ .
DPM [16]. DPM injects noise into the original data step by step until it turns out to be a known distribution and reverses these steps when sampling. DPM respectively defines these processes as the forward process  $q_{\theta}(\mathbf{x}_t|\mathbf{x}_{t - 1})$  and the reverse process  $p_{\theta}(\mathbf{x}_{t - 1}|\mathbf{x}_t)$  with  $t\in \{1,\dots ,T\}$ :
$$
q \left(\mathbf {x} _ {1: T} \mid \mathbf {x} _ {0}\right) := \prod_ {t = 1} ^ {T} q \left(\mathbf {x} _ {t} \mid \mathbf {x} _ {t - 1}\right), \quad q \left(\mathbf {x} _ {t} \mid \mathbf {x} _ {t - 1}\right) = \mathcal {N} \left(\mathbf {x} _ {t}; \sqrt {1 - \beta_ {t}} \mathbf {x} _ {t - 1}, \beta_ {t} \mathbf {I}\right), \tag {7}
$$
$$
p _ {\theta} (\mathbf {x} _ {0: T}) := p (\mathbf {x} _ {T}) \prod_ {t = 1} ^ {T} p _ {\theta} (\mathbf {x} _ {t - 1} | \mathbf {x} _ {t}), \qquad p _ {\theta} (\mathbf {x} _ {t - 1} | \mathbf {x} _ {t}) = \mathcal {N} (\mathbf {x} _ {t - 1}; \mu_ {\theta} (\mathbf {x} _ {t}, t), \sigma_ {t} ^ {2}),
$$
where  $\mathbf{I}$  is an identity matrix,  $\{\beta_t\}_{t=1}^T$  are fixed variance schedules with ascending values, and  $\alpha_t = 1 - \beta_t$ .  $\{\sigma_t^2\}_{i=1}^T$  are also fixed values. Similar to VAE, with training the variational lower bound of the marginal log likelihood and reparameterization, we finally obtain the condition-guided DPM objective:
$$
\min  _ {\theta} \left[ \mathbb {E} _ {\mathbf {x}, \mathbf {c}, \epsilon \sim \mathcal {N} (\mathbf {0}, \mathbf {I}), t \sim \mathcal {U} (\{t \} _ {t = 1} ^ {T})} \left[ \| \epsilon - \epsilon_ {\theta} (\mathbf {z} _ {t}, \mathbf {c}, t) \| ^ {2} \right] \right]. \tag {8}
$$
Following classifier-free guidance [17], we initialize  $\mathbf{z}_T$  and denoise it by  $\hat{\epsilon}_t = (1 + w)\cdot \epsilon_\theta (\mathbf{z}_t,\mathbf{c},t) - w\cdot \epsilon_\theta (\mathbf{z}_t,t)$ , and accelerate the sampling process with fewer sampling steps by DDIM [45].
B.2 Training and Sampling
Adam [25] is used to train in both generation and connection phases. Specifically, Alg. 1 and Alg. 2 illustrate the training processes of these two phases, while Alg. 3 shows the sampling process.