SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\bar {\sigma} _ {\phi} ^ {2} \left(\mathbf {x} _ {0}, t, i\right) := \alpha_ {t} \bar {\sigma} _ {\phi} ^ {2} \left(\mathbf {x} _ {0}, t - 1, i\right) + (1 - \alpha_ {t}) \sigma_ {\phi} ^ {2} \left(\mathbf {x} _ {0}, t, i\right), \tag {17}
$$
which correspond to Eq.8 and Eq.9 of the main paper, and we further obtain
$$
q \left(\mathbf {x} _ {t} ^ {i} \mid \mathbf {x} _ {0} ^ {i}\right) = \mathcal {N} \left(\mathbf {x} _ {t} ^ {i}; \sqrt {\bar {\alpha} _ {t}} \mathbf {x} _ {0} ^ {i} + \bar {\mu} _ {\phi} \left(\mathbf {x} _ {0}, t, i\right), \bar {\sigma} _ {\phi} ^ {2} \left(\mathbf {x} _ {0}, t, i\right) \mathbf {I}\right), \tag {18}
$$
which is the Eq.7 of the main paper.
A.2 Reverse process
Assuming we are given the guidance networks  $\pmb{\mu}_{\phi}$  and  $\sigma_{\phi}$  and the noise scales  $\sigma_{t}$ s of the reverse process, we follow a style similar to DDPM [14] to derive a sampling step in the reverse process. At each sampling step in the reverse process,  $p_{\theta}(\mathbf{x}_{t - 1}^{i}|\mathbf{x}_{t}^{i})$ , the Gaussian distribution to sample  $\mathbf{x}_{t - 1}^{i}$  from is supposed to have the same mean as  $q(\mathbf{x}_{t - 1}^{i}|\mathbf{x}_{t}^{i},\mathbf{x}_{0})$  defined by the forward process, which is also Gaussian, to minimize their KL divergence. As the joint distribution of  $\mathbf{x}_{t - 1}^{i}$  and  $\mathbf{x}_t^i$  conditioned on
$\mathbf{x}_0$  is Gaussian, the mean of  $q(\mathbf{x}_{t - 1}^i |\mathbf{x}_t^i,\mathbf{x}_0)$  can be easily derived with the following closed form [30, Chapter 8.1.3]:
$$
\sqrt {\bar {\alpha} _ {t - 1}} \mathbf {x} _ {0} ^ {i} + \bar {\mu} _ {\phi} \left(\mathbf {x} _ {0} ^ {i}, t - 1, i\right) + \frac {\sqrt {\alpha_ {t}} \bar {\sigma} _ {\phi} ^ {2} \left(\mathbf {x} _ {0} , t - 1 , i\right)}{\bar {\sigma} _ {\phi} ^ {2} \left(\mathbf {x} _ {0} , t , i\right)} \left(\mathbf {x} _ {t} ^ {i} - \sqrt {\bar {\alpha} _ {t}} \mathbf {x} _ {0} ^ {i} - \bar {\mu} _ {\phi} \left(\mathbf {x} _ {0}, t, i\right)\right). \tag {19}
$$
Since  $\mathbf{x}_t^i = \sqrt{\bar{\alpha}_t}\mathbf{x}_0^i +\bar{\mu}_{\phi}(\mathbf{x}_0,t,i) + \bar{\sigma}_{\phi}(\mathbf{x}_0,t,i)\epsilon^i$  for  $\epsilon^i\sim \mathcal{N}(\mathbf{0},\mathbf{I})$ , Equation 19 can be rewritten as
$$
\begin{array}{l} \sqrt {\bar {\alpha} _ {t - 1}} \mathbf {x} _ {0} ^ {i} + \bar {\boldsymbol {\mu}} _ {\phi} (\mathbf {x} _ {0} ^ {i}, t - 1, i) + \bar {\boldsymbol {\sigma}} _ {\phi} (\mathbf {x} _ {0}, t, i) \frac {\sqrt {\alpha_ {t}} \bar {\boldsymbol {\sigma}} _ {\phi} ^ {2} (\mathbf {x} _ {0} , t - 1 , i)}{\bar {\boldsymbol {\sigma}} _ {\phi} ^ {2} (\mathbf {x} _ {0} , t , i)} \boldsymbol {\epsilon} ^ {i}. \\ = \frac {1}{\sqrt {\alpha_ {t}}} \left(\sqrt {\bar {\alpha} _ {t}} \mathbf {x} _ {0} ^ {i} + \bar {\sigma} _ {\phi} (\mathbf {x} _ {0}, t, i) \frac {\alpha_ {t} \bar {\sigma} _ {\phi} ^ {2} (\mathbf {x} _ {0} , t - 1 , i)}{\bar {\sigma} _ {\phi} ^ {2} (\mathbf {x} _ {0} , t , i)} \epsilon^ {i}\right) + \bar {\mu} _ {\phi} (\mathbf {x} _ {0} ^ {i}, t - 1, i) \\ = \frac {1}{\sqrt {\alpha_ {t}}} \left(\sqrt {\bar {\alpha} _ {t}} \mathbf {x} _ {0} ^ {i} + \bar {\sigma} _ {\phi} \left(\mathbf {x} _ {0}, t, i\right) \frac {\bar {\sigma} _ {\phi} ^ {2} \left(\mathbf {x} _ {0} , t , i\right) - \left(1 - \alpha_ {t}\right) \sigma_ {\phi} ^ {2} \left(\mathbf {x} _ {0} , t , i\right)}{\bar {\sigma} _ {\phi} ^ {2} \left(\mathbf {x} _ {0} , t , i\right)} \boldsymbol {\epsilon} ^ {i}\right) + \bar {\mu} _ {\phi} \left(\mathbf {x} _ {0} ^ {i}, t - 1, i\right) \tag {20} \\ = \frac {1}{\sqrt {\alpha_ {t}}} (\sqrt {\bar {\alpha} _ {t}} \mathbf {x} _ {0} ^ {i} + \bar {\sigma} _ {\phi} (\mathbf {x} _ {0}, t, i) \pmb {\epsilon} ^ {i} + \bar {\mu} _ {\phi} (\mathbf {x} _ {0} ^ {i}, t, i) - \bar {\mu} _ {\phi} (\mathbf {x} _ {0} ^ {i}, t, i) - \frac {(1 - \alpha_ {t}) \pmb {\sigma} _ {\phi} ^ {2} (\mathbf {x} _ {0} , t , i)}{\bar {\sigma} _ {\phi} (\mathbf {x} _ {0} , t , i)} \pmb {\epsilon} ^ {i}) \\ + \bar {\boldsymbol {\mu}} _ {\phi} (\mathbf {x} _ {0} ^ {i}, t - 1, i) \\ = \frac {1}{\sqrt {\alpha_ {t}}} (\mathbf {x} _ {t} ^ {i} - \bar {\boldsymbol {\mu}} _ {\phi} (\mathbf {x} _ {0} ^ {i}, t, i) - \frac {(1 - \alpha_ {t}) \pmb {\sigma} _ {\phi} ^ {2} (\mathbf {x} _ {0} , t , i)}{\bar {\pmb {\sigma}} _ {\phi} (\mathbf {x} _ {0} , t , i)} \pmb {\epsilon} ^ {i}) + \bar {\boldsymbol {\mu}} _ {\phi} (\mathbf {x} _ {0} ^ {i}, t - 1, i). \\ \end{array}
$$
As our formulation directly parameterizes  $\bar{\sigma}_{\phi}$ , we further simplify Eq. 20 by defining  $\bar{\sigma}_{\phi}(\mathbf{x}_0, t, i) \coloneqq \sqrt{1 - \bar{\alpha}_t} C(\mathbf{x}_0, i)$  where  $C(\mathbf{x}_0, i)$  is independent of the timestep  $t$ , thus implicitly defining  $\pmb{\sigma}_{\phi}(\mathbf{x}_0, t, i) = \bar{\pmb{\sigma}}_{\phi}(\mathbf{x}_0, T, i) = C(\mathbf{x}_0, i)$ . Such a parameterization makes  $\{\bar{\sigma}_{\phi}(\mathbf{x}_0, t, i)\}_{t}$  simply an interpolation between 0 and  $\bar{\sigma}_{\phi}(\mathbf{x}_0, T, i)$  and further simplifies Eq. 20 as
$$
\frac {1}{\sqrt {\alpha_ {t}}} \left(\mathbf {x} _ {t} ^ {i} - \bar {\boldsymbol {\mu}} _ {\phi} \left(\mathbf {x} _ {0} ^ {i}, t, i\right) - \bar {\sigma} _ {\phi} \left(\mathbf {x} _ {0}, t, i\right) \frac {1 - \alpha_ {t}}{1 - \bar {\alpha} _ {t}} \boldsymbol {\epsilon} ^ {i}\right) + \bar {\boldsymbol {\mu}} _ {\phi} \left(\mathbf {x} _ {0} ^ {i}, t - 1, i\right). \tag {21}
$$
During inference, we replace  $\bar{\mu}_{\phi}(\mathbf{x}_0,t,i)$  and  $\bar{\sigma}_{\phi}(\mathbf{x}_0,t,i)$  with  $\bar{\mu}_{\phi}(\hat{\mathbf{x}}_0,t,i)$  and  $\bar{\sigma}_{\phi}(\hat{\mathbf{x}}_0,t,i)$  respectively and replace  $\epsilon^i$  with the denoising network  $\epsilon_{\theta}^{i}$  to define the mean of sampling distribution  $p_{\theta}(\mathbf{x}_{t - 1}^{i}|\mathbf{x}_{t}^{i})$  and derive a sampling step in the reverse process as
$$
\mathbf {x} _ {t - 1} ^ {i} = \frac {1}{\sqrt {\alpha_ {t}}} \left[ \mathbf {x} _ {t} ^ {i} - \bar {\boldsymbol {\mu}} _ {\phi} (\hat {\mathbf {x}} _ {0}, t, i) - \bar {\boldsymbol {\sigma}} _ {\phi} (\hat {\mathbf {x}} _ {0}, t, i) \frac {1 - \alpha_ {t}}{1 - \bar {\alpha} _ {t}} \boldsymbol {\epsilon} _ {\theta} ^ {i} (\mathbf {x} _ {t}, t, \mathbf {y}) \right] + \bar {\boldsymbol {\mu}} _ {\phi} (\hat {\mathbf {x}} _ {0}, t - 1, i) + \boldsymbol {\sigma} _ {t} \mathbf {z} ^ {i}, \tag {22}
$$
which is the Eq.10 of the main paper.
B Additional implementation details
In this section, we complement §4 of the main paper by providing the complete implementation details of PolyDiffuse for the two different tasks.
B.1 Guidance networks
The implementation details of the guidance networks are shared across the two tasks.
Model architectures: In §4 of the main paper, we parameterize  $\bar{\mu}_{\phi}(\mathbf{x}_0,T,i)$  and  $\bar{\sigma}_{\phi}(\mathbf{x}_0,T,i)$  with two Transformers  $\mathrm{Trans}_{\bar{\mu}_{\phi}}(\mathbf{x}_0,i)$  and  $\mathrm{Trans}_{\bar{\sigma}_{\phi}}(\mathbf{x}_0,i)$ , and define  $\bar{\mu}_{\phi}(\mathbf{x}_0,t,i)$  and  $\bar{\sigma}_{\phi}(\mathbf{x}_0,t,i)$  with Eq.14. The two Transformers are implemented with a DETR-style [3] Transformer decoder: The Transformer decoder contains two shared attention-based decoder layers and two separate linear projection heads for  $\bar{\mu}_{\phi}$  and  $\bar{\sigma}_{\phi}$ , respectively. Each decoder layer consists of an intra-element self-attention layer and a global self-attention layer, whose outputs are fused by addition. Each vertex in the input  $\mathbf{x}_0$  becomes an input node of the Transformer decoder, and the node feature consists of three positional encodings [44]: 1) positional encoding of the X-axis coordinate, 2) positional
$^{2}$ https://github.com/facebookresearch/detr/blob/main/models/transformer.py
encoding of the Y-axis coordinate, 3) positional encoding of the vertex index inside the element. The sequence of each element starts with a special dummy node (similar to the SOS token in language modeling), whose final encodings are used to compute the  $\bar{\mu}_{\phi}$  and  $\bar{\sigma}_{\phi}$  of the element. The dimension of the positional encoding is 128, and the hidden dimension of the Transformer decoder is 256.
Training details: The guidance training is summarized by Algorithm 1 of the main paper. We train the Transformer decoder for 3125 iterations with batch size 32. Adam optimizer is employed with a learning rate of 2e-4 and a weight decay rate of 1e-4.
B.2 Denoising networks
Since the implementations of the denoising networks are based on the corresponding state-of-the-art task-specific models, we describe them separately.
Floorplan reconstruction: The denoising network for the floorplan reconstruction task refers to RoomFormer [47] and its official implementation<sup>3</sup>. RoomFormer is a DETR-based [3] model consisting of a ResNet50 [12] image backbone, a Transformer encoder to process and aggregate image information, and a Transformer decoder with two-level learnable embeddings/coordinates to predict a set of polygon vertices from the image information. The Transformer has 6 encoder layers and 6 decoder layers with an embedding dimension of 256, and deformable-attention [50] is employed for all cross-attention layers and the self-attention layers in the Transformer encoder.
We follow the same architecture as RoomFormer, and have discussed the key modifications to turn the model into a denoising function ( $\S 4$  of the main paper) and our improvements to lift up its overall performance ( $\S 5$  of the main paper). A minor modification omitted in the main paper is that we add an intra-element attention layer to each Transformer decoder layer to increase the model capacity, as we found the model converges obviously slower under the denoising formulation than the regression/detection formulation of RoomFormer. In each decoder layer, the outputs of the intra-element attention are added to the outputs of the original global attention. The intra-element attention increases the number of overall parameters by  $\sim 5\%$ , and the corresponding ablation study is in  $\S D.1$  of this appendix, showing minor but recognizable improvements.
For training the denoising network, we keep the same setups as RoomFormer except that we increase the number of training iterations (ablation study is in §5 of the main paper). Adam optimizer is employed with a base learning rate of 2e-4 for all parameters, and the learning rate decays by a factor of 0.1 for the last  $20\%$  iterations.
HD map reconstruction: The denoising network for the HD map reconstruction task refers to MapTR [23] and its official implementation<sup>4</sup>. The overall design of MapTR is very similar to RoomFormer, where the keys are the "hierarchical" or "two-level" query embeddings for DETR-style Transformer and a loss based on "hierarchical bipartite matching". We use the same model config file provided in MapTR's official codebase and convert the model into a denoising function as described in §4 of the main paper. Similar to what we did to RoomFormer above, we add an intra-element attention layer to each Transformer decoder layer to facilitate convergence.
We employ an Adam optimizer with a base learning rate of 6e-4 and a weight decay factor of 1e-4. A cosine learning rate scheduler is used. To further facilitate convergence under the denoising formulation, we load the ResNet image backbone and Transformer encoder from the pre-trained MapTR, and set the initial learning rate of the image backbone to be 0.1 of the base learning rate.
When using MapTR as the proposal generator to produce the initial reconstruction for PolyDiffuse, we only consider predicted instances with a confidence score higher than 0.5 as MapTR's positive predictions. PolyDiffuse takes and updates the positive predictions of MapTR and keeps the remaining low-confidence predictions unchanged. Only the positive predictions are visualized for the qualitative results of MapTR and PolyDiffuse.
<sup>3</sup>https://github.com/ywyue/RoomFormer
4https://github.com/hustvl/MapTR
B.3 Permutation loss
Directly computing the  $L_{\mathrm{perm}}(\phi)$  as defined in Eq.14 of the main paper requires finding  $\mathbf{x}_0^* = \arg \max_{\mathbf{x}_0' \in \mathcal{P}!(\mathbf{x}_0) \setminus \{\mathbf{x}_0\}} L_{\mathrm{Triplet}}(\mathbf{x}_t, \mathbf{x}_0, \mathbf{x}_0')$  and  $\mathbf{x}_t^* = \arg \max_{\mathbf{x}_t' \in \mathcal{P}!(\mathbf{x}_t) \setminus \{\mathbf{x}_t\}} L_{\mathrm{Triplet}}(\mathbf{x}_0, \mathbf{x}_t, \mathbf{x}_t')$ , where the size of  $\mathcal{P}!(\mathbf{x}_0)$  and  $\mathcal{P}!(\mathbf{x}_t)$  is  $N!$ . This enumeration over all  $N!$  permutations of  $\mathbf{x}_0$  and  $\mathbf{x}_t$  immediately becomes computationally prohibitive when  $N$  gets large.
To reduce the computational cost and make Eq.14 practically feasible, we propose to approximate  $L_{\mathrm{perm}}(\phi)$  with element-level triplet losses. Concretely, we first enumerate all pairs of elements  $\mathbf{x}_0^i$  and  $\mathbf{x}_t^j$  for  $i,j = 1,\dots ,N$  to compute a  $N\times N$  distance matrix  $D$ . The entry  $D(i,j)$  is the minimum distance between two elements  $\mathbf{x}_0^i$  and  $\mathbf{x}_t^j$ , considering all possible vertex-level permutations (i.e., two variants for a polyline, and  $2(N_{i} - 1)$  variants for a polygon with  $N_{i}$  vertices, similar to the "point-level matching" in MapTR [23]). We then replace the sample-level triplet loss in  $L_{\mathrm{perm}}(\phi)$  with  $N$  element-level triplet losses, and define the computationally feasible proxy loss  $\hat{L}_{\mathrm{perm}}(\phi)$  as:
$$
\hat {L} _ {\text {p e r m}} (\phi) = \sum_ {i = 1, \dots , N} \max  _ {1 \leq j \leq N, j \neq i} L _ {\text {T r i p l e t}} \left(\mathbf {x} _ {t} ^ {i}, \mathbf {x} _ {0} ^ {i}, \mathbf {x} _ {0} ^ {j}\right) + \max  _ {1 \leq j \leq N, j \neq i} L _ {\text {T r i p l e t}} \left(\mathbf {x} _ {0} ^ {j}, \mathbf {x} _ {t} ^ {i}, \mathbf {x} _ {t} ^ {j}\right), \tag {23}
$$
$$
L _ {\text {T r i p l e t}} \left(\mathbf {x} _ {t} ^ {i}, \mathbf {x} _ {0} ^ {i}, \mathbf {x} _ {0} ^ {j}\right) = \max  \left(0, \alpha + D (i, i) - D (j, i)\right), \tag {24}
$$
$$
L _ {\text {T r i p l e t}} \left(\mathbf {x} _ {0} ^ {i}, \mathbf {x} _ {t} ^ {i}, \mathbf {x} _ {t} ^ {j}\right) = \max  (0, \alpha + D (i, i) - D (i, j)). \tag {25}
$$
$\alpha$  is the soft margin hyperparameter of the hinge-style triplet loss [15, 34] and we set  $\alpha = 0.1$ . All coordinate values are re-scaled into  $[-1,1]$ . In this way, we reduce the computational cost from  $O(N!)$  to  $O(N^2 M)$ , where  $M = \max_{i=1}^{N} N_i$  is the maximum number of vertices of an element of  $\mathbf{x}_0$ . In practical implementation, the guidance training (Algorithm 1 of the main paper) uses  $\hat{L}_{\mathrm{perm}}(\phi)$  rather than  $L_{\mathrm{perm}}(\phi)$ .
B.4 Diffusion models framework
Karras et al.[19] (EDM) presents a general diffusion model framework, where DDPM [14] and SDE-based DM formulations from Song et al.[41] can all be viewed as specializations of the proposed framework. We borrow its official codebase<sup>5</sup> to implement our GS-DM as it provides a general and clean base implementation suitable for all DM-based formulations. We then describe how we adapt the GS-DM into the EDM-based framework and list the relevant hyperparameter settings. This subsection follows the notations in Karras et al., where  $\pmb{y}$  is the data sample,  $\pmb{n}$  is the sampled Gaussian noise,  $\pmb{x}$  is the noisy sample,  $\sigma$  is the noise level (equivalent to the timestep), while  $c_{\mathrm{skip}}(\sigma)$ ,  $c_{\mathrm{in}}(\sigma)$ ,  $c_{\mathrm{out}}(\sigma)$ , and  $c_{\mathrm{noise}}(\sigma)$  are the preconditioning factors [19, Section 5]. Similar to the notations of our main paper, we let  $\pmb{y}^i$  and  $\pmb{x}^i$  denote the  $i^{\mathrm{th}}$  element of  $\pmb{y}$  and  $\pmb{x}$ , respectively. The sensor condition is omitted for notation simplicity.
To adapt our GS-DM into the EDM framework, we first set  $\sigma_{\mathrm{data}} = 1.0$  for EDM [19, Table 1], and then adapt the preconditioning equation [19, Section 5, Eq.7] based on §3 of our main paper:
$$
D _ {\theta} \left(\boldsymbol {x} ^ {i}; \sigma , \boldsymbol {y}\right) = c _ {\text {s k i p}} (\sigma) \boldsymbol {x} ^ {i} + c _ {\text {o u t}} (\sigma) F _ {\theta} ^ {i} \left(\left\{c _ {\text {i n}} (\sigma) \boldsymbol {x} ^ {i} + (1 - c _ {\text {i n}} (\sigma)) \bar {\boldsymbol {\mu}} _ {\phi} (\boldsymbol {y}, \sigma , i) \right\}; c _ {\text {n o i s e}} (\sigma)\right), \tag {26}
$$
where the per-element noise injection is defined as  $\pmb{x}^i = \pmb{y}^i + n\bar{\sigma}_{\phi}(\pmb{y}, \sigma, i)$ , and noise  $\pmb{n} \sim \mathcal{N}(\mathbf{0}, \sigma^2\mathbf{I})$ .  $D_{\theta}(\pmb{x}^i; \sigma, \pmb{y})$  is the reconstructed  $\pmb{y}^i$ .  $F_{\theta}$  is the denoising network taking all elements of a noisy sample  $\pmb{x}$ , and  $F_{\theta}^{i}$  is the output of the  $i^{\text{th}}$  element. With the above modifications, we implement our GS-DM with the general EDM framework. We then describe the concrete hyperparameter settings [19, Table 1] for our two tasks. Please refer to Karras et al. for detailed explanations of each hyperparameter.
Floorplan reconstruction: For the guidance training, we set the  $P_{\mathrm{mean}} = 1.0$  and  $P_{\mathrm{std}} = 4.0$  to ensure sufficient coverage of the forward process. For the denoising training, we set the  $P_{\mathrm{mean}} = -0.5$  and  $P_{\mathrm{std}} = 1.5$ . For inference (sampling), we set  $\sigma_{\mathrm{max}} = 5.0$  and  $\sigma_{\mathrm{min}} = 0.01$ . Instead of the  $2^{\mathrm{nd}}$  order Heun ODE solver adopted by the original EDM, we simply employ the  $1^{\mathrm{st}}$  order Euler solver for sampling, which is equivalent to DDIM [39]. All other hyperparameters are kept unchanged.
HD map reconstruction: The settings of  $P_{\mathrm{mean}}$  and  $P_{\mathrm{std}}$  for guidance and denoising training are the same as the floorplan reconstruction task. We set  $c_{\mathrm{skip}}(\sigma) = 0$  and  $c_{\mathrm{out}}(\sigma) = 1$  so that the denoising
<sup>5</sup>https://github.com/NVlabs/edm
CD matching
+OAD matching
+OAD matching
Figure 6: Illustration of the matching results with MapTR [23] predictions. CD matching: the original Chamfer distance (CD) matching criterion with a threshold of  $1.0m$ . +OAD matching: the original CD criterion with a threshold of  $1.0m$ , augmented with the order-aware angle distance (OAD) criterion with a threshold of  $10^{\circ}$ . The matched instances (i.e., true positives) are marked in red except for the two endpoints.
network directly estimates the sample  $\pmb{y}$ , as we found this choice stabilizes the denoising training on the HD map construction task. For inference (sampling), we set  $\sigma_{\mathrm{max}} = 5.0$  and  $\sigma_{\mathrm{min}} = 0.1$ , and use the Euler solver. All other hyperparameters are kept unchanged.
C The augmented matching criterion for HD map reconstruction
In §5.2 of the main paper, we augment the Chamfer-distance (CD) matching criterion for the mean AP (mAP) metric used in previous works [23, 25] with an order-aware angle distance (OAD) to better evaluate the structural regularity and directional correctness of the results. This section first discusses the limitations of the original Chamfer-distance matching criterion and then provides complete implementation details of the order-aware angle distance.
C.1 The limitations of the Chamfer-distance matching criterion
Figure 6 shows examples of the Chamfer-distance (CD) matching (threshold is  $1.0m$ ). Since CD considers each vertex separately, it mostly evaluates the global location of the instance and ignores the structural/directional information of the prediction. In the figure, some predictions with obviously wrong shapes are considered true positives under the CD matching criterion, while the augmented OAD matching criterion can effectively reject these bad shapes and align better with human judgment.
The computation of the order-aware angle distance is based on an optimal vertex-level matching between a predicted element (instance) and a ground truth (G.T.) element, similar to the "point-level matching" in the loss computation of MapTR [23]. Concretely, we enumerate through all equivalent representations of a G.T. map element and compute the average vertex-level  $L1$  distance between the G.T. vertices and the predicted vertices. The variant with the smallest average vertex-level  $L1$  distance forms the optimal matching with the predicted element. There are two equivalent variants for a polyline and  $2(N_{i} - 1)$  equivalent variants for a polygon with  $N_{i}$  vertices. After obtaining the optimal one-to-one vertex-level matching, we trace along the vertices of the G.T. and predicted elements to compute their average angle distance.
With the augmented OAD matching criterion, we change the three-level thresholds of the original CD-based AP from  $\{0.5m, 1.0m, 1.5m\}$  into  $\{(0.5m, 5^\circ), (1.0m, 10^\circ), (1.5m, 15^\circ)\}$ . However, compared to polylines (i.e., road dividers and boundaries), we noticed that the angle direction of polygons (pedestrian crossings) is much more challenging to recover. With an OAD threshold of  $5^\circ$ , MapTR's average precision for the pedestrian crossing class  $(\mathrm{AP}_p)$  is zero. Therefore, we loosen the OAD thresholds for the pedestrian crossing class by a factor of 2 (i.e.,  $\{(0.5m, 10^\circ), (1.0m, 20^\circ), (1.5m, 30^\circ)\}$ ) while not changing the thresholds for the other two classes.
This section provides additional experimental results, including extra ablation studies and qualitative comparisons.
Table 5: Ablation study for the intra-element attention layer on the floorplan reconstruction task. Note that PolyDiffuse uses the RoomFormer from the first row to produce the initial reconstruction.
Evaluation Level →	Room	Corner	Angle
Method	Intra-element-attn	Prec.	Rec.	F1	Prec.	Rec.	F1	Prec.	Rec.	F1
RoomFormer	X	96.3	96.2	96.2	89.7	86.7	88.2	85.4	82.5	83.9
RoomFormer	✓	96.9	96.4	96.6	90.3	87.0	88.6	84.9	81.9	83.4
PolyDiffuse(Ours)	X	98.4	97.8	98.1	92.4	89.0	90.7	90.2	87.0	88.6
PolyDiffuse(Ours)	✓	98.7	98.1	98.4	92.8	89.3	91.0	90.8	87.4	89.1
Table 6: Ablation study for the intra-element attention layer on the HD map reconstruction task. Note that PolyDiffuse uses the MapTR from the first row to produce the initial reconstruction.
Matching Criterion →	Chamfer distance	+ Ordered angle distance
Method	Intra-element-attn	APp	APd	APb	mAP	APp	APd	APb	mAP
MapTR	X	55.8	60.9	61.1	59.3	46.1	43.4	41.9	43.8
MapTR	✓	56.5	59.9	60.2	58.8	48.6	44.8	41.6	45.0
PolyDiffuse(Ours)	X	56.9	59.2	60.6	58.9	50.8	48.3	44.9	48.0
PolyDiffuse(Ours)	✓	58.2	59.7	61.3	59.7	52.0	49.5	45.4	49.0
D.1 Additional ablation studies
Table 5 presents the ablation study of the intra-element attention with the floorplan reconstruction task. Comparing the first and second rows, adding intra-element attention to RoomFormer slightly improves the room and corner results but deteriorates the angle-level performance. Table 6 provides a similar comparison for the HD map reconstruction task. Augmenting MapTR with intra-element attention worsens the mAP with the CD matching criterion while slightly improving the mAP with the OAD-augmented matching criterion. These comparisons provide a potential empirical explanation for the architecture choice of RoomFormer and MapTR – they only employ global attention layers without extra attention layers at the element or instance level.
On the contrary, as indicated by the third and fourth rows of both Table 5 and Table 6, applying intra-element attention consistently boosts PolyDiffuse across all metrics, although the improvements are not huge. A potential explanation here is that the denoising task of PolyDiffuse is more challenging than the regression/detection task of RoomFormer and MapTR, thus benefiting from extra modeling capacities.
D.2 A toy experiment with standard DM