Symbol as Points: Panoptic Symbol Spotting via Point-based Representation
Authors: WENLONG LIU
Tianyu Yang
Yuhan Wang
Qizhi Yu
Lei Zhang
Venue: ICLR 2024 poster (2024)
Keywords: CAD
Panoptic Symbol Spotting
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: This work studies the problem of panoptic symbol spotting, which is to spot and parse both countable object instances (windows, doors, tables, etc.) and uncountable stuff (wall, railing, etc.) from computer-aided design (CAD) drawings. Existing methods typically involve either rasterizing the vector graphics into images and using image-based methods for symbol spotting, or directly building graphs and using graph neural networks for symbol recognition. In this paper, we take a different approach, which treats graphic primitives as a set of 2D points that are locally connected and use point cloud segmentation methods to tackle it. Specifically, we utilize a point transformer to extract the primitive features and append a mask2former-like spotting head to predict the final output. To better use the local connection information of primitives and enhance their discriminability, we further propose the attention with connection module (ACM) and contrastive connection learning scheme (CCL). Finally, we propose a KNN interpolation mechanism for the mask attention module of the spotting head to better handle primitive mask downsampling, which is primitive-level in contrast to pixel-level for the image. Our approach, named SymPoint, is simple yet effective, outperforming recent state-of-the-art method GAT-CADNet by an absolute increase of 9.6% PQ and 10.4% RQ on the FloorPlanCAD dataset. The source code and models will be available at \url{https://github.com/nicehuster/SymPoint}.

Published as a conference paper at ICLR 2024
SYMBOL AS POINTS: PANOPTIC SYMBOL SPOTTING VIA POINT-BASED REPRESENTATION
Wenlong Liu', Tianyu Yang', Yuhan Wang', Qizhi Yu² Lei Zhang' 'International Digital Economy Academy (IDEA)
Vanyy Tech
ABSTRACT
This work studies the problem of panoplic symbol spotting, which is to spot and parse both countable object instances (windows, doors, tables, etc.) and uncountable stuff (wall, railing, etc.) from computer-aidedt design (CAD) drawings. Existing methods typically involve either rasterizing the vector graphics into images and using image-based methods for symbol spotting, or directly building graphs and using graph neural networks for symbol recognition. In this paper, we take a different approach, which treats graphic primitives as a set of 2D points that are locally connected and use point cloud segmentation methods to tackle it. Specifically, we utilize a point transformer to extract the primitive features and append mask2former like spotting head to predict the final output. To better use the local connection information of primitives and enhance their discrininability, we further propose the attention with connection module (ACM) and contrastive connection learning scheme (CCL). Finally, we propose KNN interpolation mechanism for the mask attention module of the spotting head to better handle primitive mask downsampling, which is primitive-level in contrast to pixel-level for the image. Our approach, named SymPoint, is simple method GAT-CADNet
yet effective, outperforming recent state-of-the-ae by an absolute increase of 9.6% PQ and 10.4% RQ on the FloorPlanCAD dataset. The source code and models will be available at https://github. http:/nicchuueerrSSee /Sa.errrt/...t.......
INTRODUCTION
Vector graphics (VG), renowned for their ability to be scaled arbitrarily without succumbing to issues like blurring aliasing of details, have become a staple in industrial designs. This includes their prevalent use in graphic designs(Redd) et al., 2021), 2D interfaces (Carlier et al., 2020), and Computer-aided design (Fan et al., 2021). Specifically, CAD drawings, consisting of geometric primitives arc, circle, polyline, etc.), have established themselves the preferred data representation method in the realms of interior design, indoor construction, and property development, promoting higher standard of precision and innovation in these fields.
Symbol spotting (Rezvanifar et al., 2019; 2020; Fan et al., 2021; 2022; Zheng et al., 2022) refers to spotting and recognizing symbols from CAD drawings, which serves as foundational task for reviewing the error of design drawing and 3D building information modeling (BIM). Spotting each symbol, a grouping of graphical primitives, within CAD drawing poses a significant. challenge due to the existence of obstacles such as occlusion, clustering, variations in appearances, and a significant imbalance in the distribution of different categories. Traditional symbol spotting usually deals with instance symbols representing countable things (Rezvanifar et al., 2019), like table, sofa, and bed. Fan et al. (2021) further extend it to panoptic symbol spotting which performs both the spotting of countable instances (c.g., single door, a window, a table, etc.) and the recognition of uncountable stuff (c.g., wall, railing, etc.).
Typical approaches (Fan et al., 2021; 2022) addressing the panoptic symbol spotting task involve first converting CAD drawings raster (aaphicssRG)) and then processing it with

---

Published as a conference paper at ICLR 2024
powerful image-basee based detection segmentation methods (Ren et al., 2015; Sun et al. 2019). Another line of previous works (Jiang et al., 2021; Zheng et al., 2022: Yang et al., 2023) abandons the raster procedure and directly processes vector graphics for recognition with graph convolutions networks. Instead of rastering CAD drawings to images or modeling the graphical primitives with GCN/GAT, which can be computationally expensive, especially for large CAD graphs, we propose a new paradigm that has the potential to shed novel insight rather than merely delivering incremental advancements in performance.
Upon analyzing the data characteristics of CAD drawings, we can find that CAD drawing has three main properties: 1). irregularity and disonderliness. Unlike regular pixel arrays in raster graphics seaphics/mmags, CAD drawing consists of geometric primitives(e.g, arc, circle, polyline, etc.) without specific order. 2). local interaction among graphient primitives. Each graphical primitive is not isolated but locally connected with neighboring primitives, forming symbol. 3). invariance under transformations. Each symbol invariant to certain transformations. For example, rotating and translating symbols do not modify the symbol's category. These properties are almost identical to point clouds. Hence, we treat CAD drawing as sets of points (graphical primitives) and utilize methodologies from point cloud analysis (Qi et al., 2017;;;; Zhao et al., 2021) for symbol spotting.
8-dimensional data point with the information of position and primitive's properties (type, length, etc.). We then utilize methodologies from point cloud analysís for graphic primitive representation learning. Different from point clouds, these graphical primitives are locally connected. We therefore propose contrastive connectivity learning mechanism to utilize those local connections. Finally, we borrow the idea of Mask22ormerrCheen Cheng et al., 2021; 2022) and construct a masked- attention transformer decoder to perform the panoptic symbol spotting task. Besides, rather than using bilinear interpolation for mask attention downsampling as in (Cheng et al., 2022), which could cause information loss due to the sparsity of graphical primitives, we propose KNN interpolation, which fuses the nearest neighboring primitives, for mask attention downsampling. We conduct extensive experiments on the FloorPlanCAD dataset and our SymPoint achieves 83.3% PQ and 91.1% RQ under the panoptic symbol spotting setting, which outperforms the recent state-of-tho-art method GAT-CADNet (Zhong et al., 2022) with large margin.
In this work, first consider each graphic primitive
2 RELATED WORK
Vector Graphics Recognition Vector graphics are widely used in 2D CAD designs, urban designs, graphic designs, and circuit designs, to facilitate resolution-free precision geometric modeling. Considering their wide applications and great importance, many works are devoled to recognition lasks on vector graphics. Jiang et al. (2021) explores vectorized object detection and achieves a superior accuracy to detection methods (Bochkovskiy et al., 2020; Lin et al., 2017) working on raster graphics while enjoying faster inference time and less training parameters. Shi et al. (2022) propose unified vector graphics recognition framework that leverages the merits of both vector graphics and raster graphics.
Panoptic Symbol Spotting Traditional symbol spotting usually deals with instance symbols representing countable things (Rezvanifar et al., 2019), like table, sofa, and bed. Following the idea in (Kirillov et al., 2019), Fan et al. (2021) extended the definition by recognizing semantic of uncountable stuff, and named it panoptic symbol spotting. Therefore, all components in CAD drawing are covered in one task altogether. For example, the wall represented by a group of parallel lincs was properly handled by (Fan et al., 2021), which however was treated as background by (Jiang et al., 2021; Shi et al., 2022; Nguyen et al., 2009) in Vector graphics recognition. Mcanwhile, the first large-scale real-world FloorPlanCAD dataset in the form of vector graphics was published by (Fan et al., 2021). Fan et al. (2022) propose CADTransformer, which modifies existing vision transformer (ViT) backbones for the panoptic symbol spotting task. Zheng et al. (2022) propose GAT-CADNet which formulates the instance symbol spotting task as a subgraph detection problem and solves by prodicting the adjacency matrix.
Point Cloud Segmentation Point cloud segmentation aims to map the points into multiple homogeneous groups. Unlike 2D images, which are characterized by regularly
2

---

Published as a conference paper at ICLR 2024
12
(x2,)2
line
4x line
Dj (x 1/2
Xx [01 x,) (n y2)/2]
arc
ellipse
fx [eg. 4. onchot(:;)] R
(a) primitive position.
(b) primitive feature.
Figure 1: Illustration of constructing point-based representation.
arranged dense pixels, point clouds are constituted of unordered and irregular point sets. This makes the direct application of image processing methods point cloud segmentation an impracticable approach. However, in recent years, the integration of neural networks has significantly enhanced the effectiveness of point cloud segmentation across a range of applications, including semantic segmentation (Qi ct al., 2017a;a; Zhao et al., 2021), instance segmentation (Ngo et al., 2023; Schult et al., 2023) and panoptic segmentation (Zhou et al., 2021; Li et al., 2022; Hong et al., 2021; Xiao et al., 2023), etc.
3 METHOD
Our methods forgo the raster image or GCN in favor of a point-based representation for graphical primitives. Compared to image-basod representations, it reduces the complexity of models due to the sparsity of primitive CAD drawings. In this section, we first describe how to form the point-based representation using the graphical primitives of CAD drawings. Then we illustrate a baseline framework for panoptic symbol spotting. Finally, we thoroughly explain three key techniques, attention with local connection, contrastive connection learning, and KNN interpolation, to adapt this baseline framework to better handle CAD data.
3.1 FROM SYMBOL TO POINTS
Given vector graphies represented by set of graphical primitives {pa}, we treat it collection of points (px (æt, fk)), and each point contains both primitive position (æx) and primitive feature {fx} information; hence, the points set could be unordered and disorganized.
Primitive position. Given a graphical primitive, the coordinates of the starting point and the ending point are (11. and (x1, v2), respectively. The primitive position 2 IR2 is defined a
zt [xx x22 (11 + 92)/2),
(1)
We take its center as the primitive position for a closed graphical primitive(circle, ellipse). as shown Fig. la.
Primitive feature. We define the primitive features fk E IR as:
fx (ou, onehol(((())))
(2)
where ag is the clockwise angle from the positive axis Tk: and 4 represents the distance between U1 and v2 for linear primitives, as shown Fig. 1b. For circular primitives like circles and ellipses, l is defined the cireumference. We encode the primitive type to (line, are, circle, or ellipse) into a one-hot vector to make up the missing information of segment approximations.
3.2 PANOPTIC SYMBOL SPOTTING VIA POINT-BASED REPRESENTATION
The baseline framework primarily comprises two components: the backbone and the symbol spotting head. The backbone converts raw points into points features, while the symbol
3

---

Published as a conference paper at ICLR 2024
CAD drawing
Synbol Spotting
Updated Queries
Query Updating
Queries
Predicting
Mask Predic
Fo
Results
F
Queries
Prinitive points
ACM
Backbone
Figure 2: The overview of our method. After transfering CAD drawings to primitive points, we use backbone to extract multi-resolution features F, and append symbol spotting head to spot and recognize symbols. During this process, we propose attention with connection module(ACM), which utilizes primitive connection information when performing self-attention in the first stage of backone. Subsequently, we propose contrastive connection learning( CCL.) to enhance the discriminability between connected primitive features. Finally, we propose KNN interpolation for attention mask downsampling( (atnnaamplin((MM))) (MD) to effetively downsample the high-resolution attention masks.
spotting head predicts the symbol mask through learnable queries (Cheng et al., 2021; 2022). Fig. 2 illustrates the the whole framework.
Backbone. We choose Point Transformer (Zhao et al., 2021) with a symmetrical encoder and decoder as our backbone for feature extraction due to its good generalization capability in panoptic symbol spotting. The backbone takes primitive points as input, and performs vector attention between each point and its adjacent points explore local relationships. Given point Pi and ils adjacent points M(p:), we project them into query feature qi, key feature kj and value feature By, and obtain the vector attention as follows:
Wig = w(r(gi
patta
Softmax (Wi); @ Bg
P.EM(P))
where y serves as a relational function, such subtraction. w is a learnable weight encoding that calculates the attention vectors. @ is Hadamard product.
Symbol Spotting Head. We follow Mask2Former (Cheng et al., 2022) to use hierarchical multi-resolution primitive features F, E IR N. from the decoder of backbone as the input to the symbol spotting predition head, where N, is the number of feature tokens in resolution r and D is the feature dimension. This head consists of L layers of masked attention modules which progressively upscales low-resolution features from the backbone to produce high-resolution per-pixel embeddings for mask prediction. There are two key components in the masked attention module: query updating and mask predicting. For each layer I, query updating involves interacting with different resolution primitive features F, to update query features. This process can be formulated as,
Xi softmax(Ar-1 + QiK?")) V + XI-1.
where X₁ E BOXD is the query features. O is the number of query features. Qi QI= (o(X)-)); Ki fx (F.) and === fv(F.) are query, key and value features projected by MLP layers. A 1 is the attention mask, which computed by,
A Av- -0
0
if Mi)((u) 0.5,
otherwise.
where e is the position of feature point and M,-1 is the mask predicted from mask predicting part. Note that need to downsample the high-resolution attention mask to adopt the query updating on low-resllution features. In practice, we utilize four coarse-level primitive features from the decoder of backbone and perform query updaling from coarse to fine.
4

---

Published as a conference paper at ICLR 2024
d= |0| of
piotiio past
(a) Construct connections.
(b) Attend to connections,
(c) Noisy connections.
Figure 3: (a) Set of primitives and its connection, primitives are disintegrated for clarity. (b) Locally connected primitives are considered in the attention layers. (c) Locally connected primitives do not always belong the same category.
During mask predicting process, we obtain the object mask Mj E ROXNo and its corresponding category Yi E ROxC by projecting the query features using two MLP layers fv and fas, where C the category number and No is the number of points. The process is as follows:
Yi = fXii)
M = fM()
The outputs of final layer, YE and Mr, are the predicted results.
3.3 ATTENTION WITH CONNECTION MODULE
The simple and unified framework rewards excellent generalization ability by offering a fresh perspective of CAD drawing, a set of points. can obtain competitive results compared to previous methods. However, it ignores the widespread presence of primitive connections in CAD drawings. It precisely because of these connections that scattered, unrelated graphical clements these connections between each primitive, we propose Attention with Connection Module (ACM), the details are shown below.
together to form symbols with special semanties. In order to utilize
It is considered that these two graphical primitives(ps, Pý) are interconnected if the minimum distance di between the endpoints (v,, By) of two graphical primitives (p.. Pj) is below certain threshold where:
d, min Eni.oeep...pp. |0, -, <.
(7)
To keep the complexity low, at most K connections are allowed for every graphical primitive by random dropping. Fig. 3a demonstrates the connection construction around the wall symbol, the gray line is the connection between two primitives. In practice, we set to 1.0px.
The attention mechanism in (Zhao al., 2021) directly performs local attention between each point and its adjacent points to explore the relationship. The original attention mechanism interacts only with neighboring points within a spherical region, as shown in Fig. 3b. Our ACM additionally introduces the interaction with locally connected primitive points during attention (pink points), essentially enlarging the radius of the spherical region. Note that experimentally found that crudely increasing the radius of the spherical region without considering the local connections primitive points does not result in performance improvement. This may be explained by that enlarging the receptive field also introduces additional noise at the same time. Specifically, we extend the adjacent points set M(p.) in Eq. (3) to A(p.) M(p:) UC(P:), where C(p:) = (pildij < <),, yielding, pattn
Softmax Softmax(W.); 2 Uj: P:E((P.)
Σ
(8)
In practice, since we cannot directly obtain the connection relationships of the points in the intermediate layers of the backbone, we integrate this module into the first stage of the backbone to replace the original local attention, shown in Fig. 2.
3.4 CONTRASTIVE CONNECTION LEARNING.
Although the information primitive connection are considered when calculating attention of the encoder transformer, locally connected primitives may not belong to the same instance,
5

---

Published as a conference paper at ICLR 2024
in other words, noisy connections could be introduced while take primitive connections into consideration, as shown in Fig. 3c. Therefore, order to more effectively utilize connection information with category consistency, we follow the widely used InfoNCE loss (Oord et al., 2018) and its generalization (Frosst et al., 2019; Gutmann & Hyvärinen, 2010) to define the contrastive learning objective on the final output feature of backbone. We encourage learned representations more similar to its connected points from the same category and more distinguished from other connected points from different categories. Additionally, also take neighbor points M(p;) into considcration, yielding,
(xpxpp)))_ixeep((dd(A LCCL - log EprEA(p.) exp( exp(-d(f:.. fi)/T)
(9)
where fi is the backbone feature of pi, d(:,.) is a distance measurement, T is the temperature in contrastive learning. we set the by default.
3.5 KNN INTERPOLATION
During the process of query updating in symbol spotting head Eq. (4) Eq. (5), we need to convert higl-resolution mask predictions to low-resolution for attention masks computation as shown in Fig. 2 (AMD on the right). Mask2Former (Cheng ct al., 2022) employs the bilincar interpolation on the pixcl-level mask for downsampling. However, the masks of CAD drawings are primitive-level, making it infcasible to dircctly apply the bilinear interpolation on them. To this end, we propose the KNN interpolation for downsampling the attention masks by fusing the nearest neighbor points. A straight forward operation is max pooling or average pooling. We instead utilize Srtance-based interpolation. For simplicity, we omit layer index in A,
A' EpIEK(p)) (p)pp/p(p.
(10)
Σpjek(p.) 1/d(p..P,)
where, A and A" are the full-resolution attention mask and the r-resolution attention mask repectively. d(.,.) is a distance measurement. K(p:)) practice, we set K- 4" in our experiments.
is the set of K nearest neighbors, In
3.6 TRAINING AND INFERENCE
Throughout the training phase, we adopt bipartite matching and set prediction loss to assign ground truth predictions with the smallest matching cost. The full loss function can be formulated as L = λBCELBCE + binary cross-entropy loss (over the foreground and background of that mask), Lutice the Dice loss (Deng et al., 2018). Lets is the default multi-class cross-entropy loss to supervise the queries classification, Leer is contrastive connection loss. our experiments, we empirically set λBCE Adice Xets Acos. 5 5 2 8. For inference, we simply use argmaa to determine the final panoptic results.
+ Acte + ACCLLCeL while LBCE is the
4 EXPERIMENTS
In this section, we present the experimental sctting and benchmark results on the public CAD drawing dataset FloorPlanCAD (Fan et al., 2021). Following previous works (Fan et al., 2021; Zheng et al., 2022; Fan et al., 2022), we also compare our method with typical image-based instance detection (Ren et al., 2015; Redmon & Farhadi, 2018; Tian et al., 2019; Zhang et al., 2022). Besides, we also compare with point cloud semantic segmentation methods (Zhao et al., 2021), Extensive ablation studies are conducted to validate the effectiveness of the proposed techniques. In addition, we have also validated the generalizability of our method on other datasots beyond floorplanCAD, with detailed results available in the Appendix A.
4.1 EXPERIMENTAL SETTING
Dataset and Metrics. FloorPlanCAD dataset has 11,602 CAD drawings of various floor plans with segment-grained panoptic annotation and covering 30 things and 5 stuff classes.

---

Published as a conference paper at ICLR 2024
Methods
PanCADNell (Pan et al., 2021)
CADTranslormer (Fan et al., 2022)
GAT-CADNe (Zheng al., 2022)
Point
SymPoint (ours)
(Zhao et al., 2021)
F1
80.6
82.2
85.0
83.2 80.7
86.8 85.5
wP1
79.8
80.1
82.3
Table 1: Semantic Symbol Spotting comparison results with previous works. t backbone with double channels. wF1: length-weighted F1.
Method
Backhone
AP50
AP75
mAP
# #Params
Speed
FasterRCNN (Ren et al., 2015)
R101
60.2
51.0
45.2
61M
59ms
YOLOv3 (Redmon & Farhadi, 2018)
DarkNet53
63.9
45.2
41.3
62M
11ms
FCOS (Tian et sl., 2019)
R101
62.4
49.1
45.3
51M
57ms
DINO (Zhang et al., 2022)
R50
64.0
54.9
47.5
47M
42ms
SymPoint (ours)
PointT
66.3
55.7
52.8
35M
66ms
Table 2: Instance Symbol Spotting comparison results with image-based detection methods.
Following (Fan al., 2021; Zheng et al., 2022; Fan al., 2022), we use the panoptic quality (PQ) defined on vector graphics as our main metric to evaluate the performance of panoptic symbol spotting. By denoting graphical primitive (I, 2) with semantic label and an instance index PQ is defined as the multiplication of segmentation quality (SQ) and recognition quality (RQ); which formulated as
PQ RQ x &QQ
(11)
TPI
Elees (oU 8g)
(12)
TPI + \ [|P FIN
[TP
Σ Ee (op(sp 8g) [TP+ I|
(13)
where, 8p (pp, is the predicted symbol, Sg (lg 2g) is the ground truth symbol. (TP, [FP), [F] indicate true positive, false positive and false negative respectively. A certain predicted symbol is considered as matched if it finds a ground truth symbol, with Ip = lg and loU(sp 8g) > 0.5, where the loU is computed by:
ToU(sp: 8g) = Besesnne,,ogh L(e:))
(14)
Lejes, log(1 L(e,))
Implementation Details. We implement SymPoint with Pytorch. We use Point (Zhao et al., 2021) with double channels as the backbone and stack 1, = 3 layers for the symbol spotting head. For data augmentation, we adopt rotation, flip, scale, shift, and cutmix augmentation. We choose AdamW (Loshchilov & Hutter, 2017) as the optimizer with default weight decay 0.001, the initial learning rate is 0.0001, and we train the model for 1000 epochs with batch size of 2 per GPU on 8 NVIDIA A100 GPUs.
4.2 BENCHMARK RESULTS
Semantic symbol spotting. We compare our methods with point cloud segmentation methods (Zhao et al., 2021), and symbol spotting methods (Fan et al., 2021; 2022; Zheng et al., 2022). The main test results are summarized in Tab. 1, Our algorithm surpasses all previous methods in the task of semantic symbol spotting. More importantly, compared to GAT-CADNel (Zheng et al., 2022), we achieves an absolute improvement of 1.8% F1. and 3.2% wF1 respectively. For the PointTF, we use our proposed point-based representation in Section 3.1 to convert the CAD drawing to collection of points as input. It is worth noting that PointT' has already achieved comparable results to GAT-CADNe (Zheng al., 2022), which demonstrates the effectiveness of the proposed point-based representation for CAD symbol spotting.
Instance Symbol Spotting. We compare our method with various image detection methods, including FasterRCNN (Ren al,, 2015), YOLOvv (Redmon & Farhadi, 2018),

---

Published as a conference paper at ICLR 2024
Method
Data Format
PQ
SQ
RQ
Params
Speed
PanCADNett [Fan al., 2021)
VG
RG
55.3
83.8
66.0
-42
>1.2s
CADTransformer (Fan al, 2022)
VG
68.9
88.3
73.3
>65
>1.2s
GAT-CADNet (Zheng et al., 2022)
VG
73.7
91.4
80.7
Point P Chuster et al., 2021)
VG
49.8
85.6
58.2
31M
80ms
Sym (ours, 300epoch)
VG
79.6
89.4
89.0
35M
66mms
SymPoint (ours, 500epoch)
VG
81.9
90.6
90.4
35M
66ms
SymPoint(ours. 1000epoch)
VG
83.3
91.4
91.1
35M
66ms
Table 3: Panoptic Symbol Spotting comparisons results with previous works.
VG: vector
graphics, RG: raster graphics.
Baseline
ACM
CCL
Klnter
PQ
RQ
SQ
DSampling method
PQ
RQ
SQ
73.1
83.3
87.7
72.6
82.9
87.6
linear
74.3
85.8
86.6
73.5
83.9
87.6
knn avepool
75.9
88.4
74.3
85.8
86.6
knn maxpool
77.0
88.8
77.3
87.1
88.7
knn interp
77.3
87.1
88.7
(a) Ablation studies of different techniques
(b)
Ablation studies of mask downsampling
BS
SW
I.
D)
PQ
RQ
SQ
loU
# #Params
1x
3
300
128
67.1
78.7
85.2
62.8
9M
1.5x
V
3
300
128
73.3
84.0
87.3
65.6
19M
ablation setting
2x
V
3
300
128
77.3
87.1
88.7
68.3
32M
2x
V
3
500
128
77.9
87.6
88.9
68.8
32M
final setting
2x
3
500
256
79.6
89.0
89.4
69.1
35M
2x
3 500
256
79.1
88.4
89.5
68.8
42M
2x
6 500
256
79.0
88.1
89.6
68.5
35M
(c) Ablation studies on architecture design. BS: Backhone size. SW: share weights. T: layer number of spotting head. O: query number. D: feature dimension. v in the share weights column means whether share weights for head layers.
Table 4: Ablation Stuides on different techniques, attention mask downsampling, and architecture desgin.
FCOS (Tian et al., 2019), and recent DINO (Zhang et al., 2022). For a fair comparison, we post-process the predicted mask to produce bounding box for metric computation. The main comparison results are listed in Tab. 2. Although our framework is not trained to output bounding bax, it still achieves the best results.
Panoptic Symbol Spotting. To verify the effectiveness of the symbol spotting head, we also design variant method without this head, named Point Cluster, which predicts offset vector per graphic entity to gather the instance entities around a common instance centroid and performs class-wise clustering (e.g. meanshift (Cheng, 1995)) to get instance labels as in CADTransformer (Pan et al., 2022). The final results are listed in Tab. 3. Our SymPoint trained with 300epoch outperforms both Point Cluster and the recent SOTA method CAT-CADDet(Zhenn) et al., 2022) substantially, demonstrate the effectiveness of the proposed method. Our method also benefits from longer training and achieves further performance improvement. What's more, our method runs much faster during the inference phase than previous methods. For imago-based methods, it takes approximately 1.2s to render vector graphic into image while our method does not need this process. The qualitative results are shown in Fig.
4.3 ABLATION STUDIES
In this section, carry out series of comprehensive ablation studies to clearly illustrate the potency and intricate details of the SymPoint framework. All ablations are conducted under 300 epoch training.

---

Published as a conference paper at ICLR 2024
(a) GT
(b) SymPoint
(c) CADTransformer
Figure 4: Qualitative comparison of panoptic symbol spotting results with CADTrransformer. Primitives belonging different classes are represented in distinct colors. The colormap for each category can be referenced in Fig. 8.
Effects Techniques. We conduct various controlled experiments to verify different techniques that improve the performance of SymPoint in Tab. 4a. Here the baseline means the method described Sec. 3.2. When we only introduce ACM (Attention with Connection Module), the performance drops bit due to the noisy connections. But when we combine with CCL (Contrastive Connection Learning), the performance improves to 74.3 of PQ. Note that applying CCL alone could only improve the performance marginally. Furthermore, KNN Interpolation boosts the performance significantly, reaching 77.3 of PQ.
KNN Interpolation. In Tab. 4b, we ablate different ways of downsampling attention mask: 1) linear interpolation, 2) KNN average pooling, 3) KNN max pooling, 4) KNN interpolation. KNN average pooling and KNN max pooling means using the averaged value or max value of the K ncarest neighboring points as output instead of the one defined in Eq. (10). We can see that the proposed KNN interpolation achieves the best performance.
Architecture Design. We analyze the effect of varying model architecture design, like channel number of backbone and whether share weights for the L layers of symbol spotting head. As shown in Tab. 4c, we can see that enlarging the backbone, the query number and the feature channels of the symbol spotting head could further improve the performance. Sharing weights for spotting head not only saves model parameters but also achieves better performance compared with the one that does not share weights.
5 CONCLUSION AND FUTURE WORK
This work introduces novel perspective for panoptic symbol spotting. We treat CAD drawings as sets of points and utilize methodologies from point cloud analysis for symbol spotting. Our method SymPoint is simple yet effective and outperforms previous works. One limitation is that our method needs a long training epoch to get promising performance. Thus accelerating model convergence
important direction for future work.

---

Published as a conference paper at ICLR 2024
REFERENCES
Alexey Bochkovskiy, Chien- Yao Wang, and Hong- Yuan Mark Liao. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint crXitt2004410994, 2020.
Alexandre Carlier, Martin Danelljan, Alexandre Alahi, and Radu Timofte. Deepsvg: A hier- archical generative network for vector graphics animation. Advances in Neural Information Processing Systems, 33:16351-16361, 2020.
Bowen Cheng, Alex Schwing, and Alexander Kirillov. Per-pixel classification is not all you need for semantic segmentation. Advances in Newral Information Processing Systems, 34: 17864-17875, 2021.
Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1290 1299, 2022.
Yizong Cheng, Mean shift, mode seeking, and clustering, IEEE transactions pattern analysis and machine intelligence, 17(8):790-799, 1995.
Ruoxi Deng, Chunhua Shen, Shengjun Liu, Huibing Wang, and Xinru Liu. Learning to predict crisp boundaries. In Proceedings of the European conference on computer vision (ECCV), pp. 562-52 052-578,22 2018
Zhiwen Fan, Lingjie Zhu, Honghua Li, Xiaohao Chen, Siyu Zhu, and Ping Tan. Floorplancad: A large-scale cad drawing dataset for panoptic symbol spotting. In Proceedings of the PREE/CVF International Conference on Computer Vision, pp. 10128-1011 2021.
Zhiwen Fan, Tianlong Chen, Peihao Wang, and Zhangyang Wang. Cadtransformer: Panoptic symbol spotting transformer for cad drawings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp-
2022.
Nicholas Frosst, Nicolas Papernot, and Geolfrey Hinton. Analyzing and improving repre- sentations with the soft nearest neighbor loss. In International conference on machine learning, pp. 2012 2020. PMLR, 2019.
Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 297 304. JMLR Workshop and Conference Proceedings, 2010.
Fangzhou Hong, Hui Zhou, Xinge Zhu, Hongsheng Li, and Ziwei Liu. Lidar-based panoptic segmentation via dynamic shifting network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognétion, pp. 13090-13099, 2021.
Xinyang Jiang, Lu Liu, Caihua Shan, Yifei Shen, Xuanyi Dong, and Dongsheng Li. Recog- nizing vector graphics without rasterization. Advances in Neural Information Processing Systems, 34:24569 24580, 2021.
Alexander Kirillov, Kaiming He, Ross Girshick, Carston Rother, and Piotr Dollár. Panoptic segmentation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 9404 9413, 2019.
Jinke Li, Xiao He, Yang Wen, Yuan Gao, Xiaoqiang Cheng, and Dan Zhang, Panoptic-phnet: Towards real-time and high-precision lidar panoptic segmentation via clustering pseudo heatmap. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern 2022.
Recognition, pp.
Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp. 2988, 2017.
10

---

Published as a conference paper at ICLR 2024
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv: 111117711.0111111 2017.
Tuan Due Ngo, Binh-Son Hua, and Khoi Nguyen. Isbnet: a 3d point cloud instance segmentation network with instance-aware sampling and box-aware dynamic convolution. In Proceedings of the IEEE/CVE /CVF Conference on Computer Vision and Pattern Recognition, pp. 13550 13559, 2023.
Thi-Oanh Nguyen, Salvatore Tabbone, and Alain Boucher. A symbol spotting approach based on the vector model and visual vocabulary. In 2009 10th International Conference on Documend Analysis and Recognilion, pp. 708 712. IEEE, 2009.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint ar 1fx: 08748, 2018.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guíbas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-660, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnct+ Deep hicrarchical feature learning on point sets in a metric space. Advances neural information processing systems, 30, 2017b.
Pradyumna Reddy, Michael Gharbi, Michal Lukac, and Niloy J Mitra. Im2vec: Synthesizing vector graphics without vector supervision. In Proceedings of the IEEE/CVE /CF Conference on Computer Vision and Pattern Recognition, pp. 7342 7351, 2021.
Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint orx:1:180, 02766, 2018.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cmn: Towards real-time object detection with region proposal networks. Advances in newral information processing systems, 28, 2015.
Alireza Reavanifar, Melissa Cote, and Alexandra Branzan Albu. Symbol spotting for architectural drawings: state-of-the--rt and new industry-driven developments. TPSJ Transactions on Computer Vision and Applications, 11:1-22, 2019.
Alireza Rezvanifar, Melissa Cole, and Alexandra Branzan Albu. Symbol spotting on digital architectural floor plans using a deep learning- framework. Tn Proceedings of the JEEE/CVE CCF Conference on Computer Vision and Pattern Recognition Workshops, pp. 568 569, 2020.
Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. Mask3d: Mask transformer for 3d semantic instance segmentation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 8216 8223. IEEE, 2023.
Ruoxi Shi, Xinyang Jiang, Caihua Shan, Yansen Wang, and Dongsheng Li. Rendnet: Unified 3d/3d recognizer with latent space rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognátion, pp. 5408 5408-5417, 2022.
Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep rish-tesolution representation learning for human pose estimation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 5693 5703, 2019.
Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Foos: Fully convolutional one-stage object detection. In Proceedings of the IEEE/CVE international conference on computer vision, pp. 9627-9636, 2019.
Zeqi Xiao, Wenwei Zhang, Tai Wang, Chen Change Loy, Dahua Lin, and Jiangmiao Pang. Position- point cloud panoptic segmentation transformer. arXiv preprint ar Xim:23333 13509, 2023.
11

---

Published as a conference paper at ICLR 2024
Bingchen Yang, Haiyong Jiang, Hao Pan, and Jun Xiao. Vectorfloorseg: Two-stream graph attention network for vectorized roughcast floorplan segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1368-1367, 2023.
Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung- Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.0380
2022.
Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVE international conference on compuler vision, pp. 16259 16268, 2021.
Zhaohua Zheng, Jianfang Li, Lingjic Zhu, Honghua Li, Frank Petzold, and Ping Tan. Gat- cadnet: Graph attention network for panoptic symbol spotting in cad drawings. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pallern Recognition, pp. 11747-11756, 2022.
Zixiang Zhou, Yang Zhang, and Hassan Foroosh. Panoptic-polarnct: Proposal-free lidar point cloud panoptic segmentation. Proceedings of the IEEE/CU /CVF Conference on Computer Vision and Pallern Recognition, pp. 13194 13203, 2021.
12

---

Published as a conference paper at ICLR 2024
30
without ACM
80
25
60
valltoss
val/PQ
20
40
15
with ACM
without ACM
0
300
100
200
300
epoch
epoch
Loss
(b) PQ
Figure 5: Convergence curves with/withoud the ACM Module on SESYD-foorplaas.
A APPENDIX
Due space constraints in the paper,additional techniques analysis, additional quantitative results, qualitative results, and other dataset results can be found in the supplementary materials.
A.1 ADDITIONAL, TECHNIQUES ANALYSIS
Effects of Attention with Connection Module. We conduct additional experiments in SESYD-foorplans dataset that is smaller than floorplanCAD. ACM can significantly promote performance and accelerate model convergence. We present the convergence curves without/with ACM in Fig. 5.
Explanation and Visualization of KNN interpolation Technique. While bilinear interpolation, as utilized in Mask2Former, is tailored for regular data, such as image, but it unsuitable for irregular sparse primitive points. Here, we provided some visualizations of point masks for KNN interpolation and bilinear interpolation as shown in Fig. 6. Note that these point masks are soft masks (ranging from 0 to 1) predicted by intermediate layers. After downsampling the point mask to 4x and 16x, interpolation well preserves the original mask information, while bilincar interpolation causes significant information loss, which could harm the final performance.
clearly find that KNN
A.2 ADDITIONAL QUANTITATIVE EVALUATIONS
We present a detailed evaluation of panoptic quality(PQ), segmentation quality(SQ), and recognition quality(RQ) in Tab. 5. Here, we provide the class-wis wise evaluations of different variants of our methods.
A.3 ADDITIONAL DATASETS
To demonstrate the generality of our SymPoint, we conducted experiments on other datasets beyond floorplanCAD.
Private Dataset. We have also collected a datasct of floorplan CAD drawings with 14,700 from our partners. We've meticulously annotated the dataset at the primitive level. Due to privacy concerns, this dataset is currently not publicly available. we randomly selected 10,200 as the training set and the remaining 4,500 as the test set. We conduct ablation studies of the proposed three techniques Different from the main paper, we also utilize the color information during constructing the connections, ie., locally connected primitives with the same color considered as
this dataset, and the results are shown in Tab. 6.
13

---

Published as a conference paper at ICLR 2024
Class
A
B
C
D
E
PQ
RQ
SQ
PQ
RQ
SQ
PQ
RQ
SQ
PQ
RQ
SQ
PQ
RQ
SQ
single door
83.2
90.8
91.6
82.9
90.7
91.4
83.3
90.9
91.6
86.6
93.0
93.1
91.7
96.0
95.5
double door
93.4
86.5
93.5
92.5
88.5
95.3
92.9
91.55
96.6
94.7
sliding door
87.5
91.0
93.1
87.6
91.6
92.5
$6.6
91.4
93.8
90.1 1
96.4
93.7
91.8
97.7
97.0
folding door
39.3
46.8
84.0
48.5
58.2
83.3
44.7
55.2
80.9
56.4
61.5
91.6
73.8
87.0
84.8
revolving door
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
rolling, door
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
window
64.5
77.1
83.6
67.2
80.7
83.2
69.6
83.0
83.8
73.0
$5.8
85.0
78.9
90.4
87.3
bay window
6.8
10.7
63.8
5.3
63.4
4.6
7.4
62.0
11.4
16.9
67.8
35,4
42.3
83.6
blind window
73.3
86.9
84.4
69.8
87.2
80.1
70.G
86.3
81.8
74.0
89.7
82.5
80.6
92.1
87.5
opening symbol
16.3
21.5
66.5
30.1
40.1
75.1
11.3
17.0
66.7
20.7
29.9
69.2
33.1
10.9
80.7
sofa
69.4
78.1
67.4
79.0
85.3
70.4
79.4
88.7
74.0
83.5
88.6
83.99
88.8
94.5
bed
74.7
89.3
83.6
70.5
85.9
82.1
73..4
87.9
83.5
73.2
87.4
83.7
86.1
95.9
89.8
chair
91.1
74.0
81.5
900.7
68.5
76.5
89.6
75.9
82.77
91-7
82.7
88.9
93.1
table
55.4
66.1
83.7
53.3
G4.4
82.8
49.3
59.9
82.4
60.2
70.2
85.7
70.9
79.1
89.6
TV cabinet
72.7
57.1
53.5
65.5
82.8
79.2
72.6
88.
82.4
80.1
92.9
86.3
90.1
97.0
92.9
Wardrobe
78.3
901.3
51.9
79.8
80.3
94.7
84.8
83.0
95.4
87.0
87-7
96.4
90.9
cabinet
59.1
69.7
818
63.6
75.9
83.7
63.2
75.3
83.9
67.5
80.5
83.9
73.8
86,2
85.6
gas stove
96.4
98.88
97.6
95.2
98.9
96.3
97.1
988.9
98..2
97.5
98L.
98..2
97.6
98.9
98.7
sink
89.9
83.3
91.8
90.8
86.1
92.9
92.7
refrigerstor
75.9
91.2
83.2
73.9
90.7
81.5
75.1
91.4
82.2
79.4
9-1.0
8-1.5
87.8
95.7
91.8
airoouditioner
68.6
75.9
90.4
G7.8
77.7
87.3
66.7
75.3
88.6
73.7
S0.1
92.0
80.5
84.4
95.4
bath
57.2
75.0
76.3
75.9
60.7
77.7
78.1
64.6
81.7
79.1
73.2
85.0
86.1
bath tub
62.7
78.8
79.6
57.4
78.3
73.2
61.6
80.3
76.7
65.0
82.3
79.D
76.1
91.4
83.2
washing machine
74.0
86.3
85.8
73.4
88.8
82.6
78.1
90.3
86.5
78.2
90.7
86.2
86.7
93.8
92.5
urinal
94.7
89.5
9-1.
91.9
91.4
96.0
95.2
93.8
96.7
96.9
squal toilet
88.8
95.7
92.7
90.2
96.6
93.4
89.6
96.0
93.4
90.4
96.8
93.3
93.6
97.5
96.1
toilet
86.5
91.11
91.6
88.4
96.0
92.1
$8.8
96.4
92.1
90.0
96.0
93.7
92.9
97.2
95.6
stairs
61.3
77.8
78.8
60.9
77.9
78.1
61.4
79.1
77.6
64.9
82.1
79.1
72.5
85.3
85.0
elevator
79.8
89.6
89.0
75.8
89.1
85.2
76.4
87.9
86.9
81.5
91.3
89.2
88.8
94.4
94.1
rscalator
J3.9
48.1
70.6
35.4
49.6
71.4
33.6
50.0
67.2
51.4
68.5
75.0
60.6
75.6
80.2
row chairs
85.1
90.8
93.8
80.2
86.6
92.7
84.6
90.6
93.3
83.9
89.9
93.4
84.3
89.2
94.5
parking spot
65.8
80.2
82.1
65.3
81.4
80.2
68.9
85.7
80.4
70.1
84.5
83.3
73.4
86.7
81.7
wall
35.7
55.5
64.4
31.7
51.1
62.2
41.6
64.6
64.4
38.5
50.4
64.9
53.5
77.5
69.0
curtain wall
30.1
41.6
72.3
34.4
46.2
74.4
31.5
44.2
71.2
33.8
46.1
73.3
44.2
60.2
73.5
railing
20.7
29.0
71.4
16.1
23.2
69.5
28.1
37.7
74.5
29.7
40.9
72.5
53.0
66.3
80.0
total
73.1
83.3
87.7
72.7
82.9
87.6
74.3
85.8
86.6
77.3
87.1
88.7
83.3
91.1
91.4
Table Quantitative results for panoptic symbol spotting each class. In the test split, some classes have limited number of instances, resulting in zeros and notably low values in the results. A: Baseline. B: Baseline+AAMM. C: Baselin=-ACM+CLL. D: Baselnn+
E:
Final setting + long training epoch.
Baseline
ACM
CCL
KInter
PQ
RQ
SQ
V
62.1
75.3
82.4
4
64.9
76.1
85.3
/
66.7
77.3
86.4
62.5
74.3
84.1
Table 6: Ablation Stuides on different techniques in private dataset.
valid connections. We do not use color information in the floorCAD dataset because their color information not consistent for the same category while ours consistent. Tt can be seen that applying ACM does not lead to decline in performance. In fact, there's an approximate 3% improvement in the PQ.
Vector Graphics Recognition Datasct. Similar (Jiang et al., 2021; Shi et al., 2022), we evaluate our method on SESYD, a public dataset comprising various types of vector graphic documents. This database is equipped with object detection ground truth. For our experiments, we specifically focused on the floorplans and diagrams collections. The results are presented in Tab. 7 We achieved results on par with YOLaT(Jiang et al., 2021) and RendNet(Shi et al,, 2022), which are specifically tailored for detection tasks. The aforementioned results further underscore the robust generalizability of our method. Some comparison visualized results with YOLaT are shown in Fig.
14

---

Published as a conference paper at ICLR 2024
Methods
AP50
AP75
mAP
Methods
AP50
AP75
mAP
Yolov4
93.04
87.48
79.59
Yolov4
88.71
84.65
76.28
YOLaT
98.83
94.65
90.59
YOLaT
96.63
94.89
89.67
Ret
98.70
98.25
91.37
RendNet
SymPoint
96.79
95.63
91.01
SymPoint
97.0 94.51
90.26
(a) Performance comparison on floorplans.
(b) Performance comparison on diagrams.
Table 7: Performance comparison on floorplans and diagrams.
A.4 ADDITIONAL QUALITATIVE EVALUATIONS
The results of additional cases are visually represented in this section, you can zoom in on each picture to capture more details, primitives belonging different classes represented in distinct colors. The color representations for each category can be referenced in Fig. 8. Some visualized results are shown in Fig. Fig. 10 and Fig. 11.
15

---

Published as a conference paper at ICLR 2024
prediction
(d) original point mask
0
O
0
(b) bilinear interp (down 4x)
KNNN interp (down 4x)
(c) bilinear interp (down 16x)
(f) KNN interp (down 16x)
Figure 6: KNN interp vs bilinear interp.
16

---

Published as a conference paper at ICLR 2024
Figure 7: Qualitative comparison on floorplans and diagrams with YOLaT. The left column displays YOLaT's results, while the right column showcases ours.
17

---

Published as a conference paper at ICLR 2024
single door
double door
sliding door
folding door
revolving door
rolling door
Doors
Windows
window
opening symbol
window
window
Stairs
stairs
Home
refrigerato
washing machine
gus stove
Appliances
sofa
chair
table
Furnitures
TV cabinet
sircondit:
bath
bath tub
squat toilet
urinal
toilet
Equipments
elevator
escalator
Stuff
parking
curtain
railing
spot
wall
Figure 8: visualized color map is provided for each class along with its corresponding super-class.
18

---

Published as a conference paper at ICLR 2024
(a) GT
(b) SymPoint
Figure 9: Results of SymPoint FloorPlanCAD.
19

---

Published as a conference paper at ICLR 2024
(a) GT
(b) SymPoint
Figure 10: Results of SymPoint FloorPlanCAD.
20

---

Published as a conference paper at ICLR 2024
(a) GT
(b) SymPoint
Figure 11: Results of SymPoint FloorPlanCAD.
21


Reviews:
[{"review_id": "QG70pyNIRX", "reviewer": "Reviewer_u21w", "summary": "In this paper, a method for symbol spotting from CAD vector graphics (VG), called SymPoint, is proposed. SymPoint treats graphic primitives as a set of 2D points. Two strategies, attention with connection module (ACM) and contrastive connection learning (CCL), are devised to better utilize the local connection information of primitives and enhance their discriminability.", "questions": "The authors should explain and verify the originality and technical contribution of the proposed method.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The main idea and technical detailed are clearly presented.", "weaknesses": "1. The originality and technical contribution of this work is quite limited. Point Transformer, Mask2Former and InfoNCE are all well-established methods or models.\n2. The potential application range of the proposed method can be narrow (CAD vector graphics), because it is unclear whether the idea and techniques presented in this work can be extend ed to other tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3WiyPAEUIL", "reviewer": "Reviewer_AtGi", "summary": "This paper, titled SymPoint, advocates for representing a symbol as a point and extends the previous methodology to encompass a broader range of symbol properties. The Point Transformer serves as the foundational feature extraction tool. Mask attention and a contrastive connectivity learning mechanism are integrated into the panoptic symbol spotting task, aiming to cultivate rich features that can effectively differentiate between graphic primitives. The PQ performance has been elevated from the previous method to a novel tier, as delineated in the experimental section.", "questions": "See the raised concens in Weaknesses section", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "With the advancements in point cloud processing and the Transformer architecture, the authors suggest leveraging these powerful backbones from other domains and adapting them to address the challenge of panoptic symbol spotting.\nA suite of techniques, encompassing vector graphics representations, Point Transformers, Masked Attention, Contrastive Connection Learning, and KNN Interpolation, has been integrated into the targeted task.\nExperimental outcomes reveal that SymPoint significantly outperforms existing methods, exhibiting a considerable advantage in Semantic Symbol Spotting, Instance Symbol Spotting, and Panoptic Symbol Spotting.", "weaknesses": "- A primary concern from the reviewer centers on the paper's predominant reliance on existing methodologies to address the issue. Specifically, in Sec3.1 (From Symbol to Point), many parameterizations echo those found in FloorplanCAD, albeit this paper seeks to enhance the diversity of encoded features. The point-based representation in Sec 3.2 directly employs the Point Transformer, reminiscent of CADTransformer. Both Contrastive Connection Learning (Sec3.4) and KNN Interpolation (Sec3.5) have been thoroughly examined in other scholarly works. While the \"Attention with Connection Module\" presents as novel to the reviewer, it would be beneficial to undertake a comprehensive review to discern if analogous concepts have been previous literature.\n- In Table 4, where the benchmark approach registers a PQ of 73.1, could you detail the design of how this baseline method is formulated?\n- In Table 4, it appears the newly introduced \"ACM\" module inadvertently undermines performance. Could the authors shed light on the causative factors behind this decline?\n- Again, referencing Table 4, the KInter technique emerges as a salient contributor to performance enhancement. Could the authors offer a more clear explanation and visualization? It might also be worthwhile to highlight this module within the methods section.\n- As the proposed framework incorporate a bunch of techniques for a specific application, did you submit the code for reviewing?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aio8v0FUxr", "reviewer": "Reviewer_q1T8", "summary": "The paper proposes a new method for symbol segmentation in architectural floorplans. The method is based on representing each graphical primitive as a point with a set of features and thus, relying on Point Transformer for feature extraction. Then, an adaptation of Mask2Former is used to segment and classify the symbols in the floorplan. Some specific components are introduced to adress the specificity of graphical primitives in architectural drawings. Experimental validation is performed by applying the method to a standard floorplan dataset, comparing with state-of-the-art and conducting several ablation studies.", "questions": "See above in Weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The idea of relying on a point representation of graphical primitives seems novel and makes sense in this context since symbols in architectural drawings are composed of graphical primitives. Then, using the combination of PointTransformer and Mask2Former is also a novel approach in this context that seems suitable for capturing the interaction between graphical primitives for symbol segmentation. \n- Experimental results on a standard dataset report better performance than state-of-the-art methods. A detailed ablation study shows the contribution of the different modules of the proposed framework.", "weaknesses": "Perhaps I missunderstood something, but I do not see the motivation of symbol segmentation in CAD drawings. As far as I understand CAD drawings should already contain information about the symbols included in the floorplan and where they are located. \n\nIn the description of the method and the experiments there are several points that are confusing or not well explained:\n- In equation (2) I understand that l_k is the distance between v_1 and v_2. Then, what about circles and ellipses? How is the lengh computed? And for arcs, this definition does not account for the curvature. Two arcs with very different curvature can have the same representation.\n- In equation (3), it is not clear how the neighbourhood M(p_i) is defined. Do adjacent points mean connected primitives? Or primitives inside a certain distance? Which is exactly the difference with A(p_i) defined later in section 3.3 (given that the threshold used in section 3.3 is just one pixel). In this sense, the role of the ACM module is not very clear. \n- It is not clear the motivation of the KNN interpolation described in section 3.5. As far as I understand, since points correspond to graphical primitives, interpolation of neighboring points could lead to losing information of specific primitives and I am not sure that makes sense merging different primitives into a new one. \n- Related to the previous point, It is not clear how it is performed downsamplind and upsampling in the Point Transformer. The same as in the original Point Transformer? \n- In equation (12) it is not clear what is e_i and L(e_i).\n- In the experiments, which is the difference between Semantic and Panoptic Symbol Segmentation? Why in table 1 (semantic segmentation) the evaluation measure is F1? How are F1 and wF1 defined in this context?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)