Title: Point or Line? Using Line-based Representation for Panoptic Symbol Spotting in CAD Drawings
Venue: NeurIPS 2025
Accepted: yes
Keywords: Panoptic Symbol Spotting

Representation Learning

CAD Drawings
Abstract: We study the task of panoptic symbol spotting, which involves identifying both individual instances of countable \textit{things} and the semantic regions of uncountable \textit{stuff} in computer-aided design (CAD) drawings composed of vector graphical primitives.
Existing methods typically rely on image rasterization, graph construction, or point-based representation, but these approaches often suffer from high computational costs, limited generality, and loss of geometric structural information. In this paper, we propose \textit{VecFormer}, a novel method that addresses these challenges through \textit{line-based representation} of primitives. This design preserves the geometric continuity of the original primitive, enabling more accurate shape representation while maintaining a computation-friendly structure, making it well-suited for vector graphic understanding tasks. To further enhance prediction reliability, we introduce a \textit{Branch Fusion Refinement} module that effectively integrates instance and semantic predictions, resolving their inconsistencies for more coherent panoptic outputs. Extensive experiments demonstrate that our method establishes a new state-of-the-art, achieving 91.1 PQ, with Stuff-PQ improved by 9.6 and 21.2 points over the second-best results under settings with and without prior information, respectively—highlighting the strong potential of line-based representation as a foundation for vector graphic understanding.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper is about panoptic symbol spotting, which can be seen as instance/panoptic segmentation on vector graphics. The paper proposed a line-based representation by discretizing vector graphics into a set of line segments. Then an instant/panoptic segmentation method is applied.\n\nSTRENGTHS AND WEAKNESSES:\nBoth the writing and figures are clear to illustrate the core idea. However, I have the following concerns:\n\n1. The method looks like a combination of vector graphics discretization and panoptic segmentation. For the discretization, the authors claim a novel line representation is proposed. However, to me, this is still point based which is similar to prior work. It is just people might be using different pooling methods for the handcrafted features (despite Fig 4 in the appendix).\n\n2. To prove the effectiveness of the proposed representation, I believe the authors need a better experimental setup. For example, the same network backbone (similar number of parameters), the same panoptic segmentation method, ... so that we can ablate the proposed representation does work.\n\n3. I do not know this field very well. However, I checked the competitor methods shown in this paper. They all followed a similar way (see 1). I do not believe the paper has enough novelty. The paper lacks conceptual depth and is quite incremental compared to other similar papers.\n\n4. The large-scale CAD dataset proposed by CADSpotting is missing in the paper. It might be important to test the scaling and generalization ability of the method.\n\nQUESTIONS:\nNone\n\nLIMITATIONS:\nThe claim \"but these approaches often suffer from high computational costs, limited generality, and loss of geometric structural information\" in the abstract is not well-supported. If they are in the abstract which means they are important, they should be emphasized somewhere.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles the problem of classification and counting of vector graphic CAD drawings to classify and enumerate (or group) the curve primitives within. The work uses a transformer architecture as well as other enchancements, e.g., LFE and BFR, to achieve state-of-the-art performance on the FloorPlanCAD dataset. The contribution is primarily just this: the new architecture and improved performance for this specific task. The tools used for the method, e.g., line-based features and the transformer architecture seem to be relatively natural generalizations and applications of existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper provides state-of-the-art performance on a relevant dataset.\n2. A novel architecture and set of features for the specific problem are used.\n3. Competing methods have been published at ICCV and other top-tier conferences, so while the problem seems a bit niche, it has warranted publication in such venues.\n\nWeaknesses:\n1. It will be hard to reproduce the method without a link to code or a repo. \n2. I found the presentation of the problem at hand to be unclear. It took me until the Experimental Results section to understand what semantic and instance labels would be. There are also other clarity issues with respect to basic definitions for readers unfamiliar with the Panoptic Symbol Spotting problem. See \"Questions\" below.\n3. I am not an expert in deep learning architectures and their application in this problem domain, but it's unclear to me if insights in this work will extend to other problems.\n\nQUESTIONS:\nThese are mostly clarity questions, that I think could be easily addressed with some small bits of text.\n\n1. Line 128: Please clarify the types of primitives. Is a \"line\" primitive necessarily a straight line segment, or could it be a Bezier curve? Later on \"line\" is used for line segments sampled from primitives, so I wasn't sure. Are there other types of primitives, e.g., fill regions or polygons?\n2. Line 141: Where is PTv3 used? The section outlines features that are explicitly given by formulae and then pooling. Are these then fed into PTv3? This seems to conflict with the diagram in Figure 2, where the transformer backbone precedes the pooling of line features.\n3. Line 155: What are the layer IDs? Does each one relate to the semantic label? Why does it need to be normalized?\n4. Eq (5): Is there a reason you explicitly have the first 5 entries instead of just putting the 2 endpoints of the line segment? They could be derived from just these 2, and perhaps it would lead to a more compact model (or not, who knows). Maybe an ablation would be interesting.\n5. Section 3.2.3: Could this section be expanded to be a bit more explicit, at the cost of compressing 3.2.1? The formulae in 3.2.1 are relatively obvious and could be relegated to an appendix, in my opinion.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper targets panoptic symbol spotting task which is a challenging task in the domain of vector drawing analysis aiming to predict semantic and instant labels for countable and amorphous symbols in the input vector drawing. To address this task, the authors propose to enrich the input representation available to the deep learning approaches by approximating all input primitives by sequences of straight lines. This enables a more complete input representation that pairs well with the modern deep learning architectures such as transformers. The authors then use this input representation to feed into a point transformer model and refined using a conventional refinement strategy to consolidate semantic and instance predictions and confidence scores. A thorough evaluation is performed on the FloorPlanCAD dataset that shows how the proposed approach compares both regarding the baselines and regarding various design choices.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The effectiveness of the proposed input representation is clear and compelling. I also appreciate that the authors provides the districts on the number of samples required to represent and given input drawing with various representations and how their method is more economical compared to the baseline.\n2. The effectiveness of the series of design stages in the proposed approach is demonstrated in the compelling way. \n3. The experimental validation is reasonable compelling and shows the advantages of the proposed approach quite clearly. \n4. The paper is sufficiently verbose with respect to the design of the method the experiments conducted and the outcomes. I believe that all the necessary information to reproduce the findings of this paper is presented.\n\nWeaknesses:\n1. The only major weakness that I see at this point is the performance drop from not including the Branch Fusion Refinement (BFR) strategy. This begs the question whether including branch fusion refinement in the baseline approaches would also improve their performance?\n\nQUESTIONS:\nThe only concern I have right now regarding the experimental evaluation is regarding the use of the BFR strategy used to refine results produced by the raw method. I think that the most compelling way to rebut that would be to conduct an experiment with one of the baseline methods and basically refine its results also using the BFR strategy. I do not know if this is an easy task or if it makes total sense to executive this comparison and exactly this way.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents VecFormer, a novel method for panoptic symbol spotting in CAD drawings by introducing a line-based representation instead of conventional point-based or rasterized representations. The key idea is to preserve geometric continuity and structural integrity by approximating primitives with sequences of line segments. The authors design a Transformer-based framework that leverages this representation, combined with a dual-branch decoder for instance and semantic prediction and a lightweight Branch Fusion Refinement module to improve label coherence. Extensive experiments on the FloorPlanCAD dataset demonstrate that VecFormer achieves new state-of-the-art results in terms of Panoptic Qualit and Stuff-PQ, with clear improvements over existing methods such as SymPoint-V2 and CADSpotting.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nOriginality: The proposed line-based representation is conceptually simple yet effective, addressing key limitations of point-based methods.\n\nTechnical Quality: The paper provides solid methodological details, including clear mathematical formulations, ablation studies, and implementation settings. The BFR module is a thoughtful addition to improve panoptic consistency.\n\nSignificance: The improvement on Stuff-PQ is particularly noteworthy, given that this aspect is challenging for prior methods. The approach is well motivated for practical CAD applications.\n\nClarity: The paper is generally clear, with good figures and detailed appendices for qualitative results and additional analyses.\n\nWeaknesses\n\nWhile the line-based representation is well motivated, its potential limitations are only briefly mentioned and not empirically examined.\n\nThe experiments focus solely on the FloorPlanCAD benchmark. The generalizability of VecFormer to other vector graphic domains is claimed but not tested.\n\nThe paper does not provide statistical significance analysis for the reported results.\n\nThe BFR refinement is post-processing but could potentially be integrated more tightly within the model to allow end-to-end training, which is not discussed.\n\nQUESTIONS:\n(1) On broader generalization: Have the authors considered validating VecFormer on other vector graphic tasks or datasets to strengthen the generalizability claim? Even small-scale tests (e.g., technical drawings, maps) would help.\n(2) Sampling strategy: Could the authors provide empirical evidence comparing uniform sampling with an adaptive or curvature-aware sampling strategy? This is important given the stated limitation.\n(3) End-to-end refinement: Is it possible to integrate the Branch Fusion Refinement module as a learnable layer within the model instead of using it purely as post-processing? A brief discussion or experiment would improve the contribution.\n(4) Reproducibility: Will the authors release the complete training and inference code along with pretrained models? This is essential for verifying the impressive gains.\n(5) Statistical robustness: Could the authors provide standard deviations or error bars for main metrics over multiple seeds to support the robustness of the results?\n\nLIMITATIONS:\nYes, the paper discusses a key limitation of uniform line sampling and suggests future directions to address it. However, the impact of this limitation is not empirically shown. I encourage the authors to either visualize failure cases or provide additional experiments to clarify when uniform sampling might fail.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}