Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper addresses the problem of object recognition, i.e., object localization and classification, in Vector Graphics.
The input is a text document that describes the vector graphics, and the output is a bounding box corresponding to a localized object.

The proposed approach is supervised.

The main idea is to represent the input textual document describing the Vector Graphic (VG) object using a unified representation for different primitives described in the text document (such as a Bezier curve). The VG objects are represented using a Graph, where the nodes correspond to the start and end points of the Bezier curves.

A dual-stream Graph Neural Network (GNN) is proposed to aggregate feature representation for the input VG object, which is in-turn used for classification and localization.

The method is evaluated using Average Precision metric. 

Two kinds of publicly available VG datasets are used: Floorplan and Diagrams.
Comparison against object recognition techniques that employ a CNN has been done, and the efficiency
of dual-stream GNN architecture is demonstrated.
main_review: Originality: 

I think the paper is addressing an interesting problem of object recognition in Vector Graphics input. The proposed idea, especially extracting the textual data into Bezier curves, is good. I have one major question here:
1) The VG input is in text format. How do you extract Bezier curves from there? In other words, how do you parse the input textual document to understand what kind of primitives are described in there?

There needs to be more clarity about the "Format Unification" step shown in figure 2.

Quality: 

The proposed method has shown to significantly outperform object recognition methods that employ a CNN.
Since no CNN is made use of in the proposed method, the model is relatively light-weight and yet, efficient.
In terms of writing and mathematical formulation, the paper is reasonably written. 

Clarity:

The flow of the paper is easy to follow. I would suggest the following to help improve the exposition of the paper even more:
State clearly what is the input. I think this is where the readers need to be educated. Let them know what VG format is and how is it described in general. Also describe how such a textual document of VG is parsed into a data structure that is fed to the employed neural network.
Explain how the proposal generator works. I am still not very clear about the proposal generator. I think Section 4.3 can be broken down in a better way. Perhaps start of with a different paragraph about proposal generators and then describe the training loss.
And finally, be specific about the representation of the output.


Significance:

The problem being addressed is important. Dependence on raster formats of data is both computationally heavy and resolution-constrained. Computational methods that can operate on VG data and perform visual recognition tasks as that of raster form of data are much needed and perhaps will become dominant in the next few years.
limitations_and_societal_impact: Yes, the authors have adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this paper, the authors proposed one framework for the object detection in the vector graphics instead of raster graphics (i.e., pixel images), where the main contribution maybe is to employ the graph neural networks in this area. The extensive experiments on two datasets show the effectiveness of this method, especially comparing with the methods on raster graphics. However, I do not think this comparison is fair, since the proposed methods employ more information contained in the vector graphics while other methods only use pixel information.
main_review: The work in this paper is for the vector graphics, which is interesting. One problem is that how to utilize the detailed information in the vector graphics. 

In this paper, the authors transforms all the primitives into a unified format, where the Bézier Curve is used, then graph neural networks are employed for the object detection. The experimental results are good, and I think this work is significant for the research on vector graphics.

However, some details are better to give:

(1) In the Eq. (4), based on my understanding, there are several points P between the two points of this edge, so the x^e is obtained by concatenating all of these points?

(2) It is better to give more details on how to get the regional cluster C_k, since this plays an important role in the following steps.

(3) In the Eq. (7), are all of the z_i in the same cluster C_k the same? My understanding is that neighbors in the same cluster are the same (i.e., N_i^p) because they are dense connections. Please correct me if my understanding is wrong.

(4) In the table 3b, it is not clear that the aggregation functions (last three rows) are replaced by three popular GNN methods in stroke-wise stream, or position-wise stream, or both. Moreover, why the performance of three popular GNN aggregation methods are reduced largely compared to the proposed two aggregation functions? BTW, in the second part (position-wise stream), I guess the h_j - h_i may be z_j - z_i because there is no variable h in this stream.

(5) In the proposal generation, the authors mentioned that "permute all vertex pairs on the grid mesh ... rectange region". does it mean every two nodes in the C_k will get one rectangle region? if it does, there will be a large number of rectangle regions.

limitations_and_societal_impact: I do not find any negative societal impact, and the suggestion are in the box of Main review.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper present a method to perform object detection for vector graphics images directly from their description as order three Bézier curves. It proceeds as follows:
1. it convert a vector graphics object as a set of order 3 Bézier cuvres
2. it looks at this set of curves as a graph
3. it process this graph using a GNN
4. it aggregates the features of the GNN on proposal regions, generated form the vector description of the image and classify them.

The method is evaluated on two public datasets and compared to standard CNN baselines on the rendered images.
main_review: I don't know of any other paper performing object detection in vector images directly from the vector description. If this is indeed true, I think it gives the paper enough novelty to be accepted despite the weaknesses listed bellow.

Weaknesses:
1. I would like a clear ablation showing the performance of CNN classifers (ResNet/Fast-RCNN) on the proposal extracted using the method from the paper + the proposed GNN using proposals from another method. Right now, it's unclear how much of the boost is coming from better proposals.
2. I found the technical part of the paper (part 4) very hard to parse. Notations could likely be improved, showing them on a picture might also help. 
3. Beyond readability, I felt many of the choices were kind of arbitrary, even if several are justified by ablations. Assuming this is the first paper in this area, I think it's ok even if not ideal.
4. A transformer baseline/comparison would be very natural considering the structure of the data.
5. Why not perform segmentation instead of detection? To me it would be much more natural considering the data
6. It would be nice to include a quick literature review on vectorization, e.g. based on the related work of "Shen, I. C., & Chen, B. Y. (2021). ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis. IEEE Transactions on Visualization and Computer Graphics." or "Parakkat, A. D., Cani, M. P. R., & Singh, K. (2021, May). Color by numbers: Interactive structuring and vectorization of sketch imagery. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-11)."

misc. 
- l 155 "an cubic"
- l 161-164 especially unclear
- section 3 seems completely useless to me
limitations_and_societal_impact: limitations: it would be nice to discuss more the influence of the way the vector graphics was generated on the resulting Bézier curve-based representation as well as the errors made in the conversion.
societal impact: ok
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: Update at the end of the discussion session: 
Discussion among reviewers and with the authors made me increase my score by a total of 2 points. 

The paper presents a graph-neural network based approach to detecting objects in vector graphics. The paper works directly on the vector graphics representation by first normalizing the vector graphics to bezier splines and then building a graph representation of the vector graphics. Object detection happens directly on this graph representation. Experiments show a comparison to standard computer vision models. 
main_review: ## Further updates after more discussion: 

increased score to "above decision threshold" assuming: 
 - title is changed
 - clarification about baselines not being strong baselines

## Updated review following on discussion with other reviewers and authors: 

Review up'd by 2 points because: 
positive:
 - it's an important topic
 - the model is interesting

negative: 
 - experiment baselines are lacking
 - online handwriting / drawing recognition work is relevant
 - the title is misleading

The paper presents an application of GNNs to detecting objects in vector graphics. 


REFERENCES AND RELATED WORK
While working on vector graphics is not a very common research domain at the moment, the authors do not refer to the related work that has been looking at vector graphics, e.g. 

[1] A Learned Representation for Scalable Vector Graphics https://arxiv.org/abs/1904.02632

further, the entire domain of online handwriting and drawing recognition also handles vector graphics, in in particular is often using a very similar representation: 

[2] SketchRNN  - https://arxiv.org/abs/1704.03477
[3] CoSE: Compositional Stroke Embeddings - NeurIPS 2020
[4] Fast Multi-language LSTM-based Online Handwriting Recognition - is using a Bezier representation - similar to what is proposed here for handling (vector graphics of) handwriting
[5] BézierSketch: A generative model for scalable vector sketches - ECCV 2020 https://arxiv.org/abs/2007.02190

Further, in the domain of architectural drawings there was: 
[5] PlanIT: Planning and Instantiating Indoor Scenes with Relation Graph and Spatial Prior Networks - https://dl.acm.org/doi/pdf/10.1145/3306346.3322941


Beyond that, I feel that the paper title and representation of the work is misleading: Just looking at the text would in my mind imagine just looking at the raw SVG file - however, as I am reading the paper it becomes clear that there is a lot of specialized preprocessing to normalize the vector graphics and create the graph. So, "only look at text" - is clearly not a good representation of the paper. 


TECHNICAL
in section 4, is a description of how the graph is build which includes a step in which the input drawing is converted to bezier curves. It is not clear however, how this would work in the case of a very long polyline or e.g. for a circle which are both not easily represented as a Bezier curve. It feels like for this the shape need to be split into multiple bezier splines - but it is unclear from the paper how that is done. 


EXPERIMENTAL EVALUATION
 - it is not surprising that a computer vision model that was pre-trained on very different data and then fine tuned on ~600 images doesn't perform particularly well on objects that are mostly defined by their outlines rather than by their textures. 


MINOR ASPECTS
 - lots of abbreviations are badly capitalized throughout the paper, e.g. "Yolov3" instead of YOLOv3, "faster-rcnn",...
 - similarly the references are badly formatted
 - Figure 2 uses an image of a dog as example - I feel it would have been good to use an image (or object) of the type that is used in the evaluation later for better understanding. 


limitations_and_societal_impact: I personally don't think the evaluation in the paper is sufficient to understand limitations. Experiments on two datasets of 1000 images of unknown difficulty don't give a good idea of what the limitations of the method are.
review_title: null

Meta-review: The reviewers have extensively discussed the paper with the authors and came to the conclusion that the paper studies an interesting and important problem. The reviewers have agreed to support the acceptance of the paper if some of the changes are incorporated in the final version. Specifically, the title should be changed to be more precise and less misleading, and there should be more discussion on the feasibility of potential baselines (including the ones coming from handwriting recognition).
Overall, I agree with the reviewers and recommend acceptance of the paper but I would like to encourage the authors to carefully incorporate all the raised concerns and suggestions to improve the paper.
Decision: Accept (Poster)