Paper: Leveraging Graph Neural Networks to Boost Fine-Grained Image Classification (ICLR-2024)

Reviewer comment (Novelty):
The compared methods and related work discussion are both very insufficient. In this work, the comparison and references are mainly from typical visual backbones. In both the introduction and related work sections, more recent FGVC works from the past five years, especially from top-tier conferences such as CVPR and top-tier journals such as PAMI and TIP, should be included and discussed. Besides, more comparisons should be made.

Author response:
We appreciate your suggestion. We will strive to do better in the next submissions.

Reviewer comment (Presentation):
The visualization is not good. The activation map is common for generic image recognition such as on ImageNet. The visualization should follow the conventions in FGVC tasks, to activate the fine-grained patterns, for example by using GradCAM. The reviewer also suggests referring to:  
[a] Destruction and construction learning for fine-grained image recognition. CVPR 2019.  
[b] Counterfactual attention learning for fine-grained visual categorization and re-identification. ICCV 2021.  
[c] Fine-grained object classification via self-supervised pose alignment. CVPR 2022.  
[d] Transfg: A transformer architecture for fine-grained recognition. AAAI 2022.

Author response:
Thank you for your suggestion. We will consider it for future submissions.

Reviewer comment (Theory):
Another aspect that is not clear is the choice of using GNNs to aggregate information from different images in the graph. The method proposed in the paper uses a fully connected graph, so there isn’t really a structure that the GNN can exploit. All the GNN is doing is aggregating features from a set of images without any relevant structural information, so why not use something like DeepSets [1] or attention?

Author response:
We acknowledge the omission of a key advantage of using GNNs over attention mechanisms in our previous response. GNNs have the distinct advantage of preserving the original image features while attention mechanisms only focus on the features of other images in the batch. This preservation of original image features is crucial for fine-grained image classification tasks, as it allows the model to retain the essential details that distinguish between fine-grained categories.

Reviewer comment (Theory):
My main concern is the lack of motivation for using a graph-based approach. The manuscript in its current version lacks any clear intuition for why a graph neural network on top of would improve performance in such a consistent way for specifically fine-grained image classification.

Author response:
We sincerely apologize for the omission of our motivation in the initial manuscript. Our motivation for employing a graph-based approach for feature refinement lies in the inherent challenges posed by fine-grained image classification. Conventional image classification methods, which process each image independently, often struggle to effectively capture the subtle and intricate details that distinguish between fine-grained categories. To overcome this limitation, we propose a graph-based approach that explicitly models the relationships between images within a batch, enabling the model to leverage the collective knowledge of the batch to refine feature representations and enhance classification accuracy.

Reviewer comment (Reproducibility):
How do you combine the GNN and DNN features? Is it different based on the GNN method you're using? I'm not sure how to interpret eq 2 in this regard. You only mention you "combine" the features, not how you go about this.

Author response:
In Equation 2, the combine function represents the skip connection that merges the outputs of the deep neural network (DNN) and the graph neural network (GNN) in Figure 1.

Reviewer comment (Evaluation):
What is your interpretation of the results shown for different GNN architectures in 4.2.1? Is there a clear reason you see for GraphTransformer outperforming conventional attention? Are these two approaches not identical in the case of fully connected graphs?

Author response:
- The experimental results presented in Section 4.2.1 demonstrate that incorporating various graph neural network (GNN) architectures into the proposed framework effectively enhances the performance of the deep neural network (DNN) in fine-grained image classification tasks. 
- While the GraphTransformer architecture does not consistently outperform all other GNNs across various datasets, it achieves state-of-the-art performance on the Stanford Dogs dataset. This observation suggests that the GraphTransformer architecture is particularly well-suited for capturing the intricate relationships between images in the Stanford Dogs dataset, leading to superior classification accuracy.
- The two approaches share similarities in their ability to focus on relevant features and suppress irrelevant ones