Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an end-to-end part-based deep object parsing model for fine-grained few-shot classification. It learns a compact dictionary of salient part templates and performs multi-scale template matching during testing. Different from traditional part-based works, this paper uses feature map reconstruction as supervision to localize object parts and imposes a geometric prior on part locations during matching. This work provides good model interpretability and is competitive with very recent state-of-the-arts on several popular few-shot classification datasets. 

strength_and_weaknesses: Strength: 
* The proposed model is technically sound and effective, and it's end-to-end trainable. It successfully combines the advantages of good interpretability in traditional part-based models and nice performance improvement brought by deep networks and modules.
* Generally speaking, the presentation is clear and the experiments are sound.

Weakness:
* The math formulas in Sec 3 are a bit difficult to understand. It's better to explain the variables more clearly immediately before each formula.
* Possible typos in math formulas:
  * In the 3rd paragraph of sec 3.1, z_{s,p}^y should be divided by M because it represents the mean part expression?
  * In the alpha(...) function in Formula (7), s1 and s2 should be swapped.  
* It would be nice to add more detailed comparison with most recent deep part-based models (e.g. [1], [2]) and show this work's novelty and strength.
* It's better to explain why the proposed method surpasses other methods considerably in some datasets (e.g. the Aircraft dataset) while only on part with other methods on some other datasets (e.g. the CUB dataset).

[1]: Few-Shot Classification with Feature Map Reconstruction Networks. CVPR 2021

[2]: Task Discrepancy Maximization for Fine-grained Few-Shot Classification. CVPR 2022 

clarity,_quality,_novelty_and_reproducibility: Generally speaking, the paper is in good quality and the presentation is quite clear. It's somewhat novel that the paper integrates recent effective deep modules (e.g. attention and feature map reconstruction) into traditional part-based models.

The paper hasn't released their code yet, so it's a bit hard for others to reproduce their work.

  


summary_of_the_review: I prefer to give a borderline accept score to the paper given that it clearly presents and provides a state-of-the-art part-based few-shot object classification method with good interpretability though some of its modules exist in previous works and some typos exist in the paper. It would be nice for the authors to make their code public to increase the potential impact of the work.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose a method for fine-grained few-shot recognition by finding a dictionary of K parts to represent objects. The dictionary is shared across all instances and categories. An object is represented by where are these K parts and a set of templates that reconstruct the part features. The method achieves competitive performance compared to SoTA methods.
strength_and_weaknesses: Strength:
- The method is end-to-end trainable.
- An object instance is represented as a collection of parts with locations, which is somewhat interpretable.
- The method achieves SoTA performance on some datasets.

Weaknesses:
- The novelty is on the weak side since there are previous methods that also parse objects into different salient parts for object recognition [2] [3]. The introduction doesn't discuss these methods and it's hard to clearly see the main contribution of the work. The authors write the following at the end of the related work: 
"We differ in our use of a finite dictionary of templates used to learn a compact representation of parts. Also, we use reconstruction as supervision for accurately localizing salient object parts, and impose a meaningful prior on the geometry of parts, which keeps us from degenerate solutions for part locations." From there, I see there are two main points: - a finite dictionary of templates and - using feature reconstruction. The justification here is weak and there is no analysis of these two points.
[2] Task-aware part mining network for few-shot learning
[3]  Collect and select: Semantic alignment metric learning for few-shot learning

- The motivation is not very convincing to me. I think a CNN automatically parses object instances into different parts [1] where each feature might respond to a different discriminative object part. In fact, here the authors also use a pre-trained network. Thus the main contribution here is to explicitly represent them as a fixed collection of K parts rather than directly using the implicit collection of parts learned from the deep network. I am not entirely sure why this representation is more robust than just using the CNN features. Does it come from the explicit locations of parts or is it because the method can extract the top K most informative and discriminative parts? In general, I think I didn't get the answer to "why does this method work?" from a high-level perspective. The connection between motivation and fine-grained few-shot learning is also not very obvious. Should this method work also on the standard setting of fine-grained classification? 

[1] Visualizing and Understanding Convolutional Networks

- Parsing objects into a fixed set of parts seems very challenging and requires several technical components involved, e.g., setting the right value of K, using multiple scales, or instance-dependent re-weighting. These components only address the challenges specifically posed in this particular modeling. The solution is quite heavily engineered.


clarity,_quality,_novelty_and_reproducibility: Clarity: Good. The paper is easy to follow.
Quality: Good.
Novelty: Borderline.
Reproducibility: Borderline.
summary_of_the_review: My main concern is the novelty and unclear significance of the overall contribution.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper studied fine-grained image recognition under few-shot settings. The authors proposed to learn a dictionary of parts using object parsing first and then train a few-shot learner based on the parsed features. Learning a structured aligned representation is the right way for fine-grained few-shot learning. The high intra-class variations enlarge with the limited labeled samples in each category, which is more challenging than traditional fine-grained image recognition. It is interesting that the paper proposes to learn parsed object features from the perspective of dictionary learning.
strength_and_weaknesses: Strength:
1. The idea of learning a dictionary of parts for fine-grained few-shot image classification is correct and reasonable. 
2. The experimental results reported validate the effectiveness of the proposed DOP, and DOP achieved state-of-the-art performance on CUB, Car, and Aircraft datasets.

Weaknesses:
1. The motivation of the paper is somewhat weak. The connections between fine-grained few-shot image classification and deep object parsing are not well illustrated, especially for the intra-class variations that will enlarge under the few-shot settings.
2. Some formulations in the paper are hard to follow. For example, on page 4, "The location corresponds to an $s \times s$ attention mask'', "$D_{p,c} \in R^{s \times s}$ '', what's the attention mask used for? what's the connection between the $G \times G$ 2D feature map slice and the $s \times s$ attention mask? "$ M(\mu) $ centered round the location $\mu$", what's the function of $M(\mu)$? 
3. As presented in the paper, the DOP needs to use DNNs to parse the input objects and generate the aligned-structured features. Is the DNNs in Algorithm 1 a pre-trained model or not? How to obtain these DNNs? If it is a pre-trained one, does the pre-trained dataset contains testing images or directly grab from pre-trained models on ImageNet?
4. As the DOP consists of two steps, it imports additional operations of object parsing using DNNs. The model complexity, total parameters and running time comparisons with other single models should be investigated.
5. In the experimental comparisons, the compared methods seem to vary a lot. On CUB, there are 15 baseline models. On Aircraft, Car, and Dog, there are only 5 to 7 baseline models. Moreover, equipped with complex deep object parsing, the DOP model achieves slight improvement on CUB, with only 0.03 and 0.21 boosts on 1-shot and 5-shot image classification over TDM. In Table 3, the accuracy of DOP lags behind the VFD by a wide margin on Dog, and more explanations or clarifications are encouraged.
6. There exist previous works [C1,C2,C3] that proposed aligning and learning the fine-grained features in an end-to-end way, which are also inspired by the template learning in fine-grained image classification. Comprehensive comparisons are suggested.
7. The visualization of the learned templated parts seems inaccurate in some correct cases in Figure 3. For example, the learned parts of Dog are not matched. In the top image, the locations are head, ear and body, while in the bottom image, the locations are nose, body and body.

C1. Local spatial alignment network for few-shot learning
C2. TOAN: Target-Oriented Alignment Network for Fine-Grained Image Categorization with Few Labeled Samples
C3. Learning Cross-Image Object Semantic Relation in Transformer for Few-Shot Fine-Grained Image Classification
clarity,_quality,_novelty_and_reproducibility: The clarity of the paper is not good, the novelty of the paper is non-trivial, and the reviewer is not very confident in reproducing the results of the DOP since there omit lots of implementation and training details.
summary_of_the_review: This paper proposed to solve the fine-grained few-shot image classification using deep object parsing. The idea is reasonable and interesting. However, the motivation of the paper is not well illustrated, and many other concerns exist, as discussed in the Strength And Weaknesses section. Therefore, the reviewer leans to reject the submission and encourages the authors to improve the paper and resubmit to other venues.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper addresses the problem of few-shot fine-grained recognition. The work is based on the generic assumption that while different object classes exhibit novel appearance, visual patterns are duplicated. Thus, the proposed method is forced to learn a dictionary of templates distributed across different relative locations. As a result an object can be recognised by identifying its corresponding parts in term of visual templates and their relation.  The whole process is expressed as a statistical model for parsing. Given a query image, the model re-weights the query and supports instances to find out the best matching class. The method is validated on 4 public datasets giving the results in terms of 5-way top-1 and 5-way top-5 accuracy.
strength_and_weaknesses: The strengths of the paper are:
1. The main assumption is logic and intuitive.
2. The statistical method is straightforward and technically sound.
3. The method also can facilitate its interpretability and explainability since express explicitly the visual dictionary and the relevant parts of the object.
4. The paper is well organised.
5. Validation is done on 4 public datasets.
6. Results are promising. 
7. Ablation study is presented.

Weaknesses:
1. State of the art should be completed and the method should be compared to the following works:
- Huang, S., Wang, X., & Tao, D. (2021). Stochastic partial swap: Enhanced model generalization and interpretability for fine-grained recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 620-629).
- Baffour, A. A., Qin, Z., Wang, Y., Qin, Z., & Choo, K. K. R. (2021). Spatial self-attention network with self-attention distillation for fine-grained image recognition. Journal of Visual Communication and Image Representation, 81, 103368.
- Zhu, H., & Koniusz, P. (2022). EASE: Unsupervised Discriminant Subspace Learning for Transductive Few-Shot Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9078-9088).
- Lazarou, M., Stathaki, T., & Avrithis, Y. (2021). Iterative label cleaning for transductive and semi-supervised few-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 8751-8760).
- Hong, J., Fang, P., Li, W., Zhang, T., Simon, C., Harandi, M., & Petersson, L. (2021). Reinforced attention for few-shot learning and beyond. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 913-923).
- Fei, N., Gao, Y., Lu, Z., & Xiang, T. (2021). Z-score normalization, hubness, and few-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 142-151), etc.
2. Looking at the papers mentioned above, it is not clear that the final results are outperforming the state of the art as claimed in the paper.
3.  Also, it is not clear why the authors didn't use more popular datasets for few-shot learning like miniImageNet, tieredImageNet, CIFAR-FS and FC100.
4. Curiously, the paper defines the contributions as a method that outperforms the Stanford-Car few shot fine-grained learning as well as the ablation study and the visualisation aspect of the method. It is recommended to define better the theoretical contributions.

clarity,_quality,_novelty_and_reproducibility: The paper is well structured and clearly written. Still the definition of the statistical method is hard to follow.

Looking at fig.3 it seems that the statistical model extracts too local features. How robust could be the model? For example in the last column the truck and the car are characterised with only 2 and 1 features correspondingly.
summary_of_the_review: The assumption is intuitive, the statistical model is technically sound but it is not clear that the method outperforms the state of the art and in which of the datasets (as stated the authors mentioned that only the Stanford-Car dataset is improved).
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary: The paper proposes an end-to-end method for fine-grained few-shot classification by parsing an image into a collection of K parts from a shared dictionary.  The dictionary consists of K part template features and is learned from data to capture part instances from different categories at multiple scales.   Each parsed part in the image is composed of a 2D location and a part expression vector.  During parsing, multi-scale template matching is performed so as to optimize the feature map reconstruction error.  Experiments show that the proposed method is competitive with prior work on popular few-shot classification datasets while providing interpretability.

Strengths:
- The proposed method provides interpretability of results and has competitive performance 
- The idea of using parts for few-shot classification is intuitive 
- The proposed method can be trained end-to-end

Weaknesses:
- Some relevant work are not discussed and compared with (reviewer 47uq, cctR, qRQa)
- Some popular few-shot classification datasets were not evaluated on (reviewer 47uq) 
- Reviewers found the paper to be difficult to follow at places  
justification_for_why_not_higher_score: Even positive reviewers (qRQa, 47uq) felt some relevant work was not discussed / compared with.  Reviewers also felt some parts of the paper was difficult to follow.  The authors did revise the paper to include more results and clarify some parts.  The AC feels that the changes are substantial and the paper warrants another review cycle.
Decision: Reject