Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes an SVD-dictionary learning approach for few-shot object detection, which decomposes the input image feature into a generalization map with large eigenvalues and a discriminative map with small eigenvalues, and the decomposed feature maps are used to enhance the model generalization and discrimination.  The proposed method is evaluated on standard benchmarks using PASCAL VOC and COCO. 
main_review: 1. Originality: Are the tasks or methods new? Is the work a novel combination of well-known techniques? (This can be valuable!) Is it clear how this work differs from previous contributions? Is related work adequately cited?

The few-shot object detection task is a relevant new task in the field. The proposed method using SVD in FSOD is novel.  The related work is well discussed.  I like the idea of decomposing the feature space to tackle generalization and discrimination within the categories.  It would be nice to provide more motivation and empirical examples to quantify the effect. What if we only have one branch rather than both branches? It would be good to have an ablation study and a motivating example to help the readers understand. 

2. Quality: Is the submission technically sound? Are claims well supported (e.g., by theoretical analysis or experimental results)? Are the methods used appropriate? Is this a complete piece of work or work in progress? Are the authors careful and honest about evaluating both the strengths and weaknesses of their work?

The submission is technically sound and provides ablation studies for the hyperparameter choices in the method. However, it would be great to show the evaluation on LVIS as it provides more categories and more realistic novel/base class splits. 


3. Clarity: Is the submission clearly written? Is it well organized? (If not, please make constructive suggestions for improving its clarity.) Does it adequately inform the reader? (Note that a superbly written paper provides enough information for an expert reader to reproduce its results.)

The manuscript is well written and easy to follow. 

4. Significance: Are the results important? Are others (researchers or practitioners) likely to use the ideas or build on them? Does the submission address a difficult task in a better way than previous work? Does it advance the state of the art in a demonstrable way? Does it provide unique data, unique conclusions about existing data, or a unique theoretical or experimental approach?

The proposed method outperforms previous methods.  The decomposition idea might be inspiring to others in the field. 
limitations_and_societal_impact: No
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper proposes a method for few shot object detection. It splits the feature map obtained from the backbone network into two parts. First is the component with largest valued singular values (similar to low rank matrix approximation with SVD), while other is that with the remaining lower ones. It then performs dictionary learning for coding the each column of the second matrix wrt the learned dictionary and uses the code/weight matrix as features. It fuses this with the earlier matrix and passes it to the region proposal network. Further it ‘remaps’ the object proposal features the same dictionary.
main_review: The method seems complicated and perhaps the steps are not sufficiently motivated. In particular the part after l149 is confusing. P is said to be the region proposals with dimension (z,m,o,o) with z number of and o spatial size of proposals—this is not clear. Further why are these proposals coded with a dictionary which was learned with some other data from other previous layer? Seems it works in practice but I am struggling to understand the motivation and benefits of this step.

Empirical results are provided on VOC and COCO standard settings, and the method achieves improvements when added with existing methods.

I appreciate that performances on base classes is also reported which doesn’t suffer significant degradations.

For the visualizations, is the single channels with largest average value are shown, or is it a max operation on channel dim for each spatial location? Would the behavior not be expected? Like in Eigen faces etc, the large eigen value components would have sharp focus on some features, while the lower eigen value components would be spread out. Could there be a before and after visualization wrt training. Show these maps without your architecture (with plain features SVD can also be done to generate G and D) and with your architecture, does any interesting observations emerge?

I am somehow left a bit dissatisfied by the way the method is presented and explained, it seems to work, but I would like to have some hints as to why.

limitations_and_societal_impact: n/a
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a solution for addressing the few-shot object detection problem through enhancing the generalization and discriminability capability, which relies on the Singular Value Decomposition (SVD). Moreover, the dictionary learning is employed to further enhance the high-level discriminative capability.
main_review: Pros:

1, This paper brings the methods for both enhancing the generalization and discriminability using SVD and dictionary learning from the research in domain adaptation, it is with novelty for adding such mechanisms into the detection structure to improve the few-shot object detection.

2, The concept of generalization (transferability) and discriminability is beneficial to few-shot object detection.

3, The experiment results is solid with obvious improvement compared to the two baseline model.

Cons:

1, The function of adding of SVD-based generalization and discriminability enhancement module is more like to improve the backbone capability of the object detection network, which is not directly related to few-shot object detection. It is also not quite clear why the RPN does not need the discrimination information for generating the proposals. Such discrimination information, e.g., the structures of objects as mentioned by the author, should be also important for generating reasonable object proposals. 

2, The author mentioned the dictionary learning is beneficial for strengthening the discrimination ability of detectors. However, in my view, for few-shot object detection, the dictionary learning may more helpful to address the category imbalance in few shot since the underline principle of dictionary learning is a bit like the k-means.

3, The larger singular values contribute to the generalization ability is based on the requirement that the network feature extraction module already has some extent transferability between the source domain and the target domain. In the original SVD paper [3], such transferability is obtained by the domain adversarial training. However, in the few-shot object detection, such transferability is not certainly guaranteed, which raises the concern on the transferability between base and novel categories.

4, According the SVD spectral theory, a discriminative classifier/regressor has necessary dependence on most dimensions instead of those with only top singular value. To address this, the original SVD paper [3] added the spectral penalization to minimize the top k singular values for both source and domain to prevent it from standing out. In this paper, E in Eq. (1) is used for RPN instead of F. This raises the concern if such way will harm the discriminability for the RPN as those features belong to the top k singular values is additionally larger.

limitations_and_societal_impact: N.A.
review_title: null

Meta-review: This paper offers an SVD based decomposition of a learned representation, factoring an embedding into components that are optimal for localization vs description.  On balance the reviewers find the paper acceptable, with a majority of reviewers including the most knowledgeable reviewer in the area supporting acceptance, due to the novelty of the method and the improved performance.  The AC concurs with these reviewers.
Decision: Accept (Poster)