Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The submission focuses on compositional zero-shot learning, in this context a type of zero-shot problem where instances are a pair of images and compositional labels (object/attribute information). 
The authors start with the standard prototypical networks setup, and add auxiliary losses in order to learn object and attribute prototypes. A HSIC loss is added to encourage object-attribute independence (for better compositional generalization).
Finally, a graph propagation algorithm is applied to combine the prototypes into compositional prototypes.
main_review: Positive aspects:
- The paper is well-written, the ideas are clear.
- Empirical evaluation is satisfactory: there is an ablation study and a detailed discussion of the impact of the different components.

Limitations/to improve:
- I disagree with the author's claim that attribute datasets are either noisy or do not work in the single attribute setting. As an example, Caltech Birds (CUB) can be easily modified to account for a single attribute (e.g. color). More generally, one of my main issues is that the datasets considered are quite simple compared to the more complex "real-world" datasets more commonly seen in the classical zero-shot literature. Could the authors adapt their model and baselines to e.g. Caltech Birds or a related dataset and show that their approach indeed generalizes beyond toy datasets?
- The related works section is missing some works, notably [1] which is highly relevant to the discussion on compositional generalization. They propose a method, by no means the only one, to measure the compositionality of a representation. It would be interesting to see if there is indeed a link between a measure of compositionality (not necessarily theirs) and downstream performance of the representations in this submission. In a similar vein, I would mention [2] who take a more applied standpoint on compositional generalization.

After rebuttal: looking at the author response, I have raised my score to a 7. Cf. my response to their point for justification.

References
[1] Tristan Sylvain, Linda Petrini, and Devon Hjelm. "Locality and compositionality in zero-shot learning." ICLR 2020
[2] Wenjia Xu, Yongqin Xian, Jiuniu Wang, Bernt Schiele, Zeynep Akata,"Attribute Prototype Network for Zero-Shot Learning." NeurIPS 2020

limitations_and_societal_impact: The authors have adequately addressed the limitations/potential negative societal impact. Indeed, they mention that there are no obvious issues, but that misuse can always happen, and that therefore caution should be exercised on a per-case basis.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes ProtoProp model for compositional zero-shot learning, which propagates the learned object and attribute prototypes with independency constraints by a compositional graph of GNN. The compositional prototypes are then used for detecting unseen classes. Experiments are done on AO-Clever and UT-Zappos benchmarks.
main_review: 1) The proposed ProtoProp model is validated on two zero-shot learning benchmarks with strong performance improvement.

2) The idea of compositional prototypes from attribute and object prototypes by GNN propagation is novel to me.

3) The paper writing is clear and well organized.

limitations_and_societal_impact: 1) Using HSIC loss to maintain the attribute-object independence achieves improved embedding property but it seems a direct adaptation from Atzmon et with different kernels. But it has no experiment table to validate the improvement.

2) Missing detailed discussion or comparison to some related works on learning the prototype-based representations, such as [a, b]. It would be interesting to see the influence of prototypical representations obtained by the different methods.
[a] PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment. ICCV, 2019.
[b] Prototypical Cross-Attention Networks for Multiple Object Tracking and Segmentation. Arxiv.

3) Missing detailed failure cases analysis, and qualitative visual results. 

4) In Table 2, missing discussion on why finetuning is critical to visual node features but has smaller influence on the semantic node feature?

5) What's the influence of number GNN layers/design choices to the propagation and final performance.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper deals with the problem setup of compositional zero-shot recognition. Where a visual object is specified by two labels: an attribute name and an object name. Not all attribute-object combinations are available at training time, and at test time, a model may observe new (unseen) combinations.

The approach combines ideas from two recent publications: [1] Atzmon et al, 2020, and [2] Naeem et al, 2021: It uses the idea of encouraging independent prototypes for the attribute and object (Atzmon), and then it uses them as the node features for the graph-NN used by (Naeem), replacing than the word-embedding used in (Naeem).
In addition, it proposes a spatial attention mechanism over CNN feature maps for extracting the prototypes, rather than the average-pooling used by previous baselines.


main_review: 
### Originality: 
This work provides a novel combination of recent techniques (Atzmon, 2020) and (Naeem, 2021) - the idea is simple, and may be considered as immediate, but that's what I find as appealing in this approach.

### Quality: 
The approach is technically sound, however there are issues with several key points:

* It will be beneficial to report the results of the model with the recent C-GQA benchmark (Naeem 2021).

* One claim uses a wrong terminology, but it is not a major issue because this is an architecture paper rather than a theoretical paper. Specifically, the paper claims that its loss function promotes *conditional*-independence of prototypes (as in Atzmon), where in fact, the loss term in eq (3) promotes *"full"* independence between prototypes. Promoting full independence may be problematic when used as a hard constraint (e.g. with a very large loss coefficient), because the attribute and object are not fully independent in the training data, but empirically it can work to some extent with a careful hyper-param tuning.

* Some claims are supported by the ablation study. However evidence is missing to several other claims. Specifically: (1) Superiority of Gaussian Kernel over Linear Kernel. (2) Superiority of a dot product similarity metric (3) superiority of simple linear combinations (4) benefit of cluster and separation loss.

* The reported metrics for Zappos are missing the Open Seen/Unseen metrics

* It will be more fair to cluster the compared approaches according to fine-tuned vs non fine-tuned.

* It will be beneficial to include an ablation study for the attention mechanism vs average pooling.

### Clarity:
The paper is mostly clear. Some sections  were hard to understand:

* Except for the linear activation, does the graph-NN architecture is identical to Naeem et al? Please discuss that in related work.
* For the HSIC kernel, how was the median distance between points calculated? If I understand correctly, this distance changes every time the model weights are updated. Therefore, it is not clear how to select this value if the distance keeps changing after every gradient update.
* Figure 2 and its caption is unclear.
* Paragraph in lines 147-154

### Significance:
Compositional generalization is a challenging and important problem in machine-learning models. The approach provides a simple combination of recent techniques using spatial attention, and provides promising results.



### Additional comment to authors:
* The claim "we do not use a fully connected layer" is not accurate: Each of the learned prototypes may be viewed as a vector of a matrix that parametrizes a fully-connected layer. 


### Post rebuttal:
I thank the authors for their answers. Following the rebuttal I decided to raise my score to 7.

limitations_and_societal_impact: Limitations were adequately discussed
review_title: null

Meta-review: Compositional generalization is an important problem in machine learning. The problem setup is well motivated and the paper is well written. The proposed method is simple and effective.  It combines several existing techniques with innovations. All reviewers agree the acceptance. 
Decision: Accept (Poster)