Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper intends to exploit the learned representation of the model to tackle partial-label learning tasks. The paper begins by a PILOT experiment to show that the class activation map is better at selecting the true label from the candidate set than the model output itself. To overcome the limitation that CAM cannot be applied to linear model and non-image data, they propose class activation value (CAV) to replace CAM and show similar properties. 
main_review: 1. It is innovative to use CAM to select true labels from candidates. The paper provides a new perspective to transfer PLL problems to supervised learning, which is worth deeper research in future work.
2. The proposed CAVL method is effective and efficient.
3. The experiments are sufficient, including various types of data and backbone. The ablation study on partial label generation methods strongly supports the intuition that CAM does not rely on any assumption on the collected data as previous work does.


weaknesses：
1. The main contribution is to utilize CAM to solve PLL problems and make some modifications on CAM to adapt to more data types and backbone. The original contributions do not seem to be enough and the novelty is a little weak.
2. The explanation of the power and dynamic attributes is not clear. Referring to section 2.2, how does these two attributes show that CAM can act as the guidance for PLL problems.
3. In the CAVL method, referring to section 3.2, the potential true label is selected based on CAV after training the model for only one epoch instead of several epochs. How do you decide when to select the potential true label?  What if the wrong labels were chosen, I did not see experiments or explanations in this case.
4. The paper needs to also provide the results and notice that realistic dataset instead of using generated visual partial labels only.  
e.g. the datasets and references should be considered in the related work survey or experiments:  
Partial Label Learning via Label Enhancement AAAI 19  
General Partial Label Learning via Dual Bipartite Graph Autoencoder AAAI 20  
GM-PLL: Graph Matching based Partial Label Learning TKDE 21  

questions:
The CAV is similar to adding attention values, what are the most important novelty and differences of the method with other closely related work.

summary_of_the_review: The paper proposes a novel CAV-based method to solve PLL problems and shows a promising performance than several state-of-the-art methods. Based on CAV, they propose a CAV learning method to select a potential true label for training and thus transfer PLL to supervised learning. Extensive and solid experiments prove the effectiveness of the proposed method. The paper provides a new way to tackle the PLL problem. The  main concern is the CAV is similar to the attention mechanism, and some of paragraphs are still need to explain.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper studies an interesting weakly supervised learning problem called partial-label learning, which solves the multi-class classification problem, where each training instance is assigned a set of candidate labels that include a set of candidate labels that include the true label. This paper empirically shows that the Class Activation Map (CAM), a simple technique for discriminating the learning patterns of each class in images, is better at making accurate predictions than the model itself on selecting the true label from candidate labels. Based on this motivation, this paper proposes a versatile version of CAM called Class Activation Value (CAV) and proposes a novel method to selects the true label by the maximum CAV for partial-label learning. Experiments validate the effectiveness of the proposed method.
main_review: Pros:

(1) Interesting empirical observation. CAM is well known as a simple technique for discriminating the learning patterns of classes. This paper shows that it can be also used for selecting the true label from candidate labels by calculating the number of positive elements in the CAM of candidate labels.

(2) Novel methodological contribution. This paper proposes Class Activation Value (CAV, which is versatile in models and inputs) and uses CAV to select the true label from candidate labels. In this way, a novel partial label learning method is obtained. This method is novel while keeping its simplicity.

(3) Experiments are extensive and validate the effectiveness of the proposed method.

Cons:

(1) I understand that CAM can be used for selecting the true label from candidate labels by calculating the number of positive elements in the CAM of candidate labels. However, I feel that it took me a long time to understand this operation. There is only one sentence about that. So the authors are encouraged to give more descriptions on how we use CAM to select the true label. This will make it easier for readers to understand.

Questions:

(1) I feel that this paper actually focuses on the pseudo labeling task in a weakly supervised learning problem. So I would like to how can the proposed method be used in another weakly supervised learning problem like semi-supervised learning. Can the authors provide some discussions on that?

(2) The authors mentioned the average-based strategy and the identification-based strategy for partial-label learning. I think the proposed method in this paper is an identification-based strategy, and the authors are encouraged to clearly state that in this paper.


summary_of_the_review: This paper investigates partial-label learning. This paper makes two key contributions including an interesting empirical observation and a novel methodological contribution. To the best of my knowledge, the empirical observation is disclosed by this paper for the first time and the proposed method is simple yet effective. Comprehensive experimental results support what the authors claimed in the paper.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper focuses on the problem of partial-label learning (PLL), where each training instance is assigned a set of candidate labels that include the true label.  This paper shows that class activation map (CAM) could identify the true label from candidate labels for PLL. In addition, the authors propose the class activation value (CAV) for identification via capturing the learned representation information in a more general way instead of CAM as CAM is confined for image datasets and CNN model. CAV Learning that selects the true label by the class with the maximum CAV for model training is proposed. Experiments on various datasets demonstrate that the proposed method achieves state-of-the-art performance.
main_review: Strengths:
1.	This paper introduces the class activation map (CAM) for handling identification in PLL.
2.	This paper proposes the class activation value (CAV) for identification via capturing the learned representation information in a more general way instead of CAM as CAM is confined for image datasets and CNN model.

Weakness:
1.	The authors claim that “We believe that such a mechanism (Grad-CAM or CAM) could guide f to differentiate the true label from the candidate set because it is constructed by taking advantage of internal elements in f” and conducted a pilot experiment to check whether CAM with the most foreground seeds belongs to the true label. However, the pilot experiment cannot empirically validate this claim, as the experiment is only conducted on the CIFAR-10 with artificial candidate labels generated by two simple strategies but not conducted on real-world PLL datasets or adopts some realistic candidate labels generation strategies such as class-dependent strategy. 
2.	The proposed method CAVL adopts the strategy that selects the “true” label from the candidate label set and only considers the loss on the “true” label after the first training epoch. However, the selected “true” label may be false-positive and the authors also claim that the potential true label would be updated. In this case, the training model could be affected by false-positive labels since CAVL only considers the “true” label.  [1] proves that models trained by this strategy cannot achieve good performance. 
3.     There is no theoretical result provided to validate the effectiveness of CAV for PLL.
4.	The authors adopt the PLL experimental settings in [1], but they do not compare the method in [1].
5.	The authors claim that the hyper-parameters are selected to exploit the best performance on the validation set containing 10% of the training examples. They should show the train/ validation/ test split. RC, CC and CAVL are suggested to have the same learning rate, weight decay and mini-batch size as they adopt the same model for fair comparisons.


Suggestion:
The authors are suggested to provide source code as supplementary material so that the reviewers could check the reproducibility of the experimental results.

[1] Progressive Identification of True Labels for Partial-Label Learning, ICML 2020.
summary_of_the_review: This paper introduces the class activation map (CAM) for handling identification in PLL. But the effectiveness of CAM for PLL is not validated. Several of the paper’s claims are incorrect or not well-supported. The contributions are only marginally novel.
review_title: null

Meta-review: This paper considers the so-called partial-label learning problem and proposes a class activation map that is better at making accurate predictions than the model itself on selecting the true label from candidate labels. The authors investigate the approach in experimental results on four benchmark image datasets. 

The reviewers appreciated the simplicity of the approach and its effectiveness in practice. The reviewers raised questions how to apply the approach to another weakly supervised learning problem such as semi-supervised learning and whether the approach is an identification-based strategy. The reviewers also raised several questions asking for more details.

The authors submitted responses to the reviewers' comments. After reading the response, updating the reviews, and discussion, the reviewers who took part in the discussion considered that their  “questions have been well addressed” and that the “authors’ responses basically provided the answers to the questions”.  

The feedback provided was already fruitful and the final version should be already improved. 

Accept. Poster.
Decision: Accept (Poster)