Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The work integrates human supervision into representation learning process to compute visual object representations inline with human requirements for interest tasks. The work combines existing contrastive learning method with a loss function that brings the representations of human-selected objects close to each other in the latent space. They evaluate the work on a simplified recycling setup that allows human-robot interaction.
strengths_and_weaknesses: strengths : (1) adapts to dynamic human-defined requirements by leveraging human-feedback (2) performs both online and offline evaluations, ablations, and outperforms some self-supervised learning baselines. (3) the case study has immediate relevance to real-world, though would like further explanation on the novelty of the architecture.

weaknesses : (1) requires active human supervision which is expensive  (2) the objects used for the experiments are limited in variations to fully evaluate the performance of the system. 
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: I read the paper, and used good judgement in assessing the paper.
issues: (1) would like to see if authors considered any approaches to alleviate the need of active human feedback by instead relying on approaches that incorporate scalable human feedback ( example : using intermittent oversight or meta-imitation learning agents). (2) would also like to see experiments with more diverse set of objects
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper proposes a method for learning visual object representations with human supervision. For this purpose, the authors suggest three components: (i) instance prediction for multiple views of the scene, (ii) cluster prediction for varying instances of the same cluster, and (iii) human supervision for human-selected examples. The authors evaluated their method with offline experiments as well as in real-time with a robot in a recycling domain.
strengths_and_weaknesses: Strengths:

The paper focuses on a critical topic and is well motivated. Its technical quality is good, and there are detailed ablation studies showing the contribution of each suggested component. It has also been evaluated in a case study with a recycling robot, and I appreciate the authors for deploying the system to the real world and showing its applicability. 

Weaknesses:

The paper is motivated by stating that "a robot may be asked to interact with a wide variety of objects, making it hard or even impossible to pre-program visual object classifiers suitable for the task of interest". However, the suggested method uses a pre-defined number of object categories $C$ and clusters $K$. Consequently, it focuses on a limited number of objects, and handling a wide variety of objects can be challenging for the suggested system in more complex applications.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Although the paper focuses on a limited number of objects, and it could have been improved by including more baseline methods from the literature, it presents a strong contribution to a critical topic with good technical quality and detailed analysis.
issues: - The paper nicely provides the background on self-supervised and contrastive learning. However, human-in-the-loop robot learning hasn't been mentioned in the related work section, which is one of the main contributions of this paper. It would be nice to include this literature for more comprehensive related work.
- What is $N$ in Equation 1? Is it the number of all positive pairs in a mini-batch?
- The authors mention 80% train and 20% validation set in the data collection. Was there a separate test set for including human supervision?
- Was the model shown in Figure 1 trained end-to-end?
- The offline evaluation results are not easy to follow in the text:
  - As far as I understand, Figure 4 (b) and Experiment 5 in Figure 3 represent the same condition (Cluster Loss + Human Supervision). If it is the case, it would be great to use the same notation in both figures. Also, I am not sure whether Figure 4(c) and experiment 7 in Figure 3 represent the same condition. 
  - It is not clear which conditions have been taken from the literature. I think Experiment 2 in Figure 3 is identical to SimCLR [10]; are there any others?
  - The colors in Figure 4 are not straightforward to interpret. It might be better to make this figure gray-scale.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The authors introduce a method for training a visual classifier in a few-shot manner from human preferences, for use in real-time robotics scenarios. In particular, they build upon existing methods from contrastive learning (e.g. contrastive clustering), but introduce an additional loss that tries to make the embeddings of all human-picked objects similar. They show through both offline and online experiments that their method shows small, but statistically significant, improvement over the baselines.
strengths_and_weaknesses: The experiments are a clear strength here: the authors have successfully applied their method for the proposed task of object classification on a conveyor belt, where they set up an experiment where they performed 5 epochs of real-time training on a human subject's preferences for which objects to pick. This experiment validates empirically their claims over the usefulness of their model in their proposed use case, and the paper deserves recognition for this non-toy experiment alone.

The major weaknesses of this paper are indeed the limitations discussed in the limitations section. There is still a reliance on this idea of predefined, fixed-count clusters (which by the way is not explained in detail, see issues section). The human preference is only used as an additional "cluster". The main problem with this kind of approach is that it is not clear what will happen when the human chooses some objects from within a cluster but not others, e.g. the human cluster does not fully intersect or exclude any of the existing clusters. This use case is relevant in that it is not hard to imagine there would be other criteria besides the cluster assignment for how the human picks these objects (otherwise we could just classify based on the cluster label alone), and it is likely that in these cases the similarity score-based approach for classification (which is also not described in detail, see issues section) will lead to lower classification performance. Experiments of this nature were unfortunately not performed, so it is difficult to estimate what the actual impact would be.

It is also a clear weakness of this paper that the introduced method is equivalent to simply introducing an additional, non-exclusive cluster to the existing idea of contrastive clustering, which is mostly a very incremental improvement. However, as an evaluation of this approach to a novel problem (few-shot human in the loop learning), the paper is valuable as a case study.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: While the proposed method is mostly an incremental improvement over an existing, known approach; it is also an interesting case study of how this approach can be applied to a new & interesting problem in the context of human-in-the-loop learning. As a result, I lean that the paper be accepted, but do have concerns about the limitations of the approach and how these were not exactly addressed (e.g. mostly avoided) in the experiments.
issues: After reading the paper, two clarity issues I noticed are below, and it would be great if these could be addressed.

1. The idea of predefined, fixed-count "clusters" is not adequately introduced, I believe this is due to the authors' familiarity with the related works such that this idea comes naturally. However, as a robotics reviewer not extremely familiar with contrastive learning it was difficult to understand from this paper alone where the clusters came from, and their limitations.

2. While the method for training the embeddings is described in detail, how the embeddings are used to classify an object for picking or not is not immediately obvious. While this similarity score idea is mentioned in the experiments it is still not fully clear what exactly is used for the pick/dont pick decision, and I think this is better suited for the method section.

Additionally, it is a valuable contribution (although maybe too costly at this point) to actually provide the evaluations of scenarios where the human prefers some objects in some clusters but not all.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper adapts modern visual object representations based on contrastive losses to better align with human visual categories. At a technical level, the problem is posed as a joint optimization of three objectives that maximize the agreement between: 1) multiple augmented views of the same image, 2) multiple instances from the same category of objects and 3) human-selected examples from a single category. This means that the contrastive component learns a latent space based purely on the visual features of the examples, while the human supervision loss is meant to align that latent space with more “human-like” features. Simulated experiments with perfect “human” input show that adding the supervised loss component does help the contrastive losses recover a representation that better classifies 10 object categories. A conveyor belt scenario with the same objects shows that (with pre-training) the method can correctly classify each category after 30 (expert) examples (per category). 
strengths_and_weaknesses: I thought the paper was well-written and clear, as was the supplementary video (which I really appreciated as an overview explanation of the paper before digging into the details). The paper is overall well-motivated in the introduction, and the experimental results and ablation well investigated. I also liked the fact that the experiments have object categories that are more interesting than the typical computer vision ones (crushed cans or bottles, intact cans or bottles, etc.). Lastly, I appreciated having a real robot demo on one of the more interesting applications I’ve seen in robotics recently. Despite these strengths, however, I still do have some issues with the paper that I’d like to detail.

Perhaps my biggest gripe with the paper is that the technical novelty seems relatively limited: add a supervised loss to two existing losses. The experimental results do help make up for it, in that they demonstrate the usefulness of all 3 losses together in the ablation, but I would’ve appreciated more discussion and insight to make up for the rather simple contribution: 1. What led to choosing the architecture from Fig. 1? Instead of sharing the weights have you tried training the network iteratively with each loss? What’s your intuition behind why the current architecture would work best? How do you tune the lambda terms in Eq. (4)? 2. Does this method apply more broadly to more than just object properties or robotics? It seems like the proposed pipeline could work for more than just object classification or the conveyor belt example; perhaps a discussion expanding on this was left out to artificially fit into the CoRL robotics theme more strongly, but I would appreciate having it (maybe in the Conclusion). 3. In what settings would the assumption that the human always selects objects from the same category would apply? The conveyor belt application does seem a bit contrived with this assumption in place (I’d normally pick objects from any category and just place them in different bins, rather than sequentially clear them out one category after the other).
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Overall, the technical novelty and originality of the work is weak, but the motivation is compelling and (I believe) well-timed, the results are compelling, and the application is interesting. As a result, I am leaning weak accept, but I look forward to more discussion as my opinion can be swayed in either direction.
issues: Please add more discussion addressing the comments above (architecture choices and loss optimization, broad applications beyond object categorization and robotics, feasibility of assumptions).
review_title: null

Meta-review: metareview: The paper presents a method for human-centric object recognition, with the application domain being the recycling conveyor belts, where human and robot can co-exist.

The paper's **strengths** are:
- the compelling motivation of the proposed method; 
- the clear technical description of the work;
- the interesting dataset used in this work;
- the real-world demonstration is very promising.

The **weaknesses** of the paper are: 
- while the simple addition of the supervisory seems to work, additional insights regarding the architecture are missing; 
- the paper does not discuss the state-of-the-art of human-centric robot learning, and human preference-based learning in their paper;
- some missing information regarding the method's performance given the sparse labeling from the human feedback. 

I think the authors can majorly improve the presentation of their work, once they address the raised issues in the rebuttal period.

**Post rebuttal assessment:** The paper introduces a very interesting and timely application domain. The methodological contribution is not as strong as the experimental part, however, the paper introduces an interesting system, and as such can be very beneficial for the community.
confidence: 5: The area chair is absolutely certain
Decision: Accept (Poster)