Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The paper pushes robot learning using audio input. Given a sound, a robot is tasked with creating the sound with behaviours. They use self supervised learning technique to learn latent representations of audio, then a second network to learn behaviour given a latent representation. Mean square error over produced sound is used as an accuracy measure. The paper benchmarks against a few known baselines and does ablations over supervised learning and a couple other design choices.
review: I like this paper, its very humorously written and is a surprisingly innovative approach to robot learning. That is refreshing. I am not an expert in audio tech but the methods used look reasonable to me. 
1. For the table 1, the abbreviation for methods in the lower blocks, it is not apparent to me what they stand for. Would be nice to explain below.
2. The result that self supervised learning is better than supervised pretraining in data constrained regimes is an important observation.
3. Limitations and discussions sections could be improved
4. An interesting question is, can we do audio conditioned task execution using similar methods or speech-to-actions?

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I like this paper, it's an intersecting problem set up, an unusual and infrequently explored data modality. the hypotheses generated are sound and the experiments make a reasonable shot at answering them. Qualitative results on actual audio samples will greatly improve this paper. I would accept this paper, so more roboticists can think about working with audio data.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper explores the concept of motor learning through the use of auditory inputs. The tasks examined are those typically underperformed when relying solely on visual data. This includes actions like rattling a rattle or shaking a tambourine, where the completion of contact-rich tasks is heavily reliant on sound. The authors collected a robust dataset comprising 5000 data points per task. The tasks evaluated include a range of activities including rattling a rattle, shaking a tambourine, swatting with a fly swatter, and performing horizontal and vertical strikes using a UR10 robot. The proposed model outputs actions given a desired sound as input. 

The proposed model transforms the audio signal into a spectrogram, then a CNN is used to extract features. These features are trained using the BYOL target, before a linear model translates these features into corresponding actions.
An important finding in this study was the use of self-supervised learning using the BYOL method.
review: Strengths:
- The paper is well-written and easy to comprehend
- The task of learning context-rich tasks through auditory inputs is interesting. 

Weaknesses:
- The ablation study provided within the paper lacks compelling evidence 
- I'm not fully convinced with the setting of using audio inputs without combining them with visual inputs. The motivation behind this approach needs to be clarified.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper introduces an novel approach to motor learning, where actions are learned from desired audio inputs. This novel setting could potentially contribute fresh insights to the field of multi-modal robot manipulation, which leads me to recommend acceptance. However, I must point out that the experiments presented within the paper do not fully substantiate the claims made by the authors. Additionally, the use of a self-supervised learning approach during feature extraction is not particularly groundbreaking. I would suggest the authors to enhance their experimental design and offer more compelling evidence to support their claims.

-------------
Post rebuttal: My questions has been addressed and I recommand weak accept
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper introduce AuRL, a self-supervised network for generating robot motion from a sound input.
AuRL is composed of two parts: a self-supervision/encoder network that defines latent audio features; a fine tuning network that predicts the motion to generates a similar sound as the input.
review: Globally this work lacks many details and rigorousness, which confuse the reader and drastically reduce the quality of the paper.
Strengths:
- Original and relevant research area.
- Experimental validation of the approach

Weaknesses:
- Unclear methodology. In baseline supervised/unsupervised, It is crucial for the reader to understand from the start  the inputs and outputs of the proposed method. Yet, after reading the paper, the reader is unable to understand the output of the proposed method.  How is the robot controlled (discrete primitives? velocity control? sequence?).  This is a major flaw.
- Many inaccurate/undefined terms in equations
- Experimental evaluation unclear: The MSE is computed for which behavior parameters?  What is the unit? How is the L2 distance computed? What is a test point?
- The organization of the experimental results is confusing. It start from Table 1 to Table 2 then back to Table 1...
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: Given the many technical and experimental flaws, this paper is not currently at the level of a CoRL paper.
It is highly recommended to check the soundness of all equations, improve the clarity of the paper and provide additional details about the experimental study.

Post Rebuttal Update:
The reviewer appreciates the efforts and rigorous answers of the authors in the rebuttal process.  The reviewer regrets that the same rigorousness and efforts were not put in the first version of the paper (with obvious "amateurish" flaws). If the paper is accepted, it requires some substantial reworking from version 1. Despite the potential benefit for the robot audition community, because of the above  weaknesses, the reviewer updates his recommendation to weak accept.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: In this paper, the authors collect a audio dataset and use self-supervised learning to improve performance of behavior prediction from audio.
review: Strengths:
*The authors presents a novel audio dataset for robotic manipulation.
*Sufficient experiments are conducted on five different tasks to demonstrate the increase in performance.

Weaknesses:
*Limitation in the use of this dataset. It would be more interesting to see if pretraining on this dataset will have performance increase in some more complex robotic manipulation tasks(eg. peg in hole task, training from scratch vs. pretraining on this dataset).
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: I think the framework is interesting and the dataset is quite novel since it is a visual audio dataset for robotic manipulation. Hence, I recommend accept the paper.
review_title: null

Meta-review: This submission initially received mixed reviews.  After discussion, the reviewers generally leaned toward acceptance based on the technical innovations and the experiments.  The AC agrees.  The authors are encouraged to further revise the paper based on the reviews in the camera-ready version.
Decision: Accept (Poster)