Venue: ICML 2020 Workshop SAS

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper claims to build a perception of touch sounds by mapping sound to UMAP space using some suggested features. The paper lacks rigor in experiments and evaluation about the effectiveness of the method (or using the specific suggested features).
review_title: Good Idea. Poor evaluation. No Baseline.

final_score: rating: 5: Marginally below acceptance threshold
review_content: This manuscript describes an approach to learning unsupervised representations of audio.  The central approach is to learn a space using Uniform Manifold Approximation and Projection (UMAP) and then use this space to describe pseudolabels for use in training.  

Strengths: 
* Learning a cluster based representation using a different set of features is an interesting, if not entirely novel, approach.  
* There is some evidence that this works.

Weaknesses:
* Much more time is spent describing other knowledge driven approaches to identifying sounds than is spend describing UMAP.  The rest of the approach is interesting only insofar as UMAP is trusted as a mechanism to construct an informative or useful feature representation.  I would recommend devoting more attention to this algorithm and assessing its value as a target for this task.  Consider the question, for example, why is the UMAP space preferable to a TSNE space, or even using K-Means centroids as a set of basis vectors to define a space?  The UMAP space is critical to this technique, but not compared against any alternatives.
* The evaluation is impressionistic. “The performance of the system is depicted in Fig 6, where the red square gives the average error in the x-y coordinates in projecting a test set of sounds into UMAP coordinates.”  Some testable measure along with the examples would be more convincing.
* “In order to evaluate the performance of our system, we hold out a portion of the test sounds, while not keeping the corresponding images out of the training set.”  What would happen if the images were removed from the training set?
* The spectrograms are treated as fixed sized data rather than sequences with potentially distinct sizes.  Why was that decision made? Is there a mechanism to extend this approach to treat speech as a temporal sequence rather than a static view.

Minor Presentation Comments: 
* “Consider a time domain signal x[k] and its corresponding spectogram denoted as X[k, m] where k denotes the k th frequency bin computed at time instance m typically every 10ms.”  Overloading the subscript ‘k’ here is odd.  Consider using a different subscript for the time domain x[k] and the frequency in the spectrogram X[k, m]
* The introduction does not make clear what the central approach to the problem is.  It takes reading much of the paper to have a solid understanding of how the pieces described fit together, including understanding what is background, related work and what is actually being used in the described approach.
review_title: Solid background but narrow novel contributions

final_score: rating: 2: Strong rejection
review_content: - It wasn't clear from the title or the abstract, what was this paper about. What is audio perception and what is being learned? What is the related work and gap? What are the contributions?  
- "Audio perception is a key to solving a variety of problems?" How? There aren't citations about audio perception.
- Touch sounds is not a coined term, authors should define it and provide a list of all the classes used. Authors don't even say how many classes they used. Impact sounds (e.g. hitting and tapping) is a different group from Scraping sounds (e.g. rubbing, sawing)
-  "Such approaches can be used to build a state of art perceptual model for any sound?" What is a perceptual model? Any sound? This is really a stretch of scope considering the actual state of the field of Sound Understanding.
- The introduction starts talking about "Audio scene understanding", which is a different focus to sound (event) understanding, the actual topic of the paper.
- What is the motivation of the paper, is it to  explore what has been done before on a different field or to leverage from images as a mean of unsupervised labeling of sound events? "Our  work explores how we can augment similar auditory based representations",  "In this work, we propose a method of unsupervised learning a perceptual model for signals of interest"
- Section 3.2 starts right away choosing low-level features, and justifies them afterwards, I suggest to reverse the order. Moreover, the motivations are mostly reasonable, but lack justification with any literature reference. 
- This is not exclusive of impact sounds " i) they consist of relative peaks in the energy envelope irrespective of the background noise present if any"
- No justification and evidence of the benefit of using low-level features over the sound event labels to select the initial set of sounds.
- No evidence for the justification: "Trying to map these sounds [impact sounds]  to the latent spaces, often ends up learning other sounds than the touch sounds due to data imbalancing problems",  it could instead be because of their duration or frequency range. 
- No quantitative experiments, no diagram or clear explanation of the system, and there is an assumption that the reader understands UMAP.
- The conclusions are limited, here is an example: "Given a lot of data, it can help us  perhaps in learning even finer grained distinctions present in the data."
- References to the audio are lacking in the topics of DSP, psychoacoustics and sound event labeling.



review_title: The title, abstract, motivation, related work, gap, experiments and results are unclear.

Meta-review: Dear author(s),

Thank you very much for your submission at the ICML2020@SaS workshop (https://icml-sas.gitlab.io/). Based on the scores assigned by the reviewers, we regret to inform you that the paper was rejected. We got 26 submissions and we were only able to accept 13 papers. We invite you anyway to consider the feedback of the reviewers and to follow our upcoming workshop on July 17.
Decision: Reject