Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: 3 new data augmentation strategies are proposed for compositional data and these methods are tested on microbiome data. With the proposed methods, state-of-the-art results in disease prediction tasks including colorectal cancer, type 2 diabetes, and Crohn’s disease are reached. They show that the proposed data augmentations can also be used for representation learning in a contrastive setting similar to SimCLR.

strengths_and_weaknesses: Strengths:

- In almost all of the cases the used augmentations improve the SOTA methods. This is a result of the fact that data augmentations can be used on top of the existing methods / can be orthogonally combined with existing methods.

- Paper is mostly written well. 

- Related work seems to be well-studied.

- There is a lack of comprehensive study in the field of data augmentations for compositional data.

- I liked the namings of the proposed data augmentations.

Weaknesses:

- The proposed data augmentations could be used for other types of compositional data besides human microbiome data. In my opinion, performance on other types of CoDA should be tested and reported for a machine learning conference like NeurIPS. Even though the methods might not work with other types of compositional data, this could as well be added to limitations and appendix. More importantly, there is a tendency to claim in some parts of the paper that the proposed data augmentations are for CoDa, not only for microbiome data. Here is an example from the "Conclusion": "By combining ideas from data augmentation with the principles of CoDa, we have defined 3 novel augmentation strategies for CoDa: Aitchison Mixup, Random Subcompositions, and Compositional CutMix." In my opinion, if you are making such claims, it is necessary to provide results in other CoDa datasets.

- Adapting data augmentations to CoDa is the main novelty of the paper. However, in my opinion, adaptations are to some extent straightforward and not very novel. "CoDa analogue of Mixup", "standard preprocessing step in CoDa", and "CoDa analogue of CutMix"  already give some hints regarding the originality of the method. On the other hand, the science builds on top of each other, thus I could say this is not a big but a medium-level concern.

- In most cases, the increase in the SOTA performance seems not so big, even though the datasets are pretty small. I would expect a higher increase for meaningful data augmentations for such small datasets. However, considering that it is hard to come up with data augmentations for CoDa, this could also be a minor point.

questions: I do not have any questions.
limitations: - I liked that you also reported the datasets, which your method could not achieve state-of-the-art, which might indicate transparency.

- My major concern is the one I listed in the Weaknesses as #1. I think the paper needs additional experiments using other types of CoDa for this venue as stated or at least it needs some rewriting in the claims.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper presents three data augmentation methods for compositional data. The first method (Aitchison Mixup) takes convex combinations of samples *within* the same class. The second method (Random Subcompositions - similar to applying dropout on features) takes an input, samples a subset of features, makes the values zero, and renormalizes the new point to be on the simplex. The third method (Compositional CutMix) samples two points from the same class, performs CutMix between them, and renormalizes the new point to be on the simplex. 

All three methods are adaptations of existing methods to Compositional Data (i.e., use the Simplex constraint) and are applied only intra-class (while the original Mixup and CutMix are also applied inter-class).

The paper's contribution is to adapt existing data augmentation methods to Compositional Data and show consistent performance improvement for classification tasks.
strengths_and_weaknesses: # Strengths
- **Well-written paper**. The paper is easy to follow, and the authors provide good details about the task of learning from compositional data and the proposed methods.
- **Very simple methods, with good empirical results**. The proposed methods are relatively trivial to implement and don't add any computational overload.


# Weaknesses
- **The contrastive learning experiments seem incomplete**. Although interesting, the experiment from Section 4.2 is evaluated too simplistically and leaves unanswered questions. Table 5 compares a standard MLP encoder that is 1) randomly initialized, 2) pre-trained using an encoder-decoder and 3) pre-trained with contrastive learning and the proposed data augmentation. Table 5 shows that pre-training using contrastive learning provide more linearly separable representations than the representations learned by an autoencoder.
	- However, Section 4.2 raises the question: "When should use contrastive pre-training rather than standard supervised training with the proposed data augmentation (like in section 4.1)?" because the results from Table 5 aren't compared with the performance of an encoder trained using data augmentation (like in section 4.1).
	- I suggest adding in Table 5 the following numbers: 
		1. encoder from DeepMicro + classification head-trained from scratch using data augmentation (similar to Table 5)
		2. encoder from DeepMicro pre-trained using contrastive learning + fine-tuned (with a classification head) using data augmentation — this answer the question: "Is contrastive pre-training complementary to training using data augmentation?"
- **Missing guideline for using the proposed methods on new datasets**. All three proposed data augmentation methods seem to provide a similar average performance improvement. It would be helpful to offer rule-of-thumb guidelines for choosing between the three methods (without trying all three methods). Please answer the question: "When should one use one method versus the other two?". Did you notice any pattern of a method working better for specific tasks/datasets?
	- **Possible solution**: Reproduce Figures 2 and 3 (performance improvement as a function of dataset size and the number of features) for all three proposed methods. Possibly, a pattern will emerge. If the results are different, add all figures to the appendix. If the results are similar, mention this.
	- **(Small, but related) Tables 2/3/4 are too similar** and the paper doesn't mention any trend difference between them. Although lines 237-251 describe the results, there's no explicit mention of the differences in the results for the three tables. Unless you specify clear trend differences, two tables can be moved to the appendix, and only the mean performance can be kept in the main paper.
- **(Small) Missing relation to prior work**: The proposed "Aitchison Mixup" is equivalent to performing SMOTE [1] in the Aitchison sense and shall be called Aitchison SMOTE. When Mixup takes convex combinations within the *same* class, it's equivalent to SMOTE. Please check the SMOTE method and the second paragraph of the related work on the mixup paper. If you use SMOTE, then please add it to the related work.

# Minor changes
- Line 30: Add examples of compositions data beyond naming domains (e.g., geochemistry -> proportion of chemical in a geological sample; economics -> proportion of GDP attributed to each industry), to make it easier for readers new to the area.
- Where are the limitations of the current work written? I couldn't find them in Section 3 (as mentioned in the checklist)
- Reference the peer-reviews published version of the papers (e.g., Mixup appeared in ICLR, and you've referenced the arXiv article)
- Line 250: provide the number of the SOTA tasks in parenthesis for easy reading
- Figure 3 from the appendix is really nice. Consider adding it to the main paper.
- The method "Random Subcompositions" could be renamed to "Compositional Feature Dropout", because it performs dropout on features, then renormalizes the feature to present the property of compositional data.
questions: I provide context for all questions in the “weaknesses” section

1. When to use each of the proposed methods?
2. Is any trend by comparing Figures 2 and 3 computed for all three proposed methods?
3. How does contrastive pre-training compare with training from scratch using data augmentation, and contrastive pre-training + data augmentation fine-tuning (my comment about incomplete Section 4.2)?
4. What's the motivation for applying the data augmentation only *intra-class* and not *inter-class* like the original Mixup and CutMix? Consider mentioning this in the paper as well. Also, highlight in the paper that the proposed methods are applied only *intra-class*.
5. Why does the performance after fine-tuning the model pre-trained using contrastive learning decrease from 0.78 to 0.77 (Table 5), while it increases for the other two models?
6. What's the masking percentage (for Random Subcompositions) in the contrastive learning experiments?

limitations: I couldn't find where the authors mentioned the limitations of the current work. One limitation I identified is proposing three (similar) methods without offering some rule-of-thumb when to use each technique.

I don't see any potential negative societal impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper looks at how one can do data augmentation on compositional data, in order to improve generalization of supervised learning on data of this type, as well as enable contrastive self-supervised representation learning.  The method utilizes vector space structures on the simplex provided by Aitchison geometry to define an augmentation scheme that can compose multiple compositional data types together, similar to what is done with the MixUp method for data types that can be combined together linearly.  They explore augmenting human microbiome data, which is characterized as the fraction of various microbe species in a person’s gut.  The paper finds improvement (some very convincing, others marginal) when classifying various phenotypes based on their microbiome using their data augmentations.  They also study contrastive methods, similar to SimCLR, which are enabled by the compositional data augmentations.
strengths_and_weaknesses: Strengths:

1. I like how the method utilizes the pre-existing Aitchison geometry to keep combinations of data points on the simplex.
2. It is good to see quantification via retraining on multiple train/test splits, compared to variance seen by just modifying the random seed.
3. I have not seen (nor could I find when reviewing the literature) anyone doing SimCLR-style contrastive learning on compositional data; the authors claim this is the first instance they could find, too.


Weaknesses:

1. I don’t really understand why the paper has focused so much on the human microbiome.  As the authors point out, compositional data is ubiquitous & their data augmentation methodology does not use any domain-specific knowledge or anything else that requires the focus to be on the microbiome.  I find the narrow positioning of an otherwise general method is doing the paper a major disservice.


questions: 1. How does the method perform on other compositional data types from other fields?  Why did you focus so much on the microbiome?
2. In the Aitchison Mixup section, step 2: what is the reasoning for requiring the two datapoints you combine to have the same class?  If you follow MixUp, they advocate doing a similar linear combination of labels.
3. (Minor) Did you try other simple augmentations on compositional data, such as shuffling?
4. (Minor) I might have missed it, but did you actually use the Aitchison inner product (Eq 4)?

limitations: The authors didn’t explicitly address limitations or societal impact, although they did describe some improvements they might expect to make on the type of data augmentations, e.g., using domain specific knowledge.  I can’t really think of a major negative societal impact of this paper, though.
review_title: null

Meta-review: metareview: 
This paper proposes three different, easy to implement methods for data-augmentation when learning models on compositional data (where each feature lies in a potentially high-dimensional simplex). The basic idea is that for such data, it is important to create augmentations that respect the fact that data are within the simplex. There was consensus among the reviewers that this work should be accepted. This work was simple, interesting and results in empirical improvements across various choices of models of outcomes. I think this work can have an impact for those building predictive models in applications of machine learning such as microbiome data.

confidence: Certain
Decision: Accept