Data Augmentation for Compositional Data: Advancing Predictive Models of the Microbiome
Authors: Elliott Gordon-Rodriguez
Thomas P Quinn
John Patrick Cunningham
Venue: NeurIPS 2022 Accept (2022)
Keywords: Data augmentation
compositional data
microbiome
supervised learning
contrastive learning

Abstract: Data augmentation plays a key role in modern machine learning pipelines. While numerous augmentation strategies have been studied in the context of computer vision and natural language processing, less is known for other data modalities. Our work extends the success of data augmentation to compositional data, i.e., simplex-valued data, which is of particular interest in microbiology, geochemistry, and other applications. Drawing on key principles from compositional data analysis, such as the \emph{Aitchison geometry of the simplex} and subcompositions, we define novel augmentation strategies for this data modality. Incorporating our data augmentations into standard supervised learning pipelines results in consistent performance gains across a wide range of standard benchmark datasets. In particular, we set a new state-of-the-art for key disease prediction tasks including colorectal cancer, type 2 diabetes, and Crohn's disease. In addition, our data augmentations enable us to define a novel contrastive learning model, which improves on previous representation learning approaches for microbiome compositional data.

Data Augmentation for Compositional Data: Advancing Predictive Models of the Microbiome
Elliott Rodriggn Columbia University og29120columbia edu
Thomas Quinn Independent Scientist contacttoonuiim@@aail.co
John P. Cunninghham Columbia University jpc21818columb1 a edu
Abstract
Data augmentation plays key role modern machine learning pipelines. While numerous augmentation strategies have been studied in the context of computer vision and natural language processing, less is known for other data modalities. Our work extends the success of data augmentation to compositional data, i.e., simplex- cmmple--aluued data, which is particular interest microbiology, geochem- istry, and other applications. Drawing key principles from compositional data analysis, such as Ailchison geomeiry of the simplex and subcompositions, we define novel augmentation strategies for this data modality. Incorporating our data augmentations into standard supervised learning pipelines results in consistent per- formance gains across wide range of standard benchmark datasets. In particular, we set a new state-of-the-art for key disease prediction tasks including colorectal cancer, type 2 diabetes, and Crohn's 's discase. In addition, our data augmentations enable us to define a novel contrastive learning model, which improves on previous representation learning approaches for microbiome compositional data
Introduction
Data augmentation, i.e., generating synthetic training examples be used model fitting, is core component of modern deep learning pipelines [54] [7]. In computer vision, augmentations such as image resizing and shifting have been used since as early as LeNet-5 351 These and many other augmentations have become essential to highly successful state-of-the-art architectures, ranging from AlexNet Rd and ResNet 29 to recent contrastive learning models such as SimCLR [ and MoCo 28 As such, growing body of literature has emerged to develop and characterize
data augmentation techniques, particularly in computer vision 451
65], as well as 22 [ However, defining useful data augmentations is highly
36
IK]
T0
natural language processing
[53]
domain -emain-dependent, and fewer works have studied augmentations for more general data modalities, such tabular data 66 64] The goal of our work is to extend the success of data augmentation to
previously unexplored data modality; namely, compositional data.
Compositional data (CoDa) are those that represent the parts of whole, and therefore carry only relative information. Equivalently, we can think of CoDa set of datapoints living in the simplex:
D far AP-11 11 Jen where AP-1_
R
Σ
Examples of CoDa arise throughout the sciences, including microbiology (proportions of different species a bacterial community) 22 12 50 3 geochemistry (chemical composition of a geological
sample) [ 4 42], materials science (components of matter) 440 [] and many more. As a result,
Our code is available http:///gggtuuu..ooo///////aaa//
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

starting with the seminal work of Aitchison numerous bespoke techniques have been developed for the statistical analysis of CoDa 4 1 18
A rapidly growing area of application for CoDa is the human microbiome, which comprises the populations of microorganisms that reside inside and on the human body 59) Microbiome data arise from an inexhaustive sampling procedure as result of high-throghhuut sequencing [23] [5]] In
particular, each feature typically represents the relative abundance of some species of microorganism: as such, each observation can be normalized to the simplex prior to downstream analyses 23 500
37
The microorganisms that constitute the microbiome are known to impact human physiology, both in health and in disease [2]
38 20 Thus, central problem in CoDa is to learn the association X; + yi, where x; E AP-1 denotes the microbial composition, and yi denotes the disease status of the ith subject in clinical study. For example, the composition of the gut microbiome, as recorded from a stool sample, can be predictive of colorectal cancer, which is the third most prevalent form of cancer T6 3 This observation offers the potential for a noninvasive alternative to traditional
colonoscopy procedures used for early detection of colorectal cancer [56] In turn, accurate predictive models for microbiome CoDa are a key stepping stone towards achieving this potential 39 [K] Note that cancer is just one application; the demand for improved predictive models holds more broadly across medicine and biological science B3 21
For example, the microbiome has been linked to
type 2 diabetes, Crohn's 's disease, obesity, and others
20
Classical techniques including logistic regression, support vector machines, and random forests have been widely used as predictive models for microbiome data [60] 5. 36). More recently, specialized deep learning architectures such DeepCoDa 48, MetaNN 37 and DeepMicro (41 have been
developed. However, the capacity of these deep networks and other expressive models has been limited by the low sample size and high dimensionality of typical mierobiome studies. These characteristics have also spurred the use of strong regularization through early stopping, weight decay and dropout, among others (57 48 37 A.l. However, no previous works have explored the use of data augmentation for CoDa which, as we shall demonstrate, provides a cheap and simple technique for boosting the performance of predictive models for this data modality.
Careful consideration of the sample space will motivate our novel data augmentation strategies for CoDa Our work draws foundational principles from the field of CoDa, such as the Aitchison geometry of the simplex [.. which we combine with popular techniques from the literature on data augmentation, such Mixup (6c and CutMix 65 This combination will lead to define custom data augmentation strategies for CoDa, such as Aitchison Mixup and Compositional CutMix. In turn, incorporating these novel augmentations into existing supervised learning pipelines will result in consistent performance gains across a wide range of microbiome datasets, including a dozen standard benchmarks from the Microbiome Learning Repo (60). These performance gains are particularly large for some deep models. for example, DeepCo oD gains more than 10% in test AUC for discriminating colorectal cancer from healthy controls, and over 20% for type 2 diabetes. The gains are also significant across other expressive model families, including random forests and gradient boosting machines. Overall, we set new state-of-the-arr on out of 12 benchmark learning tasks, including clinically relevant disease prediction tasks. Of the remaining 4 tasks, 2 were easily separable, with 100% test accuracy irrespective of whether data augmentation was used. Importantly, our augmentations rarely hurt model performance, and in the few instances that this was the case, the drops were typically of only 1% in test AUC.
In addition to supervised learning, our novel augmentations will allow us to define contrastive representation learning for CoDa, which to the best of our knowledge, represents the first contrastive learning model for compositional data. Our novel data augmentations are at the core of this approach; the contrastive loss uses randomly augmented training examples to define a self-supervised opti- mization objective whose labels are generated from unlabelled data [26 14] 9 28 In particular, our contrastive model is trained to discriminate between compositions that were generated as random augmentations of the same training example, and those that were generated as random augmentations of different training examples. Our implementation adapted from SimCLR ( but using a smaller network architecture together with our novel augmentations. Unlike SimCLR, we use our individual
With the possible exception of dropout, which can be viewed as a form of data augmentation. Note that applications of dropout to CoDa, such as [37]. use standard implementations that do not exploit the special structure of CoDa.
2

---

augmentation strategies in isolation, and find them sufficient to surpass previous representation learning approaches for microbiome CoDa 3 Altogether, we expect our data augmentations will enable significant future progress, possibly in combination with novel architectures, in both supervised and representation learning pipelines for microbiome CoDa.
Related Work
Data augmentation:
numerous data augmentation strategies have been proposed in the context of image data (544 and text data [17]. LeNet-5 35 used random shifts and resizing, and AlexNet 33
used reflections and color distortions, which have become widely adopted in computer vision 29
More general purpose data augmentations include Mixup, which generates synthetic samples by taking convex combinations of training examples, and has been applied to both image and tabular data [66]. Mixup is related to an earlier method called SMOTE [ which aims to address class imbalance by generating synthetic samples from the minority class, taken convex combinations between nearest neighbors within the minority class. Random masking, whereby random input features are hidden during training. has also seen diverse range of applications, including natural language processing (c.g., BERT (12)), computer vision (c.g.. Cutout (T3),, and tabular data (c.g.. VIME (44). In the context of CoDa, masking related to subcompositions, which we will use to define an analogous augmentation on the simplex. CutMix (65] is akin to both masking and Mixup, in that random patches from different images are pasted together to generate new synthetic images. We note that subsequent line of work has developed techniques to automatically select optimal data augmentations for a given dataset [1O [. somewhat akin to hyperparameter optimization. While the data augmentation strategies that we present in the next Section could also be combined and finetuned in similar ways, such experimentation falls outside the scope of our work and left to future research.
Microbiome models: random forests remain a strong baseline across many microbiome studies, due to their expressivity and robustness [57]. Gradient boosting machines such as XGBoost provide
similarly strong performance and have also enjoyed significant adoption [57] AutoML 30 has
shown potential for microbiome data, with recent approach called mAML achieving state-of-the-
art results on several benchmarks [63] mAML uses cross-validation to automatically select the best model class and hyperparameters for each learning task. In addition, several specialized deep learning architectures have been proposed for microbiome CoDa. DeepCoDa 4S] introduced the log- rggbbotleneck layer, which ensures the hidden units remain models. This architecture obtained strong predictive performance using weight decay regularization. MetaNN 3 employs multilayer architecture regularized using dropout [55]. The authors of MetaNN also considered generating synthetic data from a negative binomial distribution, but found no performance improvements beyond the use of dropout. DeepMicro 41 uses deep autoencoder architecture to learn low-dimensional representations of the microbial composition. In turn, these representations are fed to downstream classifier trained with supervised objective.
key desiderata in CoDa
the goal contrastive learning is to learn low-dimensional representations by optimizing some pretext task, where the objective function is similar to those used for supervised learning, but using labels that are derived from unlabelled data only (26] Commonly, the pretext task is to discriminate augmented instances of the same training example from augmented instances of different training examples [4] 9 28 Recent contrastive learning architectures have enjoyed tremendous success, setting the state-of-ther ofee-o--t-hhttttt art across range of computer vision benchmarks 9 28 B1 77 We highlight SimCLR ( for its relative simplicity and strong empirical performance. This method uses nonlinear projection head between the representations and the contrastive loss computed with normalized embeddings. Our own experiments on contrastive learning borrow these elements from SimCLR, but use smaller network architecture together with our specialized data augmentations for CoDa.
Contrastive learning:
3 Methods
In the following subsections, we introduce novel data augmentation techniques for CoDa. Note that many variations of our augmentation schemes could be considered; the versions we present here are intended to be as concise as possible. Our goal not to design augmentations that are empirically or theoretically optimal, but rather to demonstrate the effectiveness of our novel methodology by
3

---

establishing simple and performant baselines. For instance, whenever a random mixing parameter required, we use (O, 1) distribution, even though other choices can likely result in increased performance. For clarity, we will focus on the classification setting; generalizing regression problems is straightforward. We will use the notation v, A for simplex-value vectors, E I. a scalar, and D = (x,, Vo our training data.
3.1 Aitchison Mixup
Aitchison defined a Hilbert space structure on the sim- plex, known as the Aitchison geomelry, with the following vector addition, scalar multiplication, and inner product
V x
11 PpT
(2)
Dj=1 VyJ
1
A0X
[1 P xpl.
(3)
Σ)=
(v,x) = 2p 1=1
log
log
Tij
(4)
Uk
This geometry provides principled definition of linear combinations and distance metric on the simplex; par- allel and orthogonal lines are illustrated in Figure I The Aitchison geometry is closely related to the centered-log- ratio transformation M. which an inverse of the soft- max and defines an isometry between A and Euclidean space 44 Taken together, these properties form the basis of much of CoDa methodology [ 43 [1]].
Figure 1: Orthogonal grid on A in the Aitchison sense A4 The centroid of the simplex corresponds to the additive identity. The red lines are parallel and equally spaced by unit Aitchison dis- tance, are the blue lines. The red lines are also orthogonal to the blue lines.
Our first data augmentation strategy, which we call Aitchison Mixup, is based taking linear combinations of the training points, in the Aitchison sense. While general linear combinations may be used, for simplicity we focus intra-class pairwise convex combinations. Namely, each new datapoint is sampled as follows:
1. Draw class from the class prior and draw À ~ (O, 1).
2. Draw two training points 11, 42 such that yi, = Vis = c, uniformly at random.
3. Set =" (> @ x.,) @ ((1 X) xx,) and n
Put more succinctly, for each class we generate convex combinations of the points in that class, in the Aitchison sense. Note this augmentation strategy is a CoDa analogue of Mixup, which generates Fuclidean convex combinations of images and tabular data [66. Much like Mixup capable of boosting predictive accuracy by regularizing models towards a linear decision boundary in the regions between training examples, Aitchison Mixup aims to enforce the equivalent regularization in the Aitchison geometry of the simplex.
Note that, once we restrict to convex combinations (as opposed to general linear combinations), our augmented data will remain inside the simplex regardless of whether we operate in the Euclidean or the Aitchison geometry. However, found the latter to be more empirically effective, perhaps unsurprisingly given the associated vector space structure of the simplex.
We also note that the authors of Mixup (66] sampled À Beta(a, a), where is hyperparameter, and took inter-class as well as intra-class convex combinations, setting ge = Ay; + (1 - X)9 While Aitchison Mixup can also incorporate such generalizations, we chose not to do so in our implementation because, as discussed above, our focus was on finding the simplest performant augmentation strategies for our new data modality []
NNte that our method makes use of the Aitchison addition and scalar multiplication, but not the inner product, which is included for completeness.
"Inter-class combinations represent significant complication to our methods because some classification libraries, including mAML, do not allow for non- binary outputs (without substantial rewrites) which are obtained when mixing labels from different classes. We also observed that, on DeepCoDa and MetaNN, using inter-class combinations did not improve performance.

---

3.2
Compositional Feature Dropout
In CoDa, a subcomposition of x A refers a lower-dimensional composition, x A R-1 with & < p, formed by taking subset of the elements of x and renormalizing to unit total. Namely, ji.... ...k denotes a subset of the indices fl.....P! .p) the corresponding subcomposition is defined as:
X sub
[
Fj..
(5)
2/
2/
We can generate augmented data by taking random subcomposition of the training points; such a strategy is analogous to masking in language data or cropping image data, or more generally, feature dropout [55] Since our predictive models require inputs of fixed dimension p, rather than discarding elements of x we simply zero them out. Thus, each new datapoint is generated as follows:
1. Draw X ~ (0, 1). Draw a training point uniformly at random and set X = X;.
2. For each j E [l...... .p. draw 1; kid Bernoulli(M) A), and set ã = 0 if 1 = 3. x / [- Jby) and ym V/-
In short, we zero out random entries of the training points and renormalize. Intuitively, this strategy encourages our predictive models to become robust partially observed inputs. Note that, distinc- tively from feature dropout, our augmentation includes an additional renormalization to a unit total, and we therefore use the term Compositional Feature Dropout. Importantly, this renormalization ensures that the augmented samples remain in the support of the training data, i.e., the simplex.
Note that many CoDa models, including mAML and DeepCoDa, apply log transformations their inputs, and therefore require that these be non-zero. For this reason, we implement a eero-replacement step where add small positive quantity all the parts of each composition and renormalize. This small quantity is set to 1/L,, where L; corresponds to the library size from the high-throughhut sequencing procedure. L, can simply be thought of as a large number such that 1/L, is smaller than the non-zen zero components of x,. Note that this transformation is standard preprocessing step in CoDa [50].
3.3
Compositional CutMix
Our third augmentation scheme, which we call Compositional CutMix, combines elements of our previous 2 augmentations. Like in Mixup, we generate new datapoints by combining pairs of training points from the same class. However, instead combining these training points linearly (in the Aitchison sense), take complementary subcompositions and renormalize. Namely, generate each new datapoint as follows:
1. Draw class from the class prior and draw X ~ U(0, 1).
2. Draw two training points i,, 12 such that Vi, = Vig = c, uniformly at random.
3. For each j E {l.....p} draw Ij
sid Bernoulli()). and set á, Iiy if 1, 0 ă, = Fisf
if I = 1.
4. Set d0g
x (x 5;)) and y= c.
Note this strategy can be thought of as CoDa analogue of CutMix, whereby new images are formed by pasting together patches from different training images 665 Note also that in computer vision, CutMix and related methods take local image patches as opposed to randomly sampled pixels. Likewise, one could sample microbial subcompositions according to biologically relevant groupings, for example using phylogenetic tree (50]. Such strategy would require incorporating additional domain knowledge ledge and left to future work, but we expect it would further increase the quality of our data augmentations. This remark applies both to Compositional Feature Dropout and Compositional CutMix. With regard to inter-class combinations, a similar discussion as Section 3.1 applies.
Experiments
4.1 Supervised learning
We evaluate our augmentation strategies on 12 standard binary classification tasks taken from the Microbiome Learning Repo 60). These datasets comprise various discase and phenotype prediction
5

---

Table 1: Evaluation benchmark consisting of 12 binary classification tasks taken from the Microbiome Learning Repo [60|, after filtering to datasets containing at least 100 samples with at least 50 each class. For each task we show the number of samples (n), the number of features (p), description of the two classes and the number of samples each, together with a reference the original studies that each dataset was obtained from.
Task
p
Class 1 Class 2
# in 1
in 2
Reference
1
140
992
Crohn's disease / Without (ileum)
78
62
2
160
992
Crohn's s disease / Without (rectum)
68
92
20
3
2070
3090
Gastrointestinal tract Oral
227
1843
38
4
180
3090
Female Male
82
98
38
5
404
3090
Stool Tongue (dorsum)
204
200
38
6
408
3090
Subgingival / Supragingival plaque
203
205
38
172
980
Healthy / Colorectal cancer
86
86
33
8
124
2526
Diabetes Without
65
59
2.6
9
130
2579
Cirrhosis Without
68
62
10
199
660
Black / Hispanic
104
95
52
11
342
660
Nugent score high Low
97
245
12
200
660
Black / White
104
96
522
Crohn's disease 20 and cirrhosis 47 as well multiple body sites including the gut, skin, oral cavity, airways, and vagina 38 52 As such, this benchmark provides a comprehensive evaluation for predictive models of the human microbiome [63] 48 24. More details on these 12 learning tasks can be found in Table I note this benchmark is constructed from the original Microbiome Learning Repo by filtering datasets that contain minimum sample size of 100, with least 50 either class.
lasks, including colorectal cancer 333, type 2 diabetes 46
For each learning task, we take 20 independent 80/20 train/test splits and we fit Random Forest, XGBoost, mAML [63] DeepCoDa [.). and MetaNN 67 first the original training data, then on 3 augmented training sets obtained using our 3 augmentation strategies. Thus, we train a total of 12 20 our datasets do not present severe class imbalance (with the exception of tasks and 11, both of which were well separated by all our classifiers and therefore had no effect on the overall comparison).
4 4 800 models F
We evaluate test performance using ROC AUC, and we note that
For the augmented training sets, we generated 10 times as many synthetic samples as there were original training examples. The factor of 10 was chosen sample, in order to reduce the sampling variance from our random augmentations. In turn, we compensate for the fact that our augmented data is then much more numerous than our original training data, by downweighting the synthetic samples by a factor of 10; the total weight of the original and synthetic data then equal 1/2 each.
obtain relatively large augmented
Our results are shown Table (Aitchison Mixup) the main text, as well as Tables (Composi- tional Feature Dropout), and 6 (Compositional CutMix) in the Appendix:
Table 2 shows that Aitchison Mixup improved the test performance of existing models across large majority of learning tasks. On average, obtained 5% gain in test AUC for DeepCoDa, 2% for random forests and XGBoost, and 1 % for mAML. MetaNN remained flat, but this was also the worst performing model overall. Importantly, in the few instances where Aitchison Mixup hurt the performance of a model, the loss was typically of just 1%. Table that Compositional Feature Dropout also result in consistent performance improvements across most models and tasks, though to slightly lesser degree than Aitchison Mixup. On average, DeepCoDa enjoyed 3% gain in test AUC, random forests 2%, and XGBoost and MetaNN both 1% (mAML remained flat).
shows that Compositional CutMix obtained the strongest classification performance out of our 3 augmentation strategies on microbiome CoDa. On average, DeepCoDa saw 5% gain in test AUC, with 2% for random forests and XGBoost, and 1 % for mAMI. and MetaNN. Moreover, models trained with Compositional CutMix set on 8 out of 12 tasks, including colorectal cancer (0.76), type 2 diabetes (0.72), and Crohn's
state-of-ther the-art
We train these models in parallel CPU cluster.

---

Table 2: Data augmentation performance for Aitchison Mixup. We show the test AUC, averaged over 20 train/test bootstraps, for each learning task and predictive model, trained with and without data augmentation. Bold numbers indicate whether the version with or without augmentation performed best. Underlined numbers indicate the overall best model for that task. Models trained with Aitchison Mixup consistently outperformed those without (with the possible exception of MetaNN, which
performed worst across the board). Error bars are given in Appendix
Task
RF
Aug
XGB
Aug
mAML
Aug
DeepCoDa
Aug
MetaNN
Aug
0.72
0.79
0.76
0.79
0.72
0.74
0.73
0.79
0.74
0.74
2
0.78
0.82
0.81
0.80
0.80
0.80
0.78
0.83
0.74
0.74
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
4
0.60
0.64
0.57
0.57
0.56
0.58
0.58
0.58
0.50
0.51
5
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
6
0.81
0.83
0.82
0.83
0.84
0.83
0.78
0.82
0.75
0.76
7
0.68
0.67
0.67
0.69
0.73
0.74
0.63
0.73
0.59
0.54
8
0.62
0.65
0.66
0.68
0.64
0.65
0.45
0.70
0.64
0.64
9
0.93
0.93
0.94
0.95
0.92
0.93
0.84
0.90
0.76
0.82
10
0.53
0.60
0.57
0.61
0.61
0.62
0.62
0.63
0.63
0.61
11
0.98
0.98
0.98
0.98
0.98
0.98
0.98
0.98
0.96
0.95
12
0.55
0.61
0.58
0.65
0.61
0.61
0.66
0.65
0.58
0.60
Mean
0.77
0.79
0.78
0.80
0.78
0.79
0.75
0.80
0.74
0.74
AUC gain from sugmentation
RF
XGB
mAML
DeepCoDa
MetaNN
0.10
0.10
0.10
0.10
0.05
0.1
0.00
0.00
0.00
-0.05
-0.05
-0.05
-0.05
100
200
300
100
200
300
100
200
300
200
300
200
300
Training
Treining
Training set
Training
Training
Figure 2: Difference in test AUC between models trained with Compositional CutMix, and those without, shown function of training set size. Each point represents of our 12 benchmark learning tasks (note that tasks and are removed from the plot, since they already enjoyed 100% classification accuracy prior applying any data augmentation). The linear trendlines show somewhat greater outperformance on the smaller datasets relative to the larger datasets.
disease (0.78 and 0.83). Of the remaining 4 tasks, 3 were tied and only on was the best model one that did not use Compositional CutMix.
Figure shows the effect of data augmentation on test performance function of the number of samples in the training set, for Compositional CutMix. We note that datasets with smaller sample sizes tended to benefit more from data augmentation. This suggests that our methodology may prove beneficial to many microbiome research projects, where datasets with low-hundreds of samples are commonplace, Figure S shows a similar scatterplot, as a function of the dataset dimensionality rather than sample size; the gains from augmentation appear consistent across lower- and digher-dimensional microbiome CoDa (we observe similar trends Aitchison Mixup and Compositional Feature Dropout, and we omit the corresponding scatterplots).
In addition, we evaluated the impact of our data augmentations on the expected calibration error (ECE) (25 of our models. Previous works have noted that increasingly flexible predictive models such as deep neural networks, while enabling greater predictive accuracy, tend to become overconfident in their predictions, degrading uncertainty quantification [25] In Appendix A we show the ECE
obtained by our models, and we verify that our data augmentations not hurt model calibration overall (in fact, some modest improvements obtained).

---

AUC gain from augmertttioo Retion
RF
XGB
MAML
DeepCoDa
MetaNN
0.10
0.10
0.10
0.10
0.2
0.05
0.05
0.05
0.05
0.00
0.00
0.00
0.00
0.0
-0.05
-0.05
-0.05
-0.05
2000
1000
2000
2000 Dataset dimension
1000
1000
2000
2000
Dataset dimenssion
Dataset dimersion
Dataset dimension
Dateset dimension
Figure 3: Similar to Figure 2 Difference in test AUC between models trained with Compositional CutMix, and those without, shown as a function of dataset dimension.
4.2 Contrastive representation learning
We have shown that our novel data augmentation strategies provide consistent performance gains across supervised training pipelines. Next, we use our data augmentations order to apply contrastive representation learning to compositional data for the first time (to the best of our knowledge). Note that our experiment provides a simple proof of concept showing that contrastive learning capable of learning useful representations of microbiome CoDa, which result in better downstream classification performance than previous deep representation learning approaches. However, more extensive experiments will be required to validate the performance of contrastive learning across a wider range of model architectures for CoDa.
As point of comparison we take DeepMicro [1), deep autoencoder designed specifically for mierobiome data. DeepMicro includes an encoder and decoder that are trained jointly by minimizing reconstruction error. The encoder outputs are used as a low-dimensional representation that is passed downstream classifier. Our contrastive model discards the decoder and trains the encoder using the contrastive loss from SimCLR ! This loss function requires pairs of positive and negative datapoints, where positive pairs are constructed by taking synthetic augmentations of the same training example and negative pairs are constructed by taking synthetic augmentations of different training examples In the context of microbiome CoDa, it our novel augmentations that allow us to define positive and negative pairs required for contrastive learning. Full detail on model architecture and implementation is provided in Appendix B
After applying unsupervised pretraining (reconstructive contrastive), we evaluate the quality of learned representations under two standard protocols: linear evaluation and finetuning. linear evaluation, we freeze the encoder weights and train a linear classification head using the supervised cross-entropy loss on the original training set. In finetuning, the linear classification head trained jointly with the encoder network, again using the supervised cross-entropy loss. The finetuning protocol is evaluated both with and without using additional data augmentation (i.e., passing additional synthetic examples to the supervised loss). These evaluation protocols are applied for both DeepMicro and our contrastive model, in addition to a "no pretraining" control. This control a randomly initialized encoder network and linear head with the same architecture, trained only on the supervised objective (i.e., a neural network that did not enjoy the benefit of unsupervised pretraining).
The results are shown in Table 3 the representations learned by our contrastive model using Composi- tional Feature Dropout consistently outperform those learned by DeepMiero, both linear evaluation and finetuning. Note that finetuning with augmentation performs slightly better than without, sug- gesting that the benefits of supervised training with our data augmentations can combined with the benefits of augmentation as part of contrastive learning. Importantly, note that the comparison to DeepMicro is conservative, in the sense that we replicated the architecture and simply changed the pretraining objective from reconstruction error to contrastive loss; tuning the encoder architecture itself under a contrastive objective would likely lead to further performance improvements. We conclude that Compositional Feature Dropout provides valuable data augmentation for contrastive learning on the simplex.
Note that our other 2 augmentation strategies, Aitchison Mixup and Compositional CutMix, may also be used for contrastive learning. However, these augmentations require paired training examples to generate each synthetic sample, and the implementation is therefore slightly more involved. In
NNte such pairs are defined without reference to the (possibly unknown) true labels.

---

Table 3; Representation learning performance for DeepMicro and contrastive learning with Composi- tional Feature Dropout. For each task, we show the test AUC, averaged over 20 train/test bootstraps, under the linear evaluation protocol and finetuning. We also show the effect of using additional data augmentation as part of supervised training during the finetuning stage ("Finetuning with Aug"). In addition, a randomly initialized encoder network of the same architecture with no pretraining is shown for comparison. Note that the representations obtained via contrastive learning consistently achieve higher AUC than those learned by DeepMicro. Error bars are given in Appendix A No Prezno pretraining, DM-DrepMicro autoencoder pretraining, Contr=contrastive pretraining (ours).
Linear Evaluation
Finetuning DM
Finetuning with Aug
Task
No Pre.
DM
Contr.
No Pre.
Contr:
No Pre.
DM
Contr.
1
0.59
0.68
0.76
0.72
0.75
0.77
0.77
0.76
0.77
0.67
0.76
0.80
0.76
0.76
0.79
0.78
0.79
0.80
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.47
0.53
0.59
0.50
0.53
0.55
0.52
0.56
0.56
5 1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
6 0.75
0.76
0.77
0.77
0.76
0.77
0.78
0.78
0.78
0.59
0.68
0.63
0.59
0.59
0.57
0.57
0.62
0.60
8
0.65
0.66
0.66
0.66
0.68
0.68
0.66
0.66
0.68
0.72
0.71
0.86
0.76
0.77
0.82
0.79
0.80
0.82
10
0.53
0.58
0.62
0,60
0.61
0.63
0.62
0.60
0.63
11
0.96
0.98
0.98
0.98
0.98
0.98
0.97
0.97
0.97
12
0.66
0.68
0.66
0.62
0.64
0.64
0.62
0.65
0.65
Mean
0.72
0.75
0.78
0.74
0.76
0.77
0.76
0.77
0.78
particular, we generate each pair of positive samples by taking one pair of training examples, and drawing two random combinations (in the Aitchison Mixup or the Compositional CutMix sense) of that one pair. The training examples are randomly partitioned into pairs each epoch, and negative samples correspond to those that originate from separate pairs. The results are provided in Appendix AJand show similarly strong performance to Table
4.3 Additional benchmarks
Further to the 12 microbiome classification tasks from the previous sections, we analyze 8 additional compositional datasets, which were selected to represent diverse set of applications that are commonly encountered by CoDa practitioners. Note that, while these datasets correspond to wide range of scientific domains, they have all been analyzed as case studies specifically the CoDa literature. The datasets can be summarized follows:
Glass: a dataset from the UCI ML repo, on the elemental composition of glass particles of different types of origin. Such particles are routinely recovered from the clothing of criminal suspects; the goal is to provide forensic evidence by classifying the type of origin (broken window vs. other) given the composition recorded in a multielement analysis [5] 58]
Bayesite: classical materials science dataset due Aitchison [I] The goal is to predict the permeability of fibreboard from the mix of its ingredients.
Serum: another classical dataset due Aitchison ①I. where the composition of blood serum protein is used to discriminate between two known diseases.
Hydrochem: hydrochemical dataset measuring the composition of water samples across sites in the Llobregat river basin 42 Classification targets are formed by separating upstream and downstream sites.
Jura: geochemical dataset from the Swiss Jura, where rock types (Kimmeridgian vs.
Sequanian) are classified from the chemical composition of soil samples 2
Metabolites: gut metabolomic profiles for subjects with and without inflammatory bowel
disease 1| 49 The goal diagnose disease from the metabolite composition.
MicroRNA: contains microRNA expression data for different types of primary breast cancer
6 49). The goal discriminate between breast cancer subtypes (Luminal vs. Luminal B) from the molecular profile of each tumor.

---

Table 4: Data augmentation performance for Aitchison Mixup on non non-microbiome CoDa benchmarks. We show the test AUC, averaged over 20 train/test bootstraps, for each non-microbiome dataset and predictive model, trained with and without data augmentation. Bold numbers indicate whether the version with or without augmentation performed best. Underlined numbers indicate the overall best model for that task. Models trained with Aitchison Mixup consistently outperformed those without.
Dataset
n/p
RF
Aug
XGB
Aug
DeepCoDa
Aug
NN
Aug
Glass 15
213/8
0.84
0.86
0.84
0.86
0.83
0.84
0.84
0.84
Bayesite
0
20/5
0.86
0.86
0.84
0.90
0.69
0.54
0.53
0.58
Serum |II
30/4
0.81
0.83
0.80
0.82
0.62
0.82
0.87
0.82
Hydrochem
22
246/14
1.00
1.00
1.00
1.00
0,96
0.99
0.95
0.95
Jura 2
147/7
0.93
0.94
0.92
0.94
0.83
0.93
0.75
0.72
Metabolites 19
220/885
0.95
0.96
0.94
0.94
0.84
0.94
0.92
0.93
MicroRNA
61
717/188
0.90
0.90
0.91
0.92
0.87
0.90
0.90
0.90
Coffee
62
30/15
0.92
0.93
0.80
0.83
0.71
0.77
0.65
0.69
Mean
0.90
0.91
0.88
0.90
0.79
0.84
0.80
0.81
Coffee: a nutrition dataset describing the chemical composition of commercially available coffee samples Classification targets are taken as different blends of coffee.
In modeling these datasets, we apply the centered logratio transformation the model inputs M and follow the experimental protocol described in Section 4.1 except we no longer test mAML and MetaNN (which were not intended for non-microbiome data), and we replace the latter with a small neural network with one hidden layer of 32 units and ReLU activations, denoted as NN (note we choose keep this architecture very simple as it is trained on several heterogeneous dataset).
The results are shown in Table for Aitchison Mixup. As was the case for our microbiome benchmark, again see that our data augmentation results in consistent performance gains across wide range of classifiers. Indeed, on most datasets (6 out of 8) the best performing model was one that used single dataset (Serum) was the best model (NN)
Aitchison Mixup, and only without augmentation. Note that, in contrast to our microbiome benchmarks, Table T shows a strong outperformance for tree-based models. Importantly, Aitchison Mixup never hurt the performance of any of our tree-based models, and only did so rarely for our network models. We observe similarly positive results (albeit slightly weaker) for Compositional Feature Dropout and Compositional CutMix, provided in Tables and g from the Appendix, respectively.
trained
Conclusion
By combining ideas from data augmentation with the principles of CoDa, we have defined novel augmentation strategies for CoDa: Aitchison Mixup, Compositional Feature Dropout, and Composi- tional CutMix. Our augmentations offer cheap performance gains across a wide range of benchmark datasets, advancing the state-of-te of-the-art on standard classification benchmarks for colorectal cancer, type diabetes, and Crohn's 's disease. While we find all 3 strategies performant across a wide range of benchmark datasets, Aitchison Mixup and Compositional CutMix performed best on microbiome CoDa (with a slight outperformance for the latter), and Aitchison Mixup performed best on non- microbiome CoDa. In addition, our data augmentations allowed to define the first contrastive learning model for CoDa, which we show improves on existing representation learning frameworks for microbiome data. Data augmentation and contrastive learning have enjoyed tremendous success in other application domains such as computer vision; our novel methodology aims to help spur similar developments in the fields of CoDa and the microbiome.
Acknowledgments and Disclosure of Funding
We thank Samuel Lippl, Richard Zemel, and the anonymous reviewers for helpful discussions, and the Simons Foundation 542963, Sloan Foundation, McKnight Endowment Fund, NSF DBI-1707398, and the Gatsby Charitable Foundation for support.
10

---

References
[1] J. Aitchison. The statistical analysis of compositional data. Journal of the Royal Statistical Society: Series B (Methodological), 44(2):13-111 1982.
[2] O. Atteia, J.-P. Dubois, and R. Webster. Geostatistical analysis of soil contamination in the swiss jura. Environmental Pollution, 86(3):15-327, 1994.
[] M. Avalos, R. Nock, C. Ong, J. Rouar, and K. Sun. Representation learning of compositional data. Advances in Newral Information Processing Systems, 31, 2018.
[4] A. Buccianti and E. Grunsky. Compositional data analysis in geochemistry: Are we sure to see what really occurs during natural processes?, 2014.
[5] A. Buccianti, G. Mateu-Figueras, and V. Pawlowsky-Ginn Glahn, Compositional data analysis in the geosciences: from theory to practice. Geological Society of London, 2006.
[6] G. Cammarota, G. Ianiro, Ahern, C. Carbone, A. Temko, M. J. Claesson, Gasbarrini, and
G. Tortora. Gut microbiome, big data and machine learning to promote precision medicine for cancer. Nature reviews gastroenteralogy
hepatology, 17(10):3356648 2020.
[7] M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin. Unsupervised learning of
visual features by contrasting cluster assignments. Advances in Neural Information Processing Systems, 03:9912-9922 2020.
[8] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.
[9] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607, PMLR, 2020.
[10] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q V. Le. Autoaugment: Learning augmentation policies from data, arXiv preprint arXiv:I805, 005,09 2018
[11] E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le. Randaugment: Practical automated data augmentation with reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 702-703, 2020.
[12] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04803 2018
[13] T. DeVries and G. W. Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv: 04552, 2017.
[14] A. Dosovitskiy, J. T. Springenberg, M. Riedmiller, and T. Brox. Discriminative unsupervised feature learning with convolutional neural networks. Advances in neural information processing systems, 27, 2014.
[15] I. W. Evett and E. J. Spiehler. Rule induction in forensic science. In Knowledge Based Systems, pages 162-160. 1989.
[16] Favoriti, G. Carbone, M. Greco, F. Pirozzi, R. E. M. Pirozzi, and F. Corcione. Worldwide burden of colorectal cancer: review. Updates in surgery, 68(1):7-11, 2016.
[17] Y. Feng, Gangal, Wei, S. Chandar, S. Vosoughi, Mitamura, and Hovy. survey of data augmentation approaches for nlp. arXiv preprint 2rXiv:2105.03077 2021.
[18] P. Filzmoser, K. Hron, and M. Templ. Applied compositional data analysis. Cham: Springer, 2018.
[19] E. A. Franzosa, A. Sirota-Madi, J. Avila-Pacheco, N. Fornelos, H. J. Haiser, S. Reinker, Vatanen, A. B. Hall, H. Mallick, I.. J. Mclver, et al. Gut microbiome structure and metabolic activity in inflammatory bowel disease. Nature microbiology, 2(2):293-305, 2019.

---

[20] D. Gevers, S. Kugathasan, L. A. Denson, Y. Kázquez-Baeza, W. Van Treuren, B. Ren, E. Schwa- ger, D. Knights, S. Song, M. Yassour, al. The treatment-nan naive microbiome in new-onset crohn's disease. Cell host & microbe, 15(3):882-392, 2014.
[21] A. Gilbert, M. Blaser, J. G. Caporaso, J. K. Jansson, S. V. Lynch, and R. Knight. Current understanding of the human microbiome. Nature medicine, 24(4):392-400) 2018.
[22] G. B. Gloor, J. M. Macklaim, V. Srwwlowsk-GGlann and J. J. Egozcue. Microbiome datasets compositional: and this not optional. Frontiers in microbiology, 8:2224, 2017.
[23] G. B. Gloor and G. Reid. Compositional analysis: valid approach to analyze microbiome high-throughput sequencing data. Canadian journal of microbiology, 62(8)):92--73, 2016.
[24] E. Wordon-Rodrigun T. P. Quinn, and J. P. Cunningham. Learning sparse log-ratios for high-throughput sequencing data. Bioinformatics, 38(1):157-163, 2022.
[25] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks, In International Conference on Machine Learning, pages
PMLR, 2017.
[26] R. Hadsell, S. Chopra, and Y. LeCun. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Puttern Recognition (CVPR'06) volume 2, pages 1735-1742. IEEE, 2006.
[27] A. V. Hartstra, K. E. Bouter, F. Bäckhed, and M. Nieuwdorp. Insights into the role of the microbiome in obesity and type 2 diabetes. Diabetes care, 38(1):159-11)) 2015.
[28] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2020.
[29] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770- 778, 2016.
[30] X. He, K. Zhao, and X. Chu. Automl: A survey of the state-of-the-art. Showledge-Baser Systems, 212:10662) 2021.
[31] P. Khosla, Teterwak, C. Wang, A. Samaa, Y. Tian, Isola, A. Maschinot, C. Liu, and D. Krishnan. Supervised contrastive learning. Advances in Neural Information Processing Systems, 33:18661-18673, 2020.
[32] M. Korhoñová, K. Hron, D. Kliměfková, L. Müller, P. Bednář, and P. Barták. Coffee cooma-statistices analysis of compositional data. Talanta, 80(2):710-715, 2009.
[33] A. D. Kostic, D. Gevers, C. S. Pedamallu, M. Michaud, F. Duke, A. M. Earl, A. I. Ojesina, Jung, A. Bass, J. Tabernero, et al. Genomic analysis identifies association of fusobacterium with colorectal carcinoma. Genome research, 22(2):292-298 2012.
[34] A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
[35] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied document recognition. Proceedings of the IEEE,
1998.
[36] Y.-.. Liu, Y. Qin, T. Chen, M. Lu, X. Qian, X. Guo, and Y. Bai. practical guide amplicon and metagenomic analysis of microbiome data. Protein & cell, 12(5):315-330, 2021.
[37] C. Lo and R. Marculeseu. Metann: accurate classification of host phenotypes from metagenomic data using neural networks. Bmc Bioinformatics, 20(12):1-14, 2019.
[38] B. A. Methé, K. E. Nelson, M. Pop. H. H. Creasy, M. G. Giglio, C. Huttenhower, D. Gevers, J. F. Petrosino, S. Abubucker, H. Badger, et al. A framework for human microbiome research. nature, 286(7402):215 2012.
12

---

[39] M. Mulenga, S. A. Kareem, A. Q. M. Sabri, M. Seera, S. Govind, C. Samudi, and S. B. Mohamad. Feature extension of gut microbiome data for deep neural setwork-based colorectal cancer classification. IEEE Access,
2021.
[40] J. H. Na, M. D. Demetriou, M. Floyd, A. Hoff, G. R. Garrett, and W. L. Johnson. Compositional landscape for glass formation in metal alloys. Proceedings of the National Academy of Sciences, 11 (1(25))903190036)) 2014.
[41] M. Oh and L. Zhang. Deepmicro: deep representation learning for disease prediction based on microbiome data. Scientific reports, 10(1):1-9 2020.
[42] N. Otero, R. Tolosana-Delgado, A. Soler, V. Pawlowsky-Glan and A. Canals. Relative vs. absolute statistical analysis of compositions: a comparative study of surface waters of mediterranean river. Water research, 39(7):1404-1414 2005.
[43] V. Pawlowsky-Glahn and A. Buccianti. Compositional data analysis: Theory and applications.
John Wiley & Sons, 2011.
[44] V. Pawlowsky-Glahbn J. Egozcue, and R. Tolosana Delgado. Lecture notes on compositional data analysis. 2007.
[45] L.. Perez and J. Wang. The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:/712.046660 2017.
[46] J. Qin, Y. Li, Z. Cai, S. J. J. Zhu, F. Zhang, S. Liang, W. Zhang, Y. Guan, D. Shen, et al. A metagenome-wide association study of gut microbiota type diabetes. Nature, 490(74188:555- 60, 2012.
[47] N. Qin, F. Yang, A. Li, E. Prifti, Y. Chen, L. Shao, J. Guo, E. Le Chatelier, J. Yao, L. Wu, et al. Alterations of the human gut microbiome in liver cirrhosis. Nuture, 513(7516);59 2014.
[48] T. Quinn, D. Nguyen, S. Rana, S. Gupta, and Venkatesh. Deepcoda: personalized inter- pretability for compositional health data. In International Conference on Machine Learning. pages 7877-7886. PMLR, 2020.
[49] T. P. Quinn and Erb. Using balances to engineer features for the classification of health biomarkers: new approach to balance selection. bioRxiv, page 600122, 2019.
[50] T. P. Quinn, I. Erb, M. F. Richardson, and T. M. Crowley. Understanding sequencing data as compositions: an outlook and review. Bioinformatics, 34(16):2870-23 2018.
[51] P. Quinn, Gordon-Rodrigue and I. Erb. critique of differential abundance analysis, and advocacy for an alternative. arXiv preprint
2021.
[52] J. Ravel, P. Gajer, Z. Abdo, G. M. Schneider, S. S. Koenig, S. L. McCulle, S. Karlebach, R. Gorle, J. Russell, C. O. Tacket, et al. Vaginal microbiome of reproductive-age women. Proceedings of the National Academy of Sciences, 108(Supplement 1):4680 4687, 2011.
[53] R. Sennrich, B. Haddow, and A. Birch. Improving neural machine translation models with monolingual data. arXiv preprint (0Xi::55/1.060000 2015.
[54] C. Shorten and T. M. Khoshgoftaar. survey on image data augmentation for deep learning. Journal of big data, 6(1):1-48, 2019.
[55] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1): 1929:199991958 2014.
[56] S. Tarallo, G. Ferrero, G. Gallo, A. Francavilla, G. Clerico, A. Realis Luc, P. Manghi, A. M. Thomas, Vineis, N. Segata, et al. Altered fecal small ma profiles in colorectal cancer reflect gut microbiome composition in stool samples. Msystems, 4(5):00029-199 2019.
[57] B. D. Topçuoğlu, N. A. Lesniak, M. T. Ruffin IV, Wiens, and P. D. Schloss. A framework for effective application of machine learning to microhiome- classification problems. MBio, 1 1133)e00434-20, 2020.
13

---

[58] M. Tsagris. S. Preston, and A. T. Wood. Improved classification for compositional data using the a-transformation. Journal of classification, 33(2):243 2016.
[59] J. Turnbaugh, R. E. Ley, M. Hamady, C. M. Frager-Liggett, R. Knight, and J. I. Gordon. The human microbiome project. Nature, 449(7164):0448810 2007.
[60] Vangay, B. M. Hillmann, and D. Knights. Microbiome learning repo (ml repo): A public repository of microbiome regression and classification tasks. Gigascience, 8(5)giz 2019.
[61] N. Weinstein, E. A. Collisson, G. B. Mills, K. R. Shaw, B. A. Ozenberger, K. Ellrott, I. Shmulevich, C. Sander, and J. M. Stuart. The cancer genome atlas pan-cancer analysis project. Nature genetics, 45(10):1 13-1110, 2013.
[62] E. K. Wright, M. A. Kamm, S. M. Teo, M. Inouye, J. Wagner, and C. D. Kirkwood. Recent advances in characterizing the gastrointestinal microbiome in crohn's disease: systematic review. Inflammatory bowel diseases, 21(6):1219-122 1228, 2015.
[63] F. Yang and Q. Zou. maml: an automated machine learning pipeline with microbiome repository for human disease classification. Database, 2020, 2020.
[64] J. Yoon, Y. Zhang, J. Jordon, and M. van der Schaar. Vime: Extending the success of self- and semi-supervised learning tabular domain. Advances in Neural Information Processing Systems, 33:1 10000000000000 1043, 2020.
[65] S. Yun, D. Han, S. J. Oh, S. Chun. J. Choe, and Y. Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference computer vision, pages 6023-6033 2019.
[66] H. Zhang, M. Cissé, Y. N. Dauphin, and D. Lopez-Paz. mixup: Beyond empirical risk mini- mization. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes] Compare Abstract/Intro to Methoods/Epprimeett
(b) Did you describe the limitations of your work? [Yes] See Section and Section D in the Appendix.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] Not the
focus of our paper, but briefly discussed model calibration in Section
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A] No theoretical results.
(b) Did you include complete proofs of all theoretical results? [N/A] No theoretical results.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URLY? [Yes] See supplemen- tal material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section g or code supplemental material.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See appendix A
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Section
14

---

4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section
(b) Did you mention the license of the assets? [No] Assets are open-source.
(c) Did you include any new assets either in the supplemental material or as URL? [Yes]
Code provided supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [No] Datasets used
the public domain.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A] No identifiable or offensive content.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A] Not used.
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A] Not used.
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A] Not used.
15


Reviews:
[{"review_id": "ukTU3EbjtMa", "reviewer": "Reviewer_7STz", "summary": "This paper looks at how one can do data augmentation on compositional data, in order to improve generalization of supervised learning on data of this type, as well as enable contrastive self-supervised representation learning.  The method utilizes vector space structures on the simplex provided by Aitchison geometry to define an augmentation scheme that can compose multiple compositional data types together, similar to what is done with the MixUp method for data types that can be combined together linearly.  They explore augmenting human microbiome data, which is characterized as the fraction of various microbe species in a person’s gut.  The paper finds improvement (some very convincing, others marginal) when classifying various phenotypes based on their microbiome using their data augmentations.  They also study contrastive methods, similar to SimCLR, which are enabled by the compositional data augmentations.", "questions": "1. How does the method perform on other compositional data types from other fields?  Why did you focus so much on the microbiome?\n2. In the Aitchison Mixup section, step 2: what is the reasoning for requiring the two datapoints you combine to have the same class?  If you follow MixUp, they advocate doing a similar linear combination of labels.\n3. (Minor) Did you try other simple augmentations on compositional data, such as shuffling?\n4. (Minor) I might have missed it, but did you actually use the Aitchison inner product (Eq 4)?\n", "limitations": "The authors didn’t explicitly address limitations or societal impact, although they did describe some improvements they might expect to make on the type of data augmentations, e.g., using domain specific knowledge.  I can’t really think of a major negative societal impact of this paper, though.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. I like how the method utilizes the pre-existing Aitchison geometry to keep combinations of data points on the simplex.\n2. It is good to see quantification via retraining on multiple train/test splits, compared to variance seen by just modifying the random seed.\n3. I have not seen (nor could I find when reviewing the literature) anyone doing SimCLR-style contrastive learning on compositional data; the authors claim this is the first instance they could find, too.\n\n\nWeaknesses:\n\n1. I don’t really understand why the paper has focused so much on the human microbiome.  As the authors point out, compositional data is ubiquitous & their data augmentation methodology does not use any domain-specific knowledge or anything else that requires the focus to be on the microbiome.  I find the narrow positioning of an otherwise general method is doing the paper a major disservice.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "twb-gQTchFN", "reviewer": "Reviewer_ymr8", "summary": "The paper presents three data augmentation methods for compositional data. The first method (Aitchison Mixup) takes convex combinations of samples *within* the same class. The second method (Random Subcompositions - similar to applying dropout on features) takes an input, samples a subset of features, makes the values zero, and renormalizes the new point to be on the simplex. The third method (Compositional CutMix) samples two points from the same class, performs CutMix between them, and renormalizes the new point to be on the simplex. \n\nAll three methods are adaptations of existing methods to Compositional Data (i.e., use the Simplex constraint) and are applied only intra-class (while the original Mixup and CutMix are also applied inter-class).\n\nThe paper's contribution is to adapt existing data augmentation methods to Compositional Data and show consistent performance improvement for classification tasks.", "questions": "I provide context for all questions in the “weaknesses” section\n\n1. When to use each of the proposed methods?\n2. Is any trend by comparing Figures 2 and 3 computed for all three proposed methods?\n3. How does contrastive pre-training compare with training from scratch using data augmentation, and contrastive pre-training + data augmentation fine-tuning (my comment about incomplete Section 4.2)?\n4. What's the motivation for applying the data augmentation only *intra-class* and not *inter-class* like the original Mixup and CutMix? Consider mentioning this in the paper as well. Also, highlight in the paper that the proposed methods are applied only *intra-class*.\n5. Why does the performance after fine-tuning the model pre-trained using contrastive learning decrease from 0.78 to 0.77 (Table 5), while it increases for the other two models?\n6. What's the masking percentage (for Random Subcompositions) in the contrastive learning experiments?\n", "limitations": "I couldn't find where the authors mentioned the limitations of the current work. One limitation I identified is proposing three (similar) methods without offering some rule-of-thumb when to use each technique.\n\nI don't see any potential negative societal impact.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n- **Well-written paper**. The paper is easy to follow, and the authors provide good details about the task of learning from compositional data and the proposed methods.\n- **Very simple methods, with good empirical results**. The proposed methods are relatively trivial to implement and don't add any computational overload.\n\n\n# Weaknesses\n- **The contrastive learning experiments seem incomplete**. Although interesting, the experiment from Section 4.2 is evaluated too simplistically and leaves unanswered questions. Table 5 compares a standard MLP encoder that is 1) randomly initialized, 2) pre-trained using an encoder-decoder and 3) pre-trained with contrastive learning and the proposed data augmentation. Table 5 shows that pre-training using contrastive learning provide more linearly separable representations than the representations learned by an autoencoder.\n\t- However, Section 4.2 raises the question: \"When should use contrastive pre-training rather than standard supervised training with the proposed data augmentation (like in section 4.1)?\" because the results from Table 5 aren't compared with the performance of an encoder trained using data augmentation (like in section 4.1).\n\t- I suggest adding in Table 5 the following numbers: \n\t\t1. encoder from DeepMicro + classification head-trained from scratch using data augmentation (similar to Table 5)\n\t\t2. encoder from DeepMicro pre-trained using contrastive learning + fine-tuned (with a classification head) using data augmentation — this answer the question: \"Is contrastive pre-training complementary to training using data augmentation?\"\n- **Missing guideline for using the proposed methods on new datasets**. All three proposed data augmentation methods seem to provide a similar average performance improvement. It would be helpful to offer rule-of-thumb guidelines for choosing between the three methods (without trying all three methods). Please answer the question: \"When should one use one method versus the other two?\". Did you notice any pattern of a method working better for specific tasks/datasets?\n\t- **Possible solution**: Reproduce Figures 2 and 3 (performance improvement as a function of dataset size and the number of features) for all three proposed methods. Possibly, a pattern will emerge. If the results are different, add all figures to the appendix. If the results are similar, mention this.\n\t- **(Small, but related) Tables 2/3/4 are too similar** and the paper doesn't mention any trend difference between them. Although lines 237-251 describe the results, there's no explicit mention of the differences in the results for the three tables. Unless you specify clear trend differences, two tables can be moved to the appendix, and only the mean performance can be kept in the main paper.\n- **(Small) Missing relation to prior work**: The proposed \"Aitchison Mixup\" is equivalent to performing SMOTE [1] in the Aitchison sense and shall be called Aitchison SMOTE. When Mixup takes convex combinations within the *same* class, it's equivalent to SMOTE. Please check the SMOTE method and the second paragraph of the related work on the mixup paper. If you use SMOTE, then please add it to the related work.\n\n# Minor changes\n- Line 30: Add examples of compositions data beyond naming domains (e.g., geochemistry -> proportion of chemical in a geological sample; economics -> proportion of GDP attributed to each industry), to make it easier for readers new to the area.\n- Where are the limitations of the current work written? I couldn't find them in Section 3 (as mentioned in the checklist)\n- Reference the peer-reviews published version of the papers (e.g., Mixup appeared in ICLR, and you've referenced the arXiv article)\n- Line 250: provide the number of the SOTA tasks in parenthesis for easy reading\n- Figure 3 from the appendix is really nice. Consider adding it to the main paper.\n- The method \"Random Subcompositions\" could be renamed to \"Compositional Feature Dropout\", because it performs dropout on features, then renormalizes the feature to present the property of compositional data.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "LMHmw4-SGMy", "reviewer": "Reviewer_V2Ko", "summary": "3 new data augmentation strategies are proposed for compositional data and these methods are tested on microbiome data. With the proposed methods, state-of-the-art results in disease prediction tasks including colorectal cancer, type 2 diabetes, and Crohn’s disease are reached. They show that the proposed data augmentations can also be used for representation learning in a contrastive setting similar to SimCLR.\n", "questions": "I do not have any questions.", "limitations": "- I liked that you also reported the datasets, which your method could not achieve state-of-the-art, which might indicate transparency.\n\n- My major concern is the one I listed in the Weaknesses as #1. I think the paper needs additional experiments using other types of CoDa for this venue as stated or at least it needs some rewriting in the claims.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- In almost all of the cases the used augmentations improve the SOTA methods. This is a result of the fact that data augmentations can be used on top of the existing methods / can be orthogonally combined with existing methods.\n\n- Paper is mostly written well. \n\n- Related work seems to be well-studied.\n\n- There is a lack of comprehensive study in the field of data augmentations for compositional data.\n\n- I liked the namings of the proposed data augmentations.\n\nWeaknesses:\n\n- The proposed data augmentations could be used for other types of compositional data besides human microbiome data. In my opinion, performance on other types of CoDA should be tested and reported for a machine learning conference like NeurIPS. Even though the methods might not work with other types of compositional data, this could as well be added to limitations and appendix. More importantly, there is a tendency to claim in some parts of the paper that the proposed data augmentations are for CoDa, not only for microbiome data. Here is an example from the \"Conclusion\": \"By combining ideas from data augmentation with the principles of CoDa, we have defined 3 novel augmentation strategies for CoDa: Aitchison Mixup, Random Subcompositions, and Compositional CutMix.\" In my opinion, if you are making such claims, it is necessary to provide results in other CoDa datasets.\n\n- Adapting data augmentations to CoDa is the main novelty of the paper. However, in my opinion, adaptations are to some extent straightforward and not very novel. \"CoDa analogue of Mixup\", \"standard preprocessing step in CoDa\", and \"CoDa analogue of CutMix\"  already give some hints regarding the originality of the method. On the other hand, the science builds on top of each other, thus I could say this is not a big but a medium-level concern.\n\n- In most cases, the increase in the SOTA performance seems not so big, even though the datasets are pretty small. I would expect a higher increase for meaningful data augmentations for such small datasets. However, considering that it is hard to come up with data augmentations for CoDa, this could also be a minor point.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept