Data Augmentation for Compositional Data: Advancing Predictive Models of the Microbiome
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Data augmentation plays a key role in modern machine learning pipelines.
Whilenumerous augmentation strategies have been studied in the context of computervision and natural language processing, less is known for other data modalities.
Our work extends the success of data augmentation to compositional data , i.e.,simplex-valued data, which is of particular interest in microbiology, geochemistry, and other applications.
Drawing on key principles from compositional dataanalysis, such as the Aitchison geometry of the simplex and subcompositions , wedeﬁne novel augmentation strategies for this data modality.
Incorporating our dataaugmentations into standard supervised learning pipelines results in consistent performance gains across a wide range of standard benchmark datasets.
In particular,we set a new state-of-the-art for key disease prediction tasks including colorectalcancer, type 2 diabetes, and Crohn’s disease.
In addition, our data augmentationsenable us to deﬁne a novel contrastive learning model, which improves on previousrepresentation learning approaches for microbiome compositional data.
Introduction
Data augmentation, i.e., generating synthetic training examples to be used in model ﬁtting, is a corecomponent of modern deep learning pipelines [55, 18].
In computer vision, augmentations such asimage resizing and shifting have been used since as early as LeNet-5 [36].
These and many otheraugmentations have become essential to highly successful state-of-the-art architectures, rangingfrom AlexNet [35] and ResNet [30] to recent contrastive learning models such as SimCLR [10]and MoCo [29].
As such, a growing body of literature has emerged to develop and characterizedata augmentation techniques, particularly in computer vision [46, 67, 14, 11, 12, 66], as well asnatural language processing [54, 13, 18].
However, deﬁning useful data augmentations is highlydomain-dependent, and fewer works have studied augmentations for more general data modalities,such as tabular data [67, 65].
The goal of our work is to extend the success of data augmentation to apreviously unexplored data modality; namely, compositional data.
Compositional data (CoDa) are those that represent the parts of a whole, and therefore carry onlyrelative information.
Equivalently, we can think of CoDa as a set of datapoints living in the simplex:
[Equation]
1 , where
[Equation]
Examples of CoDa arise throughout the sciences, including microbiology (proportions of differentspecies in a bacterial community) [24, 23, 51, 4], geochemistry (chemical composition of a geologicalsample) [6, 5, 43], materials science (components of matter)
[41, 2], and many more. As a result, starting with the seminal work of Aitchison [1], numerous bespoke techniques have been developedfor the statistical analysis of CoDa [45, 44, 19].
A rapidly growing area of application for CoDa is the human microbiome, which comprises thepopulations of microorganisms that reside inside and on the human body [60].
Microbiome dataarise from an inexhaustive sampling procedure as a result of high-throughput sequencing [24, 52].
Inparticular, each feature typically represents the relative abundance of some species of microorganism;as such, each observation can be normalized to the simplex prior to downstream analyses [24, 23, 51,38, 42].
The microorganisms that constitute the microbiome are known to impact human physiology, bothin health and in disease [22, 39, 21].
Thus, a central problem in CoDa is to learn the associationx i (cid:55)→ y i , where x i ∈ ∆ p − 1 denotes the microbial composition, and y i denotes the disease status ofthe
i th subject in a clinical study.
For example, the composition of the gut microbiome, as recordedfrom a stool sample, can be predictive of colorectal cancer, which is the third most prevalent formof cancer [17, 34].
This observation offers the potential for a noninvasive alternative to traditionalcolonoscopy procedures used for early detection of colorectal cancer [57].
In turn, accurate predictivemodels for microbiome CoDa are a key stepping stone towards achieving this potential [40, 34].
Notethat cancer is just one application; the demand for improved predictive models holds more broadlyacross medicine and biological science [34, 22].
For example, the microbiome has been linked totype 2 diabetes, Crohn’s disease, obesity, and others [28, 63, 47, 21].
Classical techniques including logistic regression, support vector machines, and random forests havebeen widely used as predictive models for microbiome data [61, 7, 37].
More recently, specializeddeep learning architectures such as DeepCoDa [49], MetaNN [38], and DeepMicro [42] have beendeveloped.
However, the capacity of these deep networks and other expressive models has beenlimited by the low sample size and high dimensionality of typical microbiome studies.
Thesecharacteristics have also spurred the use of strong regularization through early stopping, weight decayand dropout, among others [58, 49, 38, 42].
However, no previous works have explored the use ofdata augmentation for CoDa, 1 which, as we shall demonstrate, provides a cheap and simple techniquefor boosting the performance of predictive models for this data modality.
Careful consideration of the sample space will motivate our novel data augmentation strategies for
CoDa.
Our work draws on foundational principles from the ﬁeld of CoDa, such as the Aitchisongeometry of the simplex [1], which we combine with popular techniques from the literature ondata augmentation, such as Mixup [67] and CutMix [66].
This combination will lead us to deﬁnecustom data augmentation strategies for CoDa, such as Aitchison Mixup and Compositional CutMix .
In turn, incorporating these novel augmentations into existing supervised learning pipelines willresult in consistent performance gains across a wide range of microbiome datasets, including adozen standard benchmarks from the Microbiome Learning Repo [61].
These performance gainsare particularly large for some deep models, for example, DeepCoDa gains more than 10% in test
AUC for discriminating colorectal cancer from healthy controls, and over 20% for type 2 diabetes.
The gains are also signiﬁcant across other expressive model families, including random forests andgradient boosting machines.
Overall, we set a new state-of-the-art on 8 out of 12 benchmark learningtasks, including clinically relevant disease prediction tasks.
Of the remaining 4 tasks, 2 were easilyseparable, with 100% test accuracy irrespective of whether data augmentation was used.
Importantly,our augmentations rarely hurt model performance, and in the few instances that this was the case, thedrops were typically of only 1% in test AUC.
In addition to supervised learning, our novel augmentations will allow us to deﬁne contrastiverepresentation learning for CoDa , which to the best of our knowledge, represents the ﬁrst contrastivelearning model for compositional data.
Our novel data augmentations are at the core of this approach;the contrastive loss uses randomly augmented training examples to deﬁne a self-supervised optimization objective whose labels are generated from unlabelled data [27, 15, 10, 29].
In particular, ourcontrastive model is trained to discriminate between compositions that were generated as randomaugmentations of the same training example, and those that were generated as random augmentationsof different training examples.
Our implementation is adapted from SimCLR [10], but using a smallernetwork architecture together with our novel augmentations.
Unlike SimCLR, we use our individual augmentation strategies in isolation, and ﬁnd them sufﬁcient to surpass previous representationlearning approaches for microbiome CoDa [4, 42].
Altogether, we expect our data augmentationswill enable signiﬁcant future progress, possibly in combination with novel architectures, in bothsupervised and representation learning pipelines for microbiome CoDa.
Related Work
Data augmentation: numerous data augmentation strategies have been proposed in the context ofimage data [55] and text data [18].
LeNet-5 [36] used random shifts and resizing, and AlexNet [35]used reﬂections and color distortions, which have become widely adopted in computer vision [30].
More general purpose data augmentations include Mixup, which generates synthetic samples bytaking convex combinations of training examples, and has been applied to both image and tabulardata [67].
Mixup is related to an earlier method called SMOTE [9], which aims to address classimbalance by generating synthetic samples from the minority class, taken as convex combinationsbetween nearest neighbors within the minority class.
Random masking, whereby random inputfeatures are hidden during training, has also seen a diverse range of applications, including naturallanguage processing (e.g., BERT [13]), computer vision (e.g., Cutout [14]), and tabular data (e.g., VIME [65]).
In the context of CoDa, masking is related to subcompositions , which we will use todeﬁne an analogous augmentation on the simplex.
CutMix [66] is akin to both masking and Mixup,in that random patches from different images are pasted together to generate new synthetic images.
We note that a subsequent line of work has developed techniques to automatically select optimal dataaugmentations for a given dataset [11, 12], somewhat akin to hyperparameter optimization.
Whilethe data augmentation strategies that we present in the next Section could also be combined andﬁnetuned in similar ways, such experimentation falls outside the scope of our work and is left tofuture research.
Microbiome models: random forests remain a strong baseline across many microbiome studies, dueto their expressivity and robustness [58].
Gradient boosting machines such as XGBoost providesimilarly strong performance and have also enjoyed signiﬁcant adoption [58].
AutoML [31] hasshown potential for microbiome data, with a recent approach called mAML achieving state-of-theart results on several benchmarks [64].
mAML uses cross-validation to automatically select thebest model class and hyperparameters for each learning task.
In addition, several specialized deeplearning architectures have been proposed for microbiome CoDa.
DeepCoDa [49] introduced thelog-bottleneck layer, which ensures the hidden units remain scale-invariant, a key desiderata in CoDamodels.
This architecture obtained strong predictive performance using weight decay regularization.
MetaNN [38] employs a multilayer architecture regularized using dropout [56].
The authors of
MetaNN also considered generating synthetic data from a negative binomial distribution, but foundno performance improvements beyond the use of dropout.
DeepMicro [42] uses a deep autoencoderarchitecture to learn low-dimensional representations of the microbial composition.
In turn, theserepresentations are fed to a downstream classiﬁer trained with a supervised objective.
Contrastive learning: the goal of contrastive learning is to learn low-dimensional representationsby optimizing some pretext task , where the objective function is similar to those used for supervisedlearning, but using labels that are derived from unlabelled data only [27].
Commonly, the pretexttask is to discriminate augmented instances of the same training example from augmented instancesof different training examples [15, 10, 29].
Recent contrastive learning architectures have enjoyedtremendous success, setting the state-of-the-art across a range of computer vision benchmarks[10, 29, 32, 8].
We highlight SimCLR [10] for its relative simplicity and strong empirical performance.
This method uses a nonlinear projection head between the representations and the contrastive losscomputed with normalized embeddings.
Our own experiments on contrastive learning borrow theseelements from SimCLR, but use a smaller network architecture together with our specialized dataaugmentations for CoDa.
Methods
In the following subsections, we introduce 3 novel data augmentation techniques for CoDa.
Note thatmany variations of our augmentation schemes could be considered; the versions we present here areintended to be as concise as possible.
Our goal is not to design augmentations that are empiricallyor theoretically optimal, but rather to demonstrate the effectiveness of our novel methodology by establishing simple and performant baselines.
For instance, whenever a random mixing parameteris required, we use a U (0 , 1) distribution, even though other choices can likely result in increasedperformance.
For clarity, we will focus on the classiﬁcation setting; generalizing to regressionproblems is straightforward.
We will use the notation v , x ∈ ∆ p − 1 for simplex-valued vectors,λ ∈ R a scalar, and D = { x i , y i } our training data.
3.1 Aitchison Mixup
Hilbert structure on the simplex, known as the Aitchison geometry , with the followingvector addition, scalar multiplication, and inner
[Equation]
This geometry provides a principled deﬁnition of linearcombinations and a distance metric on the simplex; parallel and orthogonal lines are illustrated in Figure 1.
The
Aitchison geometry is closely related to the centered-logratio transformation [1], which is an inverse of the softmax and deﬁnes an isometry between ∆ p − 1 and Euclideanspace [45].
Taken together, these properties form the basisof much of CoDa methodology [1, 45, 44, 19].
Our ﬁrst data augmentation strategy, which we call Aitchison Mixup , is based on taking linearcombinations of the training points, in the Aitchison sense.
While general linear combinations maybe used, for simplicity we focus on intra-class pairwise convex combinations.
Namely, each newdatapoint is sampled as follows:
Draw a class c from the class prior and draw λ ∼ U (0 , 1) .
Draw two training points i 1 , i 2 such that y i
=
y
i
= c , uniformly at random.3.
Set x aug = ( λ (cid:12) x i 1 ) ⊕ ((1 − λ ) (cid:12) x i 2 ) and y aug
= c .
Put more succinctly, for each class we generate convex combinations of the points in that class, inthe Aitchison sense.
Note this augmentation strategy is a CoDa analogue of Mixup, which generates
Euclidean convex combinations of images and tabular data [67].
Much like Mixup is capable ofboosting predictive accuracy by regularizing models towards a linear decision boundary in the regionsbetween training examples, Aitchison Mixup aims to enforce the equivalent regularization in the
Aitchison geometry of the simplex.
Note that, once we restrict to convex combinations (as opposed to general linear combinations), ouraugmented data will remain inside the simplex regardless of whether we operate in the Euclideanor the Aitchison geometry.
However, we found the latter to be more empirically effective, perhapsunsurprisingly given the associated vector space structure of the simplex.
We also note that the authors of Mixup [67] sampled λ ∼ Beta ( α, α ) , where α is a hyperparameter,and took inter-class as well as intra-class convex combinations, setting y aug = λy i 1 + (1 − λ ) y i 2 . While Aitchison Mixup can also incorporate such generalizations, we chose not to do so in ourimplementation because, as discussed above, our focus was on ﬁnding the simplest performantaugmentation strategies for our new data modality
3.2 Compositional Feature Dropout
In CoDa, a subcomposition of x ∈ ∆ p − 1 refers to a lower-dimensional composition, x sub ∈ ∆ k − 1with k < p , formed by taking a subset of the elements of x and renormalizing to a unit total.
Namely,if j 1 , . . .
, j k denotes a subset of the indices { 1 , . . .
, p } , the corresponding subcomposition is deﬁnedas:
[Equation]
We can generate augmented data by taking a random subcomposition of the training points; sucha strategy is analogous to masking in language data or cropping in image data, or more generally,feature dropout [56].
Since our predictive models require inputs of ﬁxed dimension p , rather thandiscarding elements of x we simply zero them out.
Thus, each new datapoint is generated as follows:
Draw λ ∼ U (0 , 1) .
Draw a training point i uniformly at random and set ˜x = x i .
For each j ∈ { 1 , . . .
, p } , draw I j iid ∼ Bernoulli ( λ ) , and set ˜ x
j = 0 if I j = 0 .3.
Set x aug =
˜x / ( (cid:80) pj
=1 ˜ x j ) and y
aug
= y i .
In short, we zero out random entries of the training points and renormalize.
Intuitively, this strategyencourages our predictive models to become robust to partially observed inputs.
Note that, distinctively from feature dropout, our augmentation includes an additional renormalization to a unit total,and we therefore use the term Compositional Feature Dropout .
Importantly, this renormalizationensures that the augmented samples remain in the support of the training data, i.e., the simplex.
Note that many CoDa models, including mAML and DeepCoDa, apply log transformations to theirinputs, and therefore require that these be non-zero.
For this reason, we implement a zero-replacementstep where we add a small positive quantity to all the parts of each composition and renormalize.
This small quantity is set to 1 /L
i , where L i corresponds to the library size from the high-throughputsequencing procedure.
L i can simply be thought of as a large number such that
/L
i is smaller thanthe non-zero components of x i .
Note that this transformation is a standard preprocessing step in
CoDa [51].
3.3 Compositional CutMix
Our third augmentation scheme, which we call Compositional CutMix , combines elements of ourprevious 2 augmentations.
Like in Mixup, we generate new datapoints by combining pairs of trainingpoints from the same class.
However, instead of combining these training points linearly (in the
Aitchison sense), we take complementary subcompositions and renormalize.
Namely, we generateeach new datapoint as follows:
Draw a class c from the class prior and draw λ ∼ U (0 , 1) .
Draw two training points i 1 , i 2 such that y i
=
y
i
= c , uniformly at random.3.
For each j ∈ { 1 , . . .
, p } , draw I j iid ∼ Bernoulli ( λ ) , and set ˜ x
j = x i 1 j if I j = 0 or ˜ x j = x i 2 j
if I j = 1 .4.
Set x
aug =
˜x / ( (cid:80) pj
=1 ˜ x j ) and y aug = c .
Note this strategy can be thought of as a CoDa analogue of CutMix , whereby new images are formedby pasting together patches from different training images [66].
Note also that in computer vision,
CutMix and related methods take local image patches as opposed to randomly sampled pixels.
Likewise, one could sample microbial subcompositions according to biologically relevant groupings,for example using a phylogenetic tree [51].
Such a strategy would require incorporating additionaldomain knowledge and is left to future work, but we expect it would further increase the quality of ourdata augmentations.
This remark applies both to Compositional Feature Dropout and Compositional
CutMix.
With regard to inter-class combinations, a similar discussion as in Section 3.1 applies.
Experiments
4.1 Supervised learning
We evaluate our augmentation strategies on 12 standard binary classiﬁcation tasks taken from the
Microbiome Learning Repo [61].
These datasets comprise various disease and phenotype prediction [Table] tasks, including colorectal cancer [34], type 2 diabetes [47], Crohn’s disease [21] [Table]
and cirrhosis[48]
[Table]
as well as multiple body sites including the gut, skin, oral cavity, airways, and vagina [39, 53].
As such, this benchmark provides a comprehensive evaluation for predictive models of the humanmicrobiome [64, 49, 25].
More details on these 12 learning tasks can be found in Table 1; note thisbenchmark is constructed from the original Microbiome Learning Repo by ﬁltering datasets thatcontain a minimum sample size of 100, with at least 50 in either class.
For each learning task, we take 20 independent 80/20 train/test splits and we ﬁt Random Forest,
XGBoost, mAML [64], DeepCoDa [49], and MetaNN [38], ﬁrst to the original training data, thenon 3 augmented training sets obtained using our 3 augmentation strategies.
Thus, we train a total of12 × 20 × 5 × 4 = 4 800 models.
4 We evaluate test performance using ROC AUC, and we note thatour datasets do not present severe class imbalance (with the exception of tasks 3 and 11, both of whichwere well separated by all our classiﬁers and therefore had no effect on the overall comparison).
For the augmented training sets, we generated 10 times as many synthetic samples as there wereoriginal training examples.
The factor of 10 was chosen so as to obtain a relatively large augmentedsample, in order to reduce the sampling variance from our random augmentations.
In turn, wecompensate for the fact that our augmented data is then much more numerous than our originaltraining data, by downweighting the synthetic samples by a factor of 10; the total weight of theoriginal and synthetic data is then equal to 1 / 2 each.
Our results are shown in Table 2 (Aitchison Mixup) in the main text, as well as Tables 5 (Compositional Feature Dropout), and 6 (Compositional CutMix) in the Appendix:
• Table 2 shows that Aitchison Mixup improved the test performance of existing modelsacross a large majority of learning tasks.
On average, we obtained a 5% gain in test AUC forDeepCoDa, 2% for random forests and XGBoost, and 1% for mAML.
MetaNN remainedﬂat, but this was also the worst performing model overall.
Importantly, in the few instanceswhere Aitchison Mixup hurt the performance of a model, the loss was typically of just 1%. • Table 5 shows that Compositional Feature Dropout also result in consistent performance improvements across most models and tasks, though to a slightly lesser degree than AitchisonMixup.
On average, DeepCoDa enjoyed a 3% gain in test AUC, random forests 2%, andXGBoost and MetaNN both 1% (mAML remained ﬂat).• Table 6 shows that Compositional CutMix obtained the strongest classiﬁcation performanceout of our 3 augmentation strategies on microbiome CoDa.
On average, DeepCoDa saw a5% gain in test AUC, with 2% for random forests and XGBoost, and 1% for mAML andMetaNN.
Moreover, models trained with Compositional CutMix set a new state-of-the-arton 8 out of 12 tasks, including colorectal cancer (0.76), type 2 diabetes (0.72), and Crohn’s
[Table]
[Figure]
Figure 2 shows the effect of data augmentation on test performance as a function of the number ofsamples in the training set, for Compositional CutMix.
We note that datasets with smaller samplesizes tended to beneﬁt more from data augmentation.
This suggests that our methodology may provebeneﬁcial to many microbiome research projects, where datasets with low-hundreds of samples arecommonplace.
Figure 3 shows a similar scatterplot, as a function of the dataset dimensionality ratherthan sample size; the gains from augmentation appear consistent across lower- and higher-dimensionalmicrobiome CoDa (we observe similar trends Aitchison Mixup and Compositional Feature Dropout,and we omit the corresponding scatterplots).
In addition, we evaluated the impact of our data augmentations on the expected calibration error (ECE)[26] of our models.
Previous works have noted that increasingly ﬂexible predictive models suchas deep neural networks, while enabling greater predictive accuracy, tend to become overconﬁdentin their predictions, degrading uncertainty quantiﬁcation [26].
In Appendix B, we show the ECEobtained by our models, and we verify that our data augmentations do not hurt model calibrationoverall (in fact, some modest improvements are obtained). [Figure]
4.2 Contrastive representation learning
We have shown that our novel data augmentation strategies provide consistent performance gainsacross supervised training pipelines.
Next, we take our data augmentations one step further to providethe ﬁrst application of contrastive representation learning to microbiome CoDa.
As a point of comparison, we use DeepMicro [42], a deep representation learning architecture tunedspeciﬁcally for microbiome data.
DeepMicro trains a deep autoencoder to learn a low-dimensionalrepresentation of the microbial composition.
This learned representation is fed to a classiﬁcation headdownstream.
As per the author’s implementation, the encoder has 2 hidden layers with 256 and 128units respectively, and ReLU nonlinearities, and a 64-dimensional output.
The decoder architecture isa mirror image of the encoder.
The decoder output is of the same dimension as the encoder input, andthe model is trained to minimize the mean squared error between the two, using the Adam optimizerwith default parameters for 2,000 epochs.
To ensure a fair comparison, our contrastive model uses the same encoder architecture from DeepMicro.
However, we can discard the decoder layers; instead, the 64-dimensional latent representationis passed to a multilayer projection head that is trained jointly with the encoder using a contrastiveloss, as is done in SimCLR [10].
Our projection head contains one 32-dimensional hidden layer with ReLU activations, and a 16-dimensional projection output, which is normalized to 1 unit in L2 normand passed to a temperature-scaled cross-entropy loss.
Intuitively, this loss function is designed todraw the representations of positive pairs of examples close together, and push those of negative pairsfar apart.
Positive pairs refer to two synthetic samples generated by random data augmentations ofthe same training example; negative pairs refer to synthetic samples generated from different trainingexamples.
In our implementation, at each training step we sample two Compositional Feature Dropoutfrom each training example, and compute the contrastive loss over all such pairs.
In particular, ourbatch size corresponds to the entire training set, which is reasonable given the small sample sizes inour data.
We again used the Adam optimizer with default parameters for 2,000 epochs.
We evaluate the quality of our learned representations under two standard protocols.
First, linearevaluation, where the encoder weights are frozen and a linear classiﬁcation head is trained usingthe supervised cross-entropy loss.
Second, ﬁnetuning, where the linear classiﬁcation head is trainedjointly with the encoder network.
The ﬁnetuning protocol is evaluated both with and without usingadditional data augmentation.
These evaluation protocols are applied for both DeepMicro and ourcontrastive model, in addition to a “no pretraining” control.
This control is a randomly initializedencoder network and linear head with the same architecture, trained only on the supervised objective.
The results are shown in Table 3; the representations learned by our contrastive model consistentlyoutperform those learned by DeepMicro, both in linear evaluation and ﬁnetuning.
Note that ﬁnetuningwith augmentation performs slightly better than without, suggesting that the beneﬁts of supervisedtraining with our data augmentations can be combined with the beneﬁts of augmentation as part ofcontrastive learning.
Importantly, note that the comparison to DeepMicro is conservative, in the sensethat we replicated the architecture and simply changed the pretraining objective from reconstructionerror to contrastive loss; ﬁnetuning the encoder architecture itself under a contrastive objective wouldlikely lead to further performance improvements.
We conclude that Compositional Feature Dropoutprovides a valuable data augmentation for contrastive learning on the simplex.
Note that our other 2 augmentation strategies, Aitchison Mixup and Compositional CutMix, mayalso be used for contrastive learning.
However, these augmentations require paired training examples [Table] to generate each synthetic sample, and the implementation is therefore slightly more involved.
Inparticular, we generate each pair of positive samples by taking one pair of training examples, anddrawing two random combinations (in the Aitchison Mixup or the Compositional CutMix sense) ofthat one pair.
The training examples are randomly partitioned into pairs at each epoch, and negativesamples correspond to those that originate from disjoint pairs.
Note that we no longer require thatthe pairs are drawn from the same class for the contrastive objective.
The results are provided in
Appendix B and show similarly strong performance to Table 3.
4.3 Additional benchmarks
In addition to our microbiome benchmark, we are adding an additional experiment that demonstratesthe performance of our data augmentations on non-microbiome CoDa.
This experiment is describedin Appendix A, which will be copy-pasted here after the review period (note the extended 10-pagelimit).
Conclusion
By combining ideas from data augmentation with the principles of CoDa, we have deﬁned 3 novelaugmentation strategies for CoDa: Aitchison Mixup, Compositional Feature Dropout, and Compositional CutMix.
Our augmentations offer cheap performance gains across a wide range of benchmarkdatasets , advancing the state-of-the-art on standard classiﬁcation benchmarks for colorectal cancer, type 2 diabetes and Crohn’s disease.
While we ﬁnd all 3 strategies performant across a wide range of benchmark datasets, Aitchison Mixup and Compositional CutMix performed best on microbiome CoDa (with a slight outperformance for the latter) and Aitchison Mixup performed best on non- microbiome CoDa. In addition, our data augmentations allowed us to deﬁne the ﬁrst contrastive learning model for CoDa, which we show improves on existing representation learning frameworks for microbiome data.
Data augmentation and contrastive learning have enjoyed tremendous success in other application domains such as computer vision; our novel methodology aims to help spur similar developments in the ﬁelds of CoDa and the microbiome
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes] Compare Abstract/Intro to Methods/Experiments
(b) Did you describe the limitations of your work?
[Yes] See Section 3 and Section D in
the Appendix. (c) Did you discuss any potential negative societal impacts of your work?
[Yes] Not the
focus of our paper, but brieﬂy discussed model calibration in Section 4.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A] No theoretical results.
(b) Did you include complete proofs of all theoretical results?
[N/A] No theoretical results.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
[Yes] See supplemen- tal material.
(b) Did you specify all the training details (e g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See Section 4 or code in supplemental material.
(c) Did you report error bars (e g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] See appendix B.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 4. [Table]
If you are using existing assets (e [Table] g., code, data, models) or curating/releasing new assets...
[Table] (a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 4. [Table]
(b) Did you mention the license of the assets?
[No] Assets are open-source.
[Table] (c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
[Table]
Code provided in supplemental material.
[Table] (d) Did you discuss whether and how consent was obtained from people whose data you’re [Table]
using/curating?
[No] Datasets used are in the public domain. [Table]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable [Table]
information or offensive content?
[N/A]
No identiﬁable or offensive content.
[Table] If you used crowdsourcing or conducted research with human subjects... [Table]
(a) Did you include the full text of instructions given to participants and screenshots, if [Table] applicable? [N/A] Not used.
[Table] (b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A] Not used.
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A] Not used.
[Table]
A Additional benchmarks
Further to the 12 microbiome classiﬁcation tasks from the previous sections, we analyze 8 additionalcompositional datasets, which were selected to represent a diverse set of applications that arecommonly encountered by CoDa practitioners.
Note that, while these datasets correspond to a widerange of scientiﬁc domains, they have all been analyzed as case studies speciﬁcally in the CoDaliterature.
The datasets can be summarized as follows:
• Glass : a dataset from the UCI ML repo, on the elemental composition of glass particles ofdifferent types of origin.
Such particles are routinely recovered from the clothing of criminal
[Table]
suspects; the goal is to provide forensic evidence by classifying the type of origin (brokenwindow vs. other) given the composition recorded in a multielement analysis [16, 59].
[Table]
• Bayesite : a classical materials science dataset due to Aitchison [2].
The goal is to predict
[Table]
the permeability of ﬁbreboard from the mix of its ingredients.
[Table]
• Serum : another classical dataset due to Aitchison [2], where the composition of blood serum
[Table]
protein is used to discriminate between two known diseases.
[Table]
• Hydrochem : a hydrochemical dataset measuring the composition of water samples across
[Table]
sites in the Llobregat river basin [43].
Classiﬁcation targets are formed by separating
[Table]
upstream and downstream sites.
[Table]
• Jura : a geochemical dataset from the Swiss Jura, where rock types (Kimmeridgian vs.Sequanian) are classiﬁed from the chemical composition of soil samples [3].
• Metabolites : gut metabolomic proﬁles for subjects with and without inﬂammatory bowel
[Table]
disease [20, 50].
The goal is to diagnose disease from the metabolite composition.
[Table] • MicroRNA : contains microRNA expression data for different types of primary breast cancer [Table] The goal is to discriminate between breast cancer subtypes (Luminal A vs. LuminalB) from the molecular proﬁle of each tumor.
[Table]
• Coffee : a nutrition dataset describing the chemical composition of commercially available
coffee samples [33].
Classiﬁcation targets are taken as different blends of coffee.
In modeling these datasets, we follow the experimental protocol described in Section 4.1, except weno longer test mAML and MetaNN (which were not intended for non-microbiome data), and wereplace the latter with a small neural network with one hidden layer of 32 units and ReLU activations,denoted as NN (note we choose to keep this architecture very simple as it is trained on severalheterogeneous dataset).
The results are shown in Table 4 for Aitchison Mixup.
As was the case for our microbiome benchmark,we again see that our data augmentation results in consistent performance gains across a wide rangeof classiﬁers. Indeed, on most datasets (6 out of 8) the best performing model was one that used Aitchison Mixup, and only on a single dataset (Serum) was the best model (NN) one trainedwithout augmentation. Note that, in contrast to our microbiome benchmarks, Table 4 shows a strongoutperformance for tree-based models.
Importantly, Aitchison Mixup never hurt the performance ofany of our tree-based models, and only did so rarely for our network models. We observe similarlypositive results (albeit slightly weaker) for Compositional Feature Dropout and Compositional CutMix, provided in Tables 7 and 8 from the Appendix, respectively. [Table]
Tables 5 and 6 show the performance of Compositional Feature Dropout and Compositional CutMixon our microbiome benchmarks and are discussed in the main text. Tables 7 and 8 show theperformance of Compositional Feature Dropout and Compositional CutMix on our non-microbiomebenchmarks. Tables 9, 10, and 11 show the effect on expected calibration error (ECE) of adding data augmentationto our various models.
Notice that overall calibration error stays the same or improves slightly afterincorporating data augmentation.
Tables 14, 15, and 16, show error bars for Tables 2, 5, and 6,respectively.
In Tables 12 and 13, we show the performance of our contrastive models deﬁned with Aitchison Mixupand Compositional CutMix, respectively.
The latter performs equally strongly as Compositional
Feature Dropout, as shown in Section 4.2, the former performs noticeably worse, however still noworse than DeepMicro, thus demonstrating the robustness of our contrastive framework to multipleaugmentation strategies.
Further gains may be obtained by combining augmentations in a contrastiveloss, a direction which is left to future work.
Table 3 shows error bars for Table 17, respectively.
[Table]
Mean 0.
[Table] [Table]
Coffee
[Table]
C Other augmentation strategies
We propose an additional augmentation strategy, Multinomial Resampling , which can be thought ofas a CoDa analogue of image blurring.
The goal of this augmentation is to inject noise across thecoordinates of x in a principled manner.
Additive Gaussian noise is clearly ill-suited for CoDa, as it isnot constrained to the simplex.
Instead, inspired by the recording mechanism for microbiome CoDa,we propose generating new datapoints from a multinomial distribution.
Note that high-throughputsequencing technologies record the microbial composition present in a specimen by subsampling thelarger population.
The total number of reads in this subsample, known as the sequencing depth, isan artifact of the measurement process.
Assuming the subsample is small relative to the population,the multinomial distribution provides a crude approximation for this generative process, with eachtrial being drawn according to the true proportions in the underlying population.
In this way, we cansample new datapoints from a multinomial distribution where the number of trials corresponds to thesequencing depth.
Namely, each new datapoint is generated as follows:
Draw a training point i uniformly at random.
Draw ˜x ∼ Multinomial
( L i , x i ) , where L i is the sequencing depth (a.k.a.
library size).3.
Set x aug =
˜x / ( (cid:80) pj
=1 ˜ x j ) and y
aug
= y i .
[Table]
Task
RF Aug XGB Aug mAML
Aug
DeepCoDa
Aug
MetaNN
Aug 0.
[Table]
Notice that the number of trials L i controls the noise level; as L i → ∞ , x aug → x i .
When applied to
CoDa that does not arise from high-throughput sequencing, L i can be speciﬁed arbitrarily, or treatedas a hyperparameter.
In our microbiome datasets, the sequencing depth is typically on the order of
L ∼ 10 000 .
In addition to those mentioned in Section 3, we note the following limitations of our work:
• Although rare, there were some datasets where data augmentation hurt the classiﬁcationperformance for certain models.•
We did not study the effect of combining multiple augmentations in a single pipeline, whichis left to future study.
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
Mean 0.
[Table]
Mean 0.
[Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table]
Mean 0.