dataesr/scientific-paragraphs-categorization huggingface dataset:
Second, almost two thirds state that they make use of alternative medicine in addition to or instead of conventional medicine. Third, self-medication is a prevalent feature in our survey sample (86% make use of it). Fourth, 87% of respondents state that they return their unused medication to a pharmacy, due to the specificity of French "Cyclamed" initiative 4. Finally, 70% of respondents return their medication based on environmental concerns and, of those, 79% consider that residues from pharmaceutical products pose a serious risk to human health and the environment. To capture how sustainable practices are intertwined with medicine use, we constructed a relevant dataset consisting of 40 categorical variables, of which 25 are indicators of sustainability.
Results and discussion We apply the ClustOfVar method to the 40 categorical variables extracted from our questionnaire with the R dedicated package  (Chavent et al. , 2017). The dendrogram of the variables thus obtained shows the successive aggregations between them, and the way in which they are linked together (Fig. 2 ). We observe that a four-cluster partition is the first suitable choice for our data, thus revealing four main groups of medicine consumption practices. For a finer analysis of practices, we go slightly further down the tree until the next relevant choice of number of clusters: the 9-cluster partition. We thus highlight nine Synthetic Variables that are interpreted through the four groups of practices and read as a gradient, using both the variables that compose the corresponding cluster and the plotting of the Synthetic Variables.
, 2009). Finally, in line with recent findings (see for instance  Foon et al. (2020) ), demographic variables do not clearly impact disposal behaviour, which seems to be a more complex cognitive decision implying perceived convenience, personal and social norms, and so on. These three SVs seem to indicate the central actions of the daily consumption of medicines to treat "ordinary" pathologies, outside of those usually managed by health professionals and without real consideration of the environment of these practices. Examining in detail the intercorrelations between the SVs of this first group (see Appendix 5 for details), we see how these structuring variables are correlated among individuals of our dataset. First, the slight negative correlation between SV4 and SV5 shows that when medicine consumption of the surveyed individuals is daily (positive values of SV5), their behaviour tend to be more precautionary on average (negative values of SV4).
Acknowledgements This work was financially supported by the  National Office for Water and the Aquatic Environment (Onema , now part of the  French Biodiversity Agency -OFB) , the  French Ministry of Ecology and the Water Agencies , as part of the  call for projects entitled  " Innovations and changes in practices: combating micropollutants in urban waters " (2013)(2014)(2015)(2016)(2017)(2018)(2019). The authors would like to thank  Antoine Ferron  and  Romain Valadaud  for their contribution to the survey and the REMPAR project. Finally, the authors wish to thank warmly the two anonymous reviewers for their helpful comments and suggestions on earlier versions of the document.
Comparison of batch effect correction methods across three diverse datasets demonstrated that BERNN models consistently showed the strongest sample classi cation performance. However, the model producing the greatest classi cation improvements did not always perform best in terms of batch effect removal. Finally, we show that overcorrection of batch effects resulted in the loss of some essential biological variability. These ndings highlight the importance of balancing batch effect removal while preserving valuable biological diversity in large-scale LC-MS experiments.
In this context, we propose a suite of models to overcome these various batch correction problems, all based on autoencoders executable in parallel. Our approach to countering batch effects is different from most other solutions, as we do not rely on a single solution that we claim to be superior to all others. Instead, we acknowledge that not all problems require the same solution and propose multiple potential solutions to address batch effects. Thus, we aim to empower researchers to easily try multiple methods simultaneously and pick the optimal approach for their dataset and scienti c question. Amongst this suite of models, we present the rst use of Variational Autoencoders (VAE), Domain Adversarial Neural Networks (DANN) and Domain Inverse Triplet Loss (invTriplet) for batch correction in LC-MS.
Model descriptions Our models, that we call Batch Effect Removal Neural Networks (BERNN), are composed of different modules, each with different objectives: the autoencoder, the batch classi er and the label classi er (Fig. 1 ; for more details Fig  S1 ). The autoencoder aims at nding representations, usually smaller than the inputs, that can be used to reconstruct the inputs, as can be seen in Fig. 1A  and 1B. The autoencoder, by itself, can improve classi cation generalization by removing the noise and by providing a smaller representation. To further remove batch effects, we use either an adversarial loss or a modi ed version of the triplet loss to nd feature representations that cannot discriminate between batches. The adversarial strategy is already used by NormAE 18 , but we are using the Gradient Reversal Layer (GRL) to make the training more straightforward 28.
4. 4). With recent technologies, datasets tend to have a very high number of features, but a much smaller number of samples. VAE enables the model to perform data augmentation in datasets that are highly dimensional, but with few samples. VAEs can reduce over tting in overparameterized neural networks  29  , which is important for optimal performance. Most batch effect correction methods return the corrected expression data, but  30  does not provide the corrected features: it creates an integrated embedding. All our methods also create a new embedded representation, but can also return a vector of the corrected data, although the utility of these data is not guaranteed. The corrected data is obtained by reconstructing the original input from the new embedding using the decoder (the purple module in Fig.
Evaluation and selection To get an accurate picture of the model performances benchmarked in this study, we used three datasets with different characteristics (Fig. 2 ). They are different in terms of number of features (889, 6461 and 17887), number of batches (3, 7, and 21)  and the degree of batch effect (Adjusted Mutual Information (AMI) from 0. 13 to 1. 0). The differences in initial batch effects are also visually evident (Fig. 2B ). These datasets, which have very different characteristics, achieved top performances with different models, supporting the concept that different problems might require different solutions. Models were evaluated on their classi cation performances using the accuracy (Fig  S3 ) and Matthews Correlation Coe cient (MCC) (Fig. 3A , 4A, 5A). Because some datasets were highly imbalanced (more samples in one class than the other), we selected the top performing models based on their MCC scores.
The MCC produces high scores only if predictions are good across all four confusion matrix categories (true positives, false negatives, true negatives and false positives)  31. Class imbalance was strongest in the Adenocarcinoma dataset, where 87. 5% of the samples in the dataset were of the dominant class. As can be seen in Figure  S3B , the best accuracy for this dataset is obtained using the raw data, because the model always predicts the dominant class. The main drawback of using MCC is its high sensitivity to misclassi cation of the samples in the minority class. When the imbalance is very high, misclassi cation of a single sample can have a big impact on the score. This can partly explain why the error bars for MCC were sometimes quite large. In comparison, error bars are smaller for accuracies (Fig  S3 ).
To evaluate the performance of batch effect correction, we used two main categories of metrics: batch mixing metrics and quality control (QC) metrics. To assess the batch mixing performance, we used Batch Entropy (BE), Adjusted Rand Index (ARI) and AMI (see Methods section 5. 2. 2). QC metrics measure how much the QC samples are different from each other. Because QC samples are the same for a given dataset, they should in theory be at the same position in a Euclidean space, and each of the features should have perfect Pearson Correlation Coe cients (PCC). The metrics nMED and aPCC are derived from these assumptions, respectively (see methods section 5. 2. 3). Note that in results presented in Figs. 3 4 5 , for BERNN models, all the batch effect correction metrics are for models where the MCC was maximized.
Training scenarios As explained in Fig. 1 , there are 2 scenarios to train any BERNN. Both scenarios start with a warming up phase (step 1), which uses the complete dataset (including the validation and test set) to train all components of the model that are unsupervised, which include the autoencoder and the batch classi er (if an adversarial loss is used). In the rst scenario, when the warmup is over, the autoencoder is frozen and only the labels classi er is updated using the training set (step 2a). In the second scenario, after the warmup, the training alternates between step 1 and step 2b, which is the same as step 2a, except that the backpropagation is allowed to ow through the encoder. The rst scenario was used only for the Alzheimer dataset, the two others used the second scenario.
When the second scenario is used with the Alzheimer dataset, the over tting problem is too important and only random predictions are made for the validation and test sets. However, because the labels classi er cannot in uence the representation learned, it probably causes an under tting of the data. It may be because of the data augmentation they provide that VAEs, particularly VAE-DANN, perform better on this dataset (Fig. 3A ). On the Adenocarcinoma and Aging Mice datasets, the second scenario performed best. The rst scenario made the model under tting too much, thus the second scenario performed much better (Fig. 4A ). In this particular case, the AE-based models performed much better than the VAE-based models, especially AE-invTriplet. For the Aging Mice dataset, all BERNN models were better than the other methods, but their performances were almost exactly the same, with only negligeable differences (Fig.
Most batch correction methods improve (decrease) batch entropy (BE). AMI and ARI are also consistently improved (decreased in value) in datasets with the greatest initial batch effects, but not in the Alzheimer dataset, which had moderate batch effects in the raw dataset. For every dataset, the best validation MCC score always used a transformation that improved BE (Fig. 3D , Fig. 4D  & Fig. 5C ). Although in the case of the adenocarcinoma dataset many VAE-based networks performed worse than a classi cation using the raw data (see methods section 5. 5. 4), VAE and AE networks are both the most e cient in at least one dataset. In all three datasets, all the best classi cation performances were reached using neural networks models that also signi cantly improved BE. It is also true that some of the methods that led to some of the best BE improvements had some of the worst MCC scores, sometimes far worse than when using the raw data.
The best example of this is the Aging Mice dataset, where the classi cation scores using the raw data reached high MCC scores, despite having a very bad BE (Fig. 5 ). On the other hand, Combat and harmony greatly reduced batch effects at the cost of very bad MCC scores. This reduction can lead to the loss of important biological information, which could have negative consequences even in situations where classi cation is not a primary concern, such as differential quanti cation/expression analysis. When evaluating differences between samples from two conditions, classi cation performance provides evidence of the signi cance of a set of biomarkers identi ed to discriminate between the conditions. In both datasets that used replicate QC samples (Alzheimer and Adenocarcinoma), the networks with the best classi cation performances had better nMED than the uncorrected data, but the networks that did best on these metrics did not necessarily result in the best classi cation metrics.
AE outperforms all other methods While it is not possible to con rm a single model as the best choice to improve classi cation, all the best MCCs were obtained using a version of our BERNNs. All of them performed almost identically on the Aging Mice dataset. On the Alzheimer dataset, VAE-DANN performed the best, followed closely by NormVAE and invTriplet-VAE. It is possible that the reverse triplet loss (revTriplet; for de nition see supplementary methods), which did not perform as well as the other models developed in this study, would perform best in other datasets (For the complete results, see Fig S4  -S7 ). In all datasets, one of revTriplet or invTriplet was always part of the best of the BERNN models for batch correction according to the batch entropy (Fig  S4 ). Combat or harmony were often the best according to the three batch mixing metrics, but always at the cost of very poor classi cation performances.
The triplet losses require an additional hyperparameter (called margin), which makes hyperparameter optimization more complex. It has been shown to have important consequences on optimization in scRNAseq where inverse triplet loss was also used to overcome batch effects  30. It is possible they might require more hyperparameters optimization to achieve even better performances. Nevertheless, one of the invTriplet models was either the top performer (Fig. 4 ) or was always very close to the best performance (Fig. 3  and Fig. 5 ). When batch effects hindered classi cation generalization, using one of BERNN's models was always bene cial in all three datasets. However, when a batch effect is evident, but the model can generalize predictions in new batches, the course of action is less clear. The AgingMice dataset provides a good example of this scenario, as LinearSVC achieved very good classi cation scores with unnormalized data.
Discussion It is well known in machine learning that no single model can claim to be the best at solving any task. In this study, we propose a suite of deep learning architectures models to enable users to nd the optimal solution for different problems. This suite introduces batch correction in LC-MS using VAE-based models, the use of GRL for implementing a DANN and triplet losses, all of which were part of the best model in at least one dataset. The inverse triplet loss is particularly interesting because it is the only loss that is effectively minimized. The other losses that use GRL attempt to minimize the batch classi cation, but the loss increases until it reaches the loss corresponding to a random classi cation of the batches. This property is particularly useful in the context of multitask learning  32  , because it requires all losses to be minimized to function properly.
If the end goal is not the classi cation in itself, but to identify biomarkers that can be used to search for new therapeutical compounds, then it makes sense to focus on denoised representations. We argue, however, that the bottleneck should be used in combination with other methods, like SHAP 26 or LIME  34  (Fig S2 ), that can identify the most useful features using methods. We found that the original NormAE available online had poor classi cation results, especially on the Alzheimer dataset (unpublished data). Instead of using the implemented version available online, it was implemented in our suite of models, as it is very similar to the DANN method, except without the GRL (see methods 5. 4. 2 for more details), which we believe to be a crucial component to the success of the methods that use DANN.
Because they are all trained with the same training scripts, we can make fair comparisons between the methods. Indeed, the original NormAE had more layers and a different training strategy than we used. All methods should be trained using the same architectures so that the only difference between each method is the method itself, not the architecture. We were able to surpass the classi cation performance of the default con guration which was developed the adenocarcinoma dataset (we obtained the dataset from  35  , as 18 did not have its dataset available, but the dataset description suggest they are the same dataset). In future developments, the reasons why some models are better suited for a particular dataset will be studied. For example, the VAE-based models performed much better than our other models in the dataset that had the least batch effect and the most batches, whereas it was the opposite in the dataset with the most batch effect and least number of batches.
This aspect will be further developed to get improved guidelines that could be used to reduce the amount of time needed to explore the architectures and have an idea from the start of what should work in a particular framework. In conclusion, we propose a new tool, BERNN, that integrates multiple solutions to remove batch effect in LC-MS analyses while allowing an optimal classi cation of the biological samples. Using three different proteomic and metabolomic datasets, we benchmarked BERNN models to six other tools available in the literature and found that they outperformed them in all cases while not only considering reduction of batch effect but also classi cation performances. Finally, at the difference of most of batch correction tools which provide a corrected version of the data, here we rely on the encoded version for data classi cation.
Alzheimer Disease dataset Cerebrospinal uid (CSF) samples were obtained according to standardized collection and processing protocols through the Mass General Institute for Neurodegenerative Disease (MIND) biorepository, following written informed consent for research biobanking (IRB: 2015P000221). This cohort represents a clinically complex cohort spanning a wide variety of neurological disorders, which closely aligns with a real-life diagnostic situation. The raw data for this dataset is available on ProteomeXchange with accession number PXD043216 CSF proteins were trypsin-digested prior to LC-MS/MS analysis on an Orbitrap Fusion Tribrid (Thermo Fischer Scienti c) mass spectrometer operating in Data Independent Acquisition (DIA) mode. The 923 samples were injected in duplicate across 22 different batches.
A QC sample was also generated by mixing a small aliquot of all CSF samples and analyzed in the same conditions at the beginning and at the end of each batch. This QC sample was also used to generate a Gas Phase Fractionation (GPF) library. The raw les were then processed with DIA-NN 36 software (version 1. 8. 1) for protein identi cation and quanti cation. DIA-NN was used in two steps: 1) Library-free search on the GPF les using a Uniprot Reference Homo Sapiens database to generate a spectral library; 2) Library-based search on the sample and QC sample les using the spectral library generated in step one. The main report generated by DIA-NN was used with the DIA-NN R package  37  to get quanti cations of proteins corresponding to unique genes. Common contaminants were removed from the dataset to avoid any bias in the classi cation or in any subsequent analysis.
A complete list of removed protein IDs is available in the supplementary content (Table  S1 ). The dataset consists of a total of 923 samples, with 84 QC samples and 839 samples that were obtained from 408 subjects, with 22 subjects having more than one sample from repeat clinic visits. The samples were distributed into 22 batches with an average of approximately 41 samples per batch (mean = 41. 25, std = 15. 18). The cohort was subdivided into 6 different disease classes. Cognitively unimpaired patients (CU), Alzheimer's Disease with dementia (DEM-AD), Alzheimer's Disease with mild cognitive impairment (MCI-AD), dementia with causes other than Alzheimer's disease (DEM-other), mild cognitive impairment with causes other than Alzheimer's disease (MCI-other) and patients with Normal Pressure Hydrocephalus (NPH).
Adenocarcinoma dataset The dataset is composed of a total of 642 samples, with 74 QC samples and 568 patients, comprising 497 colorectal cancer and 71 chronic enteritis patients. There were 192, 192, and 184 subject samples with 25, 25, and 24 QCs in three batches, respectively. The raw MS le were converted to mzXML using ProteoWizard 38 , then preprocessed using the R package XCMS. After data processing, the nal dataset has 6461 metabolite peaks. More details on this dataset in the original paper  35. The dataset is available at https://github. com/dengkuistat/WaveICA_2. 0/tree/master/data.
Aging Mice dataset This dataset was introduced by  3939. The dataset is made of a total of 372 samples, of which 171 received a high fat diet and 201 has a chow diet. There were only 3 QC samples, all in the same batch, and thus these samples were discarded. The samples were distributed into 7 batches with an average of 53 samples per batch (mean = 53. 14, std = 26. 91). Each sample has 17887 features that represent the peptides' precursors. The raw data for the this AgingMice dataset is available with ProetomeXchange accession number PXD009160. To reconstruct the data matrix used in this study, rst download to Github repository https://github. com/symbioticMe/batch_effects_work ow_code. Then, download the le http://ftp. pride. ebi. ac. uk/pride/data/archive/2021/11/PXD009160/E1801171630_feature_alignment_requant.
Visual diagnostic The rst category for evaluating batch effect is visual diagnostic (Fig. 2 ). This is usually done with methods such as PCA, UMAP  40  or t-SNE  41. It is often how batch effect is rst noticed. The presence of a visually observable batch effect means it is a high source of variance in the data. However, these visualizations can be incomplete, thus the absence of visually observable batch effect does not mean it is inexistant. For example, for the Alzheimer dataset, the batch effect is not as obvious as in other datasets. If only visual diagnostics is done, batch effects might go unnoticed.
Effect Removal methods We normalizing each dataset using three different methods: minmax, standard and robust standardization. The rst methods to counter batch effects that we tried was applying the same three normalization methods, but to counter batch effects, they were applied individually to each batch individually, which we named minmax_per_batch, standard_per_batch and robust_per_batch. The latter two have more potential to remove batch effect, transforming the values into z-scores, thus forcing each batch to have a mean of 0 and unit variance. We also used combat 8 and Harmony  13  , as they are popular methods in microarrays/RNAseq and scRNAseq respectively. Combat is also used to remove batch effects from LC-MS datasets  42. We used two implementations of combat: a R version https://rdrr.
io/bioc/sva/man/ComBat. html and a python version https://github. com/epigenelabs/pyComBat, which we named pycombat in this manuscript. Intriguingly, the two implementations had very different outcomes. We also used WaveICA  25  and NormAE 18 , as they are state-of-the-art methods is batch effect correction of LC-MS data. We used our own implementation of NormAE, which has a slightly different architecture. We reduced the number of layers to a single hidden before and after the bottleneck to make the comparison with our own models. Unlike NormAE, we consider the number of neurons in each layer to be hyperparameters that are optimized. To give it a fair chance to outperform our own methods, we also use all the same hyperparameter optimization as our models.