dataesr/scientific-paragraphs-categorization huggingface dataset:
Simulation of Common Database and Data Preprocessing The datasets employed in this paper are generated using the STochastic Residential water End-use Model (STREaM) developed by Cominola et al. [16]. It is a stochastic water consumption simulation tool with a high capability to model the real-world water consumption patterns of the end-uses, such as toilet, shower, faucet, dishwasher, washing machine and bathtub. STREaM is calibrated using observed and disaggregated water consumption data from 300 single-family houses in nine U. S. cities. With a stochastic model based on Monte Carlo simulation, time series with different temporal resolutions (up to 10 s) and number of inhabitants can be emulated. To the best of our knowledge, STREaM is the most advanced framework for realistic water consumption simulation to date.
In this work, the chosen temporal resolution is ten seconds, which corresponds to the highest possible resolution of the simulation tool. Six datasets are generated with a varying number of inhabitants from one to six. In addition to the resolution and the number of inhabitants, the efficiency of the present end-uses and the time horizon of the datasets are fixed to "standard fixture" (i. e. , fixtures which are not specified to improve the water use efficiency) and one year, respectively. These datasets are publicly available (see supplementary materials). The output of the simulation includes six time series corresponding to the end-use toilet, shower, faucet, washing machine, dishwasher and bathtub. An additional time series aggregating the six above mentioned end-uses is also generated.
Threshold-Based Clustering Using Dynamic Time Warping The algorithm presented in  [2]  is based on the idea that the similarity of two events (or sequences) of a time series can be approximated by their distances to a reference event. This reference event is chosen arbitrarily from the dataset as described in  [8]  and will not be clustered. First, DTW distances of all water use events in the dataset to the reference event are calculated. Subsequently, relative distances are computed for each sequence. A threshold α is introduced to assign the event to a cluster. For algorithmic details, we refer to the original work published in  [2]. In this study, the threshold α is manually fine-tuned to 0. 6. Water 2021, 13, 236 6 of 16.
Results and Discussion The datasets described in Section 2. 1 are addressed as 1P, 2P, 3P, 4P, 5P and 6P, referring to the number of inhabitants in the simulated household. The following sections provide the evaluation of the supervised and unsupervised techniques using these six datasets. The employed code is executed with python 3. 7, making use of the ELKI version 0. 7. 5 and the scikit-learn version 0. 23. 1. ELKI is run with OpenJDK 13. 0. 33. Upon calling CASH with OpenJDK, 12 GB are allocated for its execution. All experiments were con-ducted with the same hardware, equipped with an Intel(R) Core(TM) i9-8950HK CPU @ 2. 90GHz (6 cores), 16 GB RAM and a 64 bit Windows 10 operating system.
Results and Discussion The datasets described in Section 2. 1 are addressed as 1P, 2P, 3P, 4P, 5P and 6P, referring to the number of inhabitants in the simulated household. The following sections provide the evaluation of the supervised and unsupervised techniques using these six datasets. The employed code is executed with python 3. 7, making use of the ELKI version 0. 7. 5 and the scikit-learn version 0. 23. 1. ELKI is run with OpenJDK 13. 0. 33. Upon calling CASH with OpenJDK, 12 GB are allocated for its execution. All experiments were conducted with the same hardware, equipped with an Intel(R) Core(TM) i9-8950HK CPU @ 2. 90GHz (6 cores), 16 GB RAM and a 64 bit Windows 10 operating system.
For the end-uses toilet and faucet, binary SVMs tend to have high false positive and false negative rates. It indicates, that these two events are often confused with other end-uses. Both of them are abrupt and short events corresponds to a sharp curve of consumption volume and flow rate. Thus, additional features with more discriminative power should be utilized to increase the performance of the classification. Hence, we can conclude that the classifier is not the single critical factor for the accuracy and the precision, as data preprocessing and feature extraction also affect the overall result. Moreover, datasets solely including the water consumption data might not be sufficient for the identification of complex end-uses (e. g. , washing machine or dishwasher), since significant prior knowledge, i.
Evaluation of the Unsupervised Techniques In contrast to the evaluation of supervised techniques, accuracy and precision cannot be computed for the evaluation of unsupervised techniques directly, as the clusters recognized by clustering methods are not necessarily equivalent to the end-use classes. For instance, we identified 10 clusters by using the k-means method; however, there are only seven end-uses in the simulated database. Furthermore, the identified clusters are not assigned to any end-use classes inherently; hence, the list of clusters will be a permutation of the list of end-uses. For these reasons, the Adjusted Rand Index (ARI) and the Adjusted Mutual Information (AMI), which are commonly used to assess whether detected clusters correspond to ground truth classes, are utilized as evaluation metrics.
Discussion The evaluation results for all clustering algorithms show that the identified clusters do not correspond to the ground truth end-use classes. One reason why the unsupervised techniques are not able to estimate the end-uses in the given datasets is that clustering methods generally take the most significant differences in a dataset into account. This might not be the end-uses but the variation of consumer behaviors in the household or other effects which are not related to the end-use classification. Moreover, single end-uses might possess a variety of different consumption patterns. For example, a washing machine shows different patterns depending on its program settings, the wash load or even the different sections of a wash cycle. These differences within individual end-uses make the task of separating end-use categories through clustering more challenging, especially since the most significant differences in a dataset determine the outcome of clustering.
In summary, unsupervised methods alone are not sufficient to detect the correct enduses fully automatically. Similar to the experiments with supervised methods (refer to Section 3. 1. 2), the dataset has a single specific temporal resolution. As a consequence, the quantitative results cannot be applied to datasets with lower temporal resolution directly. Furthermore, the synthetic dataset utilized in this study is generated stochastically, based on the real-world consumption data from cities within the U. S. Hence, it encodes the country specific water consumption behaviors and other factors (e. g. , economy, weather etc. ) For instance, end-uses like washing machine might have a lower variety of different consumption patterns in countries or region with lower per capita income, as entry-level washing machines with less program settings are more affordable.
Conclusions In this work, we perform a comprehensive quantitative comparison of several supervised and unsupervised ML techniques for residential water end-use classification. A common database is created with a stochastic simulation tool based on real consumption data. One of the most important findings of the quantitative results is that the unsupervised methods alone, i. e. , clustering techniques, are not sufficient to detect the correct end-uses of domestic water consumption fully automatically. This is somewhat consistent with the implications of the state-of-the-art literatures: In the context of end-use classifications, clustering techniques are commonly employed in combination with manual processing  [1]  or supervised techniques  [2] [3] [4]. Another conclusion we can draw from this study is that supervised ML techniques pose an efficient way to perform water end-use classification.
A further implication derived from our study is that the database for water end-use classification needs to be representative. As stated in the introduction, one of the major limitations of the state-of-the-art literature is that the databases are commonly acquired in a specific country/region. Although our study provides a first comprehensive comparison between supervised and unsupervised ML methods, the utilized database still suffers from the same limitation (refer to Sections 3. 1. 2 and 3. 2. 2). Hence, the quantitative results in the state-of-the-art as well as in our study can be hardly applied to other datasets. In order to increase the generalizability and reproducibility of ML techniques for end-use classi-fication, we suggest to establish a large representative dataset comprising raw data and annotations from various countries or regions for future research.
For this purpose, end consumers in different circumstances (e. g. , housing and household situation, age, gender etc. ) need to be encouraged to participate. Considering the above-mentioned aspects, crowdsourcing approaches which have been applied for medical applications  [29]  could be a possible solution. A suitable framework to implement such approaches successfully is the so-called citizen science project, where the citizens are contributing to a scientific project actively with their resources and knowledge. Scientific results and other output of the project are accessible to the participants as an exemplary reward. Previously, we concluded that one shortcoming of our study with regard to its generalizability is the temporal resolution of the utilized synthetic database, since it is fixed to 10 s.
However, Cominola et al. [16]  found that accurate end-use disaggregation is only possible with a temporal resolution of less than a minute. As end-use disaggregation and end-use classification are linked closely (disaggregation, unlike classification, separates overlapping events into the individual end-uses), we infer that a lower temporal resolution would also give less accurate results for end-use classification. For example, an end-use event with a duration shorter than one minute (e. g. , faucet or toilet) cannot be described by water consumption data with a lower frequency, i. e. , less temporal resolution. Therefore, the choice of the temporal resolution in our dataset is based on the requirement of the use-case, since we are aiming at recognizing all common residential water end-uses.
This leads us to our next implication: The characteristics of the underlying database should meet the requirement of the water end-use classification task. An orientation for the estimation of an optimal temporal resolution (i. e. , frequency) is given by the Nyquist sampling theorem. It states that a system with uniform sampling can reconstruct a continuous function (i. e. , analog signal) adequately, only if the sampling rate is at least two times the maximum frequency of the original signal. A final recommendation derived from our study is that we should be aware of the "class imbalance" problem for water end-use classification. In common residential households, events like faucet and toilet occur more frequently than bathtub, washing machine or dishwasher. Furthermore, bathtub, washing machine or dishwasher are not always available in each household.
Consequently, the water end-use classification problem generally suffers from an imbalanced class distribution, which could result in a lower classification accuracy of ML models. Effective ways to handle it are upsampling of the minority class or downsampling of the majority class. The former can be realized with data augmentation techniques, whilst the latter can be achieved by removing redundant data. In future work, it would be desirable to test the implemented methods on datasets from other regions of the world. Another promising future direction to improve the accuracy and generalizability of residential water end-use classification would be the systematic exploration and evaluation of hybrid techniques combining supervised and unsupervised methods. Encouraging results of hybrid techniques have already been presented in previous studies  [2] [3] [4] , on which comparative experiments could be conducted.
Last but not least, we would like to point out the potential of deep neural networks to resolve end-use classification. Recently, deep neural networks have achieved tremendous success in various disciplines, such as medical image processing, computer vision, space or geoscience. Although artificial neural networks have been applied for end-use classification with promising results (refer to  [11] [12] [13] [14] ), the networks employed as classifiers still have a shallow structure. Sophisticated neural networks in combination with deep learning techniques may benefit the entire end-use classification process comprising data preprocessing, feature extraction and classification. A potential candidate is the Recurrent Neural Network (RNN), which is well-suited to time series data, since it has the capability to maintain the internal state from time-step to time-step.
, 2010; Martens et al. , 2011; Scott et al. , 2011). However, except for the Bacteroidetes phylum, there are still very few enzymatic systems being described and characterized. Annotating the glycosyl hydrolase genes and their loci organization/synteny is mandatory to characterize the human gut microbial capacity to breakdown glycans. Today, the CAZyme database of sequences and subfamilies is a reference for such genes. The database is built by an academic laboratory using manual expert editing of the annotation  (Cantarel et al. , 2008) , while an open-source annotation tool called dbCAN is also available  (Yin et al. , 2012; Zhang et al. , 2018). Annotating CAZyme genes is not an easy task due to the modularity of the gene structure  (Cantarel et al. , 2008). Beyond difficulties to annotate CAZyme genes, the practical question to further investigate the link between the metabolic disorders (phenotype-level) and the genomic content of the microbiome is still an open question.
, 2015). Therefore, glycosyl hydrolases are encoded within operons that are not taken into account when simply annotating CAZymes. Currently, obtaining PUL annotation from a genome is not straightforward, and we found two available published resources. The PULDB database  (Terrapon et al. , 2017)  of experimentally and non-experimentally proven PULs in Bacteroidetes is built as an extension to the CAZy database. The other one is a prediction tool called PULpy, identifying CAZymes that are co-localized with susCD gene pairs. The authors present their tool as a public version of the PULDB algorithm  (Stewart et al. , 2018). These current resources have drawbacks for non-experts: the first database has to be queried with either fibers or known species. Therefore, it is hardly usable for any new isolated strain.
Carbohydrate-Active Enzyme Annotation Annotation of CAZymes was done using the open-source dbCAN2 pipeline  (Yin et al. , 2012; Zhang et al. , 2018) , which relies on three different algorithms: (1) hidden Markov models (HMMER)  (Eddy, 2011) , (  2 ) alignment (DIAMOND)  (Buchfink et al. , 2015) , and (3) peptide recognition (Hotpep)  (Busk et al. , 2017). The outputs are the genes associated with a CAZy gene based on each of the three algorithms. We chose to use the majority consensus rule: when a hit is found by two algorithms out of the three, then the gene is considered as a CAZy gene. As expected, the alignment algorithm (DIAMOND) provides more hits because of the modular structure of GH genes. In this manuscript, the pipeline dbCAN2 was used with the default settings. In addition, we obtained CAZyme annotations from the CAZy database from B.
Henrissat. The annotation was done in two steps: first, a BLASTP analysis of the predicted ORFs against the full-length sequences included in the CAZy database is performed  (Cantarel et al. , 2008). Second, the remaining sequences are manually analyzed by both (i) a BLAST search against individual GH, PL, CE, CBM, and GT modules and (ii) a HMMER3 search using hidden Markov models built for each CAZy module family. Raw CAZy annotations are presented in Supplementary Tables  S1, S2. The strain selection was performed using the available growth dataset used in this manuscript.
Prediction of Polysaccharide Utilization Loci Using FiberGrowth The FiberGrowth tool automatically predicts growth using fiber-specific PULs on a given bacterial genome. As input, it either takes a genome in fasta format or a gff file with gene locations together with their amino acid sequences in fasta format. If only a genome is provided, gene prediction will be performed using prodigal  (Hyatt et al. , 2010). In the next step, members of each fiber-specific reference PUL are identified using hmmscan of the HMMER package  (Eddy, 2011). Based on the location and function, spatially clustered genes of carbohydrate active enzymes are determined by performing single linkage hierarchical clustering on the gene position using Euclidean distance. PUL candidates are retrieved by using a 5-kb threshold on the gene distance tree, allowing unannotated genes to be part of a candidate PUL.
In the last step, candidates having all required core genes are reported (Supplementary Figure  S0 ). Running the FiberGrowth tool on one bacterial genome takes about 1 min on a 2. 3-GHz Intel Core i9 using one core. FiberGrowth is implemented in R (R Core Team, 2018) and makes use of the  packages data. table (Dowle and Srinivasan, 2019) , docopt (de Jonge, 2020), DT  (Xie et al. , 2020) , gggenes  (Wilkins, 2019) , ggplot2  (Wickham, 2016, p. 2), knitr  (Xie, 2015) , magrittr  (Bache and Wickham, 2014) , rhmmer  (Arendsee, 2017) , rmarkdown  (Allaire et al. , 2020) , and vroom  (Hester and Wickham, 2020). The FiberGrowth code and PUL models are available on Github at https://github. com/wholebiome/FiberGrowth.
Validation of FiberGrowth Pipeline With External Bacterial Growth Datasets We compared the predictions of FiberGrowth to new external experimental measures of 28 bacterial strains' abilities to degrade a wide variety of dietary and host-derived polysaccharides performed by Eric Martens (unpublished data, but kindly shared to benchmark FiberGrowth performance) and from a previous work  (Desai et al. , 2016 ; Supplementary Table  S6 ). Briefly, these authors documented the growth of 534 strains including 28 types of strains, for which the genome sequence is available, on several polysaccharides and glycans as sole carbon sources (n = 2 replicate cultures per glycan substrate). Strains were grown on (i) a glucose-rich growth medium (PYG), (ii) a carbon-free minimum medium (PY), and (iii) a minimum medium with a polysaccharide as the only carbon source (PY + polysaccharide).
CAZy and dbCAN Annotation Comparison Since two possible strategies exist to annotate glycosyl hydrolases, the automatic open-source pipeline dbCAN and a manually curated CAZyme database, we assessed the annotation differences between them. Fifty-four different Bacteroidetes genomes were analyzed, on which the annotations were performed while screening for 87 different GH families. The majority of the predictions regarding GH gene counts were identical and the diversity of the GH was evidenced by the large number of GH detected per genome, ranging from 100 to 400 (Figure  2A ). However, the two annotation tools showed discrepancies. On a total of 2,889 genes detected on the 54 different genomes, 107 are only detected by CAZy and 49 only by dbCAN (Figure  2B ). Beyond the gene count differences, it is noteworthy that, under our settings, some GH were not detected by one of the tools, and it depends on the GH family.
For example, the CAZy detected GH24 and GH142 while dbCAN did not. Conversely, GH99 was annotated by dbCAN whereas CAZy was not, for this dataset. A detailed visualization is available on Supplementary Figure  S3. The details of the GH families found by only one of the methods can be found in Supplementary Table  S3. Using only the glycosyl hydrolase count to predict growth did not lead to meaningful results, with too many false positives (see Supplementary Figures  S3, S4 ).
To better fit the experimental results, a 0. 01 OD 600-nm threshold was used. Aware of these limits originating from the experimental data, we calculated FiberGrowth performance for each PUL model (Figure  4  and Table  1 ). The best performance was measured for arabinoxylan with a TPR of 0. 89 and FPR of 0. Only nine strains grew on arabinoxylan and only one false negative was detected (for Bacteroides cellulosilyticus). Our model, designed based on Martens et al. 2008) , takes into account a total 10 genes from the operon, including the regulator of a two-component system (Figure  3 ). Here, the information is taken from Figure  2  (expression data for genes bacova_03417, bacova_03421-36, bacova_03437-40, and bacova_03448-50) and Supplementary Table  S2  (gives more precisely the necessary three enzymes) with Bacteroides ovatus gene information.
thetaiotaomicron involved outer membrane bound-attached starch-binding sites and periplasmic starch-degrading enzymes, rather than only extracellular enzymes. Since then, starch catabolism has been largely characterized, and Foley et al. (  2016 ) described the Sus operon as the model system for starch uptake in Bacteroidetes. The complexity and substrates' diversity of starch-related polysaccharides hardly fit the CAZyme database. GH13 displays a wide phylogenetic diversity (as described by  Stam et al. , 2007)  that is now classified under GH13 subfamilies in CAZyme. The visualization of the genes' organization of our PUL models for few Bacteroides genomes is consistent with the large diversity and lack of synteny between close genomes. In the in vitro experiments we considered, starch was from potato starch (Eric Martens' data), and the results would be different from a different origin or amylose/amylopectin ratio.
The PUL model was based on transcriptomics data, and because of the complexity of mucin, the GH genes over-expressed at this time point were limited. Another issue is cDNA annotation. For instance, one of the main causes of misannotation of the fuc genes is their similarity to the genes for rhamnose utilization. Both the FucK and RhaB proteins belong to the FGGY family of carbohydrate kinases (Pfam: PF02782). Mucin used in experiments is very frequently pig gastric mucin type III. However, mucin composition depends upon the individual and the GI tract segment, and its complexity has only recently been acknowledged. Mucin is at a crossroad between dietary substrates and human secretion. Indeed, using droplet microfluidics, we recently demonstrated and characterized a new GH enzyme, active on human gangliosides, with similar structure to human mucin and milk oligosides and overrepresented in IBD patients' metagenomes  (Tauzin et al.
This pathway does match our model, which includes enzymes and carbohydrate-binding and import proteins with distinct substrate specificities, which could not have been predicted previously based on sequence data alone. The discrepancy between databases regarding GH annotation can be highlighted in the case of levan PUL, looking at the domain level. Our results showed that taking into account three GH32 distinct gene copies and a levanase within a PUL model was sufficient to predict growth on levan, but the model overpredicted growth in Bacteroides genomes. However, the PUL was built using transcriptomics data, which raises questions on crossannotation of cDNA and genomes. Indeed, Pfam annotation from the transcriptome provided an N-terminal domain for the GH32 used in this model, instead of a catalytic domain.