dataesr/scientific-paragraphs-categorization huggingface dataset:
This problem, known as "dataset shift" or "concept drift"  (Moreno-Torres et al. , 2012) , is a very common pitfall in the application of ML to marine ecology problems (e. g. Langenkämper et al. , 2020)  and for the trustworthiness of ML models in general  (D'Amour et al. , 2020). Indeed, it leads to poor predictive performance that is not necessarily detected when the model is evaluated. Detecting it requires specific validation methods, such as computing evaluation metrics per sample, to capture the inter-sample variability in distribution  (Gonzalez et al. , 2017). When such a shift is detected, retraining the model with a new training set incorporating more of the natural variability  (Langenkämper et al. , 2020)  or discarding low confidence predictions  (Plonus et al. , 2021)  can help reduce its effect.
For classification-type problems, the transition towards quantification approaches, which estimate abundance per class directly, rather than classifying each object, and use the class distribution, can help alleviate it  (Gonzalez et al. , 2019; Orenstein et al. , 2020). Overall, the transferability of a model learned on a given dataset to a dataset with different characteristics is called "domain adaptation" and is also an active field of research  (Kouw and Loog, 2019). Finally, ML models are only as good as the datasets they are trained on. Those training datasets are generated by humans, who can make mistakes. For example, in the hard task of discriminating among six dinoflagellate species with large intraspecies morphological variations from images, trained scientists achieved 67-83% self-consistency and only 43% consensus  (Culverhouse et al.
, 2003). For the estimation of benthic cover from quadrat pictures in coral reefs, self-consistency of experts ranged from 50 to 90% depending on the type of cover  (Beijbom et al. , 2015). One upside is that, in both cases, ML models trained on a reference dataset reached performance similar to or higher than human labellers when their inconsistency is taken into account. Another use of ML can be to resolve ambiguous labels and present such potential mistakes to new experts  (Schmarje et al. , 2022). Finally, a way of alleviating the effect of inconsistent labels is to use additional, independent "ground truth" validation information. For example, in  Lekunberri et al. (2022) , estimations of species abundance and size inferred with ML were compared with independent samplings and counting at port when fish were landed.
General outlook As remarked above, so far, applications of ML in marine ecology have closely followed the development of techniques in computer sciences (Figure  7 ). However, innovation in ML is accelerating and it may be difficult for marine ecologists to keep track of it. Current developments include transformerbased architectures and diffusion models. Transformers can be seen as an alternative to LSTM recurrent neural networks (Table  1 ) for sequential inputs, such as language; a wellknown everyday application is ChatGPT (Chat Generative  Pre-Trained Transformer). Their extensions to images are called vision transformers (ViTs), which can be considered as alternatives to CNNs; they have been topping the ImageNet classification challenge since their release  (Dosovitskiy et al. , 2021).
com/sota/image-classification-on-imagenet), which require more data to train  (Dosovitskiy et al. , 2021). Such massive datasets and the computing power to train on them are often only available in large private companies (ChatGPT and CLIP are from OpenAI, the first ViT is from Google, etc. ). While the performance benefit is measurable on well-defined challenges (8% increase in accuracy on ImageNet between the two models above), the actual gains on the smaller, noisy, imbalanced datasets of marine ecology, for which global accuracy may not even be a relevant metric, remain to be demonstrated; effort may turn out to be better spent elsewhere. Actually, the relative performance of existing solutions is already difficult to assess in most subdomains described above because of the lack of standard benchmarks (see also  Irisson et al.
, 2022 for plankton imaging; and Politikos et al. , 2023 for macrolitter). Such benchmarks depend on the availability of published (and labelled) datasets. The field is progressing on that end, with the release of datasets on e. g. plankton  (Sosik et al. , 2015)  and fish  (Fisher et al. , 2016)  images, remotely sensed images  (Kikaki et al. , 2022) , or ship noise  (Santos-Domínguez et al. , 2016). However, other datasets, such as images from electronic monitoring on ships, are gathered by private companies that aim to use them to develop and sell electronic monitoring solutions, which are either made mandatory by authorities or desired by fishing companies to reduce costs compared to human observers. For researchers, the effort of gathering and labelling a dataset consistently is often huge and makes some people reluctant to distributing the result openly, although the availability of referenceable repositories (e.
g. Zenodo) and citation tracking via Digital Object Identifiers helps. After releasing datasets, the next steps would be to define evaluation metrics following guidelines for proper benchmarking  (Weber et al. , 2019)  and to provide tools to easily track the results of those, now comparable, studies. So, overall, releasing high-quality public datasets, defining benchmarking studies, and centralizing their results are necessary to assess (i) the current state of ML tools in marine ecology and (ii) the tradeoff between gains from new architectures and their cost in complexity. Transferring innovations from computer sciences to marine ecology also depends largely on efficient collaboration across disciplines. However, establishing interdisciplinary research teams is difficult and takes a long time  (Haapasaari et al.
, 2012). Once again, public datasets are an efficient first step for marine ecologists to garner interest from computer scientists. For example, after the WHOI-Plankton dataset was released  (Sosik et al. , 2015) , it was used in many papers on this topic in computer science conferences. In the assembled database, about 15% of references are from such computer science conferences or engineering journals, but very few are from high-level ones. This can indicate that marine ecology questions have not gained enough interest from the ML research community to generate significant new developments that would be published at high-profile conferences, unlike other applications such as face recognition, customer tracking, or self-driving cars. It could also simply reflect differences in overall funding for research in those fields, linked to the potential commercial applications of some research.
Acknowledgements All authors acknowledge the support of  ICES  through the  Working group on Machine Learning in Marine Science (WGMLEARN). Soykan ,  C. U. ,  Eguchi ,  T. , Kohin, S. , and  Dewar , H. 2014. Prediction of fishing effort distributions using boosted regression trees. Ecological Applications, 24: 71-83. Spampinato ,  C. ,  Giordano ,  D. ,  Di Salvo ,  R. ,  Chen-Burger , Y. - H. J. , Fisher, R. B. , and  Nadarajan , G. 2010. Automatic fish classification for underwater species behavior understanding. Proceedings of the First ACM International Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Streams, pp. 45-50. New York, NY. http://doi. acm. org/10. 1145/1877868. 1877881 (accessed 7 August 2019). Stephens, D. , and Diesing, M. 2014. A comparison of supervised classification methods for the prediction of substrate type using multibeam acoustic and legacy grain-size data.
Marine mammal species classification using convolutional neural networks and a novel acoustic representation. In Machine Learning and Knowledge Discovery in Databases, pp. 290-305. Ed. by U. Brefeld, E. Fromont, A. Hotho, A. Knobbe, M. Maathuis, and C Robardet. Springer International Publishing, Cham. Thoya, P. , Maina, J. , Möllmann, C. , and Schiele, K. S. 2021. AIS and VMS ensemble can address data gaps on fisheries for marine spatial planning. Sustainability, 13: 3769. Trifonova, N. , Kenny, A. , Maxwell, D. , Duplisea, D. , Fernandes, J. , and Tucker, A. 2015. Spatio-temporal Bayesian network models with latent variables for revealing trophic dynamics and functional networks in fisheries ecology. Ecological Informatics, 30: 142-158. Trifonova, N. , Maxwell, D. , Pinnegar, J. , Kenny, A.
Funding TC acknowledges support from the Swiss National Science Foundation (#31003A_179125), the European Research Council (#818449 AGENSI), and the Horizon Europe programme (#101094924 ANERIS). JAF-S has received funding from the project H2020 FutureMARES (#869300) and Sus-TunTech (#869342). NOH acknowledges support from the CRIMAC centre funded by the Research Council of Norway #309512. KH is supported by the Mission Atlantic project funded by the European Union's Horizon 2020 Research and Innovation Programme (#862428). MS acknowledges funding from the European Union's H2020 programme #7553521 (SMARTFISH); European's Maritime and Fisheries Fund and the Danish Fisheries Agency, #33112-I-19-076 (AutoCatch); and Fully Documented Fisheries, funded by the European Maritime and Fisheries Fund (EMFF).
LVdB acknowledges support from the Sand Fund of the Federal Public Service Economy. SC, RML, and SV acknowledge support from the H2020 project AtlantECO (#862923). RML acknowledges support from CNPq, Brazil (grant number 315033/2021-5). RK acknowledges support via a "Make Our Planet Great Again" grant of the French National Research Agency within the "Programme d'Investissements d'Avenir" (#ANR-19-MPGA-0012), by the Heisenberg programme of the German Science Foundation (#469175784), and from NOAA (#NA21OAR4310254). JBR acknowledges funding from the IFREMER Scientific Direction project DEEP. KM's participation was funded by the Norwegian Ministry of Trade, Industry and Fisheries. JOI acknowledges funding from the Belmont Forum project WWWPIC (#ANR-018-BELM-0003-01).
Stunting affects one-in-five children globally and is associated with greater infectious morbidity, mortality and neurodevelopmental deficits. Recent evidence suggests that the early-life gut microbiome affects child growth through immune, metabolic and endocrine pathways. Using whole metagenomic sequencing, we map the assembly of the gut microbiome in 335 children from rural Zimbabwe from 1-18 months of age who were enrolled in the Sanitation, Hygiene, Infant Nutrition Efficacy Trial (SHINE; NCT01824940), a randomized trial of improved water, sanitation and hygiene (WASH) and infant and young child feeding (IYCF). Here, we show that the early-life gut microbiome undergoes programmed assembly that is unresponsive to the randomized interventions intended to improve linear growth. However, maternal HIV infection is associated with over-diversification and over-maturity of the earlylife gut microbiome in their uninfected children, in addition to reduced abundance of Bifidobacterium species.
Metagenome sequencing performance Overall, 875 unique whole metagenome sequencing datasets were used. On average, 12 million ± 4. 2 million quality-filtered read pairs were generated per sample. Sixteen negative controls produced a mean of 655 quality-filtered reads (range = 149 to 1,425; SD = 456). The median percent of human reads detected was 0. 05% but ranged widely by age group and decreased over time (Supplementary Fig. 2a ). The median percent un-annotatable reads detected in each sample was 58. 6% and increased over time (Supplementary Fig. 2b ). Thirty-six samples were subject to repeated extraction and metagenome sequencing to assess technical variation. These samples originated from 4 unique children, each with 3 visit samples, where each visit sample was extracted and sequenced in 3 replicates.
Microbiome functionality, but not composition, predicts linear and ponderal growth We next examined the relationship between taxonomic and functional features of the gut microbiome and attained growth (length-for-age Z-score [LAZ] and weight-for-height Z-score [WHZ]) and growth velocity (WHZ and LAZ velocity, increase in z-score increments per day between visits) from 1-18 months of age using XGBoost models. These models were used to test whether the composition or function of the gut microbiome could predict child growth. Models were run separately for each of the six age groups, stratified by maternal HIV status and run in two combinations of predictive features: (i) microbiome features alone (species or pathways); and (ii) microbiome features and epidemiological variables, which included maternal anthropometry, baseline WASH and infant diet variables, amongst others (Supplementary Data File 3).
Importantly, these microbiome differences between CHU and CHEU were not reversed by WASH and IYCF interventions, suggesting that novel microbiota-directed interventions that enhance Bifidobacteria colonization or prevent the over-diversification of the early-life gut microbiome are warranted for future clinical trials in CHEU. There are several limitations to this analysis: (i) Machinelearning prediction models at inherent risk of over-fitting and bias depending on cross-validation approaches and model complexity. Our conservative approach to model tuning and feature selection aimed to minimize overfitting. The age models, which included an out-of-sample ('healthy') test dataset, produced comparable performance metrics, making overfitting less likely. However, it remains possible that some degree of over-fitting occurred in growth models, since an out-of-sample test dataset was not used, thereby possibly affecting the strength of our reported gut microbiome-related growth predictions.
(ii) the SHINE microbiome sub-study intentionally included more women living with HIV than the main SHINE trial (30% versus 15%), and the proportion of HIVexposed infants varied by age group. This resulted in some small subgroups in some of our age categories by HIV exposure analyses, which likely resulted in unstable predictions in certain XGBoost models; (iii) a significant proportion of the sequencing reads included in our datasets were not annotatable (median 58. 6%) using the specified bioinformatic pipelines (MetaPhlAn3 and HUMAnN3). This large abundance of unknown sequences is common in samples derived from non-Western populations  63  and leads to inferences solely being made from the assignable fraction, potentially missing important microbiota features that are predictive of infant growth but are currently not represented in databases.
Kmer-based bioinformatic tools (e. g. Kraken) and marker-based tools (e. g. MetaPhlAn) each have their own advantages and disadvantages. Tools relying on metagenome assembly may help in identifying some of these unknown sequences in datasets from less-studied populations in the future; (iv) Escherichia coli was one of the most prevalent bacterial species across all age groups; however, MetaPhlAn3 cannot differentiate between E. coli pathotypes. Different E. coli pathotypes, such as enteropathogenic and enteroaggregative E. coli have been associated with intestinal pathology and EED; however, we previously reported in the same cohort that some of these pathotypes were not associated with growth 14 ; (v) we attempted to account for the agerelated confounding of the microbiome-growth relationship by predicting attained growth and growth velocity in discrete age groups, but residual confounding may still be present, influencing our ability to identify microbiome features independently associated with growth; (vi) data on infant antimicrobial use, aside from cotrimoxazole use in CHEU, was incomplete (but infrequent), limiting our ability to confidently assess this as a potential confounder in early-life gut microbiome maturation; (vii) differences in microbiome composition and function may also be driven by differences in intestinal microbial load, motility and biogeography, which were not assessed; and (viii) finally, we chose mother-infant pairs with the most complete sample collection during follow-up, in order to strengthen our inferences about development of the gut microbiome over time, but which may introduce selection bias.
SHINE trial design The study design and methods for The Sanitation Hygiene Infant Nutrition Efficacy (SHINE) trial and for the corresponding microbiome analyses, have been reported previously  64, 65. Briefly, SHINE was a 2 × 2 cluster-randomized trial, conducted between 2012 and 2017, to determine the independent and combined effects of improved infant and young child feeding (IYCF) and WASH on child stunting and anaemia in two rural Zimbabwean districts, Chirumanzu and Shurugwi (NCT01824940). 5280 pregnant women were clusterrandomized to one of four interventions: WASH, IYCF, WASH + IYCF, and Standard of Care (SOC). The SOC interventions, included in all trial arms, comprised exclusive breastfeeding promotion for all infants up to 6 months and strengthened prevention of mother to child transmission (PMTCT) of HIV services.
The primary outcomes (LAZ and haemoglobin concentrations at 18 months) were assessed by comparing length and age of children against standardized World Health Organization growth standards for LAZ and using a Hemocue hemoglobinometer, for haemoglobin concentrations. Infants were followed up at study visits at 1, 3, 6, 12 and 18 months of age. Length and weight were measured at each infant visit, as described previously  39. Length-for-age z scores (LAZ) and weight-forheight z scores (WHZ) were calculated from length and weight measurements at each visit according to WHO Child Growth Standards. Epidemiologic data for the infants was collected from the baseline and follow-up visits using trial questionnaires that included maternal anthropometry, birth outcomes, baseline household WASH facilities, household wealth, maternal education, religion, parity, household size, dietary diversity, changes in breastfeeding and complementary feeding practices, food security, 7-day and 3-month infant health status, and antimicrobial use.
Microbiome sub-study All CHEU and a subgroup of CHU from the SHINE study were enrolled into an Environmental Enteric Dysfunction (EED) sub-study (n = 1,656 mother-child pairs); these infants underwent intensive biological specimen collection at 1, 3, 6, 12 and 18 months of age  65. The EED substudy was therefore enriched for mothers living with HIV, by design. Sample selection for inclusion into the current microbiome study was conducted to enhance longitudinal profiling of the mother and infants gut microbiota. Of the mother-infant pairs within the EED substudy, those with least one maternal fecal specimen (of 2 possible) and at least 2 infant fecal specimens (of 5 possible) were included in the gut microbiome analyses. An additional 94 samples collected at the 1 and 3-month visits, that did not meet these criteria, but had microbiome sequencing data available from a separate study examining rotavirus vaccine immunogenicity in the SHINE trial  66  , were also included in these analyses.
Bioinformatics Quality control and bioinformatic processing of raw sequencing data was conducted using the publicly available KneadData (https:// huttenhower. sph. harvard. edu/kneaddata/). Sequenced reads were trimmed of adaptors and filtered to remove low-quality, short (<70% raw read length), and duplicate reads, as well as those of human, other animal or plant origin, with default settings. Species composition was determined by identifying clade-specific markers from reads using MetaPhlAn3 with default settings  67. Relative abundance estimates were obtained from known assigned reads, and unknown read proportions were estimated from total, assigned and unassigned, reads. Percent human DNA was estimated from KneadData output, using the proportion of quality-filtered reads that align to the human genome.
Given the smaller viral genome sizes, sequencing depth, and limitations of MetaPhlAn3 for virus identification, we did not include viruses in our current analyses. We applied a minimum threshold of >0. 1% relative abundance and ≥5% prevalence for all detected species. Metabolic pathway composition was determined using HUMAnN3 with default settings against the UniRef90 database 67. Pathway abundance estimates were normalized using reads per kilobase per million mapped reads (RPKM) and then renormalized to relative abundance. We applied a minimum relative abundance threshold of 3 × 10 -7 % and ≥5% prevalence for all metagenomic pathways.
Statistical analysis All data were analysed using R (v. 4. 0. 5). Microbiome data were handled using the phyloseq package (v1. 34. 0). Alpha diversity metrics were calculated using the vegan package (v2. 5. 7). A permutation test, implemented using the coin package in R, was performed to compare alpha diversity metrics between unbalanced age categories. Betadiversity was estimated using the Bray-Curtis dissimilarity index and analysed by permutation analysis of variance (PERMANOVA). Differential abundance analysis of species or functional pathways was assessed using multiple regression analyses using the MaAsLin2 v1. 5. 1 package  68  and applying default arguments (significance thresholds -p < 0. 05, q < 0. 25). Four covariates were chosen for adjustment in diversity analyses, microbiome maturity analyses and MaAsLin2 regression models and included age in days at stool sample collection, exclusive breastfeeding status (recorded at 3 months old), delivery mode (caesarean section versus other), and randomised trial arm.
XGBoost models Relationships between the infant microbiome and age or growth (attained LAZ/WHZ and WHZ/LAZ velocity) were evaluated using extreme gradient boosting machines (XGBoost). XGBoost builds an optimized predictive model by creating an ensemble from a series of weakly predictive models. XGBoost is also non-parametric, can capture non-linear relationships, and can accommodate high-dimensional data  69. The XGBoost models were developed using microbiome relative abundances (species or pathways) and all epidemiologic variables. XGBoost model selection was performed in 3 stages  30. In brief, the BayesianOptimization function of the rBayesianOptimization package was used with 10-fold cross-validation to select model hyperparameters by minimizing the mean squared error (MSE). This 3-stage hyperparameter tuning and model building was performed for two feature sets, one comprising microbiome features and a second comprising microbiome plus epidemiologic features; this was done to assess model performance and to examine the contribution of epidemiologic versus microbiome features.
In stage one, the model dataset included either microbiome features alone or microbiome and epidemiologic features. Models with the lowest MSE (in the 5 th percentile) were retained, and from these models the variables that contributed to the top 95% of variable importance by proportion were retained; this filtering was applied separately to the microbiome and epidemiologic features. In stage two, models were built with the retained features obtained from stage one, and BayesianOptimization was re-run as for stage one but using leave-one-out cross-validation. Models containing microbiome features alone, or together with epidemiologic features, were again filtered according to feature importance as for stage one. In stage three, all retained epidemiologic variables, microbiome features, and hyperparameters selected in stage two were used to fit our final models, using leave-one-out crossvalidation to minimize the MSE.