dataesr/scientific-paragraphs-categorization huggingface dataset:
All procedures were carried out in accordance with clinical best practices, applicable laws and regulations as well as the Declaration of Helsinki. Ground-truth labels for the dataset were generated from clinical records based on validated laboratory results and clinical parameters (c-reactive protein (CRP), body temperature, antibiotic response for bacterial, sputum or sweat polymerase chain reaction (PCR) and/or absence of bacterial infection signs for viral) as well as clinical assessment of specialist paediatricians/neonatologists not involved in image evaluation. Model training. Privacy-preserving processing of dataset statistics. For the training of neural networks, data is typically pre-processed by mean subtraction and division by the standard deviation. In federated learning, dataset statistics from the local nodes or aggregated statistics from all nodes can be used.
Additionally, the provision of the final model in an inference setting requires these statistics for rescaling incoming images. However, dataset statistics can contain private information that should not be shared, especially in case non-imaging data is included (for example, age in the case of clinical record data). Hence, we propose and implement differentially private secure aggregation of dataset statistics. Here, sensitivity-calibrated Laplacian noise is added to the statistics to satisfy a user-defined ϵ DP value before SMPC is used to average them, and they are then stored on the central server for later use. Before inference starts, the data is rescaled with the (differentially private) securely aggregated mean and standard deviation of the training set. For training, the nodes use their local dataset statistics.
Thus, data leakage is prevented, especially in the case individual nodes contain few, or just one, dataset(s). Model architecture, hyperparameters and augmentation. We used the ResNet18 architecture  19  , pretrained on ImageNet  48  , with the final average pooling layer replaced by a single linear layer with 512 units and randomly initialized with the Kaiming Uniform initializer  49. Images were cropped to squares such that the entire chest section of the radiograph is preserved and resized to 224×224 pixels. The following standard augmentation techniques were employed: random horizontal flips, random affine transformations, Gaussian noise injection. In extension, we used the Albumentations library  50  to apply the following transformations: random changes in the gamma value and brightness, blurring, optical distortions, grid shuffles/dropouts/distortions, elastic transforms, changes in hue-saturation-value (HSV) colour space, inverting images, cutouts of the image, artificial shadows, fog, solarizations and sun flares.
We also provide the option for histogram equalization or contrast-limited adaptive histogram equalization (CLAHE), both as an augmentation and a standardization technique. The individual augmentations were introduced with a probability p 1 and augmentation was activated overall with a probability p 2. Furthermore, we applied a modified variant of MixUp augmentation 51 by which the mixing parameter (λ) is randomly sampled from a uniform distribution similar to that in ref. 36. Training was performed for 40 epochs using the Adam optimizer  52  with a log-linearly decreasing learning rate initially set at 10 -4. PriMIA caches models automatically after each round, and selects the model with the highest validation set Matthews correlation coefficient (MCC). The centralized model was trained by pooling all data on a single machine and training the model on the accumulated dataset.
Personalized models were trained on the respective nodes using only the local dataset. PriMIA implements the ability to carry out centrally coordinated automated hyperparameter tuning on the entire federation or locally, which was used to determine the best model in every case according to highest validation set MCC. An example is provided in Supplementary Section 1 and Supplementary Fig. 23  based on clipping the gradient L 2 -norm of each individual sample, then adding calibrated Gaussian noise. This process occurs on each node independently with independent noise sources (local DP). We considered the paediatric pneumonia dataset private, therefore did not perform hyperparameter optimization based on multiple training runs. Furthermore, due to the relatively small size of the dataset, we determined it would not be possible to train the model with sufficient utility while maintaining acceptable privacy guarantees.
Hence, we used the pre-training technique described previously  23  and employed a publicly available dataset trained on a related task to determine the optimal parameters for the DP mechanism and pre-train the model. Details can be found in Supplementary Section 8. 2. 2 and Supplementary Fig. 4. Training topology, gradient descent and secure aggregation. We selected the hub-and-spoke system topology due to its reported improved final model performance over techniques such as incremental or cyclical training  5, 43  and its higher flexibility with respect to node availability and asynchronous training  53. In PriMIA training is carried out asynchronously in rounds. Initially, the model is sent from the central server to all computation nodes. During each round, nodes locally perform a variant of gradient descent in which gradient updates are weighted inversely by the frequency of the individual dataset classes present on the node (class weighted gradient descent).
We provide further details on these findings in Supplementary Section 10 and Supplementary Fig. 6. Measures against FL training deterioration. Literature findings and our own evidence indicate that, in case one of the federation's nodes contains less data than others, continuing training beyond convergence until other nodes have completed training can lead to overfitting or training collapse. Alternatively, not including the updates from this node can lead to catastrophic forgetting  56  of the node's data and reduced generalization performance. We empirically determined that local early stopping, that is, terminating training on the local node once the node's local dataset is exhausted, then using the state of the node's local model for all future update steps until a full round of training is completed, led to improved training performance.
PriMIA's DP training procedure provides effective protection against such attacks  30, 32, 59  to the individuals whose data was used to train the model used for inference. Classification performance assessment. Classification performance was evaluated as follows. For expert readers, accuracy, sensitivity/specificity (recall) and MCC  27  were calculated on test set 1. The model's performance was evaluated in terms of accuracy, sensitivity/specificity (recall), ROC-AUC MCC on the validation set and on both test sets. MCC was employed due to its invariance to class imbalance and its indication of prediction concordance alongside quality of classification, leading to recent recommendations for its use over the usually employed accuracy or F1-Score metrics  61. McNemar's test was used to test for statistical significance in classification performance.
As an active attack, iDLG can be executed by an adversarial client or central server. We note that in the case of an adversarial central server, the usage of SMPC prevents the disclosure of individual model updates, therefore only allowing the adversary to utilize averaged model updates instead. For the attacks on the FL system we assumed that one out of three data owners is an adversary. For the 'baseline' attack on the centralized model, we used a batch size of 1. Attacks were performed against 100 randomly selected images from the training set. For the gradient norm experiments, 100 gradient samples were taken at equispaced intervals during model training. Batch size experiments were carried out under identical circumstances only varying batch size. Model and dummy image initialization was deterministically set for all experiments.
21 Outside of the above-mentioned PPML techniques, it offers solutions to common challenges in medical imaging analysis workflows, such as dataset imbalance, advanced image augmentation, federation-wide hyperparameter tuning functionality. Furthermore, it provides an accessible user interface for applications ranging from local experimentation on the user's machine to distributed training on remote compute nodes to facilitate the application of PPML best practices in medical consortia. The source code and documentation for the library and the publicly available data are provided at https://doi. org/10. 5281/zenodo. 454559918. We present a case study for the application of PriMIA on clinical data by training an 11. 1 million parameter ResNet18 CNN19on the paediatric pneumonia dataset originally proposed by Kermany et al.
Acknowledgements We acknowledge funding from the following sources, funders played no role in the design of the study, the preparation of the manuscript or the decision to publish. The  Technical University of Munich, School of Medicine Clinician Scientist Programme (KKF) , project reference  H14  (G. K. ). German Research Foundation ,  SPP2177/1 ,  German Cancer Consortium (DKTK)  and  TUM Foundation, Technical University of Munich  (R. B. and G. K. ). European Community Seventh Framework Programme  (FP7/2007-2013 grant no. 339563 -CryptoCloud ) and  FUI   ANBLIC  Project (T. R. ). UK Research and Innovation London Medical Imaging & Artificial Intelligence Centre for Value Based Healthcare  (D. R. and G. K. ). Technical University Munich/ Imperial College London Joint Academy of Doctoral Studies (D.
code availability The current version of the PriMIA source code is publicly available at https://github. com/gkaissis/PriMIA and permanently archived at https://doi. org/10. 5281/zenodo. 4545599  18. PriMIA includes source code from PySyft (https://github. com/OpenMined/PySyft), PyGrid (https://github. com/ OpenMined/PyGrid) and Opacus (https://github. com/pytorch/opacus) re-used under open-source licence terms.
71 0. 83 0. 48 0. 47 0. 88 0. 19 0. 25 0. 88 0. 71 0. 65 Personalized 2 0. 87 0. 69 0. 58 0. 88 0. 85 0. 91 0. 90 0. 74 0. 67 0. 80 0. 51 0. 37 0. 85 0. 65 0. 29 0. 87 0. 41 0. 50 Personalized 3 0. 87 0. 68 0. 66 0. 86 0. 68 1. 00 0. 90 0. 75 0. 79 0. 80 0. 50 0. 48 0. 90 0. 79 0. 72 0. 84 0. 53 0. 00 Expert 1 - 0. 79 - - 0. 96 - - - - - 0. 70 - 0. 47 0. 88 Expert 2 - 0. 79 - - 0. 96 - - - - - 0. 68 - 0. 84 0. 41 37derated, model trained with federated learning; DP+/-, model trained with (+) or without (-) DP gradient descent; SecAgg+/-, model trained with (+) or without (-) SecAgg; Centrally trained, model trained on the entire dataset on a single machine. Personalized 1-3, models trained only on the data owner's local data set. Expert 1/2, human experts. Sensitivity/specificity metrics refer to normal/ bacterial/viral, respectively.
NATURE MAchINE INTELLIGENcE | VOL 3 | JUNE 2021 | 473-484 | www. nature. com/natmachintellFunctionalimprovements to FL. To increase framework usability and flexibility as well as FL model performance, our framework includes the following functional improvements. (1)Besidesincorporatingadaptiveclientoptimization in the form of the Adam optimizer recently shown to yield improved convergence results35, we include a wide range of advanced image augmentation techniques including MixUp, which has been shown to encompass privacy-enhancing attributes36. (2)Weimplementtechniquestoaddress imbalances in data volume between nodes (local early stopping), as well as between dataset classes (class-weighted gradient descent and federated averaging37). (3) We include facilities to carry out centrally coordinated hyperparameter optimization.
e. , zooplankton such as copepods). We used a principal coordinates analysis (PCoA), based on Bray-Curtis dissimilarity, to illustrate their distribution through sampled sites (Figure  S1 ). To assess the sources of variation (i. e. , taxonomy and diet) in the Bray-Curtis matrix, we used a PERMANOVA analysis based on 1000 permutations  [47]  with the function adonis, in the vegan package  [48].
Sequence Processing All analyses were carried out with R software 3. 6. 2 (https://www. r-project. org/, accessed on 9 August 2021)  [50]  and are available on GitHub: https://github. com/mccheutin/Seychelles. git, accessed on 9 August 2021. Sequence reads were processed using the DADA2 pipeline (v. 1. 12. 1) in R  [51] , following the pipeline's tutorial (https://benjjneb. github. io/dada2/tutorial. html, accessed on 9 August 2021). Briefly, sequences were trimmed and filtered based on read quality profiles (maxN = 0; maxEE = (2, 2); truncQ = 2; and truncLen = (240, 240)), error correct, dereplicated and amplicon sequence variants (ASVs) were inferred  [52]. Forward and reverse ASVs were merged and pooled in a count table where chimera were identified and removed. Taxonomy assignment was performed using the SILVA reference database (release 132)  [53].
The ASVs count table, their taxonomy, and their sequences were organized in a phyloseq object using the phyloseq package (v. 1. 28. 0)  [54] , on R. ASVs assigned to the kingdom Eukarya, Archaea, and to chloroplast, were removed before computing any further analysis. Bacterial genera known as potential kit contaminants were also removed from our datasets using the list described in  Salter et al. (2014)   [55]. Overall, 40 genera corresponding to 12% of the total reads were removed (Supplementary File 1). Our final dataset consisted of 1,042,080 sequences belonging to 5129 ASVs.
Inference of ASVs Habitat Preference We used a BLASTn approach on the nr/nt database and the habitat-associated metadata to the closest ASV match to infer the habitat preference of the 254 ASVs constituting the enteric core bacteriome of reef fishes  [22, 28]. Only blast results with an identity >95% and a sequence coverage >95% were kept. Information concerning the isolation source contained in the GenBank fields "isolation source", "host", and "title" of each closest blast were extracted using a dedicated python script and parsed into "Animal", "Environment" (i. e. , free living bacteria associated with sediment, soil or water), and "Other" habitat categories. For ASVs associated with animals, we further categorized the isolation sources into specific hosts (i. e. , fish, marine invertebrates, terrestrial vertebrates, and unknown animals) and organ (i.
e. , gut, tissue, and other organs) categories (Supplementary File 2). In order to associate these habitat preferences to the phylogenetic affiliation of each ASV, core bacteriome ASVs were aligned against the silva. nr_v132 reference database using mothur v. 1. 35. 1 ([63] ; https://mothur. org/, accessed on 9 August 2021) before being imported into the ARB software (  [64] ; http://www. arb-home. de/, accessed on 9 August 2021) and loaded with the SILVA (v. 138) reference database  [53]. A base frequency filter was applied to exclude highly variable positions before adding sequences to the maximum parsimony backbone tree using the parsimony quick add marked tool implemented in ARB. The tree and the associated categories were drawn and visualized using the interactive Tree of Life (iTOL) web server (  [65] ; https://itol.
Computation of Alpha and Beta-diversity of Bacteriomes In order to correct for the uneven sequencing depth among samples, 1041 sequences were randomly sub-sampled within each sample using the "rarefy_even_depth" function from the phyloseq R-package v. 1. 28 [54]  (Figure  S3A ). Good's coverage estimator  [66]  was 99. 9 ± 0. 1 indicating that the coverage was still excellent after rarefaction. Taxonomic diversity of each microbial community (fish gut or macroalgae swab) was measured using the richness (number of ASV) and the Shannon's index H, computed on ASV relative abundance, and later exponentially transformed to express it as effective number of species (ENS)  [67]. Taxonomic dissimilarities between pairs of bacteriome samples were assessed using the Bray-Curtis dissimilarity computed on relative abundances of ASV.
Functional Diversity Predictions of Bacteriome Using the 16S rRNA gene information, predictions of metabolic functions for Bacteria were performed using Tax4Fun2 v. 1. 1. 5 [68]  with a clustering threshold set at 99%, following the tutorial of the algorithm (https://github. com/bwemheu/Tax4Fun2, accessed on 9 th August 2021). In order to account for all ASVs, the predicting functional profiles were then proceeded using the minimum blast identity to reference at 78%. Among the 7279 KOs predicted by Tax4Fun2, about 23% are involved in at least two different metabolic pathways (until 15 for some KOs) and 33% are unknown or hypothetical proteins. These KOs are thus not indicators of a particular function and are a source of an additional and false functional redundancy, hardly ever taken into account in the literature.
Statistical Tests First, the gut and macroalgal bacteriomes were compared in richness and in composition by measuring the alpha and beta-diversity. In the same way, to understand the influence of the reef condition on the bacteriomes, we compared the same measures between reef conditions for both macroalgae and fishes. Since the effect of reef condition could have been masked by the effect of diet or phylogeny, we removed this by analyzing the dataset at different community levels (i. e. , inside trophic guilds, family and species level). Only levels with at least a triplicate per reef condition were tested. Comparison of alpha diversity indices (richness and entropy) was achieved using a Kruskal-Wallis test (999 permutations) in the vegan R-package followed by a post-hoc Dunn test (999 perm, pvalue corrected by Bonferroni's method) in order to identify which group means differed.
Sampling Size and Composition of Fish Catch Between Reef Conditions The fish species distribution and sample size were highly variable among reefs. The sampling size of caught fish species was higher in recovering coral reefs (RCR), with 27 species sampled compared to 17 species in macroalgae shifted reefs (MSR). Fish community composition and fish diet behavior differed significantly between RCR and MSR (Figure  S1A, B ) with a higher abundance of the scrapers Scarinae in RCR while grazers Siganidae are more abundant in MSR (Table  S1 ), conforming with the underwater visual census (UVC) data  [42]. In contrast, carnivorous species, overall represented by the Lethrinidae and Lutjanidae families, were distributed in both reefs with 11 Lethrinidae in RCR and 15 in MSR and six Lutjanidae in RCR and eight in MSR.
e. , invertivores and piscivores) (Figure  S6B ). The reef condition significantly affected the composition of gut bacteriome of herbivorous fishes (R2 = 0. 07; p = 0. 001, Figure  4 , Table  1 ). However, part of this effect may be driven by the strong fish species turnover among herbivores between coral and macroalgae dominated reefs. Indeed, 16 out of 17 of Siganidae and Acanthuridae fishes were distributed in MSR, while 19 out of 22 Scarinae fishes (mainly represented by Scarus ghobban) were present in RCR (Figure  4A ). Fish phylogeny was a strong and significant determinant of bacteriome composition at the family (R2 = 0. 14; p = 0. 001), genus (R2 = 0. 24; p = 0. 001), and species (R2 = 0. 43; p = 0. 001) levels. In order to exclude this effect, we analyzed the herbivore dataset at the family and species levels (for Scarinae and Scarus ghobban, the only herbivores distributed in both reef conditions).
In this way, we corroborated the fact that gut bacteriome composition did differ as a result of reef condition (Figure  4B, C ). This effect was marginal at the species level probably due to the low number of samples (R2 = 0. 28; p = 0. 078, N = 7). Differences in the composition of herbivore core bacteriomes among reef conditions was driven by changes in the relative abundance of biomarkers of the Scarinae and Siganidae families (Figure  5 ). Bacteriome abundance in MSR was lower for six of the seven Scarinae biomarkers and one (i. e. , Anaeroplasma) was totally absent. Fusobacterium and Odoribacter biomarkers were only present in Siganidae in MSR (Figure  5 ). These biomarkers accounted for 0. 8% on average of the Scarinae bacteriomes and 0. 4% of the whole dataset. The decrease in Scarinae biomarkers was paralleled by a significant decrease in the abundance of 207 specific KOs (Kegg Orthologs) mainly involved in host lipid (i.
e. , fatty acids, butanoate, propanoate, and glycerophospholipid metabolisms) and glucose homeostasis (Figure  6 ). In contrast, Siganidae biomarkers, all efficient anaerobes fermenters of plant and algal polysaccharides  [30] , showed a significant increase in MSR and one new appeared (i. e. , Brachyspira), accounting for 5. 9% on average of the Siganidae bacteriomes and 2. 0% of the whole dataset. This increase came also with an increase in KOs notably involved in carbohydrates metabolism (starch, sucrose, fructose, and mannose), DNA replication, and nitrogen metabolism suggesting higher rates of fermentation and a stimulation of bacterial growth (Figure  6 ). Reef condition neither appeared as a significant driver of herbivore bacteriome variability, nor of bacteriome diversity (Figure  S7 ).
ASVs constituting Siganidae biomarkers were closely related to sequences previously retrieved from Siganus canaliculatus (Supplementary File 2). Indeed, bacteria from the genera Desulfovibrio (sulfate reducing bacteria), Rickenella, Brachyspira (anaerobic fermentative bacteria), and Akkermansia (mucin degrading bacteria) were found to be part of the core bacteriome of Siganidae [102,103], accounting for 5. 9% on average of their bacteriomes and 2. 0% of the whole dataset. These taxa may be of importance for host digestive function in MSR, in particular for the fermentation of sulfated algal polysaccharides. For example, members of the Rikenella genus are known to degrade celluloses into short chain fatty acids (SCFA) available for the host through microbial fermentation  [30, 104]. The prevalence of these fermentative bacteria is in line with the high fermentation rates observed within herbivorous fish hindguts  [105] , particularly in Siganidae  [106]  and further suggest a well-suited adaptation of Siganidae bacteriomes to the consumption of algae.
Acknowledgments: We thank the  Seychelles Fishing Authority research team  for assistance in the field and with sample preparation ( Andrew Souffre ,  Rodney Melanie ,  Achille Pascal ,  Marie-Corinne Balett ,  Stephanie Marie ,  Clara Belmont ) and  Dr Nathalie Bodin  for assistance with study design and logistics. We are grateful to three trap fishers who provided boat time and fisheries expertise, and assisted with sample collection. We are grateful to the genotoul sequencing platform in Toulouse ( Occitanie, France ), for providing the MiSeq Illumina sequencing.
Using a culturomics approach, a strain was isolated, identified and characterised following the taxonogenomics concept. Neobacillus massiliamazoniensis sp. nov. , strain LF1 T (=CSURP1359) was isolated from human stool. The 16S rRNA gene sequence analysis of strain LF1 T (accession number: LK021124) exhibits 98. 32% similarity levels with Neobacillus bataviensis strain IDA1115 (accession number: NR_036766. 1), the phylogenetically closest related species with standing in nomenclature. The draft genome size of strain LF1 T (accession number: CVRB00000000) is 4. 6 Mbp with a G+C content of 34. 1 mol%. Analysis of phylogenic tree, genomic analysis and phenotypic criteria described here sufficiently prove that this bacterium is different from previously known bacterial species with standing in nomenclature and represents a new Neobacillus species belonging to Firmicutes phylum.