dataesr/scientific-paragraphs-categorization huggingface dataset:
Reproducibility of a deep-learning fully convolutional neural network is evaluated by training several times the same network on identical conditions (database, hyperparameters, hardware) with non-deterministic Graphics Processings Unit (GPU) operations. The propagation of two-dimensional acoustic waves, typical of time-space evolving physical systems, is studied on both recursive and non-recursive tasks. Significant changes in models properties (weights, featured fields) are observed. When tested on various propagation benchmarks, these models systematically returned estimations with a high level of deviation, especially for the recurrent analysis which strongly amplifies variability due to the non-determinism. Trainings performed with double floating-point precision provide slightly better estimations and a significant reduction of the variability of both the network parameters and its testing error range.
Motivation and context Reproducibility, the principle that a method or experiment can be replicated in different conditions, is essential in science. Obtaining reproducible results is complex in deep learning (DL) due to the large databases, the intrinsic stochastic nature of the majority of the algorithms, the complex nature of the optimization search-spaces and the use of nondeterministic computations. As such, this topic is constantly being discussed  [25, 9, 27, 26]. Regarding the computational source of variability, numerical non-determinism comes from the rounding of numbers associated with a stochastic order of arithmetic operations  [34, 4]. Its is notably observed when the same code is evaluated in different hardware  [10]  or with different floating point precision  [29]. Currently, performance requirements make the use of Graphics Processing Units (GPUs) mandatory in DL, their major drawback being the difficulty to perform deterministic operations.
Different runs in the same GPU can also be non-deterministic due to use of algorithms that favor performance over repeatability  [8, 11]. The use of neural networks for estimating Partial Differential Equations (PDEs) is a growing trend and has been widely evaluated in the Fluid Mechanics field, many examples are available on the review by Brunton et al. [3] ; general purpose PDE solvers are also discussed on the literature  [2, 30]. In contrast with classification problems, the estimation of spatial-temporal PDEs must guarantee pixel-wise precision. Even if determinism is not mandatory, levels of accuracy and reproducibility must be mastered, and thus, further evaluated and discussed. This paper presents a detailed comparison between several DL models that estimate the evolution of acoustic waves, as a benchmark for time-space evolving systems, trained in the same context (identical database, hardware, random number seeds, hyperparameters, software and hardware) in a GPU with non-deterministic operations.
Similar analysis in the context of Reinforced Learning  [21]  found that the GPU non-determinism has an impact analogous to changing the starting model's parameters. Tests are performed for evaluating and comparing the generalization capacity of the trained models. To the authors knowledge, no previous work has discussed the reproducibility of a neural network when estimating time-space evolving systems. Document is organized as follows. Methodology is presented in Section 2, with the description of the physical system, the network architecture, and the training and testing procedures. Models properties (losses convergence, kernels weights and featured fields) are discussed in Section 3. In Section 4, results for the recurrent analysis are presented for solution benchmarks and a database with random acoustic pulses.
Model is coded using the PyTorch 3  framework  [24]. Input is composed by 4 acoustic density fields, called frames, issued from the LBM simulations, ordered in time. Output is the following frame, representing the field to be estimated. A jump of 4 simulation timesteps is considered between each frame. A datapoint is a group of these 5 frames, as illustrated in Figure  2. The simulated fields are split with no overlap, that is, every datapoint has a group of unique frames. At each forward pass, data of all frames is divided by the standard deviation of the first frame before performing the calculations. Loss, presented next, is equally scaled. As in  [18] , the L2-norm of the scalar fields and their spatial gradients compose the loss L:   L = 1 M M k=1 (λ L2 L L2 + λ GDL L GDL ) where L L2 = (ρ k -ρ k ) 2 L GDL = ∂ ∂x ρ k -ρ k 2 + ∂ ∂y ρ k -ρ k 2 ( 2 ) where ρ is the estimated acoustic density, subscript k represents a grid point (or pixel), M = N × N is the number of grid points and λ are the loss factors, set as λ L2 = 0.
Training procedure and databases The density fields are initiated with 1 to 4 Gaussian pulses, as described by Equation  1 , centered inside [0. 1 , 0. 9 ], both directions. Both the number and position of the pulses are defined randomly following uniform distributions. The training and validation databases are build by 500 simulations (400 for training and 100 for validation), with a total of 4400 datapoints (3200 + 1200); combined databases size is 6. 5 GB. During the learning phase, batches of 32 datapoints (160 frames) are considered. Random rotation operations by multiples of π/2 are performed for the group of fields composing the datapoints before the training. That is, there is no virtual expansion of the database, only more diversity is added to it. Reproducibility of the network is evaluated by performing several trainings with the exact same database and hyper-parameters.
Seeds of random number generators are equally set at the beginning of each training. For current framework, deterministic algorithms are not available for all the operations present in the model  [33]. Even if determinism could be achieved by using another framework or model architecture, the use of non-deterministic algorithms is deliberate as it mimics training the same model on different devices or distinct versions of libraries and drivers, where determinism may not be guaranteed  [22, 23]. Since non-determinism is associated with truncation error, training is done considering two floatingpoint precision: single (32-bits, framework's default, referrenced as FP32) and double (64-bits, FP64). Model and database are exactly the same, framework default type is modified at the beginning of the runs.
Data is generated in double precision and converted to single precision for the single precision trainings. The number of runs (a run is defined as an independent training) is of 10 for single precision and 5 for double. Such number of experiments is in accordance with number of runs used in traditional hyperparameter optimization  [17]. A total of 1500 epochs are calculated on each run. In order to have a representative range of the observed behaviors, models are saved at every 125 epochs so a total of 12 checkpoints are available at the time of the post-processing. Training is performed on a NVIDIA V100 GPU. The average time for the completion of an epoch (training + validation) is of about 30 seconds in single precision and 100 seconds in double precision, so respectively about 12 and 40 hours are necessary per run.
Testing procedure and databases Two distinct testing datasets are used for the evaluation of the quality of the trained models: Benchmarks database: Three simulations with benchmarks that are unknown to the network, aiming at evaluating the capacity of generalization of the models. The benchmarks are illustrated in Figure  3  and described next (pulse half-width of h w = 12∆x is maintained for all cases): • single Gaussian pulse at the center of the domain, amplitude of ε = 0. 001 (same properties of the Gaussian pulses used for generating the trainining database); • two opposite Gaussian pulses (ε = ±0. 001), placed at the y symmetry axis with ±20∆x of offset from the center of the domain, since only positive pulses have been seen during training; and • a plane wave of Gaussian contour in x (ε = 0.
Featured fields in non-recurrent mode In order to quantify the difference between the models, the featured fields produced for a sample input are also compared. Due to the size of the network, the analysis is limited to the outputs of each scale (quarter, half and full), the latter being the output of the model, used as new input when evaluating the network in recurrent mode (Section 4). The first 4 frames (simulation timestesps 0, 4, 8 and 12) of the Gaussian pulse benchmark are selected as input. Quantification of the deviation among different models is made using the criterion defined on Equation  4 , here applied to each value that composes the fields. Analysis is split between the runs in single and double precision and the results are shown in Figure  7. Even if restricted by the small size of the dataset, this analysis leads to two conclusions.
Benchmarks Evolution of the total loss in the recurrent estimation of the benchmarks for the models trained with single and double precision are presented in Figure  8. The capacity of the neural network to capture the physical phenomenon is evidenced by the fact that similar performance is obtained for features that were not present in the training database (negative and planar pulses). Loss (and consequentially the error) remains relatively small even after 100 recurrences, showing the overall quality of the estimation. The shape of the curves are similar when comparing the runs with same floating-point precision or the two groups of runs. Losses approach L = 0. 01 in the two graphs, that is, both single and double precision resulted in similar overall preciseness for the tested benchmarks.
Random pulses A statistical approach is used to compare the recurrent test results for the random database. For each simulation, the same recursive test used for the benchmarks database is performed. The evolution of the testing losses are represented for a subset of recurrences (0, 5, 10, 25, 50) in Figure  10. Each boxplot describes the distribution of the loss for the 100 simulations that compose the database, at the given number of recurrences. Note that runs 1 to 3 with double precision do not return identical distributions despite having the same loss (Figure  4 ). For all models, the median loss lies lower than the converging amplitude in training (L around 2 × 10 -6 ). At the recurrence 0 (evaluation of the model as it was trained), comparison of average losses returns a factor 2. 06 between the worst (run 1) and the best (run 2) models trained with single precision and 1.
In Figure  11 , average error and min-max bands for the 100 simulations in the database are plotted against validation losses for recurrence numbers 0, 10 and 50. Linear regression coefficients for the single and double precision models and the combined set of points are presented in Table  3. Evolution of the coefficients A and B with the recurrence number r stresses the increase of the error associated with the recurrent evaluation. After several recurrences, small deviations observed at the time of training, here quantified by the model's training loss, are significantly amplified. Despite the global trend, variability is elevated for any given abscissa and such conclusion can not be used to directly compare models with close validation loss as observed single (FP32) double (FP64)   in the analysis of the box-plots.
The regression coefficients indicate that models trained with double precision are, on average, more prone to amplify the initial errors, suggesting that the accumulation of error is larger for double precision. Such regression curves are issued from a smaller and much less spread dataset (points concentrated at L val < 10 -5 , many of them superposed), thus, less significant and more subjected to the influence of outliers. Besides, recurrent estimation (until r = 100) of the benchmarks performed with single precision for a model trained with double precision (run 1) returned a disparity in RMSE lower than 1% when compared to the double precision inference. In other words, there is no apparent increase in error accumulation due to the use of single or double precision in testing. This indicates that inference may be performed with lower precision with reduced impact in accuracy while gaining in performance (factor 1/3 in calculation time for current application).
This exercise aims at monitoring the genetic integrity of its banana accessions and combines evidence f rom morphological and molecular characterization to determine genetic integrity. To do so, plantlets that were maintained in vitro f or more than ten years are sent to the Musa Genotyping Centre (MGC) and to a partner field collection, USDA-ARS. In the lab, leaves are analyzed by f low cytometry and SSR markers, while in the f ield, plants are grown, characterized and photo documented. Once all results are obtained after a year or two, a panel of taxonomists check the morphological and molecular data and compare them to known ref erence inf ormation f or each accession. Given that plants that have undergone somaclonal variations are expected to change in morphology, f rom changes in color to obvious growth issues, the panel of experts then assesses whether the accession is True-To-Type (TTT) or Of f-Type (OT)  [3].
Current advances in Next Generation Sequencing (NGS) technologies and associated Genotyping-by-Sequencing methods allow the generation of high-throughput genetic markers f or a large number of samples in a fast and cost-effective way. These datasets can therefore be used to study many events, including large chromosomal variations as recently reported in wheat and barley  [8]. Such chromosome changes (e. g. aneuploidy) have also recently been reported in bananas  [9] [10] [11] [12]. Although not all these variations may be the cause of off-type phenotypes, likely due to polyploidy that can mitigate them, it remains a change in DNA integrity of the plant that should be f lagged, as well as a good reason for accession regeneration from backup or reintroduction with new material. These large chromosomal variations will be the focus of this chapter.
Plant Material Passport data of accessions held at the ITC are published on the Musa Germplasm Information System (MGIS; www. crop-diversity. org). Available material f rom the ITC collection can be ordered at no cost f or research, education and breeding purposes using a straightforward workflow (Figure  1 ). The MGIS website offers several options to search for germplasm that presents specific criteria such as taxonomy, country of origin, and it is also possible to identify accessions that have been evaluated f or pest and diseases resistance, biotic or abiotic stresses and genome composition. To do so: 1. Go to https://www. crop-diversity. org/mgis/accession-search 2. First ref ine your search by selecting "Yes" with the exchange availability filter 3. Use the other f ilters to refine your search based on other criteria.
Data generation To detect changes in chromosome ploidy, SNP ratio changes need to be monitored along the chromosomes. It is therefore necessary to choose a technology that will generate high-throughput SNPbased markers f or genome-wide marker discovery. Technologies based on restriction enzyme-mediated genome complexity reduction such as Genotyping by Sequencing (GBS)  [13] , Restriction Site Associated DNA Sequencing (RADseq)  [14] , Diversity Arrays Technology Sequencing (DArTseq™)  [15]  have advantages and disadvantages (Table  1 ) and are all appropriate to detect aneuploidy events. The choice of the technology can be inf luenced by existing datasets, in-house f acilities or affordable solutions offered by service providers. Theref ore, we don't provide a protocol f or data generation but list the main points related to the main methods previously tested for such analyses.
• Filtering • Split VCF by chromosome • Generate f igure and assignation to each genome The preprocessing pipeline presented in Figure  2  shows the dif ferent steps that are compatible with f our different sequencing technologies as listed in the previous section. The pipeline has to be done sample per sample according to the best practice of the software GATK  [19]  used for the SNP calling part. -b AGATCGGAAGAGC (universal sequence for Illumina). Sequence of an adapter that was ligated to the 5' or 3' end. The adapter itself is trimmed and anything that follows too if located at 3' end. the read is not modified. This reduces the no. of bases trimmed purely due to short random adapter matches. -m 30: Discard trimmed reads that are shorter than 30. --quality-base=64: Assume that quality values are encoded as ascii (quality + QUALITY_BASE).
In the absence of annotations, this option is strongly recommended. The tool generates a folder for each accession, (names filled in column 3 "genome_name") filled in the conf iguration file, which contained the SAM f iles (converted in BAM f ile) of aligned reads and a. f inal. out f ile of mapping statistics f or each library. In addition, a (--pref ix) f older containing a mapping statistics f ile (--prefix + mapping. tab) for all accession is generated. Website: https://broadinstitute. github. io/picard/ The tool generate a bam (*_rmdup. bam) f iles f or each accession with duplicated reads removed. In addition, a f ile named (--prefix + rmdup*stat. tab) f ile containing duplicate statistics f or each accession was generated in the (--prefix) folder. • Index BAM with Samtools Description: This step reorders the bam file according to the genome index position.
Step 2: Molecular karyotyping and ploidy change detection Based on known sequence variability, SNP variants can be assigned to the ancestral genomes in order to plot the genome allele coverage ratio and to calculate the normalized site coverage along chromosomes as described in  [10]. This method can detect chromosome changes such as homoeologous exchanges  [9]  but also powerf ul enough to detect ploidy variation along the chromosomes as illustrated in Figure  3. The method has been developed within the VCFHunter software  [10]  and can be used with the following procedure (Figure  4 ). --MinCov, --MaxCov (int): min and max coverage for each genotype. If not, converted into missing data. --MinAl (int): min coverage for each allele. If not, converted into missing data. --MinFreq (f loat): minor allele frequency --nMiss (int): max number of accession with missing data--RmAlAlt : keeping diallelic sites.
•• Website: https://github. com/alexdobin/STAR Variant discovery Add read group and (accession name from the fq. gz filename) sort BAM with Picard Tools Description: This step replaces the reads Groups which describe the reads mapped on the reference, the sequencing technology, samples names, and library number are added. ID = Read Group identifier (e. g. FLOWCELL1. LANE1) PU = Platf orm Unit (e. g. FLOWCELL1. LANE1. UNIT1) SM = Sample (e. g. DAD) PL = Platform technology used to produce the read (e. g. ILLUMINA) LB = DNA library identifier (e. g. LIB-DAD-1) Website: https://broadinstitute. github. io/picard/ The tool generates a bam (*_rmdup. bam) files for each accession with the RG (Read Group) modified. Mark duplicate reads and index BAM with MarkDuplicates from PicardTools Description: PCR duplicate removal, where PCR duplicates arise from multiple PCR products f rom the same template molecule binding on the f low cell.
The accurate and consistent border segmentation plays an important role in the tumor volume estimation and its treatment in the field of Medical Image Segmentation. Globally, Lung cancer is one of the leading causes of death and the early detection of lung nodules is essential for the early cancer diagnosis and survival rate of patients. The goal of this study was to demonstrate the feasibility of Deephealth toolkit including PyECVL and PyEDDL libraries to precisely segment lung nodules. Experiments for lung nodules segmentation has been carried out on UniToChest using PyECVL and PyEDDL, for data pre-processing as well as neural network training. The results depict accurate segmentation of lung nodules across a wide diameter range and better accuracy over a traditional detection approach. The datasets and the code used in this paper are publicly available as a baseline reference.
Currently, there are various segmentation algorithms that are automated or semi-automated working on segmentation, detection and classification of lung nodules, but it's difficult to analyse and inter-compare the robustness of them all. A study was conducted using NELSON's data and three different software systems for the malignancy risk assessment task. These systems calculated different tumor doubling time and the results conclude that due to this variation the classification of lung nodules is affected  [26]. New lung segmentation algorithms based on learning methods are introduced periodically. Hence there is a dire need of a platform where the user can access different deep learning and computer vision algorithms, analyse them and use them off-the-shelf. This is where our contribution comes in: here we introduce Deephealth toolkit, a complete deep learning and computer vision solution that can be easily used by all developers, includes most commonly available deep learning and computer vision algorithms and provides easy integration of data between them.
DeepHealth toolkit (DHt) Owing to the success of Artificial Intelligence (AI) and learning-based methods in the health sector, The European Union is funding AI based projects to cater to this challenging field. Deephealth  [5, 16]  is a health-centered project, part of this effort, that aims to do large-scale experimental research using AI. For this purpose multiple large-scale open access datasets are gathered, UniToChest  [17]  is one them  [2, 17, 9]. The DeepHealth toolkit (DHt) is the framework of Deephealth that is developed to provide one platform for easy deployment of Deep Learning and Computer Vision applications  [3]. It uses High Performance Computing, Big Data and Cloud Computing for providing off the shelf services for all Image Processing related tasks. The Deephealth toolkit consists of European Computer Vision Library (ECVL), European Distributed Deep Learning Library (EDDL), and a front-end interface.
ECVL The European Computer Vision Library (ECVL) is a general purpose library that is Image centered. It supports basic functionalities like Image read/write, Image manipulation, integration, parallel image augmentation to advance features like dataset parsing and batch creation. The main objective of ECVL is to provide integration and data exchange between existing Computer Vision (CV), Image processing libraries, and EDDL. Different operating systems including Windows, Mac, and Linux are supported on ECVL. The ECVL contains generic algorithms employed with the Deep Learning library. The Image class in ECVL has a generic tensor model and provides an Hardware Abstraction Layer (HAL) allowing it to run on GPU and FGPU. It also has Memory Management flexibility support for different devices.
Moreover, there is also a Visualiser for 3D volumes like CT scan slices in ECVL. The user can observe different slices from various views using this visualizer. A basic Image Editor also comes with ECVL, including functionalities like brightness and contrast adjustment, rotation, flipping, etc. The programming language of ECVL is C++ but a Python version is also developed using a wrapper class, called the PyECVL, to support Python Ecosystem. The latest versions of ECVL and PyECVL along with documentation are publicly available on GitHub [6] [19].