dataesr/scientific-paragraphs-categorization huggingface dataset:
Imaging and analysis Images were acquired using an automated Yokogawa confocal fluorescence microscope. A 40X objective was used to acquire 16 fields/well using Z-stacks consisting of 3 Z-slices separated by 2 µm. Exposure times and laser intensities were adjusted for each of the 4 fluorescent channels separately to obtain an optimal dynamic range of the fluorescent intensities and prevent signal saturation. Images were stored as TIFF files. Image segmentation and phenotypic feature extraction were required for the creation of quantitative phenotypic profiles. Our in-house developed software PhenoLink was used to extract quantitative information from the multichannel images (Table  S1 ). Image segmentation was performed on illumination corrected raw images based on fluorescent channel intensity thresholds empirically determined per plate.
Data set composition, processing and statistics Data was generated using two biological replicates representing the separate thawing and culture of mDANs from two cryovials. Within each biological replicate, at least three technical replicates (wells) were generated. The used dataset contains 126 columns with continuous phenotypic feature data derived from the image segmentation workflow described above. Additional columns include categorical data that detail the experimental conditions used, such as the position on the plate or the chemical treatment applied. Each row in the dataset represents the mean values per well of a 384-well plate, derived from 16 images. To be suitable for ML, all data was scaled per phenotypic feature using the RobustScaler method in the Python package scikit-learn.
RobustScaler scales the data according to the interquartile range (IQR). The IQR is the range between the 1st quartile (25th quantile) and the 3rd quartile (75th quantile). Continuous data was graphically reported by dose-response graphs showing the technical replicate data points, the mean and the 95% confidence interval (CI). Since the data were normally distributed, analysis of variance (ANOVA) was used to determine statistically significant differences between the different concentration of each ED on mDAN phenotypical features. When statistical differences were found, a Tukey post-hoc test was employed and adjusted p-values were considered to check where difference occurred. Results were considered significant when p < 0. 05. We provide a Python-based Jupyter notebook to reproduce all data standardization and plotting steps (Table  S1 ).
ML classification In brief, a pipeline was created for preprocessing and classification using the LDA, LightGBM and XGBoost classifiers. The data was split into training and testing sets, with 10% of the data being used for testing. Grid search cross-validation with 5 folds was used to find the best hyperparameters for each classifier. The pipeline was then trained on the full training set using the best hyperparameters found. Features importance was calculated and plotted to show the most important features used by the models. Predictions were made on the test set, and the probabilities of each sample belonging to the "control" class were calculated. The accuracy of the classifier on the test set was evaluated, and a confusion matrix was computed and plotted as a heatmap. Tenfold cross-validation scores were also calculated and plotted across the whole dataset.
Generating a Training Dataset We first built a training dataset made of noisy low-SNR images (x_train) paired with their corresponding high-SNR images In srSMLM, the diffraction-limited image (PSF) of a single emitter is recorded in a spatial spaceallowing its localization to be estimated with high accuracy. Simultaneously, part of the emission light is diffracted and the emission spectrum is recorded in a spectral space on the same detector-giving access to spectroscopic information of the emitter. b) srUnet architecture. A U-net is a convolutional neural network architecture made of a contracting path (left part of the U) and a symmetric expanding path (right part of the U) with cross connections between some same-sized parts of the down-sampling and of the up-sampling paths (horizontal arrows).
The training set was composed of pairs of images acquired at different laser intensities to collect different levels of signal and/or noise. We also augmented the training set with images in which the spectral container box was randomly shifted along the spectra axis-mimicking a shift of the emission spectra of the emitter and we introduced about 15% of images without any localizations in the training dataset. Taken together, we built a training matrix of dimensions (265 000, 16, 64, 2) (Figure  2c ) that we split in a training dataset (90%) and a validation dataset (10%).
Training the Network The U-net was trained on ≈240 000 pairs of images of the training set. The training time was about 60 min (200 epoch of 15-20 s each) on a GPU (Nvidia GTX 1080 Ti) using keras data generator. [42] The ≈25 000 pairs of images of the validation dataset were used to evaluate the performance of the network. Restored images (y_pred) were generated from the x_test images and compared to the y_test images used as a ground truth (G. T. ) (Figure  2d, e ). Visually, predicted images were very similar to the targeted G. T. images, although slightly smoothed. Several examples of individual raw and srUnet enhanced images are shown in Figure  S3 , Supporting information. As compared to the raw images, the restored images predicted by the network ( Y or y_pred) exhibited an average improvement of The PSF of a single emitter in the spatial channel can be associated to its spectral decomposition in the spectral channel.
The total signal of a single emitter is split in between these two distinct channels. b) Cluster of "mergeable" PSF on consecutive frames corresponding to the signal of the same molecule. In single molecule microscopy, fluorescent molecules can be detected in their "on" state for more than one frame. Molecules appearing on several consecutive frames are usually merged as a single localization. c) Building srUnet training dataset. Signal appearing on consecutive frames were used to generate the srUnet training dataset. All but one images of a cluster were fused using a weighting based on their spatial frequencies to generate a reference image with high signal to noise ratio (ground truth image or y_train). The image left was used as a train low SNR image (x_train). The training set was composed of k ≈ 265, 000 pairs of images and split in a train and a test dataset.
d) Representative images and e) signal profiles of the test dataset and its srUnet prediction. The y_pred image predicted by the srUnet for the x_test image appears very similar to the y_test reference image used as a target. f) Peak signal-to-noise ratio (PSNR) and g) structural similarity index (SSIM). The restored images predicted by the network (pred) exhibited an average improvement of ≈26 dB, a value very similar to the improvement between the raw images and their corresponding ground truth (G. T. ). Similarly, the SSIM values between predicted images and their ground-truth target were very close to one, showing that srUnet was able to predict ground truth quality images from raw noisy data.
Exploring the Reconstruction Performance of the U-Net To evaluate further the reconstruction performance of the trained network (SSIM evaluated differences at the pixel level only), we applied srUnet to the previously unseen images of a validation set built independently from the training set. We first explore the ability of srUnet to reconstruct properly images from raw noisy images. We explored whether we could fit a PSF in the reconstructed image if a localization was initially truly present in the raw image. From the confusion matrix (Figure  3a. ) we calculated a sensitivity (or recall) of 99. 5%. The specificity (or true negative rate) was also very good (Sp= 0. 988%). Together, the accuracy reached 99. 4%, showing that discrepancies relative to the G. T. data were minimal. To explore the properties of the reconstructed PSF, we fitted the PSF of the G.
T. images and of the reconstructed images of the validation dataset using the same software (Peakfit) and same setup parameters. Compared to the G. T. PSF, we observed a small lag on the center position of the 2D-Gaussian fits of about 4 and 2 nm on the X and Y-axis, respectively (Figure  3b ). The standard deviation (SD) distributions of the circular 2D-Gaussian fits were on the opposite very similar (Figure  3c ). The presence of a few nanometer change in the estimation of the localization is hard to interpret because the true positions of the emitting molecules in the sample are unknown. Further explorations (see Figures  S5  and S6 , Supporting Information) showed that the shifts in localizations likely correspond to systematic bias which seems to be independent of the position of the emitters in the field of view and to be signal independent.
Image stacks ranging from 1000 to 4000 frames with exposure time 40 ms were recorded. Images were processed by PeakFit (part of GDSC SMLM2 plugin) and/or ThunderSTORM, two open source software packages for Fiji in order to retrieve the molecule localization coordinates. The procedure described in Bongiovanni et al. [8]  was used to calibrate the wavelengths using multicolor fluorescent beads. All subsequent image processing was performed using self-written scripts in Python and are available at https://github. com/hmanko/ srSMLM_DeepL. DNA Origami Nanorulers: Green (ATTO 542) nanorulers with markto-mark distances of 80 nm (Gatta-Paint) were purchased (GATTAquant GmbH, Germany) and used as-is. Culture and Sample Preparation of Bacteria: P. aeruginosa (PAO1 -DSM 22644) were grown overnight in 5 mL lysogeny broth (LB) (L3152, Sigma Aldrich) at 30 °C under 220 rpm orbital shaking.
Fifty µL of fixed PAO1 cell solution were allowed to settle on PLL treated coverslips for 10 min before being washed once with PBS buffer. The slide was transferred to the microscope stage using an Attofluor cell chamber (A-7816 Thermofisher) and optically coupled to the objective lens through index-matching immersion oil (n = 1. 518, Olympus, Japan). Diluted solutions of Nile Red (Invitrogen) or POPO-III-iodide (Invitrogen) dyes were added directly on the sample on the microscope stage. srUnet: The srUnet architecture was adapted from ref. [34] and built using Keras library package (2. 4. 0) running on Tensorflow backend (2. 5. 0). The model was composed of seven 6 × 6 convolution layers activated by Leaky Rectified Linear Unit (Leaky ReLU) functions followed by batch normalization (Figure  1b ).
The image left was used as a train low SNR image (x_train). The training set was composed of k ≈ 265, 000 pairs of images and split in a train and a test dataset. d) Representative images and e) signal profiles of the test dataset and its srUnet prediction. The y_pred image predicted by the srUnet for the x_test image appears very similar to the y_test reference image used as a target. f) Peak signal-to-noise ratio (PSNR) and g) structural similarity index (SSIM). The restored images predicted by the network (pred) exhibited an average improvement of ≈26 dB, a value very similar to the improvement between the raw images and their corresponding ground truth (G. T. ). Similarly, the SSIM values between predicted images and their ground-truth target were very close to one, showing that srUnet was able to predict ground truth quality images from raw noisy data.
Acknowledgements This research was supported by the  ITI Innovec  ( IdEx  ( ANR-10-IDEX-0002 ) and  SFRI  ( ANR-20-SFRI-0012 )) and  CNRS. H. M. was funded by a 3-years contract (Contrat doctoral) from  Ministére de la Recherche, France. Y. M. is grateful to the  Institut Universitaire de France (IUF)  for support and providing additional time to be dedicated to research.
All but one images of a cluster were fused using a weighting based on their spatial frequencies to generate a reference image with high signal to noise ratio (ground truth image or y_train). The image left was used as a train low SNR image (x_train). The training set was composed of k ≈ 265, 000 pairs of images and split in a train and a test dataset. d) Representative images and e) signal profiles of the test dataset and its srUnet prediction. The y_pred image predicted by the srUnet for the x_test image appears very similar to the y_test reference image used as a target. f) Peak signal-to-noise ratio (PSNR) and g) structural similarity index (SSIM). The restored images predicted by the network (pred) exhibited an average improvement of ≈26 dB, a value very similar to the improvement between the raw images and their corresponding ground truth (G.
SyntenyViewer is a public web-based tool relying on a relational database available at https://urgi. versailles. inrae. fr/synteny delivering comparative genomics data and associated reservoir of conserved genes between angiosperm species for both fundamental (evolutionary studies) and applied (translational research) applications. SyntenyViewer is made available for (i) providing comparative genomics data for seven major botanical families of flowering plants, (ii) delivering a robust catalog of 103 465 conserved genes between 44 species and inferred ancestral genomes, (iii) allowing us to investigate the evolutionary fate of ancestral genes and genomic regions in modern species through duplications, inversions, deletions, fusions, fissions and translocations, (iv) use as a tool to conduct translational research of key trait-related genes from model species to crops and (v) offering to host any comparative genomics data following simplified procedures and formats.
SyntenyViewer database interface SyntenyViewer is a tool relying on a relational database (DB), aiming at displaying and making publicly available the previously described comparative genomics data at https://urgi. versailles. inrae. fr/synteny. The Java web application uses the Google Web Toolkit (GWT) framework for graphical dynamic web content processing. On the back end, the web server uses   named  with its primary key term (referenced below). Tables colored in green store data in an append only fashion when a new synteny dataset is submitted, blue tables contain new data as well as data shared between different datasets, orange table stores new data as well as updated data from a previously inserted dataset (i. e. 'dataset_t' that handles several versions of a dataset: a new tuple is inserted for Version 2 of Dataset A, while the tuple with Version 1 is marked obsolete).
Genome synteny between angiosperm species and within major botanical families Genome synteny has been obtained following a four-step method consisting in the identification of conserved genes (Step 1), orthologous relationships (Step 2), SBs (Step 3) and CARs (Step 4), Figure  1   (33) ]. All datasets are licensed under the Open Licence Version 2. 0 (CC-BY-compatible) as described in the 'Terms' tab of each dataset from the French national scientific data repository Recherche Data Gouv (RDG) https:// recherche. data. gouv. fr/en. For example, see the Terms tab of 'PlantSyntenyViewer Solanaceae submission file' where the license is also prompted when a user attempts to download the associated file (Table  1 ). Genome version and information are available at the RDG portal (see references).
Data integration and query in SyntenyViewer Previous synteny data are integrated into the SyntenyViewer tool relying on a DB with a Java web application for graphical dynamic web content processing, a web server (Apache HTTP and Apache Tomcat) and a PostgreSQL 9. 6 instance to store the data (Figure  2 ). A spreadsheet-based data exchange format allows synteny data submission to SyntenyViewer, available at https://urgi. versailles. inrae. fr/Data/Synteny/Datasubmission. It consists of a four-sheet file (in addition to a README that explains how to properly complete the whole file) regarding (i) the person to contact and authors of the data, (ii) genomic features (genes, position on chromosomes, annotation and genome versions) mainly from the Phytozome database (  34 ), (iii) phylogenic relatedness between chromosomes of extant species and chromosomes of their inferred ancestors and (iv) homology groups that are used to store relationships between genes of several species, each gene being declared in the genome description sheet (see Point ii).
Excel format is provided as an example for users to be completed, but text format is also possible for simplicity in the data submission process. An Extract-Transform-Load (ETL) toolbox, using open-source Talend Open Studio, is dedicated to validate the dataset consistency and completeness as well as its database insertion. This ETL is able to manage data updates on previously integrated datasets, by only inserting the changes, masking the previous versions and ultimately validating consistency and unicity of the data provided by users. Part of this validation step to avoid conflicts is handled into the ETL tool directly before integration into the database. The database itself relies heavily on unicity constraints over identifiers, some composite keys and some concatenations of entity (gene, dataset and chromosome) name and version.
It contains several constraints used to ensure complete consistency over time between several integrations of updated versions. The version of the application sticks to the GnpIS information system  (35). Dataset's versions are displayed in the dataset form and detailed in the associated downloadable dataset from the RDG repository. Synteny data is then made publicly available when the format described on our website (https://urgi. versailles. inrae. fr/Data/Synteny/Data-submission), and the aims of Syn-tenyViewer are met. The usage of an all-in-one Excel file simplifies the data exchange between both parties. It also eases its manipulation by scientists for filling and submitting a unique file, which includes some static data extracted from the database (i. e. taxon scientific names), and hence guides the submitter with correct data at the beginning of the process and reduces the need for interactions between both parties.
SyntenyViewer functionalities There are two main entry points to visualize SyntenyViewer data (cf Supplementary Video). The first allows for selecting a dataset that provides gene conservation among a given botanic family or at a larger scale to whole monocot (grasses) or eudicot phylum, which then shows a query form described later. The other entry point allows for searching for a gene name (via its entire form or a prefix) across all datasets available. A search displays a popup with a short description of matching genes referenced in the database, and clicking on a selected gene loads the associated orthologous genes in the selected botanic family of interest, with an additional form for querying data in several manners as well as customizing display parameters. This query form offers users to enter the database through a gene ID, an extant or ancestral chromosome number and species of interest.
The customized display parameters offer users to display windows with a specific number of (extant or ancestral) genes, to produce a compact view when having numerous genes visible, to swap gene order on chromosome or to hide chromosomes. Orthologous genes are given the same color code and are linked in a top-down manner between chromosomes, facilitating the identification of orthologous groups between genomes and species. Left clicking on a gene updates the synteny display centered on the selected gene. Right clicking a gene provides the associated gene information with its ID, its coordinates as well as links for redirecting toward numerous international databases  (36, 37)  following the Findable, Accessible, Interoperable, Reusable principles  (38). Dedicated buttons make it possible to browse along the chromosomes.
There is also a specific mode, which makes searches wider when no hit can be found for a specific gene on syntenic chromosomes: in such case, flanking genes are serially searched for orthologs on syntenic chromosomes until a first match is found, highlighting all relationships observed between all displayed genes of a given genomic region. At any stage, the SyntenyViewer's Uniform Resource Locator is dynamically updated, and it can be bookmarked in the browser for sharing the visualization link, which makes users able to go back to previous work and pursue exploration. Finally, a download button, available after dataset selection, allows users to access its publicly available submission file along with relevant metadata (authors, description and DOI) through the RDG repository or through any link provided by the submitter.
Funding The ' Région Auvergne-Rhône-Alpes ' and ' Fonds Européen de Développement Régional ' (# 23000816  project ' SRESRI-2015 '), the  Institut Carnot Plant2Pro  (project ' Synteny Viewer '), the  Initiative-Science-Innovation-Territoires-Economie (ISITE)   CAP2025  (# 00002146 SRESRI 2015 ,  Pack Ambition Recherche Project  ' TransBlé '), the  Agence Nationale de la Recherche (ANR)  projects ' BREEDWHEAT'  ( ANR-10-BTBR-03-06 ), the Biomass For the  Future  project ( ANR-11-BTBR-0006 ), the  ANR  project , for Plant and Animal Genome Evolution (PAGE)  ( ANR-11-BSV6-0008 ) and the  Plant Bioinformatics Facility  (https://doi. org/10. 15454/1. 5572414581735654E12 ).
Among the difficulties encountered in building datasets to evaluate intrusion detection tools, a tricky part is the process of labelling the events into malicious and benign classes. The labelling correctness is paramount for the quality of the evaluation of intrusion detection systems but is often considered as the ground truth by practitioners and is rarely verified. Another difficulty lies in the correct capture of the network packets. If it is not the case, the characteristics of the network flows generated from the capture could be modified and lead to false results. In this paper, we present several flaws we identified in the labelling of the CICIDS2017 dataset and in the traffic capture, such as packet misorder, packet duplication and attack that were performed but not correctly labelled.
This evaluation relies extensively on the use of benchmark datasets of network traffic. These datasets consist of two parts: the network data (either raw network packets or a more high-level network flow description) and the labels, i. e. , the class (benign or attack) in which each packet or flow belongs. Due to privacy and confidentiality reasons, there are only a few public datasets of real traffic for evaluating NIDS  [12]. To circumvent those constraints, other datasets are generally obtained by generating network traffic in a testbed. One of these datasets is CICIDS2017  [14]. Though currently considered to be of good quality and widely used, it has nevertheless been criticized. Engelen et al. [4]  notably pointed out some flaws in CICFlowMeter, the tool used to create flow descriptions from raw traffic capture, as well as issues with labels of some network flows that should not be labelled as attacks (network flows without a payload).
We discovered several new problems in CICIDS2017: most notably, several port scan attacks were not properly labelled, and a non-negligible part of the traffic capture was duplicated, leading to feature extraction and labelling issues. In addition to providing corrected traffic captures and labels, we took advantage of this opportunity to investigate why some references of the literature  [11]  exhibited high recall and precision values even though the dataset has serious labelling issues. We thus present in this paper three contributions: we first release a fixed version of the CICIDS2017 dataset for both labels and network captures, we propose a patch for CICFlowMeter that avoids processing malformed input data, we evaluate the consequences of the different dataset corrections on the evaluation of several popular intrusion detection models.