dataesr/scientific-paragraphs-categorization huggingface dataset:
• Conversely, on the seventh fashion sequence, as the first increase of the external signal led to a decrease of the main sequence in the train set, HMM models using the influencers signal showed diffi-   culties to leverage the external signal. Consequently, even simple statistical models like ets or tbats outperformed the HMM-based models on this specific sequence. • As the 10 fashion sequences are short and some of them do not have a strong seasonal component, seasonal variations of HMMs did not reach the best global level of accuracy. Finally, over the 10 time series, HMMs including the external signal show the same high level of accuracy than benchmarck models like hermes-es and lstm-es while these two benchmarks have several thousand of parameters, have been train on the whole fashion dataset gathering 10000 time series and include the external signal.
A. 4 HMM-based model training We propose a complete overview of the training process used in Section 4. 2 for the HMM approaches. For each HMM-based model, given a fashion time series, the following estimation procedure is used. 1. Parameter θ 0 is randomly initialized. 2. A GEM is run for 10 iterations. 3. Using the resulting parameter, 10 predictions of the last year of the train set are computed and evaluated using a MSE. 4. The average MSE over the 10 forecasts is computed 5. Steps 1-4 are repeated 30 times and the best run is saved based on the average MSE computed in step 4. 6. Starting with the initial parameter of the best run, 500 iterations of the EM algorithm are run and the final parameter θ is saved. The complete code is developed in Python and Tensorflow and available at https://github.
When researchers are faced with building machine learning (ML) radiomic models, the first choice they have to make is what model to use. Naturally, the goal is to use the model with the best performance. But what is the best model? It is well known in ML that modern techniques such as gradient boosting and deep learning have better capacity than traditional models to solve complex problems in high dimensions. Despite this, most radiomics researchers still do not focus on these models in their research. As access to high-quality and large data sets increase, these high-capacity ML models may become even more relevant. In this article, we use a large dataset of 949 prostate cancer patients to compare the performance of a few of the most promising ML models for tabular data: gradient-boosted decision trees (GBDTs), multilayer perceptions, convolutional neural networks, and transformers.
Introduction Quantitative image analysis as a tool to improve decision-making and management in healthcare has been growing steadily. A specific analytic technique called Radiomics, where predefined mathematical features are calculated from regions of interest (ROIs) in medical images, has recently gained particular attention in radiation oncology as logistic regression, random forests, basic support vector machines, or cox regression. These generally work well for small datasets (e. g. under 100 samples) but their performance likely deteriorates in high dimensions and non-linear scenarios  [8] [9] [10] [11] [12]. Prediction models for tabular data (like the ones constructed from radiomic features) are a staple in machine learning (ML) research since considerable amounts of real-world data are collected in the form of spreadsheets.
However, when evaluated on a wide range of different tasks, many of these models have shown little to no improvement over simpler baselines like 1D convolutional neural networks (CNNs) or multilayer perceptrons (MLPs)  [11, 12, [20] [21] [22]. Therefore, standard MLPs and CNNs may be preferred when evaluating the utility of DL on a new task such as radiomics, particularly when comparing many models is impractical. The question remains as to whether these DL models have the potential to compete with GBDTs in radiomics, where the datasets are characterized by few samples, many variables, and a mix of categorical, ordinal, and continuous data. In clinical prostate oncology, it is recommended practice to characterize the tumor via both MR-image-related parameters (such as prostate volume, number of dominant lesions, PI-RADS score, EPE value, and ADC value) and biopsy-related parameters (such as initial PSA, ISUP grade group, tumor stage, lymph node status, and risk class) in order to select an appropriate treatment and follow-up schedule.
To do this, we compare the performance of different prediction models and learning techniques on a large dataset of 949 prostate cancer patients with nine different pathology endpoints of clinical value in prostate cancer care. Our chosen methods are a mix of common high-performance ML models: gradient-boosted decision trees (GBDTs), multilayer perceptron (MLP), one-dimensional convolutional neural network (1DCNN), and a transformer model with a feature tokenizer (FT-Transformer)  [20]  specialized for tabular data. To combat the problem of overfitting and overoptimistic performance estimates, we employ rigorous training, validation, and test procedures in all experiments. We also evaluate the benefit of multitask learning, which is a training technique in which the model learns to predict every available endpoint simultaneously, leading to a potential improvement in both performance and speed.
Dataset Patient data were retrospectively collected from 949 prostate cancer patients who had undergone multiparametric prostate MRI and prostatectomy in the European Institute of Oncology (IEO) from 2015 to 2018. For each patient, we used the T2-weighted MRI sequences and all potentially relevant clinical variables. The MRI images were acquired using a 1. 5 T MR scanner with slice thickness 3. 0-3. 6 mm, slice gap 0. 3 mm, pixel spacing 0. 59 × 0. 59 mm, echo time 114-118 ms, and repetition time 3780 ms (median). Eight out of the 949 images were acquired with non-habitual protocol sequences or a second scanner from another manufacturer. The clinical variables included age, initial PSA, and comorbidity as well as MRI-related data such as prostate volume, number of dominant lesions, PI-RADSv2 score, extraprostatic extension score, and ADC (apparent diffusion coefficient).
Radiomic feature extraction and pruning Radiomic features were extracted from the whole prostate using PyRadiomics  [34]  3. 0 in Python 3. 7. All available features were extracted from all available filter classes (Laplacian of Gaussian, wavelet, square, square root, logarithm, exponential, gradient, local binary pattern 2D, and local binary pattern 3D). For the Laplacian of Gaussian filter, we calculate features for three different values of sigma corresponding to 1, 2, and 5 times the in-plane spacing (0. 59375 mm). No resampling was performed since all the images had nearly identical resolution and spacing. In total, 1967 features were extracted. Features were removed if their variance was lower than 10 -6 or if they had an absolute Spearman correlation above 0. 98 with any other feature.
Model training & feature selection 2. 5. 1. CatBoost The CatBoost model was trained with a nested 5-by-5 stratified cross-validation (CV) and a standard binary cross-entropy (log-loss) loss function. The inner 5-fold was used to search for parameters and the outer 5-fold was used to estimate the performance. This eliminates the selection bias from selecting the lowest error model. To search for parameters, we used the Tree-structured Parzen Estimator (TPE) in the Optuna (v2. 10) python package  [35]  and ran it for 600 trials with default parameters. Among the trials, we selected the model with the highest Matthews correlation coefficient (MCC) on the validation sets and proceeded to retrain it on the full training+validation set. The performance of the retrained model was then evaluated on the previously held-out test sets.
Deep learning models The DL models were trained with a similar cross-validation procedure, but instead of a full 5-fold validation in the inner loop, we used a single train-test split (80%-20%) due to computational constraints. Moreover, the DL hyperparameter search was carried out with 64 trials instead of 600. All random seeds were fixed so every model was trained and evaluated on the same data (including the CatBoost model). The hyperparameter search spaces and complete training details for the different DL models are shown in Appendix B. 2. Like for the CatBoost model, we directly optimize the number of radiomic features to use within the parameter search (see Appendix C for details). We trained the DL models to directly optimize a differentiable version of the MCC, which can be achieved by defining continuous versions of the true positives/negatives (TP and TN), and false positives/negatives (FP and FN).
In other words, if we let 𝑦 be the real target label and ℎ(𝑥) be the network's prediction: 𝑇 𝑃 = ∑ 𝑖 𝐲 𝑖 ℎ(𝐱 𝑖 ) (1) 𝑇 𝑁 = ∑ 𝑖 (1 -𝐲 𝑖 )(1 -ℎ(𝐱 𝑖 )) (2) 𝐹 𝑃 = ∑ 𝑖 (1 -𝐲 𝑖 )ℎ(𝐱 𝑖 ) (3) 𝐹 𝑁 = ∑ 𝑖 𝐲 𝑖 (1 -ℎ(𝐱 𝑖 )). (4) In the above formulae, it is assumed that the last operation in ℎ is a logistic function such that the outputs are in the (0, 1) range. This loss function has been argued to exhibit many attractive properties compared to its alternatives, particularly for imbalanced and medical datasets  [36] [37] [38].
In the medical field, these instabilities are largely overlooked, even though the small datasets may exacerbate the problem. Due to limited computational resources, this study focused on finding high-performance models instead of repeated experiments, which is currently the standard practice. To ensure the reliability of this approach, we performed informal repeated tests (only on the first target variable due to the required computational investment) and concluded that this variability would not invalidate the results. It is worth noting that another way of improving the reliability of predictions is to create model ensembles, but this also requires an increased investment of computational resources. The optimization objective and different performance measures: We chose to optimize the MCC because it is known to be a more reliable and informative measure of performance than other metrics like accuracy, F1-score, and AUC  [36] [37] [38] 42].
The differences in b-values may have had a similar effect on the calculated ADC values. That said, our images were acquired in the course of routine clinical practice, and should therefore reflect realworld context. Finally, even though the dataset of 949 patients is the largest one to date (the previous one being 489 patients, according to a recent review covering 57 different prostate cancer radiomics studies  [6] ), it is still not big enough to make conclusive statements about the validity of these models in clinical practice. It is conceivable that the small dataset contributed to the somewhat subpar performance of the DL models given that DL models are known to be data-hungry. A potential solution for this that was not studied in this article is data augmentation, which is standard practice for most non-tabular DL models.
Acknowledgments IEO, the  European Institute of Oncology , is partially supported by the  Italian Ministry of Health  (with '' Ricerca Corrente' ' and '' 5x1000 '' funds) and by  Institutional  grants from  Accuray Inc. Lars J. Isaksson  is a Ph. D. student within the  European School of Molecular Medicine (SEMM), Milan ,  Italy. Mattia Zaffaroni  and  Maria Giulia Vincini  supported by a research fellowship from the  Associazione Italiana per la Ricerca sul Cancro (AIRC)  entitled '' Radioablation ±hormonotherapy for prostate cancer oligo-recurrences (RADIOSA trial): potential of imaging and biology '' registered at  ClinicalTrials. gov  NCT03940235. Prof. Jereczek-Fossa  reports grants from  Accuray ,  AIRC  and from  Fondazione IEO-CCM  ( Istituto Europeo di Oncologia- Centro Cardiologico Monzino)  personal fees for lecturers, presentations or participation on data safety monitoring or advisory boards from  ESTRO ,  ESO ,  ASCO ,  ROCHE ,  BAYER ,  JANSSEN ,  CARL ZEISS ,  ACCURAY ,  ASTELLAS ,  ELEKTA ,  IBA  and ACCMED all outside the submitted work.
Quantitative PCR Lungs were mechanically dissociated using Lysing Matrix D tubes (FastPrep ® , MP biomedical) and RNA were isolated using the miRNEasy ® micro kit, Quiagen) following manufacturer instructions. RNAs were quantified by Nanodrop similar amounts of RNA were used to perform RT-PCR using the QuantiTect ® Reverse transcription kit (Quiagen) and the SyberGreen ® Master Kit (Roche). Amplicons were quantified using a LightCycler 480 II (Roche). Primers were designed according to PrimerBank (Harvard University). mRNA cytokine levels were normalized to GAPDH using LightCycler software (Roche).
Statistical analysis Student's t-test and Mann-Whitney-Wilcoxon test were performed to assess statistical significance of Gaussian and non-Gaussian distributed data respectively. hours later and the percentage of CMRA + cells was measured after gating on neutrophils (Siglec-F -CD11c -CD103 -CD11b + Ly6G high ), inflammatorymonocytes (Siglec-F -CD11c -Ly6G -CD11b high MHCII -CD64 -Ly6C + ), patrolling monocytes(Siglec-F -CD11c -Ly6G -CD11b high MHCII -CD64 -Ly6C -), interstitial macrophages(Siglec-F -CD11c -CD11b high MHCII + CD64 + CD24 -) and CD11b + DCs (Siglec-F -CD11c + CD11b high MHCII + CD64 -CD24 + ). Percentage of CMRA + in individual mice normalized to the mean valuesdetermined in SE mice. Mean ± s. e. m. two experiments. S1. RNA-seq transcriptomic profiling of CMRA + andCMRA -Ly6C + cells.
Funding This work was funded by grants from the  Agence Nationale de la Recherche  ( AsthmaTreat, Programme Blanc 2012 ), the  Institut National de la Santé et de la Recherche Médicale  ( Programme Environnement Santé Travail 2013) , the  Fondation de la Recherche Médicale (FRM) , and the  Canceropole Provence Alpes Côte d' Azur (PACA). M. G. was supported by a postdoctoral fellowship from the  Fondation ARC pour la Recherche contre le Cancer. This work was also supported by the LABEX  SIGNALIFE  (# ANR-11-LABX-0028-01 ).
5 EGTA, 2 ATP, 0. 4 GTP, pH 7. 3 (280-290 mOsm). Pipette and neuronal capacitive currents were canceled and, after breakthrough, the series resistance was compensated and monitored. Recordings were digitized on-line and filtered at 10 kHz through a Digidata 1322A interface using Clampex 10. 3 software (Molecular Devices). The main characterization of I NaP was accomplished by slow ramp increase from À70 mV to À10 mV, slow enough (12 mV/s) to prevent transient sodium channel opening. All experiments were designed to gather data within a stable period (i. e. , at least 5 min after establishing whole-cell access). For the whole spinal cord preparation, motor outputs were recorded from lumbar ventral roots by means of glass suction electrodes connected to an AC-coupled amplifier. The ventral root recordings were amplified (31,000), high-pass filtered at 100 Hz, low-pass filtered at 5 kHz, and sampled at 10 kHz.
Assessment of motor behaviors Walking. The CatWalkXT (Noldus Information Technology, Netherlands) was used to measure walking performance. Each animal walked freely through a corridor on a glass walkway illuminated with beams of light from below. A successful walking trial was defined as having the animal walk at a steady speed (no stopping, rearing, or grooming), and three to five successful trials were collected per animal. Footprints were recorded using a camera positioned below the walkway while a second camera placed parallel to the corridor captured hindlimb movements of the right side. The paw print and kinematic parameters were then analyzed using the CatWalk and the deepLabCut softwares, respectively. Rotarod test. Mice were placed on a rotarod (Bioseb) accelerating from 4 to 40 rpm over a span of 5 min.
Slices were then washed 3 3 5 min in PBS and incubated for 2 h with Alexa Fluor 488-anti-mouse or 546-anti-rabbit or 546-anti-goat IgG secondary antibodies (1:800, 1:400, 1:400, #A-11017, #A-11071, A-11056, Lifetechnologies). After 3 washes of 5 min in PBS, they were mounted with a gelatinous aqueous medium. Sections were scanned using a laser scanning confocal microscope (Zeiss LSM700) in stacks of 0. 3-1 mm-thick optical sections at 3 63 and/ or 320 magnification respectively and processed with the Zen 12. 0 software (Zeiss). Each optical section resulted from two scanning averages. Each figure corresponds to a projection image from a stack of optical sections. We used identical settings, finely tuned to avoid saturation, for the whole series.
Protein concentrations were determined using a detergent-compatible protein assay (Bio-Rad). Equal protein amounts (30 mg) from samples were size fractionated by 4-15% Mini-PROTEAN TGX stain-free gels (Bio-Rad), transferred to a nitrocellulose membrane and probed with the monoclonal anti-pan Na v (1:500, #S8809, Sigma from mouse) at 4 C overnight in Tris-buffered saline containing 5% fat-free milk powder. The blot was then incubated for 1 h at 22 C with a polyclonal horseradish peroxidase-conjugated anti-mouse IgG secondary antibody (1:40000; #31430, ThermoFisher). Signal intensities were measured with the image analysis software Quantity-One (BioRad). Drug list and recording solutions: Normal aCSF was used in most cases for in vitro electrophysiological recordings (see above). Ca 2+ -free solution was made by removing Ca 2+ chloride from the recording solution and replacing it with an equimolar concentration of magnesium chloride.
Data analysis Electrophysiological data were analyzed offline with Clampfit 10. 7 software (Molecular Devices). For whole-cell recordings, several basic criteria were set to ensure optimum quality of intracellular recordings. Only cells exhibiting a stable resting membrane potential, access resistance (no > 20% variation) and an action potential amplitude larger than 40 mV under normal aCSF were considered. Passive membrane properties of cells were measured by determining from the holding potential the largest voltage deflections induced by small current pulses that avoided activation of voltage-sensitive currents. We determined input resistance by the slope of linear fits to voltage responses evoked by small positive and negative current injections. Firing properties were measured from depolarizing current pulses of varying amplitudes.
Voltage dependence and kinetics of I NaP were analyzed from normalized voltageramp data by fitting them with Boltzmann functions. We defined the voltage-dependent activation threshold of the I NaP as the membrane potential at which the slope of leak-subtracted current first becomes negative. We measured the magnitude of I NaP as the peak of the leak-subtracted inward current during the ascending phase of the voltage command. For extracellular recordings, alternating activity between right/left L5 recordings was taken to be indicative of fictive locomotion. To characterize locomotor burst parameters, raw extracellular recordings from ventral roots were rectified, integrated and resampled at 50 Hz. Peak amplitude and duration of locomotor bursts were measured and the cycle period was calculated by measuring the time between the first two peaks of the auto-correlogram.
Behavioral analyses: For walking, bottom view recordings were processed by the CatWalk XT software (Noldus Information Technology, Netherlands) to measure a broad number of spatial and temporal gait parameters in several categories. These include (1) the paw print area that refers to the total area of the glass in contact with the paw, measured when the area of contact with the glass is maximal; (2) dynamic parameters related to individual paw prints, such as duration of the step cycle; (3) parameters related to the position of paw prints with respect to each other, for example the hindlimb base of support (the width between the paw pairs); (4) parameters related to time-based relationships between paw pairs. These parameters were calculated for each run and for each paw. To estimate five hindlimb joints positions (toe, ankle, knee, hip and iliac crest) on lateral view videos of freely walking mice, we trained a model using the DeepLabcut toolbox.
A total of 320 image frames (16 videos, 20 frames/video) were used to label and train the model. A pre-trained network with 50 layers (ResNet50) was selected and trained for 200 000 iterations. Parameters describing gait timing and joint kinematics were extracted for each gait cycle using a custom-written Python script. Gait cycles were defined as the time interval between two successive paw contacts of one limb. Individual steps were identified within the run by the acceleration of the paws to identify the ''stance'' and ''swing'' phases. We measured the total angular excursion for the hip, knee and ankle joints across the cycle for the hindlimbs. In the rotarod test the average latency of the subject to fall was recorded (in seconds). All behavioral tests were carried out with the experimenter blind to the experimental group.
Immunohistochemistry analysis: Quantification of staining intensities were performed with the Zen 12. 0 software (Zeiss). Measurements were performed on the axon initial segment (AIS) from interneurons located in the ventromedial part of upper lumbar segments (L 1 -L 2 ) near the central canal and from motoneurons identified as the biggest cells located in the ventral horn (L 4 -L 5 ). The AIS was identified as linear structures labeled by pan Na v -specific antibodies, and for which the beginning and the end of the structure could be clearly determined. Immunofluorescence intensity profiles were obtained from a line traced along the AIS. For each staining, the background intensity was first subtracted, and immunofluorescence values along each AIS were then normalized to the maximum intensity.
Statistics No statistical method was used to predetermine sample size. Group measurements were expressed as means ± s. e. m. When two groups (control vs. transgenic mice) were compared, we used the unpaired Mann-Whitney test or the Unpaired t test. Fisher test was used to compare the percentages of bistability. When two conditions (control vs. drugs) were compared, we used the Wilcoxon matched pairs test. We also used the one-way or two-way ANOVA tests for multiple comparisons. For all statistical analyses, the data met the assumptions of the test and the variance between the statistically compared groups was similar. The level of significance was set at p < 0. 05. Statistical analyses were performed using Graphpad Prism 7 software and are indicated in.
ACKNOWLEDGMENTS We are grateful to  Je  ´remy  Verneuil  for coding videos, Hele ne Bras  for technical advice on the administration of the viral vector,  Jean-Charles Viemari  for his assistance in fictive locomotor experiments, and  Genevieve Rougon  and  Nejada Dingu  for their valuable input to the manuscript. This research was supported by  Agence National de la Recherche Scientifique  ( SpasT-SCI -T ,  ANR-21-CE17-0060  to F. B. ) and the  French Institut pour la Recherche sur la Moelle  epiniere et l'Encephale (to F. B. ).
Discussion Our study in ambulatory settings analyzed pertussis and parapertussis cases from 2014 to 2022 when important changes in the vaccine strategy occurred during the COVID-19 pandemic. Several observations deserve to be highlighted. The number of cases reported with a 2 + 1 schedule was higher than that of cases with a 3 + 1 schedule. However, this difference could be due to the natural fluctuation of epidemiological pertussis cycles. The age of confirmed cases with a 3 + 1 or a 2 + 1 schedule was not significantly different (3. 8y ± 1. 4 vs 4. 2y ± 1. 5). A recent modelling study, using data collected between 2012 and 2018 from French national surveillance sources, analyzed the PCR test results of symptomatic outpatients aged 2-20 years with suspected pertussis. The model that best fit the 2012-18 epidemiological data supported a faster waning of immunity following vaccination with the new 2 + 1 vaccine schedule  [7].