dataesr/scientific-paragraphs-categorization huggingface dataset:
Model Development For the development of end-to-end transformer models, we used the Libri-trans  [38]  and VoxforgeIT  [39]  datasets in conjunction with the Espnet toolkit  [4]. Libri-trans is a subset of the LibriSpeech database, which was created as part of the Librivox project (librivox. org) [40]. It contains 236 h of annotated English utterances from audiobooks. VoxforgeIT, on the other hand, is an Italian dataset developed within the Voxforge project (voxforge. org) and provides 20 h of audiobooks. Both datasets are divided into three sets: a training set, a development set, and a test set. For the Libri-trans dataset, 231 h are used for model training, while 2 h and 3. 45 h are allocated for the development and evaluation sets, respectively. In the case of VoxforgeIT, 80% of the data are used for training, and the remaining data are evenly split between the development and test sets.
To augment the data, we applied speed perturbation  [41]  with ratios of 0. 9, 1. 0, and 1. 1. This technique effectively multiplies the quantity of data by three, providing additional variations for training. Espnet is an open source toolkit that integrates Kaldi  [42]  tools for data processing and parameter extraction, along with PyTorch (pytorch. org) modules for model estimation. For the input features, we computed 80 filter bank coefficients, which were then normalized based on the mean and variance. Filter bank (F-bank) parameters enable the extraction of relevant information from speech signals in the frequency domain. By capturing the spectral content of speech, they provide a representation that is more suitable for various speech processing tasks including speech recognition. As for the representation of transcripts, we used subword units.
Experiments and Results The evaluation of various pruning techniques, including global, local, and variable scale (i. e. , adaptive), was performed using the transformer models described in Section 3. 2. Unstructured pruning with the L1 norm were employed to prune connections. Individual weights were set to zero according to their magnitude. The adaptive pruning parameters for the Libri-trans and VoxforgeIT datasets were determined by utilizing the development data. Subsequently, the evaluation was conducted using the test data.
Baseline Systems For the purpose of conducting a comparative analysis, we conducted preliminary experiments on both global and local pruning techniques utilizing the test sets from the Libri-trans and VoxforgeIT datasets. The outcomes of these experiments are illustrated on Figures  9  and 10. An observation we made is that when the pruning rate is 30%, there is a slight increase in the error rate of approximately 4. 7% relative for Libri-trans and 2. 1% for VoxforgeIT. Additionally, it is worth noting that the attention layers contribute significantly to the model size, as indicated on Figure  4. As a result, we will at least fix the attention layer pruning rate at 30% and focus on varying the pruning rate of the feedforward layers fin the subsequent analysis.
Pruning Settings The adaptive pruning technique was applied in a single step to the feedforward and to the attention layers after the model was trained. The encoder and decoder pruning rates are represented by the arithmetic sequences U and V. To fix the initial pruning amounts U 0 and V 0 and the differences α and β, adaptive pruning was applied to the development data as follows: 1. fix α and β; 2. for each pair (U 0 , V 0 )-(a) use Formulas (  5 ) and (  6 ) to prune the feedforward layers; (b) evaluate the system using the pruned model and the development dataset; (c) note the error rate and the Sparsity. The algorithm was executed three times on each dataset, with different pruning rates applied to the attention layers. Specifically, for the Libri-trans dataset, the pruning rates used were 30%, 35%, and 40%.
For the VoxforgeIT dataset, the pruning rates employed were 35%, 40%, and 45%. α and β were chosen empirically. For example, for a pruning rate of 30% and α = 0. 01, the first encoder layer is pruned with the coefficient U 0 = 0. 3; the second is pruned with U 1 = U 0α = 0. 3 -0. 01 = 0. 29. Thus, for a 12th encoder block , the pruning rate of the last layer is U 12 = 0. 19 (i. e. , a pruning rate of 19%). The results of the adaptive pruning technique applied to development data are reported in Appendix A. They are illustrated by Tables A1-A6. Among these results, the best tradeoffs between WER and Sparsity are identified and highlighted in bold. To determine the optimal tradeoff, one must select a specific WER value from the table and then choose the lowest associated sparsity level. Based on the obtained tradeoffs, we deduced the encoder parameters U 0 and V 0 , which correspond to the optimal initial pruning amounts.
For both the Libri-trans and VoxforgeIT settings, we extracted the optimal values of the (WER, Sparsity) pairs from Tables A1-A6. Subsequently, we organize and present these optimal values comparatively in Tables  2  and 3. This arrangement allows us to effectively demonstrate the relative trade-offs achieved for each set of parameters. In both the Libri-trans and VoxforgeIT datasets, adaptive pruning demonstrates superior performance compared with local and global pruning methods. This outcome was expected due to the estimation of variable scale pruning parameters using the development data.
Pruning Evaluation The three types of pruning, namely local, global, and variable scale, were applied to the test data without any parameter modifications. The values of U 0 , V 0 , α, and β remained the same as those set on the development data. Figures 13 and 14 present the error rates for varying Sparsity levels, reaching up to 52. 5% for Libri-trans and 62. 5% for VoxforgeIT. It is noteworthy that, beyond these thresholds, the application of global pruning results in a significant increase in the error rate. Specifically, the error rate exceeds 24% for Libri-trans and 13. 9% for VoxforgeIT. In all cases, variable scale pruning demonstrates superior performance compared with global and local pruning techniques. The Word Error Rate (WER) of the non-pruned Libri-trans system is reported to be 6.
It is worth noting that VoxforgeIT's transformer model demonstrates a higher level of over-parameterization compared with that of Libri-trans, as evidenced by its larger Sparsity at the same increase in error rate. Overall, these results highlight the effectiveness of variable scale pruning in achieving a favorable trade-off between Sparsity and error rate for both datasets. Setting a maximum allowable relative increase of 10% in the error rate (i. e. , WER = 7. 3% for Libri-trans and CER = 10. 3% for VoxforgeIT), the most optimal pairs of (error rate, pruning rate) were found to be (7. 6%, 43%) for the Libri-trans system and (10. 3%, 59. 5%) for the VoxforgeIT system.
Leveraging this insight, our proposed approach, named variable scale pruning, applies a pruning rate that decreases progressively as the layer depth increases. To validate the effectiveness of our method, we conducted experimental evaluations on diverse databases. The results showcased the superiority of our approach over traditional local and global pruning methods. Remarkably, our technique achieved an impressive model compression rate of up to 59. 5%, all while maintaining a less than 10% relative reduction in accuracy. Looking ahead, there are exciting research directions to explore. One such avenue involves investigating the application of variable scale pruning on attention head layers, which play a vital role in the transformer's performance. Additionally, we intend to delve into fine-tuning techniques for the pruned models, aiming to further optimize their performance without sacrificing compression gains.
Data Availability Statement: This research utilizes the Libri-trans and VoxforgeIT datasets, both of which are openly available. The datasets can be accessed at the following URLs: https://zenodo. org/record/6482585. YsmV (accessed on 9 August 2023) and http://www. repository. voxforge1. org/ downloads/forLibri-trans(accessed on 9 August 2023) and VoxforgeIT respectively.
Moreover, it is less expensive to use the Allan variance in terms of required hardware resources  [APFB18]. Definition 3 (Allan variance of the counter output). The Allan variance of the counter output is defined as follows: AVAR(N p ) := 1 2 E (N p+1 -N p ) 2 , ( 4 ) where (N p ) p∈N are realizations of the counter value. In practice, we observed that the Allan variance and classical variance are very close to each other -more details in the data log files in [SDB + 23]. It goes to show that the PLL naturally filters out low frequency noises through its feedback loop. Due to this correspondence, we are able to keep the less expensive implementation of Allan variance while the model still uses classical variance.
To find all possible PLL-TRNG configurations under the selected design constraints, we use the Python code presented in  [CBBB20] , which we upgraded as explained in Section 3. 3. In our experiments, we used the two-PLL TRNG configuration with the input frequency f in = 125 MHz. While searching for possible configurations, we limited the reference clock frequency to f 0 < 250 MHz to guarantee timing constraints in counters and embedded tests. These requirements limited the design space for each device to hundreds of thousand possible configurations: more than 500 thousand configurations in Intel Cyclone V and Cyclone 10 FPGAs, about 100 thousand in Xilinx Spartan 6 and Spartan 7 FPGAs, and more than one million in Microchip SmartFusion2 FPGA. The exact values depending on input and output frequency constraints can be obtained using the Python code available at the GitHub repository [SDB + 23].
Another problem concerning placement and routing is the position of the first flip-flop of the sampler and the way the clock signal clk 1 arrives to its data input. Indeed, the PLLs are aimed at generating clock signals (and not data) and are thus preferably connected to clock trees. We placed the first flip-flops as close as possible to the PLL1 output and used a regional clock tree for the Cyclone V family, and local routing resources for Spartan 6 and SmartFusion2. We used the signal x i from Fig. 3 , which was output from the device via dedicated LVDS pins, to acquire 32 Mb of sampler output bits at high speed. The acquired files were used to reconstruct images of the period T 1 (see the right panels in Fig. 10   Implementation results for the three PLL-TRNG configurations with one and two PLL1 outputs implemented in three available FPGA families are presented in Table  3.
Effectiveness of the Online test and threshold selection As we explained in Section 4. 2, the result of the Online test depends on the required threshold value for the Allan variance of the counter values, derived from the required entropy rate (e. g. H 1 = 0. 9998 or H ∞ = 0. 98 according to  [PS22] ). We use the stochastic model from Section 3 to compute this threshold to obtain 1. 06 for the required Shannon entropy (H 1 ) and 1. 09 for the required min-entropy (H ∞ ). The developer will have the responsibility to claim either one or the other (or both) of these required entropy rates. To assess the effectiveness of the Online test, we collected data samples over a grid of environmental parameters regulated by temperature and power supply. We considered three values for the temperature, which correspond to the used commercial version of devices (low -0 • C, nominal -24 • C and high -85 • C [Int19,  Xil15, Mic18] ) and three values for the power supply (low, nominal, and high) depending on the device specification (1.
07, 1. 10, 1. 13 for Cyclone V  [Int19] , 1. 14, 1. 20, 1. 26 for Spartan 6  [Xil15]  and SmartFusion 2  [Mic18] ). For each combination of card, temperature and power supply, we collected 8 data files containing 32 MB of continuous samples. We then computed the NIST min-entropy estimate and the Allan variance value for each of them, for a total of 216 = 8 × 3 × 3 × 3 acquisitions and measurements. Figure  9a  presents the min-entropy estimates against the Allan variance for different temperature and power values. We framed the problem as a binary classification task of high (at least 0. 98) and low (below 0. 98) min-entropy rate, based on the Allan variance. The Online test is expected to pass under high entropy and raise an alarm under low entropy. Various decision thresholds give rise to a trade-off between the false-positive rate (FPR) and false-negative rate (FNR).
Dependencies in raw bits and counter values The stationarity of output bits (Section 6. 2) enables testing of possible long-term dependencies in the counter output. In each experiment, 32 MB of data from the relevant configuration were used to run Ljung-Box test on both counter values and output bits, implemented in the package statsmodels v0. 13. 5  [SP10]. The results are presented in detail in the right part of Table  4 ; they demonstrate that: • there are no short or long-term dependencies in the output bits, except on configuration SF_C where Table  3  shows the most correlation, • there may be dependencies in the counter values pointed out by the LjungBox test. Assumption 4 is therefore validated. The tests used default implementation parameters. Note that the confidence in the claim of possible dependencies in the counter values is weak, as the test may falsely reject data due to the long-term fluctuations in stationarity.
12  for similar results in both Spartan 6 devices). In addition, the stationarity of the TRNG output is not an issue, primarily thanks to the behavior of the PLLs. Last but not least, since PLLs occupy a separate device area and are powered from independent power supplies, the logic running in the programmable area has practically no impact on the generated numbers. The cost of the new PLL-TRNG version including the TDC and new embedded tests is very similar in the three tested FPGA families, especially concerning the use of PLLs, registers and DSP modules: besides the two PLLs, about 58 registers are needed to implement the core of the PLL-TRNG including two sampling registers per each PLL1 output, two times 9 registers for counters, and up to 34 registers for the FIFO. The number of required combinational blocks depends on the family: 41 ALUTs in Cyclone V, 14 LUT slices in Spartan 6 and 55 LUTs in SmartFusion2.
The two embedded tests require 131 registers and one DSP module in each family and an additional 74 ALUTs in Cyclone V, 69 LUT slices in Spartan 6, and 133 LUTs in SmartFusion2. We also tested the speed (latency) and efficiency of embedded tests. To test performance of the Total failure test, we modified the frequency of the reference clock signal clk 0 -the PLL0 was replaced by an external clock signal generator. The left panel in Fig. 14  depicts the oscilloscope screenshot when the frequency of clk 0 was modified in configuration CV_B from the nominal frequency of 237. 98 MHz to 200 MHz (to simulate locking with clk 1 ). The Total failure alarm was triggered in 103. 5 µs. In this case, the Online test alarm was triggered much later (not visible in the figure). The right panel in Fig. 14  depicts the case when the nominal frequency was changed to 231 MHz (not sufficient to trigger the Total failure alarm).
Acknowledgements This work was supported by the  ARSENE  project funded by the " France 2030 " government investment plan managed by the  French National Research Agency , under the reference " ANR-22-PECY-0004 ". It was also supported by the PSPC Regions AAP 1 -AURA 2019 call, funded by  "Pôle de compétitivité MINALOGIC", France. The fourth author was supported by the  French National Research and Technology Agency  under the  CIFRE program   N°2021/0413.
O B S E RVAT I O N S A N D DATA R E D U C T I O N χ Dra (HD 170153, HIP 89937, HR 6927) was originally observed as part of the BRITEpol project  (Neiner et al. 2017  ) which, as support for the BRITE (BRIght-star Target Explorer) satellite campaign  (Weiss et al. 2014 ) , observed every star brighter than V = 4 in circularly polarized light to look for stellar magnetic fields. The primary of χ Dra (F7V) was shown to host a magnetic field with a longitudinal magnetic field strength of B l = -6. 3 ± 1. 2 G  (Seach et al. 2020 ) , in agreement with the earlier observations of  Lee et al. ( 2018 ) , and thus was chosen for follow-up studies in order to map the magnetic field topology. The stellar parameters for χ Dra A used/found in this study are given in Table  1. 2 , 3 ,  and 4  ). We used the linear polarization measurements to determine the radial velocity (Section 3 ) and measure the Ca II H&K emission (Section 4 ).
Data reduction The NARVAL observations were reduced and calibrated in-house using the LIBRE-ESPRIT software reduction package. LIBRE-ESPRIT is based on ESPRIT ( Échelle spectra reduction: an interactive tool) which is detailed in  Donati et al. ( 1997 )  and produces normalized wavelength calibrated spectra. For all observations, a Stokes I spectrum was created by combining each of the four sub-exposures together for e very observ ation, while a Stokes V , Q , or U spectra was created, for the rele v ant observ ations, by separating sub-exposures with orthogonal polarization states. A null spectrum is also created by separating sub-exposures with the same polarization states and acts as a check on the validity of any magnetic signature detected in the Stokes V , Q , or U profile. More details on the data reduction methodology can be found in  Donati et al.
Based on observations obtained at the T élescope Bernard Lyot operated by the Universit é de Toulouse (Observatoire Midi-Pyr én ées) and the CNRS (Institut National des Sciences de l'Univers). DE has been funded by a University of Southern Queensland (UniSQ) International Stipend Research Scholarship and a UniSQ International Fees Research Scholarship. ELB is supported by an Australian Postgraduate Award Scholarship. SVJ acknowledges the support of the German Science Foundation (DFG) priority program SPP 1992 'Exploring the Diversity of Extrasolar Planets' (JE 701/5-1). This research w as undertak en using the UniSQ F a wkes HPC which is co-sponsored by the Queensland Cyber Infrastructure Foundation (QCIF), see www. usq. edu. au/hpc. The research has made use of NASA's Astrophysics Data System, and the SWMF tools developed at The University of Michigan Center for Space Environment Modelling (CSEM) and made available through the NASA Community Coordinated Modelling Center (CCMC).
13  This outcome set comprises physicianassessed clinical signs; patient-assessed symptoms; healthrelated QoL; and disease long-term control. In a clinical research setting, EASI has been documented to be the best outcome measure for assessing clinical signs,  14  POEM for measurement of symptoms  15  and DLQI for QoL. However, it is still unclear whether other PRO instruments, such as the PO-SCORAD, may be helpful in the clinical practice setting, notably whether PO-SCORAD provides a more accurate picture of disease severity between two visits than SCORAD. A key issue is data interpretability from these instruments, that is how a clinician would translate a score into clinically meaningful information. Overall, relevant interpretability research has been conducted for both the POEM and the EASI instruments.
The present study had three principal objectives: 1. To evaluate the correlation of PO-SCORAD with SCORAD, POEM and IGA in a population of children with mild-to-moderate AD. 2. To evaluate the interpretability of SCORAD. 3. To evaluate how PO-SCORAD could be helpful in clinical practice, by comparing AD disease severity between two visits, captured by two different methods (PO-SCORAD and SCORAD) and by assessing the extent to which PO-SCORAD data could predict flares.
M ATER I A L S A N D M ETHODS Data were retrospectively extracted from a European multicentre, randomized, parallel-group (two emollient groups and one non-emollient group), non-blind study in 335 children aged 2-6 years with mild-to-moderate AD, diagnosed according to UK Working Party criteria. 22  The study was conducted in accordance with Good Clinical Practice and was approved by relevant Ethics Committees. A parent provided written informed consent for each child to participate. Full details of study methodology have been published previously. 23, 24  This section provides only a brief overview of principal Materials and Methods. In brief, children were included during a flare and treated with a potent topical corticosteroid (0. 1% desonide; Locatop®, Pierre Fabre Dermatologie, France) applied twice daily until complete resolution of inflammatory signs.
Statistical analyses Because of the exploratory nature and multiplicity of the analyses, all statistical results (issued from SAS® software, version 9. 3) are to be considered only from a descriptive perspective. PO-SCORAD and SCORAD correlations were assessed using the Pearson correlation coefficient. Interpretability of the SCORAD was assessed by stratifying it against the IGA (which acted as a severity anchor). The kappa coefficient of agreement was calculated for each set of bands. To compare AD disease severity over time, captured by two different methods (PO-SCORAD and SCORAD), areas under the score versus time curve (AUC) were compared for the PO-SCORAD and SCORAD indices over 8 weeks using paired-samples Student's t-tests. To assess the extent to which PO-SCORAD data could predict flares, logistic regression was used to compare PO-SCORAD values 7 days before a flare with similar values (from the same time) for patients who did not proceed to have a flare 7 days later.
Conditional analyses and heritability estimates in European ancestry individuals To identify additional signals, Genome-wide Complex Trait Analysis (GCTA, v1. 26. 0)  14  was performed using European ancestry UK Biobank (UKB) participant meta-analysis summary statistics from 33,960 individuals. The analyses identified conditionally independent variants at 4 loci for spQRSTa and at 2 loci for fQRSTa (Supplementary Data 6). Common SNP-based heritability was estimated in the same set of UKB participants with BOLT-Restricted Maximum Likelihood (BOLT-REML, v2. 3. 2) software  15. Heritabilities of spQRSTa and fQRSTa were 22. 3% and 6. 8%, respectively (standard error [SE] 1. 0%). European ancestry lead and conditionally independent variants explained 4. 0% and 0. 5% of the variance of spQRSTa and fQRSTa, respectively.
Variants at previously unreported loci are green. Fig. 3  | Manhattan plot for the fQRSTa multi-ancestry meta-analysis. Manahattan plot for the frontal QRS-T angle (fQRSTA) meta-analysis. Two-sided P-values are plotted on the -log 10 scale (Y-axis). The red horizontal line indicates genome-wide significance (P < 5 × 10 -8 ). Variants within the boundaries of loci previously reported for the spatial QRS-T angle are labelled with the candidate gene and colored blue. Variants at previously unreported loci are green. (TWAS) was performed with S-PrediXcan software. The expression of 33 genes was significantly associated with the spQRSTa (Bonferroni corrected threshold; P < 3. 1 × 10 -6 ), 26 of which mapped within GWS loci, and 10 were significant in multiple tissues (Supplementary Data 10). Increased expression was associated with an increase in spQRSTa for 17 genes, whereas an inverse relationship was found for 15 genes (Supplementary Fig.