dataesr/scientific-paragraphs-categorization huggingface dataset:
B. ROI PROCESSING The aim of the ROI processing is to collect pixels containing the most informative signal components for heart rate estimation. Typically, best regions to extract PPG-related information encompass the entire face or are predetermined rectangular patches including, for instances, forehead, nose or cheeks. ROI selection is a critical process often requiring refinements in order to remove noise and artifacts, while preserving reliable elements for beat detection  [10] ,  [22]. In the pyVHR framework, we implement both rectangular ROIs and a skin detection module. As to the latter, we use simple thresholding on HSV color space (see  [39] ), providing two options for thresholds: fixed user-defined values, or adapted threshold calculated according to color statistics of the video frame at hand.
F. THE GROUND TRUTH SIGNAL Public datasets provide ECG or BVP signals as ground truth, while methods usually provide BPM estimation on a video. It is therefore necessary to estimate HRV from ECG or BVP signals in order to compare method outcomes with ground truth. HRV measurement is not simple and this problem is well known in literature ( [50]-  [52] ). A variety of recording techniques have been proposed, which can roughly be categorized into time and frequency domain-based. In time domain, HRV is calculated from RR intervals occurring in a chosen time window (usually between 0. 5 and 5 min). In frequency domain, HRV is computed by calculating the spectrogram of the BVP signal (by STFT computation). Both techniques present advantages and disadvantages (see  [50]  for an in-depth analysis).
IV. DATA AND STATISTICAL ANALYSES In this section we report a comprehensive statistical comparison of the algorithms outlined in the previous section over multiple datasets by safe, yet robust non-parametric tests. As motivated by many works in different scientific domains, we apply the Friedman test  [53]  instead of standard ANOVA, because it relaxes the assumptions of normality and equality of variances of the residuals. Moreover, we are not only interested in knowing whether any difference exists among algorithms, but also in discovering which method is significantly different from each other (and the FT is not designed for this purpose). To this end, we apply the so-called post-hoc tests to find out which methods actually differ  [31]. In the rest of this section we describe the evaluation metrics used to assess the performance quality, and the nonparametric hypothesis testing procedure as applied to a pool of six benchmarking datasets.
A. BENCHMARK DATASETS The framework accounts for a multi-dataset analysis. Namely, we consider data from 6 datasets, briefly described in the following. Mahnob  [26]. Although this database was mainly conceived for emotion analysis, it has been adopted for testing rPPG algorithms,  [54] ,  [55] , even though it applies a strong compression on the videos. UBFC  [25]. This dataset is composed of 50 videos, synchronized with a pulse oximeter finger clip sensor for the ground truth. Each video is about 2 min long recorded at 30Hz with a resolution of 640 × 480 in uncompressed 8bits RGB format. The authors divided this dataset into two subsets: the first one, UBFC1 is composed by 8 videos, in which participants were asked to sit still; the second one, UBFC2 is composed by 42 videos, in which participants were asked to play a time sensitive mathematical game that aimed at augmenting their heart rate while simultaneously emulating a normal human-computer interaction scenario.
Participants were sitting frontal to a camera placed at a distance of about 1 meter. LGI  [15]. This database is designed for the heart rate estimation from uncompressed face videos acquired in the wild. It is recorded in four different sessions: 1) a resting scenario with neither head motion or illumination changes, 2) head movements are allowed (with static lighting), 3) a more ecological setup, where people are recorded while performing exercises on a bicycle ergometer in a gym; 4) urban conversations are recorded including head and facial motions as well as natural varying illumination conditions. Videos were captured at 25Hz while the pulse sampling rate was 60Hz. It's worth remarking that although the original dataset nominally provides 25 subjects, at the time of writing only 6 are officially released and therefore used in the analysis.
In literature, all these datasets have been adopted to test rPPG algorithms. However it has also been pointed out how the compression can destroy and pollute the subtle pulsatile information essential to rPPG. In  [56]  it has been claimed that uncompressed videos could increase SNR due to information being lost during the video compression process. Similarly, in  [29]  a more in depth analysis has been conducted, aiming at finding an acceptable level of compression, indeed necessary in real world applications.
C. NONPARAMETRIC STATISTICAL TESTS The aforementioned performance measures are now used to perform the statistical analysis. By following  [31]  each metric is analyzed via the Friedman Test (FT) followed by the associated post-hoc analysis. To perform the FT, we apply the repeated measures design in which k classifiers are compared on multiple datasets. The observed data is arranged in a tabular form, where the columns represent the classifiers (i. e. , "groups" in standard statistical test notation) and the rows the datasets ("blocks"). Observations in different blocks are assumed to be independent, but obviously this assumption does not apply to the observations within a block. Denote x j,d the performance measure for the j-th method on the d-th dataset (with j = 1,. , k and d = 1,. , n).
The x j,d values are sorted with respect to j so that each observation within a block receives a distinct rate among the first k integers, thus yielding the values r j,d ∈ {1,. , k} indicating the rank of j-th algorithm on the d-th dataset. A rank of r j,d tells that the method j outperformed k -r j,d methods on the dataset d. The average rank for any j over the datasets is defined as R j = (1/n) i r j,d. Under the null hypothesis, i. e. , no difference between the algorithms, their R j should be equal, and the statistic is χ 2 F = 12 n k(k + 1)   j R 2 j - k(k + 1) 2 4   , (8) VOLUME 4, 2016 which follows a chi-squared distribution with k-1 degrees of freedom. The FT rejects the null-hypothesis at a pre-specified significance level α when the test statistic (8) exceeds the 100(1 -α)th percentile of the limiting chi-squared distribution of χ 2 F with k -1 degrees of freedom  [53].
D. EXPLORATORY ANALYSIS OF PERFORMANCE We provide a summary, via box plots, of the overall performances achieved by the eight rPPG algorithms over the six benchmark datasets (UBFC dataset is split in two parts, namely UBFC1 and UBFC2, as described in previous section). By measuring the central tendency via the median we are able to elicit information about the underlying distribution as well as to identify possible outliers (their character, the amount, etc. ). Figures  6  and 7  present the standard boxplots computed by pyVHR framework and associated to MAE and PCC metrics, respectively. Data are plotted in logscale in order to emphasize the best values for each metric; the boxes are put in gray scale with the intensity proportional to the median value. The general consideration that can be drawn at a glance (besides the log-scale) is that the fences defined by the whiskers are far too small with respect to outliers, and probably asymmetry or tail heaviness is a distinctive character of all distributions.
It is also evident that the extremities of upper whiskers go beyond those of the lowers in almost all cases. It's worth to notice also that the high variability of the results does not lay down an absolute winner or loser among methods against all datasets. Instead, it is beyond doubt that methods perform consistently better on uncompressed video datasets (PURE, LGI and UBFC) whereas it is quite impossible to establish a sound ranking of the methods for compressed video datasets (MAHNOB and CHOFACE). Besides the median, also the interquartile range, IQR, (defined as the difference between the third and first quartile and representing the box size), covering the central 50% of the data, provides useful insights for the assessment procedure. By inspecting the IQRs depicted in the figures, it is worth noticing that in general the methods POS and CHROM provide better median MAE and PCC values, albeit showing less spread with respect to the others.
1) Single Dataset Analysis In all experiments in the single dataset condition, the FT, whose statistics is defined in  (8) , rejected the null hypothesis with very low (p < 10 -3 ). To establish the significant differences between the algorithms post-hoc analysis has been performed via Nemenyi test, Critical values (Eq. (  9 )) were computed followed by pairwise comparisons. These are reported, for each dataset, in Table  2. As suggested in  [31] , differences arising from post-hoc tests can be visually represented with simple diagrams connecting groups of methods that are not significantly different. Figures  8  and 9  display the critical differences through the so-called critical differences diagram (CD), a succinct way to display the differences in methods' performance. The top line in the diagram is the axis where the average ranks of methods are plotted.
The axis is turned so that best performing methods are displayed to the right. Note that depending on the metric adopted, either MAE or PCC, the best ranking method can be the one with the lowest or highest rank respectively. The figures display the CD diagrams obtained from the FT followed by the post-hoc Nemenyi test with a significance level of 95%. A line connecting two or more methods indicates that there are no statistical differences between them. The CDs are also shown above the graph. CDs show a wide ranking variety depending on dataset and metric used. The groups of rPPG methods that behave the same change accordingly, providing a clear picture of the impossibility to establish an absolute pool of winners across datasets. The investigation also give evidence of the usefulness and strength of multiple comparison statistical procedures to analyse and select the best methods for a single dataset.
2) Cross-dataset Analysis Thorough this paper, we have considered the results obtained in a experimental study regarding 8 well-known algorithms using a benchmark suite of 5 datasets we have selected among the most representative. To finally highlight the significant statistical differences among the various algorithms, we further split the datasets into 15 subdatasets, as shown in Table  3. The reason is twofold: on the one hand this split reflects the peculiar differences within each dataset, also motivated by the different algorithm behaviour on each subdataset. On the other hand it allows to increase the number of blocks that, as a rule of thumb, for the FT should be greater than 10  [31]. In addition, our analysis considers a further partition between an uncompressed video collection (top 12 of the table) and a compressed video one (least 3 at the bottom).
The rationale of this distinction relies on the fact that compression certainly is the feature that more than anything else affects both the performances of each rPPG approach and the results of nonparametric statistical tests. Table  3  also reports the MAE and PCC obtained for the 8 algorithms over the 15 datasets considered. For the compressed datasets (top of the table), the lowest median is obtained by POS method for the MAE metric (lowest IQR by CHROM) , whereas the highest PCC median values is produced by POS algorithm (the lowest IQR by others). In turn CHROM provides the best values when results are extended to all datasets (complete table), but the same does not hold for for IQR which has various winners. Boxplot of Figures  11  and 10  synthetically visualize all comparisons. As for the FT, the p-values = 9 • 10 -6 computed through the χ 2 statistics, strongly suggests the existence of significant differences among the algorithms considered.
Nemenyi tests results with critical values at 95% are provided in Figs,  11-(c ) and 11-(d). Surprisingly, it turns that the performances achieved by the four best methods, namely POS, CHROM, PCA and SSR, are not significantly different from a statistical standpoint. Using a three different levels of significance, namely α ∈ {0. 05, 0. 01, 0. 001}, Figures  10-(e ) and 10-(f) display in the form of heatmaps the various hypotheses rejected/accepted by the Nemenyi method for the uncompressed video datasets. In particular, the heatmaps collect a family of 28 hypotheses (all pairs of algorithms) highlighting which algorithms achieve improvements with respect to others, at each given level of significance. The value of significance levels are marked with intensity proportional blue color; alternatively, they are marked with NS to claim that the difference is not significant.
Under such circumstances, we surmise that an open algorithmic framework such as the one we have presented here, may help in promoting a good practice about the design, the experimental analysis and the general assessment of rPPGbased algorithms. We also believe that this work may lead to a sort of standardization of algorithm evaluation process overcoming the uncertain different experimental approaches seen so far. Similar concerns have been reported in the machine learning field  [30] -  [32] ,  [57] ,  [58] , where by and large there is no golden standard for making comparisons and tests based on solid statistical foundations, often leading to unwarranted and unverified conclusions. A clear indication has been put forward in the direction of sound statistical assessment of method performances through statistical tests and post-hoc procedures devised to perform multiple comparisons across many datasets.
In particular, when a single dataset is used (or many, but separately) for experiments, due to dependencies between the samples of examples drawn, there is the concrete risk to incur into biased variance estimations, thereby increasing Type 1 error in hypothesis testing. Conversely, over multiple datasets the variance comes from the differences between the datasets, which are usually independent, and this fact can be better faced with some families of nonparametric statistical tests. The pyVHR open-source framework we have introduced here, to substantiate the proposed methodology, is a flexible and extensible tool for creating, tuning and evaluating any kind of rPPG-based methods. It already implements the most representative methods developed for this purpose and incorporates a relevant amount of results from experiments conducted on five known datasets, either with compressed or uncompressed videos.
It is also endowed with standard tools for preprocessing and postprocessing the data, as well         as to visualize both partial and final results. Cogently, pyVHR includes multiple comparison statistical procedures, based on Friedman and Nemenyi hypothesis tests, that can be employed to carry out sound statistical assessments. As a final remark, we point out that the pyVHR has been developed in Python, a language that enjoys a widespread popularity and a ease of use, qualities that facilitate further developments. This leaves open the possibility to contribute with new and, at the moment, lacking features, such as real time pulse rate estimation or advanced video processing capable of compensating subject movements leading to better prediction.
MSE and PCC medians for the 8 rPPG algorithms over 15 datasets. The first eleven (top of the table) is a group of uncompressed video subdatsets, the last 3 are compressed video subdatsets. MAE CC CHROM GREEN ICA LGI PBV PCA POS SSR CHROM GREEN ICA LGI PBV PCA POS SSR LGI-PPGI-talk 10. 86 18. 25 12. 46 10. 86 13. 40 10. 75 11. 07 13. 79 -0. 03 -0. 07 0. 08 -0. 04 -0. 01 0. 18 0. 02 0. 11.
Transcriptomic Analysis Transcriptomic analysis was performed by the Get-TRIX platform (INRA, Toulouse, France) using Agilent Sureprint G3 Mouse microarrays (8 × 60K, design 028005). For each sample, Cyanine-3 (Cy3) labeled RNA was prepared from 25 ng of total RNA using the One-Color Quick Amp Labeling kit (Agilent), followed by Agencourt RNAClean XP (Agencourt Bioscience Corporation, Beverly, MA, United States). Cy3-lebeled RNA (600 ng) was hybridized on a microarray slide. After washing, the slides were scanned on Agilent G2505C Microarray Scanner using Agilent Scan control A. 8. 5. 1 software and the fluorescence signal was extracted using Agilent Feature (extraction software v10. 10. 1. 1 with default parameters). One sample from lean controls didn't pass the quality check and was excluded from the analysis.
Spikes and Force Analysis Data were analyzed offline using custom routines in Matlab (MathWorks). First, the position signal was extracted by the video using a white squared marker on the slide as reference. The recording frequency of the video was 25 Hz. After applying an antialiasing FIR lowpass filter, a uniform linear resample of the movement of the slide was performed, in order to synchronize the position signal with the force data, recorded at 100 Hz. To identify the timing of the voluntary activity of the animal, a threshold method was used to detect force peaks during the pulling phase of the task. For the following analysis, we picked out peaks that produced a displacement of the slide, in addition to crossing the threshold; and we calculate the onset of these peaks as the minimum of the force derivative just before the respective peak  (Spalletti et al.
, 2014). The electrophysiological signal, recorded at 40 kHz as sampling rate, was analyzed by Offline Sorter (Plexon, Dallas, TX). First, for each channel of the probe, we sorted waveform that crossed a detection threshold of the mean ± 3 standard deviations. Then, detected spikes were clustered using an automatic process based on principal component analysis. Starting from these clusters, a manual sorting was executed to isolate all single units which could be identified in the recorded multi-units signal. The time stamp of each unit was synchronized with the data of the robot. To evaluate the temporal behavior-related spike activity, the peristimulus time histograms (PSTHs, NeuroExplorer, Plexon) was generated with bins of 20 ms in an interval of 1 s around the onset of force peaks. In addition, the resting activity of each unit was evaluated selecting intervals of at least 0.
Phase Coherence and Functional Connectivity Functional connectivity (FC) among cortical regions was inferred from phase coherence of activity measurements, and used to determine changes in brain activity in "stroke" and "rehab" condition, as compared to the healthy mice. These inferred activity changes were used to parameterize simulations of the BNM built over the Allen Brain Atlas mouse connectivity data (http://connectivity. brain-map. org/; Oh et al. , 2014, below  referred to as the Allen Mouse Brain Atlas -AMBA), incorporated in the extended virtual mouse brain  (Melozzi et al. , 2017). In each animal, the camera field-of-view used for activity measurements was placed in a standard position using the sagittal suture and its intersection with the coronal suture of the skull (bregma) as anatomical landmarks.
To spatially correlate our activity measures with the structural connectivity data  (Oh et al. , 2014) , the camera field-of-view (Figure  3B ) was spatially translated to the Allen Common Coordinate Framework  (CCF, v3, 2015; Wang et al. , 2020). Since the CCF lacks stereotactic skull landmarks, these were introduced by spatially co-registering all diagrams from a standard stereotaxic mouse brain atlas  (Franklin et al. , 2008)  to the CCF coordinate space with affine transformations defined using the QuickNii tool  (Puchades et al. , 2019). Using bregma and the sagittal suture as a reference, the four corners of the downsampled 128x128 pixels field-of-view of the recorded images were positioned in CCF, taking the 5 degree lateral tilt of the camera view into account. Delineations of layer IV cortical regions were then projected onto the camera field-ofview, and used as a custom atlas reference for all activity maps.
Embodied Mouse in the Neurorobotics Platform The full musculoskeletal model of the virtual rodent controlled by the spinal cord model was simulated in the Neurorobotics Platform (NRP) developed in the Human Brain Project  (Falotico et al. , 2017). The main components of the NRP are a world simulator, a brain simulator and the mechanism that enables the data flow between the two in a closed-loop. The connection between the body and the brain is specified through a domain specific language  (Hinkel et al. , 2015 (Hinkel et al. , , 2017)) , via Python scripts called Transfer Functions. In these scripts the output of devices that read neuronal output data can be processed and passed as input for the virtual body actuators, and vice versa, the sensory information from the virtual body sensors, in this case muscle length data, can be passed to devices that map sensory data to neural input.
Simulation of the Experiment on Goal-Directed Movements As the first component of the proposed framework ("Movementdriven models"), we simulated the experiment on goal-directed forelimb pulling in the virtual environment and validated the simulation on experimental data. In the in-vivo experiment, two healthy mice were trained on the M-Platform to perform active pulling of the forelimb. As we expected, the contralateral motor cortex showed a highly coherent activation with the kinetic data. The coherence between the force applied by the animal and the signal recorded in the CFA was evident both in the low and in the high frequencies band (Figure  8 ). For the data that were later used in simulations we focused on the high band (300 to 40k Hz); in particular we found an high activation of the motor cortex around the force peaks for both multi-unit activity and single units analysis.
In Figure  8 , we show the results for a simulation trial and a comparison with data recorded from a physical experiment. We employed kinematic data recorded alongside neural activity in the same in-vivo experiment: position of the slide and force applied to the slide through the trial. As expected, by comparing the activation levels with the normalized force applied by the mouse to the slide we can observe that the flexor muscles are active when there is also a force recorded, and conversely, there is low activation when the slide is still. It is also worth mentioning that, although the two muscles receive the same inputs from the descending stimuli, their activation levels are different due to the feedback circuitry of the spinal cord and the activity of muscle spindles, which are different for the two muscles.
DATA AVAILABILITY STATEMENT The datasets generated for this study are openly available online at: http://doi. org/10. 25493/Z9J0-ZZQ. The raw datasets on neurophysiological recordings for this study can be found online at: http://doi. org/10. 5281/zenodo. 3546068. Processed data and source code that can be used to reproduce the neurophysiological recordings experiment can be found in the following repository: https://gitlab. com/lore. ucci/closedloop-mouse-stroke-simulation. The BNM model and all the relevant information to reproduce the simulated BNM data can be found in the following repository: https://github. com/esaps/ AllenMouse_strokeKuramoto. Data sharing license: This work is shared under a Creative Commons Attribution CC BY 4. 0 license (https://creativecommons. org).