dataesr/scientific-paragraphs-categorization huggingface dataset:
The MATLAB function chop  4  [2] and the FLOATP_Toolbox 5  for MATLAB  [27]  leverage the MATLAB random number generator to draw a random number r from the uniform distribution over the open interval (0, 1) and choose the rounding direction depending on whether r is larger or smaller than q(x). Most software favors the use of integer random numbers, integer arithmetic, and bit manipulation. The implementation of SR in the QPyTorch 6  package  [35] , for example, rounds stochastically a binary32 number y to a floating-point format with precision p < 23 as follows. First, it generates the 32-bit integer m by zeroing out the leading (p -1) + 9 bits of a 32-bit random integer, since the first 9 digits in the binary representation of y store its sign and exponent and the following p -1 store the p most significant bits of the significand using the implicit bit convention.
For each operation, they propose two algorithms, one that uses only RN, and one that combines it with RZ, RU, and RD. Both variants are faster than an implementation that relies on the GNU MPFR library, and the RN-only versions are faster on x86 architectures, where switching the rounding mode incurs a high performance penalty [17, Sec. 12. 3. 2]. Klöwer's Julia software package StochasticRounding. jl 10 defines three new Julia floating-point types that automatically include SR. These correspond to bfloat16, binary16, and binary32, and use the Xoroshiro128Plus fast pseudo-random number generator (PRNG). 11 Composability and type flexibility in Julia enable SR computations in single and half precision in a large number of numerical software and mathematical libraries. Automatic application of SR operations is extremely advantageous from a user standpoint, as it allows significant code simplification.
The SpiNNaker2 SR accelerator rounds and saturates 64-, 32-, or 16-bit to 32or 16-bit fixed-point numbers with SR. As a special case it also includes rounding from IEEE 754 binary32 to bfloat16. The random bits needed for rounding are produced using the 32-bit hardware pseudo-random number generator available on SpiNNaker2  [48]. The number of bits to be used to round the fixed-point number is programmable (it can be anything between 1 and 32 bottom bits of the input), while in the case of binary32 to bfloat16 rounding it is fixed to round the bottom 16 bits of the significand. (ii) Modifying basic floating-point algorithms to include SR Now we discuss how to modify classical algorithms for addition and multiplication of floatingpoint numbers  [17, Ch. 7 ], [49, Sec. 4. 2. 1], [50, Ch. 8 ] in order to obtain algorithms that support SR and can readily be implemented in floating-point software or hardware.
(c) Numerical verification software Numerical verification software uses SR to explore the propagation of rounding errors in applications: a particular computation is repeated multiple times and the distribution of errors from these runs is used to draw conclusions about the sensitivity of a code to rounding errors. Mode 2 SR, known as stochastic arithmetic  [72] , is used for example in the CADNA library  [73]. An approach that includes as options both mode 1 and mode 2 SR is Monte Carlo arithmetic  [74] ,  [75] , a method used by tools such as Verificarlo  [36]  and Verrou  [37] ,  [38]. Monte Carlo arithmetic is more general than SR, not least because as well as randomly rounding the result of a floating-point operation it can also randomly perturb its inputs and output.
Conclusions Hardware with stochastic rounding in computer arithmetic units is not yet widely available, but has started to appear: as we explained in Section 5(d), Graphcore and Intel have produced processors with SR built-in. Patents from AMD, NVIDIA, IBM, and other computing companies describing implementations of SR in fixed-or floating-point arithmetic units show that this rounding mode could become more widely available in the future. When hardware is not available, simulation in software of arithmetics with SR can be used to explore its behaviour. Multiple simulators have been developed, as discussed in Section 5(c). These are available in various forms in MATLAB, C, Julia, and Python. Rounding error analysis with SR, discussed in Section 6, shows that compared with the standard rounding modes SR promises smaller probabilistic errors bounds in applications that involve inner products and avoids the problem of stagnation (Section 6(a)), where small values are lost to rounding when they are added to an increasingly large accumulator.
The main components of these sensors are triangular "pixels", 50cm × 25cm in size, that allow conductance detection (Fig. 1 ). The size of the sensors is large compared to other devices (e. g. , SpeedLetiX), and therefore a solid layer of algorithmic computation is required. Each sensor performs a continuous measurement of the electrical capacitance expressed on a scale from 0 to 255, at a sampling rate of 868 Mhz. The specific zero, calibrated at 127 arbitrary units (AU), allows the expression of negative data that reflect, among other phenomena, the withdrawal of the conductive body. The signals emitted by the sensors, called activations, are encoded on a server, stored in a daily ". csv" file, and transferred post-hoc to the Walk@Home team. Each recorded activation generates the following data (Table.
1), or with two individuals engaged in a conversation (E. 2); Three participants (23 years old ± 1) voluntarily participated in the testing. They were brought to the experimental apartment, filled out the informed consent form, were told about the goal of the test, and performed the five scenarios in turn. They performed seven round-trip trials per scenario, yielding a total of 200 round-trip walking or standing trials. The MATLAB R2020a environment was used to write all the scripts for data cleansing, filtering, and processing, as well as for further analyses. These scenarios were tested in summer 2019 during a period of non-occupancy of the apartment. In addition, in December 2019, a second step involved the recording of the natural walking behaviors of a single person during 24 hours of occupancy.
V. RESULTS AND DISCUSSION As mentioned above, we first evaluated our algorithmic approach by testing five controlled scenarios. As the timestamps and the walking conditions are known, this allowed us first to verify the correct functioning of the Walk@Home algorithm. In a second step, we applied our algorithm to a specific data file representing one day of continuous recordings with only one inhabitant in the apartment. We first identified valid trajectories within the apartment, then estimated trajectory-related metrics, and finally performed a statistical analysis of the occupied surface during that day.
Global gait metrics: The locomotor information automatically extracted during data processing allowed us to obtain simple daily metrics. For our selected file, "data_8000_2019-12-21_04h20", 529 trajectories and 595 stationary standing patterns were calculated from 100,490 ground activation contacts. Trajectories were generated with an average velocity of   = 1. 4 / (  = 0. 6 /), for an average distance traveled (per trajectory) of   = 14. 5 (  = 18. 9 ) and an average walking time of   = 9. 4  (  = 6. 9 ). Each trajectory had an average of 50 activations. Furthermore, the temporal information extracted showed a dominant occupation of the living room (3. 84%) and room1 (1. 75%), for a total travel time of 8. 21% over a day (Table  III ). These metrics, particularly walking velocity (value of 1.
Spatial occupancy: The matrix shown in Table IV allowed us to obtain a representation of the number of activations per room, broken down into one-hour periods. These data are useful and allow us to (i) observe the occupancy dynamics, (ii) understand the habits of the inhabitants, e. g. , the links between occupancy and daily needs (restroom, kitchen, bathroom), and (iii) optimize in the future living spaces according to locomotor habits, personal needs and comfort. We previously tested and validated the calibration file and ensured the consistency of the occupancy data output by the algorithm. We defined here the occupancy of one room by its number of activations per time slot. For instance, the bathroom had its maximum occupancy    from 1am to 2am with 512 activations recorded, followed by another intense occupation period from 8pm to 9pm (422 activations).
Context: We study the benefits of using a large public neuroimaging database composed of functional magnetic resonance imaging (fMRI) statistic maps, in a self-taught learning framework, for improving brain decoding on new tasks. First, we leverage the NeuroVault database to train, on a selection of relevant statistic maps, a convolutional autoencoder to reconstruct these maps. Then, we use this trained encoder to initialize a supervised convolutional neural network to classify tasks or cognitive processes of unseen statistic maps from large collections of the NeuroVault database.
Researchers took advantage of the specific properties of fMRI data to build more and more sophisticated models  [4] [5] [6] [7] [8] [9] [10] [11]. However, training effective DL models using MRI data comes with many challenges  [12]. Brain images differ from natural images that are typically used in the DL community on multiple aspects: they contain quantitative information (i. e. , statistical values in our context of task fMRI that need to be considered differently during preprocessing steps such as standardization or during training with batch normalization), spatial localization is crucial information (i. e. , the same activation in different regions of the brain leads to a completely different interpretation), and the dimensionality of medical images is much larger (i. e. , an fMRI statistic map contains tens of thousands of dimensions).
This means that a technique used for natural images may fail for medical images (see  [13] ). While DL models have helped resolve important problems in subfields of brain imaging (such as the segmentation of anatomical datasets), their use in task fMRI is still limited  [14]. Performance of DL models in task fMRI has been limited by the high dimensionality, lack of diversity, and low sample size of conventional datasets  [15, 16]. In relation with the large number of trainable parameters in DL models  [17] , this makes it particularly difficult to build fair and generalizable DL models for fMRI. Indeed, fMRI datasets are typically composed of 3-dimensional (3D) volumes with hundred thousand dimensions (or voxels) for a rather small number of subjects (typically 10-100). The field also suffers from a large number of sources of variability in the data at the subject level (brain activity patterns differ across subjects), the acquisition level (fMRI scanners and protocols often vary between centers and studies), and the analysis level (different analysis pipelines lead to different brain patterns).
In our case, brain decoding models should be robust to all these sources of variability  [18]. This problem of small and uniform training sets is not limited to neuroimaging and is well known in the field of machine learning, where researchers extensively use deep transfer learning to improve classification and generalization performance of their models (see, e. g. ,  [19] ). This method consists in using the knowledge obtained from a model trained for a source task on a source dataset and applying it to a target task on a target dataset. Transfer learning proved its worth on natural images by using large, publicly available datasets  [20]  to pretrain DL models before fine-tuning them on smaller datasets of a related domain. While the availability of large datasets has enabled solving difficult machine learning problems (such as classification), collecting similarly large datasets in brain imaging (including a diversity of participants and modalities) is especially challenging.
In fact, in fMRI studies, the median sample size was still limited to N = 30 participants in 2015  [15]. Efforts for collecting large-scale datasets have arisen in the field in the past 10 years with, for instance, the Human Connectome Project (HCP)  [21]  or the UK Biobank  [22]  and give hope for an improvement of model performance. But these datasets are still much smaller than those that brought breakthroughs in computer vision and are often much less diverse  [23]. To prevent overfitting and allow for generalizable statistical inference, neuroimaging researchers proposed methods to tackle this lack of training data  [24] [25] [26]. For instance, Mensch & al. [27]  built a decoding model using data gathered from 35 studies and thousands of individuals that cover various cognitive domains.
Despite the good performance of the models, these can only be applied on restricted sets of studies, discriminating between few cognitive concepts. More annotated training data (e. g. , using large public databases) would be required to map a wider set of cognitive processes. Many studies were also made on inductive transfer learning with labeled source data as defined in  [19]  (e. g. , source task and target task are different, as well as source domain and target domain)  [28] [29] [30]. For instance, Thomas & al. [28]  pretrained 2 DL classifiers on a large, public whole-brain fMRI dataset of the HCP, finetuned them, and evaluated their performance on another task on the same dataset and on a fully independent dataset. In another study,  Gao & al. [29]  used the ImageNet database  [20] , a large, public dataset containing naturalistic images from more than 1,000 classes, to pretrain a model and adapt it to classify tasks from 2dimensional (2D) fMRI data.
This database was also used in  [31]  for pretraining a 2D structural MRI classifier. In the same study, the Kinetics dataset  [32]  was also used to evaluate the transfer learning process with 3D images. In a recent work, Thomas & al. [33]  used self-supervised learning frameworks to pretrain DL decoding models across a broad fMRI dataset, comprising many individuals, experimental domains, and acquisition sites. These studies show improved classification accuracies as well as quicker learning and less training data required. However, labeled databases are not always available in neuroimaging, despite the growing effort in data sharing to build public databases  [34] , such as OpenNeuro for raw data  [35]  and Neu-roVault for fMRI statistic maps  [36]. The unconstrained annotations and the heterogeneity of tasks and studies make them difficult to use to pretrain a supervised DL model.
By initializing the weights of a supervised classifier with the pretrained weights of an unsupervised model trained on many images, the aim is to improve the model performance by placing the parameters close to a local minimum of the loss function and by acting as a regularizer  [41]. In the field of neuroimaging, latent representations have recently been used in a task-relevant autoencoding framework. In  [42] , an autoencoder was used with a classifier attached to the bottleneck layer on a small fMRI dataset. This model outperformed the classifier trained on raw input data by focusing on cleaner, task-relevant representations. This suggests that a low-level representation of fMRI data, learned for a reconstruction task, can be helpful in a classification task, as in a self-taught learning framework.
In this work, we propose to take advantage of NeuroVault-a large public neuroimaging database that was built collaboratively and therefore displays a good level of variability in terms of fMRI acquisition protocols, machines, sites, and analysis pipelinesin a self-taught learning framework. We pretrained an unsupervised DL model to learn a latent representation of fMRI statistic maps, and we fine-tuned this model to decode tasks or mental processes involved in several studies. In the first part, we leveraged the NeuroVault database to select the most relevant statistic maps and train a convolutional autoencoder (CAE) to reconstruct these maps. In the second part, we used the final weights of the encoder to initialize a supervised convolutional neural network (CNN) to classify the cognitive processes, tasks, or contrasts of unseen statistic maps from large collections of the NeuroVault database (an homogeneous collection of more than 18,000 statistic maps and an heterogeneous one with 6,500 maps).
Material and Methods The code produced to run the experiments and to create the figures and tables of this article is available in the Software Heritage public archive  [43]. Derived data used by these notebooks are stored in Zenodo  [44]. Fig. 1  illustrates the overall process used to implement our selftaught learning framework: a CAE was first trained to reconstruct the maps of a large dataset extracted from NeuroVault. Then, the encoder part of the CAE was fine-tuned to answer a classification problem on another dataset (with labels). After hyperparameter optimization, performance of the pretrained classifier was compared to those of a classifier initialized with a default algorithm. Details regarding the datasets (NeuroVault dataset and classification datasets) can be found in the next subsection.
NeuroVault dataset NeuroVault  [36]  (RRID:SCR_003806) is a web-based repository for statistic maps, parcellations, and atlases produced by MRI and positron emission tomography studies. This is currently the largest public database of fMRI statistic maps. NeuroVault has  NeuroVault dataset is used to train a convolutional autoencoder (CAE). The encoder of this CAE is used to initialize a convolutional neural network (CNN) and to train it to classify other datasets. These classification datasets are split in 2 disjoints datasets: a "validation" one used to optimize hyperparameters and a "test" one to evaluate performance. In each one, a 5-fold cross-validation is performed. (36)  its own public Application Programming Interface that provides a full access to all images (grouped by collections) and enables filtering of images or collections with associated metadata.
At the time of the experiment (19 January 2022), a total of 461,461 images in 6,782 collections were available. Among the available metadata, some are mandatory and specified for all maps such as the modality (e. g. , "fMRI-BOLD" for blood oxygen level dependent functional MRI; dMRI for diffusion MRI, etc. ), the type of statistic (e. g. , "T map" or "Z map"), or the cognitive paradigm (e. g. , "working memory" or "motor fMRI task paradigm"), and others are optional and only available if additionally entered at the time of the upload. From this large database, relevant maps were selected based on multiple criteria. First, we chose maps for which the modality was "fMRI-BOLD" to exclude other modalities such as structural or diffusion MRI. To get comparable maps, we set 3 additional inclusion criteria and selected maps (i) for which all required metadata were provided ("is_valid" to True), (ii) that were registered in MNI space ("not_mni" to False) to ensure that anatomical structures were located at the same coordinates in each map, and (iii) that were referenced as "T map" or "Z map" to exclude maps in which voxel values did not have the same meaning (e.
g. , P value maps, chi-squared maps). Among these, thresholded statistic maps were excluded. We found that some maps in our initial dataset were wrongly referenced as T map or Z map. These misclassified maps were removed by filtering the "filename" column of the dataframe to exclude SetA_mean SetB_mean (AFNI contrast maps), con (SPM contrast maps), and cope (FSL contrast maps). Using these criteria, a total of 28,532 statistic maps were selected from the NeuroVault database and constituted our "Neu-roVault dataset. " Most of these maps were unlabeled (i. e. , cognitive processes or tasks performed described as "None/Other") or not labeled in a standardized way (i. e. , use of terms that are specific for a study instead of generic terms, such as those defined in  [38] ; e. g. , some maps were labeled as "word-picture matching task" for the cognitive paradigm, whereas others in which a similar task was performed were referenced as "working memory fMRI task paradigm," which is a label that includes other specific tasks).
HCP dataset (NeuroVault Collection 4337) NeuroVault collection 4337 [45] includes 18,070 z-statistic maps, for base contrasts (task vs. baseline), corresponding to 787 subjects of the HCP 900 release  [46]. This collection was excluded from our pretraining dataset (see subsection "NeuroVault dataset") due to missing metadata (i. e. , "is__valid" is false). All maps in this collection were grouped together and referred to as the "HCP dataset" in the following. Multiple labels were entered for each map, including mental concepts ("cog-nitive_paradigm_cogatlas"), tasks ("task"), and contrasts ("con-trast_definition") (as defined in  [38] ). For each subject, 23 contrasts distributed in 7 tasks were available: r Working memory: "0-back body," "0-back face," "0-back places," "0-back tools," "2-back body," "2-back face," "2-back places," "2-back tools" r Motors: "cue," "left foot," "left hand," "right foot," "right hand", "tongue" r Relational: "relational," "match" r Gambling: "punish," "reward" r Emotion: "faces," "shapes" r Language: "math," "story" r Social: "tom" For more details on contrasts, tasks, and mental concepts of this study, see  [46].
BrainPedia dataset (NeuroVault collection 1952) NeuroVault collection 1952  [47] , known as BrainPedia  [48] , contains fMRI statistic maps of about 30 fMRI studies from Open-Neuro  [35] , the HCP  [46] , and data acquired at Neurospin research center; together, they were chosen to map a wide set of cognitive functions. This collection contains 6,573 statistic maps corresponding to 45 unique mental concepts derived from 19 subterms (e. g. , "visual, right hand, faces" for maps associated with the task of watching an image of a face and responding to a working memory task). These images were previously used to build a multiclass decoding model  [48] , and labels corresponded to the mental concepts associated with the statistic map (e. g. , "visual," "language," or "objects"). Here we excluded the 9 classes that had fewer than 30 samples each, leaving 6,448 images corresponding to 36 classes.
Classifier training We trained 2 types of classifiers for all the experiments: r the classifier with default algorithm initialized with the original algorithm from  [56]  (i. e. , Kaiming Uniform algorithm for convolutional and fully connected layers with a parameter of √ 5) and r the classifier with pretrained CAE initialized using the weights and bias of the convolutional layers of the CAE pretrained on the NeuroVault dataset. The CNNs were trained using the Adam optimizer with a learning rate of 1e-04. We used the cross-entropy loss function for training the classifier. Both were implemented in PyTorch.
Dataset split As described in Fig. 1  (on the right), the classification datasets were split in 2 disjoint subsets: the validation dataset used to optimize the hyperparameters and the test dataset used to test the performance. Each subset contained 50% of the subjects of the overall dataset with no overlap to avoid any data leakage (see  [57, 58] ). For each experiment, the validation and test datasets were then split into 5 folds for cross-validation. Subjects were randomly sampled in each fold in order to ensure that there was no overlap of subjects across folds. The identifiers of the subjects included in the different folds were saved for reproducibility. More details on the methods used to perform the 5-fold split for each dataset are specified in section "Benefits of self-taught learning and impact of different factors".