dataesr/scientific-paragraphs-categorization huggingface dataset:
Running time Due to the ever-increasing amount of data coming from new sequencing technologies, execution time has become an important factor in evaluating computational prediction tools. In Table  1 , we present the running time of miRBoost and the other tools (except for mirExplorer, which could not be installed on our Linux machine) for predicting the 690 novel pre-miRNAs and the 8246 non-miRNA sequences considered above. miRBoost is the only method that performs well the classification, when dealing with the imbalanced data issue, in a reasonable amount of time (2 min 29 sec). This compares with several hours for related methods such as microPred, MiPred, and HeteroMirPred. Although microPred shows sim-ilar results to miRBoost in classification ability, it takes over 1400 times longer than miRBoost to achieve this.
miRBoost is also 10 times faster than CSHMM. triplet-SVM and MIReNA seem to be very rapid (three times faster than miRBoost), yet their prediction performance is much lower than that of miRBoost. All experiments were performed on a Linux machine with four six-core processors Intel Xeon X5680 of 3. 33 GHz and 20 GB of RAM, except for mirExplorer, where the tests were performed on a Windows machine with Core 2 Duo Intel E8400 of 3. 00 GHz and 4 GB of RAM. None of the programs were executed in parallel mode.
Positive data set for cross-validation The genomes of eukaryotes containing at least 100 miRNAs in the miRBase database (version 18) (Kozomara and Griffiths-Jones 2011) are studied. We take from these genomes pre-miRNAs of <400 nt. As it is known that miRBase contains a number of mis-annotated miRNAs, we first remove the sequences reported as mis-annotated in the later versions  (19 and 20). The remaining pre-miRNAs are filtered by ncRNAclassifier  (Tempel et al. 2012)  to discard the ones that are mis-annotated because corresponding to transposable elements. The obtained sequences are then considered as positive data. They include 1279 sequences for human and 3082 sequences for cross-species. To avoid overfitting, we remove the sequences that have an identity of >97% with the other ones using EMBOSS skipredundant  (Rice et al.
Negative data set for cross-validation From selected genomes, we randomly choose the exonic regions from protein-coding genes at NCBI (http://www. ncbi. nlm. nih. gov/genome). We also take the noncoding RNAs that are not miRNA, including tRNA, siRNA, snRNA and snoRNA, from fRNAdb  (Kin et al. 2007) , NONCODE  (Liu et al. 2005) , and snoRNA-LBME-db  (Lestrade and Weber 2005)  database. All of them contain <400 nt. We use miRNAFold  (Tempel and Tahi 2012)  to predict a hairpin-like structure in each selected sequence (see Fig. 3 ). miRNAFold first identifies the longest exact stem from the given sequence. The identified stem is then extended into the longest nonexact stem (i. e. , succession of exact stems separated by symmetric internal loops). The hairpin secondary structure corresponding to this nonexact stem is finally predicted.
Novel pre-miRNAs for sensitivity validation Furthermore, we evaluate the sensitivity of our algorithm, i. e. , its ability to identify pre-miRNAs, through a prediction on novel cross-species pre-miRNA sequences. These pre-miRNAs are taken from newly added sequences in the versions 19 and 20 of miRBase database. We also use ncRNAclassifier to eliminate false pre-miRNAs, after removing the misannotated ones that are reported in miRBase. Their redundancy is then reduced to 97%, giving 690 sequences.
2005) , and the average size of internal loops  (Batuwita and Palade 2009). Thus, we study in total 187 features, which give information on structure and sequence of pre-miRNAs. They can be gathered into three groups describing size, position, composition, asymmetry, and energy of the exact stem, nonexact stem and hairpin (see Table  3 ; Supplemental Data 2). For each data set, to select the consistent and nonredundant features from those 187 ones, we exploit feature selection techniques proposed by the WEKA workbench  (Hall et al. 2009 ): Best First, Linear Forward Selection, Greedy Stepwise, Scatter Search, and Subset Size Forward Selection. These search-based techniques  (Devijver and Kittler 1982; Jain and Zongker 1997)  find highly scored low-dimensional projections of the given data, and thus select the features giving the largest projections in lower dimensional spaces.
Existing software A number of computational tools were introduced for pre-miRNA classification. As mentioned in the Results, we used several of them for comparison with miRBoost: CSHMM  (Agarwal et al. 2010) , triplet-SVM  (Xue et al. 2005) , microPred  (Batuwita and Palade 2009) , HeteroMirPred  (Lertampaiporn et al. 2013) , MiPred  (Jiang et al. 2007) , mirExplorer  (Guan et al. 2011) , and MIReNA (Mathelier and Carbone 2010), which are described as follows. CSHMM used a context-sensitive hidden Markov model (HMM) to represent pre-miRNA structures and identify pre-miRNAs in genomes. CSHMM extended the idea of HMM by introducing a memory, in the form of a stack or a queue, between certain states in the model. The proposed CSHMM structure had two context sensitive states that were linked to the same pairwise-emission state through a stack in order to separate states for the stem and symmetric bulge generation and to keep information about what was emitted earlier.
The known human pre-miRNA sequences were used to assess the transition and emission probabilities for CSHMM. triplet-SVM consisted of an SVM classifier applied on the features of local contiguous structure-sequence information to distinguish real and pseudo pre-miRNAs. The SVM model trained on human miRNA data showed the ability to predict pre-miRNAs from other species across animals, plants, and viruses with high accuracy. It suggested that their 32 features of triplet elements reflected discriminative and conserved characteristics of pre-miRNAs, which were consistent across all species. microPred applied filter methods to select discriminative features (they filtered 48 initially proposed features to 21) and utilized an SVM classifier, which was able to deal with the imbalance problem via techniques of random over/under-sampling, synthetic minority over-sampling technique (SMOTE), different error costs, and z-SVM.
The prediction model of microPred was built on human data and then validated on other animal and viral data. HeteroMirPred aggregated the prediction of different heterogeneous algorithms, including SVM, k-nearest neighbors, and random forest in order to create a high level of diversity and to reduce bias of each individual classifier. The authors proposed a modified version of SMOTE to solve the imbalanced data problem. The feature selection was realized via filter methods of ReliefF, Information Gain, and Correlation-based feature selection from 125 features on sequence, secondary structure, base pairs, triplet elements, and structural robustness. This cooperative combination was validated across various organisms, from animals, plants to viruses. MiPred used random forests to separate the real pre-miRNAs from the pseudo ones.
Random forest is an ensemble classifier that consists of several decision trees  (Breiman 2001). It could be considered as a combination of a further development of the bagging technique and a random feature selection technique. For bagging, each tree was trained on a bootstrap sample of the training data, and predictions were made by a majority vote of trees. For feature selection, a hybrid feature that incorporated 34 features on the local contiguous structure-sequence composition, the minimum free energy of the secondary structure and the P-value of randomization test was used. The model was only trained on human pre-miRNA sequences and tested on data from other species. mirExplorer proposed a visual pre-miRNA prediction using a set of transition probability and miRNA biogenesis features.
Acknowledgements. The authors would like to thanks the NEMOVAR group in general and  A. Weaver  and  K. Mogensen  in particular for their participation in the early phase of the development of NEMOTAM. Additional precious help came from  R. Benshila , who guided us through the meander of the NEMO system. This work was also financially supported by  CNES  and the  French National Research Agency  ( ANR  project  VODA :  ANR-08-COSI-016 ). Edited by:  D. Ham.
We collected data among students of nine classes corresponding to the second year of such studies: 3 classes of type "MP" (MP, MP*1, MP*2), two of type "PC" (PC and PC*), one of type "PSI" (PSI*) and 3 of type "BIO" (2BIO1, 2BIO2, 2BIO3). All these students must prepare a small scientific project that they present at the final exam, and several students could build a project based on their participation to the data collection, together with the use of the collected data in some small scale analysis or numerical simulations. The active involvment of some students ensured a good participation of other students to the data collection, as also reported in other works  [9, 24]. We collected data of different nature. • we deployed the contact measurement platform developed by the SocioPatterns collaboration  [20] , which is based on sensors that are embedded in unobtrusive wearable badges and exchange ultra-low power radio packets in order to detect close proximity of individuals wearing them.
As described in detail in  [20] [21] [22] , the power is tuned so that the sensors can exchange packets only when within 1 -1. 5 meters of one another. Moreover, students were asked to wear the sensors on their chests using lanyards, ensuring that the devices of two individuals can only exchange radio packets when the persons are facing each other. The sensors are also tuned so that the face-to-face proximity of two individuals wearing them can be assessed over an interval of 20 seconds with a probability in excess of 99%. Contact data are thus collected with a temporal resolution of 20 seconds: two individuals are considered to be in contact during a 20s time window if their sensors exchanged at least one packet during that interval, and the contact event is considered over when the sensors do not exchange packets over a 20s interval.
The information on face-to-face proximity events detected by the wearable sensors is relayed to radio receivers installed throughout the high school: contacts occurring outside the school premises were not measured. Thanks to this infrastructure, we gathered contact data of the 327 participating students (out of 379 in the 9 classes, i. e. , a 86. 3% participation rate) during the week of Dec. 2-6, 2013. • At the end of one specific day, namely Dec. 5, 2013, we asked students to fill in paper contact diaries: they were asked to give the list of other students they had had contact with (where contact was defined as close face-to-face proximity) during the day in the high school, and to give the approximate aggregated duration of the contacts with each nominated individual, to choose in one of four possible categories: at most 5 minutes, between 5 and 15 minutes, between 15 minutes and 1 hour, more than one hour.
120 students returned a filled in diary. • During the period of the deployment, we moreover asked the students to fill in a survey in which they were asked to give the names of their friends in the high school. We obtained such friendship surveys from 135 students. • We asked students to use the Netvizz application (https://apps. facebook. com/netvizz/) to create their local network of Facebook friendships (i. e. , the use of the application by a student yields the network of Facebook friendship relations between this student's Facebook friends). 17 students gave us access to their local network, from which we removed all users who were not concerned by the data collection. • Each participating student gave us moreover access to the following metadata: gender, class, class in the previous year, whether s/he was smoker, and if s/he was repeating the year.
C. Data analysis The collected data sets have different nature and different resolutions. The most detailed data comes from the SocioPatterns infrastructure and consists in a temporal network of contacts between students, with a temporal resolution of 20 seconds: the nodes of this network represent students, and for each time window of 20 seconds a link is drawn between pairs of students between whom contacts are detected. This temporal network can be aggregated temporally over a given duration, for instance a day or the whole duration of the data collection: weighted daily or global aggregated networks are then obtained. In each aggregated network, a node represents an individual and a weighted link between two nodes i and j represents the fact that the two corresponding individuals have been in contact at least once during the aggregation time window.
e. , w diary ij is not necessarily equal to w diary ji. We will first perform a systematic study of these discrepancies. Then, in order to compare the contact diary data with the sensor data as in  [11] , we will consider a symmetrized version of the network, in which a link exists between i and j if at least one of the two students reported a contact, and the weight of the link is taken as the maximum of w diary ij and w diary ji. From such symmetrized network, we can also aggregate the data by class and obtain contact matrices. The friendship survey yields, as the contact diaries, a directed network between students: indeed, a student i might nominate j as a friend without being nominated by j. As we did not ask students to quantify the intensity of their friendships, we obtain a directed unweighted network of reported friendship relations.
We can also symmetrize the network in order to compare it with the aggregated contact network, and obtain a link density contact matrix between classes. Finally, the data set gathered from the local Facebook friendship networks has a slightly more complex character. Indeed, for each individual who gave us access to his/her local network, we obtain the Facebook friendship links between his/her Facebook friends. However, as only few students provided us with such data, the presence or absence of a Facebook friendship link between many pairs of students remained unknown. To understand this point, let us take the example of two students A and B, with sets of Facebook friends F A and F B. Let us now take two students i and j in the union F A ∪ F B. If both i and j are in F A , or both are in F B , we know if they are Facebook friends or not.
If however i ∈ F A \F B and j ∈ F B \F A , i. e. , i is friend with A but not with B and j is friend with B but not with A, we do not have access to the existence or absence of a friendship link between i and j (see Fig. 1 ). As a result, the data set can not be represented as a network but consists in a list of pairs of students for which we know if they are Facebook friends or not (list of "known-pairs") and a list of pairs for which the presence or absence of such a link is unknown. In particular, we know that i and j are friends on Facebook but not j and k, as i and j are both friends of A and j and k are both friends of B. On the other hand, we do not know if i and k are friends or not: the red dashed line represents the lack of knowledge about the potential existence of this relationship.
B. Contact diaries As mentioned above, the contact diaries yield a directed weighted network of reported contacts between students. Only 120 students filled in a contact diary, yielding a network of reported contacts of 120 nodes and 502 directed, weighted edges among them. Note that we ignore the contacts reported with students who did not fill a diary. Moreover, in the following we consider only students for whom the sensors also registered contact data during the day concerned by the contact diary. This corresponds to 109 students and 416 directed weighted links. 158 contacts were reported by only one student (non-reciprocated links), while the reciprocated links correspond to 129 pairs of students reporting both a contact with each other. Moreover, reciprocated links were sometimes reported with different durations by the two students involved.
Sensors Contact diaries Figure  2. Contact matrices of link densities. We compare here the contact matrices of link densities built from (a) the network of contacts obtained using the sensor data collected on Dec. 5 th and (b) the network of contacts as reported in the contact diaries. We discarded here the data corresponding to the MP*1, PC* and PSI* classes as too few students from these classes filled in a contact diary (1 for MP*1, 0 for PC* and PSI*). The similarity between these two matrices is of 97%. It is clear from the different numbers of links in the two networks that there are discrepancies between sensor-based data and contact diaries, as also found in  [11]. Overall, 70. 4% of the links obtained from contact diaries correspond to contacts registered by the sensors, while only 41.
D. Multiplex network of students' relationships The face-to-face proximity measured by different methods as described in the previous sections represent only one type of relationship between individuals. Other social ties exist, such as friendships and online relationships, and contribute to form a multiplex network in which the nodes represent students and links of different nature coexist between them. These links might a priori be related (one has more contacts with a friend, or becomes friend with someone after meeting him/her often, etc. ) but might also differ substantially: one can be very good friend with someone and meet him/her only rarely because of specific constraints (such as different schedules of classes). It is thus of interest to compare the different layers of this multiplex network and to investigate what information on actual contacts can be gathered from data describing friendship relations.
In particular, friendship survey data might be more reliable than contact diaries: it might be easier to remember the names of one's friends than the contacts occurred during a day and moreover, friendships evolve on slower timescales so that friendship surveys can more easily be gathered on several days without memory biases. As described in the Methods section, we have collected data concerning the friendships and online Facebook links of a number of students, and we compare in this section the properties of the resulting data sets. Figure  4  displays the network of contacts registered by the sensors during the week of data collection, as well as the network of reported friendships and the Facebook links  4  , using the same position for the nodes in the three representations. The friendship survey and Facebook data involve respectively 41% and 48% of the students who participated to the data collection, so that the resulting networks have also clearly much less links than the contact network.
(a) Shortest path length distributions for both networks; (b) Distributions of aggregate durations, as measured by the sensors, for different kinds of links in the contact network: (i) all links, (ii) links i -j for which only one of i or j reported a friendship with the other, (iii) links for which both students reported the friendship, and (iv) links for which no friendship was reported; (c) and (d): Contact matrices of link densities. We compare here the contact matrices of link densities built from (c) the global aggregated network of contacts obtained using the sensor data and (d) the symmetrized network of reported friendships. The similarity between these two matrices is ≈ 95%. network, while 39% (resp. 50%) of the contact network links correspond to friendships. We investigate moreover in Fig.
b. Face-to-face contacts and Facebook links We now perform a similar analysis as in the previous subsection, but focusing on a comparison between contact and Facebook data. As explained in the Methods section, only 17 students provided us with the network of their Facebook friends, so that we could not build the entire network of Facebook relationships between the students but rather work with a list of pairs of students ("known-pairs") for which we know if they have a friendship relation on Facebook or not. The corresponding data set includes 4515 known-pairs involving 156 students, with 1437 Facebook links (and 3078 known pairs of students with no Facebook link). Moreover, these 156 students have 1118 links in the aggregated contact network. 52% of the Facebook links find a corresponding link in the contact network, and 67% of the 1118 links of the contact network are between Facebook friends.
Each pair of students can be characterized by one, two, three or none of these three possible links, forming a multiplex network with three layers  5. We here perform a simple analysis of this multiplex, in which we consider only students who are part of all three corresponding data sets: they represent 82 nodes, 992 links in the contact network (aggregated over the whole study), 326 reported friendship links and 1026 Facebook links. Figure  8 (a) reports in a matrix form the conditional probability to observe a link between two students in a layer of the multiplex, given that the two students are linked in another layer  [43]. The probabilities that a link is present in Facebook or that a contact was registered, given that a friendship has been reported, are very high. The probabilities that a friendship is reported if a contact has been registered or given that a Facebook link exists are much lower.
IV. DISCUSSION In this article, we have presented an analysis of several data sets concerning interactions between students in a high school, collected both through a decentralized sensing platform based on wearable sensors and through diaries and surveys. The fact that these data sets were collected at the same time and in the same population allowed us to compare them and to quantify the overlap and complementarity of data sets of different nature. Data collected thanks to the wearable sensors developed by the SocioPatterns collaboration yielded a temporally resolved network of contacts between the 327 participating students. This network had features similar to data collected in the same context in previous years  [24]  and in other contexts  [37, 38] , with heterogeneous contact durations and inter-contact times, repeated daily activity patterns, and an aggregated contact network structure shaped by the division of students in classes.
As the present data and the ones of  [11]  concern similar contexts (high schools), it would be of great interest to confirm the robustness of these findings in other contexts. In cases such that collecting diary-based is easier than using wearable sensors, such knowledge on the differences between these two kinds of data could help obtain a more complete picture of the actual contact patterns. Friendship relations represent another kind of data for which surveys are commonly used; the resulting data sets are a priori less prone to biases due to imperfect memory, and might also be easier to collect in a given population than contact diaries. Surveys might indeed be given and collected on a less constrained time frame than contact diaries, as friendships evolve on longer time scales than contacts.