dataesr/scientific-paragraphs-categorization huggingface dataset:
Real datasets The following public datasets were used for the experiments: • Childhood Blood Lead Surveillance (CBLS)  [27]  dataset, which contains data on blood levels of lead in children from between 1995 and 2015, from several U. S. states and local health departments. • COVID-19 in Spain (COVID esp )  [28]  dataset, which contains data on daily increments split by age and gender of the COVID-19 pandemic in Spain. • Suicides in India (SI)  [29]  dataset, which contains annual suicide records of all states of India with various parameters from between 2001 and 2012. • Mavlab (MAV) dataset  [30] , which contains activities related to daily living, was collected using the MavLab testbed during March and April of 2003. This dataset captured an inhabitant's interactions with an intelligent home through sensors placed in different rooms.
• COVID-19 by country -Daily update (COVID all )  [31]  dataset, which contains data on daily increments of the COVID-19 pandemy at the country level. • Human Activity Recognition (HAR) dataset  [32]  built from the recordings of 30 subjects performing activities of daily living while carrying a waist-mounted smartphone with embedded inertial sensors. This dataset can be found in the UCI Machine Learning Repository  [33]. • DEAP dataset (DEAP)  [34, 35]  which consists of information about 32 volunteers when watching 40 music videos. Each video is labelled with affective tags. For each video of each volunteer, there are 63 recordings seconds, including EEG (32 channels) and up to 8 complementary physiological signals (electrooculography, electromyography, galvanic skin response, respiration belt, plethysmography and temperature).
Experimental scenarios Four types of experimental configurations were defined: 1. Performance in the function of the algorithm parameters. This experimental configuration is directed to test the performance of the algorithm based on its parameters, such as | Σ|, δ and max_gap, and compare the results with the state of the art approaches (i. e. Baseline). The correctness of the outputs is analysed by using the Baseline approach with a perfect scenario (PM set to 0). In so doing, the number of time intervals found by the Baseline approach and the TA4L should be the same. On the other hand, when the PM increases, the number of intervals should be reduced in TA4L. Moreover, the efficiency of the data structures proposed in TA4L is tested under the different parallelism strategies. This experiment has been applied to both synthetic and real datasets.
• To analyse the sensitivity to the size of δ, | Σ| was set to the maximum value 20, and PM was set to 1. The hypothesis is: as the smaller is the δ, the larger the time and memory consumption. • To analyse the sensitivity to the size of | Σ| , PM and δ was set to 1. The hypothesis is: as larger is |Σ|, more time and memory consumption will be required. • To analyse the sensitivity to the length of max_gap, | Σ| was set to 3. Since the effect of the max_gap depends on the number of nulls in the database, PM has been tested with 1%, 25%, 50%, 75% and 99%. The hypothesis is: as smaller is the max_gap, the larger the number of time intervals found, and consequently larger the memory and time consumption.
Performance-based on the discretisation method. To analyse the sensitivity to discretisation method, we compared the symbol assignation from SAX, EWD and KBTA methods in the Baseline and TA4L (TA4L_sl strategy) algorithms in the function of q, n sid and |Σ| parameters. Note that when we talk about applying SAX to TA4L, we are referring only to the task of assigning the symbol (as explained in Section 4. 3). The hypothesis is: there should not be a significant difference between the runtimes of the discretisation methods (since all methods assign the symbol based on the range where the value falls), but regarding the memory consumption, SAX will be the method that consumes more memory, due to the storage of statistical tables, and the breakpoints determination. Although note that the KBTA will not go far from SAX either, as it also needs space to keep the rules set by the experts; and to simulate this, logical rules have been applied to each variable of datasets and following a common sense.
3. Performance in the function of the dataset characteristics. This other configuration aims to test the algorithm's performance based on the dataset characteristics, such as PM, q, n and n sid. This experiment has been applied to synthetic datasets. In particular, the PM analysis aims to validate the approach followed in TA4L in which time intervals are fragmented according to known data. • To analyse the n sensitivity, |Σ| was set to 20; q, PM and δ to 1. The discretisation method was SAX. The hypothesis is that: as more n, more time and memory will need the algorithm. • To analyse the n sid sensitivity, |Σ| was set to 20; q, PM and δ to 1. The discretisation method was SAX. The hypothesis is: as more n sid , more time and memory will need the algorithm. • To analyse the q sensitivity, |Σ| was set to 20, δ and PM to 1.
4. Performance in terms of classification. This configuration aims to analyse how TA4L impacts classification effectiveness because, as mentioned in Section 4. 2, TA4L generates sequences different from previous approaches (SAX). Since converting LSTISs to vectors of pattern characteristics to classify is a complex process, which also changes depending on the dataset, we decided to reuse the work we have in  [36]  prepared for the DEAP data. The idea is to apply the method described in Section 3. B of  [36] , using SAX and TA4L and then comparing the results. In this work, contrary to the work in  [36] , the minimum support of the vertTIRP algorithm was fixed to 0. 7, and the Entropy threshold to 0. 6 for all the labels. To cause the sequences found by the two algorithms to be different, 10 per cent of the data has been randomly deleted from the DEAP dataset.
Algorithm behaviour In this section, we analyse experimentally the behaviour of algorithms based on input parameters, such as Σ, δ and max_gap. Fig. 8  shows the results obtained with the synthetic datasets, and Figs. 9 and 10 for the real ones. We can observe that in the case of both synthetic and real datasets, time and memory increase with increasing vocabulary size and with decreasing interval size. In real datasets (Figs. 9 and 10) the increase in time and memory is most noticeable with large datasets, such as HAR and COVID_all datasets. Multithread version by sequence (TA4L_sl_paral_seq and TA4L_se_paral_seq) consumes less time for both parameters (δ and |Σ|). The parallel version by variables is the structure that consumes slightly more memory than the rest of the TA4L versions. In general, in all real datasets, TA4L is significantly faster than Baseline for all configurations, and in terms of memory consumption, TA4L is more efficient than Baseline for relatively large datasets, such as HAR, COVID_all and IS datasets.
Synthetic datasets In the case of synthetic datasets, in general for small values of q, n, and n sid SAX and KBTA consume significantly less time and memory than the EWD approach, while for large values of q, n, and n sid EWD consumes less memory and runtime than SAX (see Figs. [13] [14] [15]. The baseline is significantly slower than TA4L (observe that time scales of plots in the figures are different in TA4L from Baseline, due to the big difference in the results achieved), and it also consumes more memory than the TA4L approach.
Real datasets In the case of real datasets, TA4L generally consumes less time and memory than Baseline (observe the differences on the time scales of plots in the figures as happened with the synthetic datasets). Nevertheless, in TA4L implementations, the behaviour of the discretisation approach requires a deepening analysis. For example, with respect to memory consumption, in the case of the HAR dataset, for large q, n, and n sid values (Figs. [16] [17] [18] , Baseline_KBTA is the one that consumes significantly more memory than Baseline_EWD. Whereas in the case of the COVID_all dataset, for large values of q, n and n sid , Baseline_SAX consumes significantly more memory than Baseline_EWD. In the case of large datasets (such as HAR and COVID_all datasets), for small values of q, n and n sid , TA4L_SAX is the least memory consuming than TA4L_EWD and TA4L_KBTA methods.
Regarding the time consumption (Figs. [19] [20] [21] , in the case of the COVID_all dataset, Baseline_EWD is significantly slower than Baseline_KBTA, especially for n sid experiments. Regarding the sensitivity to discretisation in TA4L, in the case of the HAR dataset, in both n and n sid experiments, SAX is significantly slower than the EWD and KBTA methods.
Performance according to the dataset characteristics In this section, we analyse experimentally the cost of algorithms based on dataset characteristics, such as q, n and n sid. In the case of q (Figs. 22(d)  and 22(c)), multithread version by sequence consumes less time for both data structures, which is around 3. 8 s for q 501 (worst case). The state of the art approach is that consumes more time (732. 19 s for q 501). However, the multithread version of TA4L by variables also presents a great runtime concerning other versions of TA4L, which is 35. 22 s for q 500. Therefore, if we have many sequences and few variables, it is not useful to parallelise by variables. In the case of n (Figs. the longest one (23. 75 s for n 501 and 321. 33 for PM 1). And regarding memory consumption, the state of the art approach consumes more memory.
Discussion One of the first insights from the experimental analysis regarding the dataset characteristics is that it is observed that there is a decrease in time, memory and number of time intervals constructed with an increase in PM values, since as larger is PM values, fewer data per interval, and therefore, fewer time intervals constructed, memory usage and runtime. This also applies to the state-of-the-art algorithm, in which, first, each value is discretised and then concatenated. Another point to highlight regarding missing values is that in the presence of missing values, previous algorithms (SAX) generate the sequence AB for the same series (Fig. 4 ) generates the sequence AB, while TA4L ABCB. That means that the patterns found from this input by any TIRP mining algorithm will differ.
This behaviour is reflected in all versions of datasets and algorithms. Concerning max_gap, as smaller is the max_gap parameter, the algorithm performs fewer concatenations between the adjacent intervals, and therefore more intervals are generated, and consequently, more time and memory consumption is required. As expected, this effect is less noticeable if the dataset has a very small percentage of missing values (there are no gaps) or a very large number of missing values (low number of time intervals). In case of knowing the problem, the max_gap is the maximum time allowed between two consecutive events so that these two successive events can be considered one single event. The correct choice of the size of the sigma and the max_gap parameter without expert knowledge could be made in a similar way to the selection of the delta.
In terms of discretisation, in the case of synthetic datasets, for large q, n, and n sid values EWD is the more memory efficient method, while for small values of q, n and n sid , SAX is that consumes less time nor memory. In the case of real datasets, we were able to verify that, in general, there is no significant difference between the methods in terms of time and memory consumption. However, concerning memory consumption, in the case of large datasets (such as HAR or COVID_all datasets), for large q, n, and n sid values, EWD is the more memory efficient method, while for small values of q, n and n sid , SAX is the least memory consuming method. The cost is greater for numeric variables and a sorted-list like structure. Sorting all at once has a lot more design freedom than maintaining the ordering incrementally since an incremental update has to maintain a complete order at all times, which represents more time and memory consumption.
And regarding multithreading, if we have a sudden processor and RAM, the best option is to parallelise the TA4L algorithm per sequence instead of per variable. And in terms of classification, TA4L does not always have to be better than the Baseline in downstream tasks. TA4L is suitable for datasets where situations like the one described in Fig. 4  occur. A priori, it could be datasets with missing values, and where the duration of events plays an essential role in the prediction.
Moreover, the size of the time intervals is managed by a given max_gap constraint that enables controlling the maximum separation of two points to be considered part of the same event (i. e. symbol), a property of particular interest in different application domains. Finally, TA4L considers a sorting list data structure together with several parallelism strategies (variable and sequence multithreads) that make the whole pre-processing efficient. The experimentation carried out in different synthetic and real datasets show that the TA4L cost varies in function of the data structure used and the type of the variables in the dataset (numeric or discrete). In general, TA4L provides a noticeable reduction in time and memory than the state of the art algorithm. Multithread version by variables is useless in the case the dataset has few variables and many sequences.
Acknowledgements This project received joint funding from  ERDF , the  Spanish Ministry of the Economy, Industry and Competitiveness (MINECO)  and the  National Agency for Research , under grant no. RTC 2017-6071-1  (SERAS). The work was carried out with support from the  Generalitat de Catalunya   2017 SGR 1551 , a predoctoral grant from the  University of Girona  (grants for researchers in training/IFUdG2017) and a mobility grant (additional support for the mobility of UdG researchers/MOB2019).
F1-F7 contain multiple global optima, while F8 and F9 have only one global optimum and multiple local optima of different qualities. In F10, there are several regions in which all points are local optimal solutions. The detailed information and the surface plots of the benchmark functions can be found in Appendix A. In the following experiments, if the deviation from the objective function value of an obtained optimal solution to the objective function value of a real optimal solution is below ε (level of accuracy) and the distance between these two solutions is less than the radius in Table  2  (level of precision), this real optimal solution is considered as being found. For the proposed method, a solution is considered as an obtained optimal solution only when it is stored in S opt (]the datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request).
Effect of the New Budget Size ∆ The results of the ANOVA in Figure  3  show that a small ∆ value is preferred in Problem F6  (3, 3, 3)  in terms of the total budget size, because it allows more budgets to be allocated after obtaining more information and generating more precise regions. Nevertheless, according to the discussion in Section 2. 3, a small ∆ value may increase the computational time of the proposed method. In Figure  6 , the average total budget sizes and the corresponding computational times required to obtain all optima in Problem F6  (3, 3, 3)  with different ∆ values are presented. The confidence intervals with confidence level 95% are also plotted. The experiments are executed in Matlab R2020a on a computer (Intel(R) Core(TM) i7-7700U CPU @ 3. 6 GHz and 16 GB of RAM). Budget size Computational time Figure  6.
Computational Time In this section, the PAR-MMO method, in which local search is not included, is applied to Problem F4 (2,••• ,2) (five dimensions) with ε = 1 × 10 -4. A total of 20 independent replications are executed in Matlab R2020a on a computer (Intel(R) Core(TM) i7-7700U CPU @ 3. 6 GHz and 16 GB of RAM). Figure  10  shows how the average computational time changes as the total budget size increases. It can be found that the computational time increases quadratically as discussed in Section 2. 3 (R 2 = 100. 0% for the quadratic regression). Although the proposed method will become time-consuming when the total budget size is very large, it is still efficient in the studied case (on average, 121 s for a budget of size 1 × 10 6 ) and the computational time climbs slowly before the total budget size reaches one million.
Application The optimal control problem of a nonlinear stirred tank reactor  [72]  is considered in this section. The chemical process can be modeled by the following differential equations: ẋ1 = -(2 + u)(x 1 + 0. 25) + (x 2 + 0. 5) exp 25x 1 x 1 + 2 , ( 8 ) ẋ2 = 0. 5 -x 2 -(x 2 + 0. 5) exp 25x 1 x 1 + 2 , ( 9 ) ẋ3 = x 2 1 + x 2 2 + 0. 1u 2. ( 10 ) where u(t), u(t) ∈ [0. 0, 5. 0] is the flow rate of the cooling fluid, x 1 (t) is the dimensionless steady temperature, x 2 (t) is the deviation from dimensionless steady concentration, and the interval of integration is 0 ≤ t ≤ 0. 78. The performance index to minimize is f = x 3 (0. 78), which can be evaluated using ode45 in Matlab, and the initial condition is x 1 (0) = 0. 09, x 2 (0) = 0. 09, x 3 (0) = 0. The problem has a global optimum x 3 (0.
Related Works Most approaches addressing the Relation Extraction task use deep learning methods. Historically, convolutional neural networks  [19]  and LSTM  [21]  were used first, then were replaced by graph convolution networks methods  [23] , which allow to take into account the syntactic structure of sentences. Currently, the approaches that give the best results for the RE task use pre-trained language models such as BERT  [4]  and its variants  [11, 22]. However, the performance of those approaches (with an F-score between 70 and 75% on the TACRED benchmark  [24] ) are still too low to allow a full automation. In addition, those fully statistical approaches lack of explanations for their predictions, which limits the possibilities of introducing human control in the process to improve reliability.
Experiments and Results In this section, we present the different experiments made with our relation extraction system and the subsequent results. Those experiments can be divided in three parts: 1) the LUKE-based Relation Detection module, 2) the Concepts of Neighbours-based Relation Classification module, and 3) the whole system. All experiments were made on the TACRED dataset  [24] , one of the most used dataset for Relation Extraction. This dataset is made of 106,264 examples, split into a training corpus (68,124 examples), a development corpus  (22, 631  examples) and a test corpus  (15,509 examples). Each example of this dataset is a sentence with two entity mentions (a subject and an object), each mention being typed among 23 possible types, and annotated with a relation type among 41 effective classes plus a no_relation class representing the absence of relation between the subject and the object.
Relation Classification We now evaluate our Concepts of Neighbours-based module individually on the Relation Classification task. Experiment Design These experiments are made on the positive examples of TACRED, i. e. the examples that have an annotation other than no_relation. As our method does not have any use of a development corpus, we merge this corpus with the training one. We finally obtain a dataset composed of 18,446 training examples and 3,325 test examples. The quality measure usually used on TACRED is the micro-averaged F-score. However, as there is no negative class on this task, this measure does not make sense, and therefore we use accuracy. In these experiments, as we work on a subset of TACRED we cannot compare this approach directly to other existing methods. Therefore, we compare our approach to a basic baseline in the RECENT paradigm.
First it can be observed that the baseline has an accuracy of 80. 4%, which is particularly high, which means that the dataset leaves little space for progress. Then, it can be read that for any timeout, the proposed approach has a better accuracy than the baseline, surpassing it by 2. 2 points for a timeout of over 300s. [22]  72. 7 BERT-LSTM-Base  [20]  67. 8 Ours 66. 9 C-GCN  [23]  66. 4 GCN  [23]  64. 0 In addition, this table clearly shows a saturation phenomenon: there is an important gain when timeout gets from 10s to 120s, gain that is far smaller from 120s to 1200s. It can be intuited that this comes from the fact that most concepts are computed before 120 seconds, and only a few concepts are added after 120s. This also can be seen in the proportion of examples for which the full set of Concepts of Neighbours is computed: of less than 30% for a timeout of 10s, it rises to over 80% for a timeout of 120s and to over 99% for a timeout of over 600s.
Relation Extraction Now that we have shown that our Concepts of Neighbours-based method is a valid approach for relation classification and that we have chosen a deep learning relation detection module, both can be assembled to form a full relation extraction method. In this subsection we present the experimental process to evaluate this method, as well as both quantitative and qualitative results. Experiment Design We evaluate our two-step approach on the full TACRED dataset in order to compare it to previous approaches. To do so, according to the structure presented in Figure  3 , we process the test examples of TACRED with luke-base, and obtain examples classified as positive or negative. Then, each example classified as positive is processed by our Concept-of-Neighbours module for relation classification.
Our theory was quantitatively confirmed by extensive tests on in silico lattice proteins for which the ground-truth fitness is known, and on mutagenesis datasets of 7 proteins families we have analyzed. Based on the results above, we then proposed a "focusing" procedure to optimally select the best subset of sequences from a multi-sequence alignment, and tested it on the 7 mutagenesis experiments. With this procedure, the least powerful, independent sites model, showed performances higher than fully connected graphical models trained on the same data for 4 out of 7 studied protein families, and comparable performances for the remaining ones. An important finding of the present work is the so-called bias factor J0, which relates the squared bias µ 2 to the mean Hamming distance D of the sequence data to the wild-type sequence: µ 2 ƒ J0 D.
Materials and Methods Multiple sequence alignments. Proteins families and the corresponding alignments were taken from  (37). The alignment procedure of EVmutation (https://github. com/debbiemarkslab/EVmutation) is based on a query against the UniRef100 database of nonredundant protein sequences (release 11/2015) (63) from the wild-type sequence, using the profile HMM homology search tool jackhmmer  (64)  and choosing a default score in the alignment depth of about 0. 5 bits/residue; the threshold was adjusted if the alignment had not enough coverage or number of sequences  (37). Redundant sequences were removed from the alignments, as well as sites with more than 50% of gaps in the alignment. The list of families and wild type, the sequence length, the number of sequences, and the reference to the mutational scans are given in Table  1.
ACKNOWLEDGMENTS. The authors are grateful to J. Tubiana and M. Molari for insightful discussions, and to J. Fernandez de Cossio Diaz and E. Mauri for a careful reading of the manuscript. Change in log probability following a mutation. We assume that the protein family under consideration is defined by a distribution P over the space of sequences s. Informally speaking, good sequences s, i. e. corresponding to functional proteins have large values of P , while bad sequences correspond to very low values; We further assume that the landscape associated to P includes local biases acting on residues as well as pairwise epistatic interactions. More precisely, we have P (s) = e ≠F (s) Z , [ 22 ] where Z is a normalization constant, and the "statistical energy" F (s) reads F (s) = ≠ ÿ i h i (s i ) ≠ ÿ i<j J ij (s i , s j ).