dataesr/scientific-paragraphs-categorization huggingface dataset:
Second, some of the decision variables in the MILP formulation (6)(11) are xed in advanced, and therefore eliminated. The pseudocode for this reduction strategy can be found in Figure  3. Using the grid corresponding to parameters C and a 1 , we rst build the DILSVM (1) classier yielding optimal parameters values C (1) and a (1) 1. We then build the DILSVM (2) classier using the MILP formulation (6)(11) for K = 2, with C = C (1) , a 1 = a (1) 1 and the values of a 2 in the grid. In addition, we reduce the number of zeroone decision variables in the MILP formulation. In the current version of the xing strategy, feature j will have a strong positive rating in the DILSVM (2) classier if that was the case in the DILSVM (1) classier, and similarly for a strong negative rating. For the rest of features, β + jk and β - jk , k = 1, 2, must be optimized.
We use the optimization engine CPLEX v12. 4, (CPLEX 2012), for solving all optimization problems. We have set the time limit to 300 seconds, which is enough for most of the optimization problems we have solved. For the remaining ones, the classiers derived in this way are of heuristic nature. The rest of this section is structured as follows. The tuning procedure used to choose the tradeo parameter C and the rating levels a k , k = 1,. , K, is given in Section 4. 1. The datasets used to compare the models are described in Section 4. 2. The computational results are presented in Sections 4. 3 and 4. 4.
Parameters Setting As customary in Supervised Classication, the construction of the DILSVM classier calls for tuning some parameters, namely the tradeo parameter C as well as the rating levels. The tuning procedure works as follows,  (Carrizosa et al. 2014, Carrizosa and  Romero Morales 2013). The dataset is split into three sets, the so-called training, testing and validation sets. For each vector of parameters, the DILSVM is run on the training set, which has size n. The dierent classiers built in this way are compared according to their accuracy on the testing set. The vector of parameters with the highest accuracy on the testing set is chosen, and its accuracy and sparsity on the validation set are reported, see Figure  4. Following the usual approach, for the DILSVM (1) , parameters C and a 1 are tuned by inspecting a grid of the form C n ∈ {10 -6 ,.
Datasets The performance, in terms of accuracy and sparsity, of our model is illustrated using 10 real-life large datasets: calhous from the StatLib repository  (Vlachos and Meyer 1989) , ijcnn1 and cod-rna from the LIBSVM repository (Chang and Lin 2011), and the remaining ones from the UCI repository  (Blake and Merz 1998). Datasets containing categorical features have been transformed splitting the categories into binary features (mushroom, german). Regression datasets are transformed into 2-class datasets using the median (abalone, calhous), and multi-class datasets are transformed into 2-class, treating the largest class as class +1 and the remaining classes as class -1 (careval, shuttle). Please note the features are normalized and therefore we can safely assume that there is one single rating scale for all the features, and thus for the scores.
A description of these datasets can be found in Table  1 , whose rst four columns report the dataset name, full name given in the repository, number of features (d) and total size of the dataset (|Ω|). The size of the training set (n) is set as the closest 5 • 10 s-1 Step 1. For each value of the parameter vector (C, a 1 ,. , a K ) in the grid, (i) Obtain the corresponding classifier K k=1 a k (α + k -α - k ) x + b using the training set. (ii) Let π test (C, a 1 ,. , a K ) be the accuracy of K k=1 a k (α + k -α - k ) x + b in the testing set. Step 2. Let ( C, ā1 ,. , āK ) ∈ arg max (C,a 1 ,. ,a K ) π test (C, a 1 ,. , a K ) and K k=1 āk (ᾱ + k - ᾱ- k ) x + b be the corresponding classifier using the training set. (i) Report the performance of K k=1 āk (ᾱ + k -ᾱ- k ) x+b in the validation set.
For each dataset, we report the mean validation accuracy across the ten instances, as well as the standard deviation and the median. The same statistics are presented for the sparsity. Below we discuss We start with the analysis of the mean accuracy, and show that the DILSVM is competitive against the SVM. We rst compare with the mean accuracy of the SVM. For three datasets, adult, german and careval, the DILSVM (1) outperforms the SVM by 5. 54, 2. 36 and 1. 16 p. p. , respectively. For three datasets, mushroom, gamma and shuttle, the DILSVM (1) and the SVM are comparable. In three datasets, ijcnn1, abalone and calhous, the SVM outperforms the DILSVM (1) by 1. 20, 1. 24 and 1. 61 p. p. , respectively. In cod-rna, the DILSVM (1) is clearly inecient compared to the SVM. For datasets such as cod-rna, the DILSVM (1) is too restrictive, and the accuracy may benet from the additional exibility built in by the DILSVM (2).
An improvement of more than 1 p. p. can be observed from the DILSVM (1) to the DILSVM (2) in three datasets. The rst one is careval, for which the DILSVM (1) is already better than the SVM, and the improvement on the SVM increases from 1. 66 to 3. 66 p. p. The second one is cod-rna, where now the dierence in mean accuracy between the SVM and the DILSVM has been reduced in eight datasets, where the increase in mean accuracy achieved ranges between 2. 80 and 18. 41 p. p. , while the mean accuracy of both methods is basically the same for mushroom and ijcnn1. We now focus on the second criterion, and show that the DILSVM is the best in terms of mean sparsity. Indeed, for each dataset except cod-rna, the best model is the DILSVM (1) , followed by the DILSVM (2) , and the SVM being the worse. In cod-rna,.
25%, while for the DILSVM (2) the mean sparsity is always above 35% except for calhous, with 17. 50%. Thus, except for calhous, at least half of the features are irrelevant in the DILSVM (1) , while this becomes at least one third in the DILSVM (2). The results are even more encouraging for ve of these datasets, where the mean sparsity of the DILSVM (1) is at least 70%, while the mean sparsity of the DILSVM (2) is at least 50%. In addition, in the mushroom dataset, both the DILSVM (1) and the DILSVM (2) report a 100% accuracy with a 91. 60% mean sparsity. In summary, the model we propose, the DILSVM, is competitive against the SVM in terms of accuracy, being substantially sparser than the latter. When faced with the choice between one or two rating levels, we should observe that the DILSVM (1) is, in general, more appealing: it has comparable accuracies to those of the DILSVM (2) in seven datasets, while the DILSVM (1) is at least as sparser as the DILSVM (2).
DILSVM (2) boosts the accuracy to a competitive level, at the cost of a lower sparsity. The MILP approach to build the DILSVM yields attractive results in terms of both criteria against both benchmarks. That comes with a high computational cost though, where an MILP problem needs to be solved for each parameter vector. This can be illustrated by the median ratio across the ten datasets between the building time of the DILSVM and that of the SVM, which is equal to 210. 91 for K = 1 and 429. 60 for K = 2. When comparing with the xed DILSVM (1) , the median ratio is 361. 18 for K = 1 and 1054. 53 for K = 2. We should emphasize that this high computational eort only aects the phase of building the classier (o-line). The evaluation phase (on-line), when new objects are to be classied, is even faster than with the SVM, since, due to the high sparsity of the classier, fewer terms are computed.
For K = 1, the RSVM strategy is clearly inecient against the SVM, where the loss in accuracy is at least 1 p. p. in all datasets, while this becomes 47. 97 p. p. in mushroom and 22. 81 p. p. in cod-rna. The improvement of K = 2 is dramatic for mushroom, where the RSVM strategy becomes comparable to the SVM, also in ijcnn1, while in the remaining eight datasets this strategy is still not competitive against the SVM. In Strategy 2, the RR strategy, we reduce the computational cost of solving an MILP problem for each parameter vector. Instead, the RR strategy solves the LP relaxation of the DILSVM (K) and applies a randomized rounding to its optimal solution. Below, we show that the RR strategy yields mean accuracies close to those of the SVM. For K = 1, the loss in accuracy of the RR strategy compared to the SVM ranges from 0.
51 to 17. 73 p. p. , and therefore, using this criterion, the RR strategy is dominated by the SVM. For K = 2, the RR strategy outperforms the SVM in adult by 4. 15 p. p. and in careval by 3. 71 p. p. , for three datasets both methods are comparable, while in the remaining ve the losses in accuracy (in p. p. ) with respect to the SVM are 1. 12 (ijcnn1), 1. 87 (shuttle), 1. 88 (abalone), 3. 12 (calhous) and 13. 76 (cod-rna). Except for cod-rna, we can conclude that the RR strategy with K = 2 and the SVM are comparable in terms of accuracy. In Strategy 3, the xing strategy, we rst build the DILSVM (1) , and use its classier to reduce both the number of parameter vectors to be inspected as well as the number of zeroone decision variables in the MILP. For each dataset, this strategy reduces the number of MILP problems to be solved from 286 (full grid inspected) to 2 (only grid for a 2 inspected), after solving the 143 MILPs associated with the DILSVM (1).
Roughly speaking, this halves the number of MILPs to be solved. We now analyze the mean accuracy, and show that the xing strategy yields results close to those of the SVM. Compared to the SVM, the xing strategy outperforms the SVM in adult by 4. 19 p. p. and in careval by 1. 79 p. p. , for three datasets both methods are comparable, while in the remaining ve the SVM performs better. If we ignore cod-rna with a 12. 54 p. p. loss, one can see that the improvement of the SVM for the remaining four datasets is below 1. 36 p. p. Except for cod-rna, we can conclude that the xing strategy has a comparable behavior to the SVM. In terms of mean sparsity, the strategies clearly dominate the SVM, as the latter is always fully dense (except for ijcnn1). We now take a closer look at the mean sparsity of each strategy.
For K = 1, the mean sparsity of the RSVM strategy is at least 50% for all datasets with eight datasets above 70%, while for K = 2 the mean sparsity is above 25% for all datasets with seven above 50%. For K = 1, the mean sparsity of the RR strategy is at least 65% for all datasets with eight datasets above 70%, while for K = 2 the mean sparsity is above 15% for all datasets with eight datasets above 50%. Finally, the sparsity of the xing strategy is at least 20% for all datasets with ve datasets above 50%. We now illustrate the reduction in building time achieved by the strategies. As before, we measure the ratio between the building time of a given strategy and that of the SVM, and report the median ratio across the ten datasets. For the RSVM strategy, the rounding time is negligible for small values of K.
Thus, for K = 1, 2, the ratio between the building time of the RSVM and the SVM is roughly equal to 1 for each dataset. For the RR strategy, when comparing with the SVM, the median ratio is equal to 10. 32 for K = 1 and 34. 41 for K = 2. Finally, the xing strategy is designed to reduce the building time when K = 2 using the DILSVM (1) classier, and the median ratios are very similar to the ones reported for the MILP approach with K = 1, namely 211. 60 against the SVM. In conclusion, we have presented three strategies of very diverse nature that are able to reduce the building time of the DILSVM classier. When compared to the SVM, the RR and the xing strategies are competitive in terms of mean accuracy, but less so is the RSVM, while the dominance of the three strategies in terms of sparsity is overwhelming.
The gain in visualization and sparsity achieved by the DILSVM is made without paying any price in accuracy. As our computational experience shows, the DILSVM is competitive against the SVM in terms of accuracy. Our classier is much harder to obtain than the SVM, since an MILP problem is to be solved for each parameter vector, and we have illustrated that parameter tuning is crucial if we do not want to compromise accuracy. In terms of the number of parameters, there are now one (K = 1) or two (K = 2) more parameters than in the standard SVM to tune. In order to alleviate the computational burden, we propose three reduction strategies. Our computational tests show that we are able to preserve an accuracy comparable to the SVM and signicantly better sparsities. This means that, at the expense of an increase in o-line computational cost, the DILSVM is able to extract easy-to-interpret information from datasets without sacricing classication accuracy.
from 15. 69 to 3. 22 p. p. The third one is calhous, where the SVM is better than theDILSVM(1) , but now the DILSVM (2) has a comparable performance to the SVM. Across all datasets, the DILSVM (2) outperforms the SVM in three datasets, they are comparable in ve datasets, while the SVM outperforms the DILSVM (2) in two datasets. Thus, the DILSVM (2) is competitive against the SVM in terms of mean accuracy. We now show the relevance of tuning parameters by comparing mean accuracy results between the DILSVM (1) and the DILSVM(1) with xed parameters C = ∞ and a 1 = 1. Using this criterion, the xed DILSVM (1) is not competitive and is outperformed by the DILSVM (1).
Results We have developed a new Reference Transcript Dataset for Arabidopsis (AtRTD2) for RNAseq analysis containing over 82k non-redundant transcripts, whereby 74,194 transcripts originate from 27,667 protein-coding genes. A total of 13,524 protein-coding genes have at least one alternatively spliced transcript in AtRTD2 such that about 60% of the 22,453 protein-coding, intron-containing genes in Arabidopsis undergo alternative splicing. More than 600 putative U12 introns were identified in more than 2,000 transc ripts. AtRTD2 was generated from transcript assemblies of ca. 8. 5 billion pairs of reads from 285 RNA-seq data sets obtained from 129 RNA-seq libraries and merged along with the previous version, AtRTD, and Araport11 transcript assemblies. AtRTD2 increases the diversity of transcripts and through application of stringent filters represents the most extensive and accurate transcript collection for Arabidopsis to date.
We have demonstrated a generally good correlation of alternative splicing ratios from RNA-seq data analysed by Salmon and experimental data from high resolution RT-PCR. However, we have observed inaccurate quantification of transcript isoforms for genes with multiple transcripts which have variation in the lengths of their UTRs. This variation is not effectively corrected in RNA-seq analysis programmes and will therefore impact RNA-seq analyses generally. To address this, we have tested different genome-wide modifications of AtRTD2 to improve transcript quantification and alternative splicing analysis. As a result, we release AtRTD2-QUASI specifically for use in Quantification of Alternatively Spliced Isoforms and demonstrate that it out-performs other available transcriptomes for RNA-seq analysis.
gene expression studies. Experimental validation of alternative splicing changes identified inaccuracies in transcript quantification due to UTR length variation. To solve this problem, we also release a modified reference transcriptome, AtRTD2-QUASI for quantification of transcript isoforms, which shows high correlation with experimental data. Key words: RNA-seq; Alternative splicing; Arabidopsis thaliana; Cufflinks; StringTie; Salmon; Kallisto; high resolution RT-PCR.
With an available genome sequence the quantification of transcripts usually involves mapping of RNA-seq reads to the genomic reference and then construction of transcript isoforms as the first steps. Transcript expression levels are then inferred based on the number of aligned reads. Current analysis tools that identify and quantify transcripts from reads mapped to a genome include TopHat2/Cufflinks  [29] [30] [31] [32] , RSEM  [33, 34] , eXpress  [35] , Bayesembler  [36]  and StringTie  [37]. However, the determination of transcripts from short reads is inaccurate and generates incorrect, mis-assembled transcripts and misses bona fide transcripts that impacts the accuracy of transcript quantification  [37] [38] [39]. For example, the assembly functions of two of the best performing programmes, Cufflinks and StringTie, generate 35-50% false positives, and quantification based on these transcript annotations often leads to inaccurate results  [37].
In particular, for genes with multiple isoforms, the accuracy of transcript inference and quantification is poor  [39]. Therefore, assembled transcripts and their quantification require rigorous experimental validation. Rapid and accurate quantification of known transcripts can be achieved using programmes such as Sailfish and Salmon  [40, 41]  or Kallisto  [42] , which use lightweight algorithms to quantify the abundance of previously annotated RNA isoforms  [40]. To analyse AS data in Arabidopsis thaliana we previously developed an Arabidopsis Reference Transcript Dataset (AtRTD) (now referred to as AtRTD1), and showed high correlation of AS using individual transcript abundances from Sailfish and Salmon and experimental data from high resolution (HR) RT-PCR  [43]. AtRTD1 was made up of transcripts identified during RNA-seq analysis of a normalised library from 10-day old seedlings and flower tissue  [14]  and from transcripts in The Arabidopsis Information Resource version 10 (TAIR10)  [44].
Here, we describe AtRTD2 where we have increased the number and diversity of Arabidopsis transcripts (over 82k unique transcripts from around 34k genes) by assembling ca. 8. 5 billion 100 bp pairs of reads from RNA-seq libraries of a diverse range of plant material and incorporating transcripts from Araport11 Pre-release  3 [45, 46] , hereafter Araport11. AtRTD2 contains higher confidence transcripts through the application of stringent filtering and quality control measures based on our knowledge of plant intron and splicing characteristics to reduce the number of probable false p ositive transcripts and other factors which could perturb quantification. Good correlation of quantification of AS using 127 AS events in 62 genes was observed between Salmon/AtRTD2 data and HR RT-PCR. However, for genes with transcripts which have variation in the lengths of their 5' and 3' untranslated regions (UTRs), quantification of transcript isoforms was often inaccurate.
Generation of AtRTD2 The main objective of generating AtRTD2 was to provide a new transcriptome for Arabidopsis for gene expression and, in particular, alternative splicing analysis. Wellannotated genomes and transcriptomes generally improve quantification of transcript isoforms, especially using RNA-seq analyses. We therefore wished to produce a reference transcript dataset that was as comprehensive and diverse as possible in terms of AS isoforms and contained the highest quality of supported transcripts. The quality of transcripts was achieved by applying stringent criteria to minimise the number of false, mis-assembled and poorly supported transcripts, and transcript fragments. To provide diversity of transcripts, two different extensive datasets of ca. 8. 5 billion pairs of reads obtained from 285 RNA-seq runs of 129 libraries (Methods and Supplemental Methods: Table  S1 ) were assembled into transcripts and merged with our previous AtRTD1  [43]  and the recently released Araport11 transcript set  [45, 46]  (Figure  1 ).
The two datasets were analysed separately in the different research groups and used different mapping programmes (STAR and TopHat2) but both used the transcriptome assembly functions in Cufflinks and StringTie and similar filtering approaches. To generate the AtRTD2 two different series of quality control filters were applied; the first at the transcript assembly stage and the second at the transcriptome merge stage (see Supplemental Figure  S1 ). The assembly functions of Cufflinks and StringTie can generate large numbers of false positives  [37, 38]. Following mapping of reads to the genome with STAR and TopHat2, we identified that both Cufflinks and StringTie generated novel splice junctions which were unsupported by reads. Therefore, the resulting transcriptome assemblies were initially filtered on the basis of splice junction quality.
To this end, we combined the splice junction reads mapped by STAR and TopHat2 and kept only those with canonical splice sites (GT. AG, GC. AG and AT. AC) which were present in at least ten reads in at least three sequencing samples/replicates (Dataset 1) or in three independent samples (Dataset 2), generating around 220k unique splice junctions. Applying these splice junctions to the transcript assemblies resulted in the removal of 49. 0% and 28. 3% of the transcripts from the initial Cufflinks and StringTie assemblies for Dataset 1 (see Supplemental Figure  S1a ). Similarly, using TopHat2/Cufflinks and TopHat2/StringTie on Dataset 2, 33. 5% and 33. 6% of assembled transcript models contained introns which were not supported by splice junction reads and were removed. Therefore, substantial numbers of transcripts assembled by both programmes here contained at least one intron with either non-canonical splice sites or with insufficient support by splice junction reads.
The prediction of such "introns" by these programmes highlights the need to quality control assembled transcripts prior to quantification of transcript isoforms. Following the splice junction quality filter, antisense transcripts that were completely contained within an annotated gene in TAIR10 were removed along with transcripts from unknown genes and transcripts with no or very low expression (see, for example, Supplemental Figure  S1a ). The latter were removed on the basis that they likely represent mis-assembled transcripts. To this end, RNA-seq read data were analysed using Salmon, and only the transcripts which had a TPM >1 in at least three sequencing samples were kept. RNA-seq from Dataset 1 generated a transcriptome of 48,509 transcripts with Cufflinks and 50,942 with StringTie, of which 38,133 transcripts were identical (representing 78.
6% and 74. 9% of the Cufflinks and StringTie assemblies respectively) (see Supplemental Figure  S2a ). The remaining 10,376 and 12,809 transcripts were found in only the Cufflinks and StringTie transcriptomes, respectively (see Supplemental Figure  S2a ). For Dataset 2, the Cufflinks and StringTie assemblies contained 52,984 and 47,816 transcript models, respectively, of which 37,903 (71. 5% for Cufflinks and 79. 3% for StringTie) were identical between the two assemblies (see Supplemental Figure  S2b ). To generate the final AtRTD2, the various total transcriptomes were merged toget her in a step-wise manner (Figure  1 ), and a second set of splicing and redundancy filters was applied after each merge (see Supplemental Methods and Supplemental Figure  S1b ). The application of the stringent filters will result in the loss of some transcripts with alternative transcription start sites or polyadenylation sites and of some bona fide transcripts which may be only expressed at very low levels or in a small number of specific cells.
However, they have been removed currently for the purpose of generating a robust core dataset for transcript quantification and alternative splicing analyses in Arabidopsis. The AtRTD2 contains 82,190 transcripts where each transcript model represents a unique transcript isoform. Finally, to show that the introns in the AtRTD2 transcripts predicted by the splice junction reads were bona fide introns, we searched for sequence signatures of plant U2 and U12 introns using position weight matrices (PWMs)  [14, 48]. The majority of introns (>99. 8%) define typical plant introns with PWM values at both 5' and 3' splice sites of >60 (74. 8% with PWM values >65 at each site) (data not shown). In addition, we identified 41 putative U12 AT. AC and 589 putative U12 GU. AG introns in 156 and 2,152 transcripts, respectively (PWM values of >75 and >65 at 5' and 3' splice sites, respectively).