dataesr/scientific-paragraphs-categorization huggingface dataset:
Thus we see this situation as an ideal application for our work. We recorded the flying sound of male and female insects of the species Culex stigmatosoma, which is a vector of several diseases such as the West Nile Virus and Western Equine Encephalitis  [32]. Being able to classify male vs. female mosquitoes is important because only the females actually spread disease, and different interventions are used to control females (to reduce biting now) and males (to reduce biting one generation hence). Using our pseudo-acoustic sensor  [31] , we recorded about 10,000 flights and created a dataset by randomly choosing 200 examples of each class (male/female). We then randomly split this dataset into two balanced train/test datasets of same size. As we can see in Figure  7 , our algorithm is able to achieve a lower error-rate using just two items per class, than by using the entire training dataset.
We now proceed with the rest of the experiments, in order to assess the generality of the two observations that we have made on this case study: 1. The average-based methods condense better the information about the class than the state-of-theart methods (detailed in the next sub-section: B). 2. Not only are average-based methods better at reducing the size of the training set, but they can also improve the accuracy of the classifier. This has been observed in Figure  7  where reducing the training set with the K-means algorithm allows us to derive a classifier that performs better than 1-NN using the full training set (error rate of 0. 092 vs 0. 14). This observation will be assessed in subsection C. Finally, note that all the raw material generated by our experiments (for example, the charts similar to Figure  7  for all the datasets, but also the rankings used in the reminder of this section) cannot be included in the paper due to space limitations, but are available at  [33].
B. Condensing the model of the class to a handful of exemplars To demonstrate that the results in the case study represent typical improvements over the rival methods, we will test on a very diverse collection of datasets. We have compared our approach on all the datasets in the UCR time series archive  [22]   3. A description of a representative sample of these datasets is given in TABLE  III. We want to compare the performance of the different methods when they are authorized (under the "contract") to use, say, 1 prototype per class (or # prototypes for Random, DropX and SimpleRank). To this end, we follow the standard practices for the statistical comparison of classifiers  [34]  and use the average ranking of each method over all the datasets. This will allow us to assess what algorithm exhibits, on average, the best classification performances under the contract restriction.
For every dataset and every algorithm, we compute the error-rate when constrained to use a reduced set of  prototypes per class only. Then, for every dataset, we rank the methods by error-rates: rank 1 is assigned to the best method; rank 8 is assigned to the worst one. 4 e then compute the average rank for every method (see  [34 -. This gives a direct general assessment of all the algorithms: the lowest rank corresponds to the method that, on average, obtains the best error-rate for the considered "contract". TABLE  IV  shows the average rank of all algorithms over the datasets of  [22]  (again, the raw results giving the error rate and rank for every method and every dataset is available at  [33] ). These results show unanimously that the methods that use an average sequence (K-means and AHC) significantly outperform the prior state of the art.
In addition, we test the statistical significance of these results. We want to assess if 42 datasets is a large enough sample to state that this difference in the ranking is statistically significant. We first perform a Friedman test  [34] , in order to assess if the results are significantly different. This test is used to evaluate if there is enough evidence to confidently state that the different methods are not performing equally. 𝜒 𝐹 2 = 12𝑁 𝑑 𝐴(𝐴 + 1) [∑ 𝑅 𝑗 2 - 𝐴(𝐴 + 1) 2 4 𝑗 ] (3) The values are reported in the second-to-last line of TABLE  IV ; given that the Friedman test follows a  2 distribution with  -1 degrees of freedom, these results yield a highly significant difference between the methods ( < 10 -16 ). Having rejected the null hypothesis, we can proceed with a detailed comparison of the methods.
state-of-the-art methods. Comparing 8 methods over 42 datasets,  [34]  shows that, to be statistically significant ( = 0. 05) the difference between the average rankings has to be greater than: CD = 𝑞 0. 05 ⋅ √ 𝐴(𝐴+1) 6𝑁 𝑑 = 2. 690 ⋅ √ 72 252 ≈ 1. 438. We report the difference between the average rank obtained by K-medoids and the one obtained by K-means over the 42 datasets in the last line of TABLE IV. It shows that the difference is greater than the critical one CD, regardless of the number of prototypes used. As a result, we can confidently conclude that the K-means algorithm is statistically significantly better than K-medoids, and thus that the use of averaging-based methods yield better results than state-of-the-art methods.
C. Classifying faster and more accurately We have seen in the case study on insect surveillance that average-based methods manage, with a reduced set of time series, to outperform the classification accuracy of the 1-NN classifier on the full training set. This result may be counterintuitive, so in this section we will assess this phenomenon on a wide variety of datasets. To this end, we start by performing a standard 1-NN classifier using the full training set for classification. This gives us the reference error-rate against which we compare the results of different methods. We then progressively restrict the allowed size of the reduced set (), until we find the smallest value of  for which the error-rate is smaller than the reference full 1-NN algorithm. Then, for each dataset (and similar to the experiment in the last section), we rank the methods by size of their reduced sets that are able to "beat" the full 1-NN classifier.
The results of these experiments are reported in TABLE V; note that for fairness in the ranking, we do not include the Random sampling strategy because, on average, it cannot beat the results of the full 1-NN classifier. A first look at TABLE V shows that average-based methods again outperform the prior state of the art, with the K-means algorithm obtaining an average rank of 1. 57 better than the K-medoids algorithm. Moreover, on average, the K-means method is able to condense the training set by 71%. This means that on average over the archive of datasets, our method using the K-means algorithm achieves equal or better performance that the full 1-NN classifier, while only requiring 29% of the computational complexity. This is an extraordinary result. We can now assess the statistical significance of the superiority of K-means over K-medoids (the best method that does not average time series).
VI. CONCLUSIONS AND FUTURE WORK We have shown that an obscure result on averaging "warped" time series can be augmented to allow us to create much faster and/or more accurate time series classifiers. Our results may be particularly useful for resource constrained situations, such as wearable devices and "in-sensor" classifiers  [30]. We have demonstrated the utility of our ideas on more than 40 datasets, and made all code and data freely available to allow independent confirmation and extensions of our work  [33]. Note that the classic data condensing methods such as Drop{X} occasionally do reasonably well, at least at some levels of condensation. Further note that the only operator in their search space, the deletion of items, is completely orthogonal to our proposed methods. This suggests that we may be able to further improve our search space by expanding our search space to include deletion.
Acknowledgements The authors would like to acknowledge the  Key Centre for Polymer and Colloids at the University of Sydney  for donating photo-initiators. We wish to thank  Dr Michael Phillips ,  Prof Paul Peiris ,  Julie Markham ,  Danielle Taylor ,  Adam Sutton ,  David Fania ,( UWS ) and  Prof Emily Hilder  ( UTas ) for fruitful discussions as well  Joel Thevarajah  ( UWS ) for the oligoacrylates injection.
Ontology Development and Annotation A meeting was held between renal biomedical experts, GO curators and GO editors to determine the correct representation of renal processes (renal development in particular) in the Gene Ontology. Ontology editors added the new terms and reorganized existing renal GO terms accordingly. For annotation purposes, expression data from the GUDMAP database  [8]  was used to identify 29 mouse genes expressed in the murine loop of Henle. BLAST  [19]  was used to identify similar genes, where present, in human, zebrafish, Xenopus, chicken and fly. Table  1  lists the UniProtKB accession numbers of the corresponding gene products. Curators from UniProt, Mouse Genome Informatics (MGI), FlyBase and the zebrafish model organism database (ZFIN)  [21] [22] [23] [24]  then assigned molecular function, biological process and cellular component GO terms to the gene products, based on experimental evidence in scientific papers.
GO Term Enrichment Analysis Two term enrichment tools were used for the analyses; GO-Elite (http://www. genmapp. org/go_elite/) [26, 27]  and Ontologizer (http://compbio. charite. de/index. php/ontologizer2. html) [28] ; GO term enrichment analysis was performed using annotations to biological process terms only. We took the gene data set for our reanalysis from the investigation into the differential gene expression in glomeruli from human kidneys with diabetic nephropathy by Baelde et al. [29]. The gene identifiers used in this 2004 study were mapped to current UniProtKB accession numbers (Table  S1  in File S1). Some of the gene identifiers, for example, D87002, mapped to multiple UniProtKB accessions (Q14390, Q5NV78, Q5NV77) because both 'reviewed' and 'un-reviewed' sequences in the UniProtKB database cross-referenced to the same gene identifier.
However, in such cases only one of the accession numbers for the gene product was curated; usually the reviewed UniProtKB/SwissProt entry or, if all entries were un-reviewed, the longest UniProtKB/TrEMBL sequence. The protein accessions in Table  S1 , in File S1, constitute the 'Input' list for the GO term enrichment analysis. For the GO-Elite analysis, we used the ORA-pruned analysis with a z-score cut-off of. 1. 96, the minimum number of changed genes was set at 3 and the permuted p-value cut-off was ,0. 1. GO-Elite uses the Z-score/hypergeometric statistical method and Benjamini-Hochberg (BH) correction for multiple hypothesis testing  [27]. For the Ontologizer analysis, term enrichment was calculated using the parent-child intersection analysis method using a modified Fisher's exact analysis.
Data Files Ontology files were downloaded from: http://cvsweb. geneontology. org/cgi-bin/cvsweb. cgi/go/ontology/ gene_ontology. obo. Versions of the ontology files downloaded from the above location, used in OBO-Edit for creating Figure  1  The 'Background' list of protein accessions used in the GO term enrichment analyses was obtained from the relevant Gene Association File. For example, the background list for the 2009 analyses was the unique protein accessions in the 'gene_association. goa_human. 72. gz' file. Annotation data sets for renal specific gene products and GO terms can be viewed via the QuickGO browser at www. ebi. ac. uk/ QuickGO, using either gene product name(s), UniProtKB accession number(s) or GO term(s).
An OBO-Edit 'Ontology Tree Editor' view showing the relationship and position of the new GO term 'branching involved in ureteric bud morphogenesis'. By placing the new term 'branching involved in ureteric bud morphogenesis' as a sub-type of 'morphogenesis of a branching structure', it puts the renal branching into the context of other types of branching morphogenesis within the Gene Ontology (using the Gene Ontology file from March 19 th 2012). The [+] icon beside each term denotes that there are further child terms that can be viewed; the  [2]  icon denotes no further child terms; (P) denotes a part_of relationship; (I) denotes an is_a relationship. doi:10. 1371/journal. pone. 0099864. g004 A summary of the number of GO terms that were significantly enriched (having a p-value of ,0. 1) in the Baelde groups' differentially expressed gene dataset from glomeruli affected by DN  [29]  by both Ontologizer  [28]  and GO-Elite  [27]  term enrichment tools, using the pre-annotation (2009) and post-annotation (2012) GO annotation datasets.
Use of the New Renal Development GO Terms in Gene Product Annotation Following the improved ontology structure for renal system development, curators annotated renal-related gene products. The first annotation targets were the 29 gene products highly expressed in the mouse loop of Henle  [8]  and hence predicted to play a role in the development and/or physiology of this renal structure. To compare the function of these proteins across species, similar proteins in human, rat, zebrafish, Drosophila and Xenopus (found via BLAST run on the uniprot. org website) were also annotated. Table  1  lists the UniProtKB accession numbers for these gene products. The loop of Henle plays an important role in creating a concentration gradient in the medulla of the kidney. It is involved in reabsorption of filtered water and ions including sodium, potassium and calcium, and independently regulates both the volume and osmolarity of body fluids.
Impact of Improved Gene Ontology Annotation on Data Analysis To test the value of increasing the depth and coverage of GO annotation for renal-related proteins on interpretation of experiments, we compared the annotation dataset from immediately prior to our focused annotation project (March 2009, and referred to hereafter as the 'pre-annotation dataset') with the dataset from the end of the project (March 2012, and referred to hereafter as the 'post-annotation dataset'). To this end, we fully annotated the biological roles of a set of human proteins that were initially identified in a study of genes differentially expressed in glomeruli of kidneys affected by the later stages of diabetic nephropathy (DN), as reported by Baelde et al. [29]. We chose to test the improvement in the annotation dataset by performing a term enrichment analysis on the differentially expressed gene products identified in the original study, as this is an efficient way to get an overview of the annotations for a set of genes without losing the specificity of the added GO terms, and can be used to directly compare the results obtained by Baelde et al.
Overall Observations from the Analysis The analysis was performed separately on up-and downregulated genes since this distinction was made in the original analysis  [29]. Full term enrichment results for both up-and downregulated genes using both GO-Elite and Ontologizer are available as Tables S2-S9 in File S1. Table  2  summarizes the number of terms significantly enriched by both Ontologizer and GO-Elite term enrichment tools using the GO annotation datasets from before and after the annotation focus. The most striking observation is that, in the output from both tools, there are significantly more enriched terms using the post-annotation dataset, compared to using the pre-annotation dataset (significance cut-off values are reported in the 'Methods' section). In general, the terms that were significantly enriched using the post-annotation dataset were not enriched using the preannotation dataset, indicating that focused GO annotation using both existing and the new terms created during this initiative has had a great impact on the interpretation of this analysis.
This was likely due to lack of experimental data and/or lack of curation. Enrichment of terms not specific to kidney development. We noted also the appearance of GO terms describing biological processes that are not specific to kidney development, but are still relevant to DN. The importance of extracellular matrix proteins in the expansion of the mesangial matrix and thickening of basement membranes that occurs in DN has already been reported by Abrass  [49]. Although no related terms were reported in the original analysis by Baelde et al. [29] , in the post-annotation analysis we see the terms 'extracellular matrix organization' (GO:0030198) (Table  S6  in File S1) and 'basement membrane organization' (GO:0071711) (Table  S7  in File S1) significantly enriched. 'Extracellular matrix organization' (GO:0030198) was available for annotation in 2004 but only 7 human gene products had been associated with it, none of which were from the Baelde study set.
Basement membrane-related process terms were not added to the ontology until 2008 and so were unavailable for use at the time of the Baelde study. Together, this demonstrates ongoing improvements to the Gene Ontology and GO annotations since 2004, as well as this focused annotation project, contributed to the improved results for the postannotation dataset. DN can occur due to longstanding diabetes mellitus, a metabolic disease in which an individual has high blood sugar, either because the body does not produce enough insulin, or because cells do not respond to the insulin that is produced  [50, 51]. Therefore it is encouraging to see the appearance of 'positive regulation of insulin secretion involved in cellular response to glucose stimulus' (GO:0035774) and 'response to insulin' (GO:0032868) (Table  S6  in File S1) following the focused annotation.
There is increasing evidence that there is an inflammatory aspect to DN  [52]  and this is reflected in the terms that were significantly enriched in the post-annotation analysis, such as 'inflammatory response' (GO:0006954) (Tables  S6  and S8  in File S1), 'regulation of cytokine secretion' (GO:0050707) (Table  S6  in File S1), 'immune system process' (GO:0002376) and the regulation thereof (Tables S7-S9 in File S1), 'T cell mediated immunity' (GO:0002456) (Table  S7  in File S1), 'interleukin-10 production' (GO:0032613) (Table  S7  in File S1) and 'myeloid leukocyte activation' (GO:0002274) (Table  S6  in File S1). The only terms relating to immunity or inflammation in the pre-annotation dataset were 'T cell homeostatic proliferation' (GO:0001777), 'lymphocyte apoptotic process' (GO:0070227) and 'immune system process' (GO:0002376) (Tables  S2  and S3  in File S1).
Only one of these terms, 'VEGF receptor signaling pathway' (GO:0048010) was present in the analyses using the pre-annotation dataset (Table  S5  in File S1). These results suggest that genes influencing endothelial cell proliferation may be down-regulated in DN, whereas genes influencing endothelial cell apoptosis may be up-regulated. A second difference between the up-and down-regulated gene sets was the appearance of nitric oxide-type terms. One of the roles of nitric oxide is to help control blood pressure in the kidney, so the presence of these terms suggests that nitric oxide may play a role in the progression of DN. It has recently been shown that nitric oxide is reduced in diseased kidneys  [53] , however the increased severity of endothelial dysfunction in DN has been demonstrated in a mouse diabetic model, which has an endothelial nitric oxide (eNO) synthase gene knock out  [54].
We observed terms such as 'nitric oxide transport' (GO:0030185) (Table  S7  in File S1) and 'regulation of nitric oxide biosynthetic process' (GO:0045428) (Table  S6  in File S1), only in the analysis of the up-regulated set of genes using the post-annotation dataset. It should be noted at this point that researchers can choose from many freely available GO analysis tools to interpret their datasets. However, each tool will give a different interpretation of the dataset, as demonstrated in this paper, with the results from GO-Elite and Ontologizer. This is usually due to the different analysis and correction methods, statistics, filters and versions of the ontology and annotation files that each tool integrates in order to analyze the gene lists  [55]. Nevertheless, our analyses using the new set of ontology terms and annotations from the focused annotation initiative, contained more specific and up-to-date results that are in line with current knowledge about DN.
This paper highlights the importance for both continued development of the Gene Ontology and comprehensive GO annotation of proteins within this resource, can enable researchers to gain improved biological insights into their particular proteins of interest and consequently guide new investigations into understanding the mechanisms of, and propose new treatments for, renal diseases. expressed gene products in the Baelde 2004 study. A description tab entitled 'Tables S2-S9 Description', has been included, defining the output from the GO-Elite and Ontologizer GO term enrichment tools presented in Tables S2-S9. Table  S1. Input protein list. Mapping of the gene product identifiers from the Baelde 2004 study to UniProtKB accession numbers. Most of the differentially expressed gene products in DN glomeruli were mapped to a UniProtKB accession number and those that could not be mapped were not annotated and are not included in the table.
The boldface terms are referred to in the manuscript text and the italicized boldface indicates new terms created during the Renal GO Annotation Initiative. The ''Study'' column shows the number of proteins in the input list with an annotation to the given term. The ''Population'' column shows the number of proteins in the background list with an annotation to the given term. Table  S7. Ontologizer analysis on up-regulated proteins in DN glomeruli, using the 2012 Gene Ontology and annotation sets. Results from the Ontologizer enrichment analysis tool on the up-regulated proteins from the Baelde list, using the Gene Ontology and Annotation files from March 2012; showing significantly enriched GO terms. The boldface terms are referred to in the manuscript text and the italicized boldface indicates new terms created during the Renal GO Annotation Initiative.
The ''Population'' column shows the number of proteins in the background list with an annotation to the given term. Table  S9. Ontologizer analysis on downregulated proteins in DN glomeruli, using the 2012 Gene Ontology and annotation sets. Results from the Ontologizer enrichment analysis tool on the down-regulated proteins from the Baelde list, using the Gene Ontology and Annotation files from March 2012; showing significantly enriched GO terms. The boldface terms are referred to in the manuscript text and the italicized boldface indicates new terms created during the Renal GO Annotation Initiative. The ''Population'' column shows the number of proteins in the background list with an annotation to the given term. The ''Study'' column shows the number of proteins in the input list with an annotation to the given term.
[27]ary of significantly enriched GO terms from the Ontologizer and GO-Elite analyses that are relevant to kidney development. thesignificantlyenrichedGO terms from the Ontologizer[28]and GO-Elite[27]analyses, which are relevant to kidney development, using the preannotation (2009; Tables S2-S5 in File S1) and post-annotation datasets (2012; Tables S6-S9, in File S1). Terms in italics indicate parent terms where the descendants are indicated directly underneath as follows:. descendant of term above in italics. Rank refers to the position of the term in the results of the enrichment analyses (see Tables S2-S9 in File S1) where significance of the enriched term has a p-value of ,0. 1. (n/a = not applicable).