dataesr/scientific-paragraphs-categorization huggingface dataset:
We present WIKIREADING, a large-scale natural language understanding task and publicly-available dataset with 18 million instances. The task is to predict textual values from the structured knowledge base Wikidata by reading the text of the corresponding Wikipedia articles. The task contains a rich variety of challenging classification and extraction sub-tasks, making it well-suited for end-to-end models such as deep neural networks (DNNs). We compare various state-of-the-art DNNbased architectures for document classification, information extraction, and question answering. We find that models supporting a rich answer space, such as word or character sequences, perform best. Our best-performing model, a word-level sequence to sequence model with a mechanism to copy out-of-vocabulary words, obtains an accuracy of 71.
Introduction A growing amount of research in natural language understanding (NLU) explores end-to-end deep neural network (DNN) architectures for tasks such as text classification  (Zhang et al. , 2015) , relation extraction  (Nguyen and Grishman, 2015) , and question answering  (Weston et al. , 2015). These models offer the potential to remove the intermediate steps traditionally involved in processing natural language data by operating on increasingly raw forms of text input, even unprocessed character or byte sequences. Furthermore, while these tasks are often studied in isolation, DNNs have the potential to combine multiple forms of reasoning within a single model. Supervised training of DNNs often requires a large amount of high-quality training data. To this end, we introduce a novel prediction task and accompanying large-scale dataset with a range of sub-tasks combining text classification and information extraction.
The dataset is made publiclyavailable at http://goo. gl/wikireading. The task, which we call WIKIREADING, is to predict textual values from the open knowledge base Wikidata  (Vrandečić and Krötzsch, 2014)  given text from the corresponding articles on Wikipedia  (Ayers et al. , 2008). Example instances are shown in Table  1 , illustrating the variety of subject matter and sub-tasks. The dataset contains 18. 87M instances across 867 sub-tasks, split roughly evenly between classification and extraction (see Section 2 for more details). In addition to its diversity, the WIKIREADING dataset is also at least an order of magnitude larger than related NLU datasets. Many natural language datasets for question answering (QA), such as WIKIQA  (Yang et al. , 2015) , have only thousands of examples and are thus too small for training end-to-end models.
Hermann et al. (2015)  proposed a task similar to QA, predicting entities in news summaries from the text of the original news articles, and generated a NEWS dataset with 1M instances. The bAbI dataset  (Weston et al. , 2015)  requires multiple forms of reasoning, but is composed of synthetically generated documents. WIKIQA and NEWS only involve pointing to locations within the document, and text classification datasets often have small numbers of output classes. In contrast, WIKIREADING has a rich output space of millions of answers, making it a challenging benchmark for state-of-the-art DNN architectures for QA or text classification. We implemented a large suite of recent models, and for the first time evaluate them on common grounds, placing the complexity of the task in context and illustrating the tradeoffs inherent in each arXiv:1608.
Wikidata Wikidata is a free collaborative knowledge base containing information about approximately 16M items  (Vrandečić and Krötzsch, 2014). Knowledge related to each item is expressed in a set of statements, each consisting of a (property, value) tuple. For example, the item Paris might have associated statements asserting (instance of, city) or (country, France). Wikidata contains over 80M such statements across over 800 properties. Items may be linked to articles on Wikipedia.
Dataset We constructed the WIKIREADING dataset from Wikidata and Wikipedia as follows: We consolidated all Wikidata statements with the same item and property into a single  (item, property, answer)  triple, where answer is a set of values. Replacing each item with the text of the linked Wikipedia article (discarding unlinked items) yields a dataset of 18. 58M (document, property, answer) instances. Importantly, all elements in each instance are human-readable strings, making the task entirely textual. The only modification we made to these strings was to convert timestamps into a human-readable format (e. g. , "4 July 1776"). The WIKIREADING task, then, is to predict the answer string for each tuple given the document and property strings. This setup can be seen as similar to information extraction, or question answering where the property acts as a "question".
Documents The dataset contains 4. 7M unique Wikipedia articles, meaning that roughly 80% of the Englishlanguage Wikipedia is represented. Multiple instances can share the same document, with a mean of 5. 31 instances per article (median: 4, max: 879). The most common categories of documents are human, taxon, film, album, and human settlement, making up 48. 8% of the documents and 9. 1% of the instances. The mean and median document lengths are 489. 2 and 203 words.
Properties The dataset contains 867 unique properties, though the distribution of properties across instances is highly skewed: The top 20 properties cover 75% of the dataset, with 99% coverage achieved after 180 properties. We divide the properties broadly into two groups: Categorical properties, such as instance of, gender and country, require selecting between a relatively small number of possible answers, while relational properties, such as date of birth, parent, and capital, typically require ex- tracting rare or totally unique answers from the document. To quantify this difference, we compute the entropy of the answer distribution A for each property p, scaled to the [0, 1] range by dividing by the entropy of a uniform distribution with the same number of values, i. e. , Ĥ(p) = H(A p )/ log |A p |.
Properties that represent essentially one-to-one mappings score near 1. 0, while a property with just a single answer would score 0. 0. Table  2  lists entropy values for a subset of properties, showing that the dataset contains a spectrum of sub-tasks. We label properties with an entropy less than 0. 7 as categorical, and those with a higher entropy as relational. Categorical properties cover 56. 7% of the instances in the dataset, with the remaining 43. 3% being relational.
Answers The distribution of properties described above has implications for the answer distribution. There are a relatively small number of very high frequency "head" answers, mostly for categorical properties, and a vast number of very low frequency "tail" answers, such as names and dates. At the extremes, the most frequent answer human accounts for almost 7% of the dataset, while 54. 7% of the answers in the dataset are unique. There are some special categories of answers which are systematically related, in particular dates, which comprise 8. 9% of the dataset (with 7. 2% being unique). This distribution means that methods focused on either head or tail answers can each perform moderately well, but only a method that handles both types of answers can achieve maximum performance. Another consequence of the long tail of answers is that many (30.
Answer Classification Perhaps the most straightforward approach to WIKIREADING is to consider it as a special case of document classification. To fit WIKIREAD-ING into this framework, we consider each possible answer as a class label, and incorporate features based on the property so that the model can make different predictions for the same document. While the number of potential answers is too large to be practical (and unbounded in principle), a substantial portion of the dataset can be covered by a model with a tractable number of answers.
Character seq2seq with Pretraining Unfortunately, at the character level the length of all sequences (documents, properties, and answers) is greatly increased. This adds more sequential steps to the RNN, requiring gradients to propagate further, and increasing the chance of an error during decoding. To address this issue in a classification context,  Dai and Le (2015)  showed that initializing an LSTM classifier with weights from a language model (LM) improved its accuracy. Inspired by this result, we apply this principle to the character seq2seq model with a twophase training process: In the first phase, we train a character-level LM on the input character sequences from the WIKIREADING training set (no new data is introduced). In the second phase, the weights from this LM are used to initialize the first layer of the encoder and the decoder (purple and green blocks in Figure  2 ).
Experiments We evaluated all methods from Section 3 on the full test set with a single scoring framework. An answer is correct when there is an exact string match between the predicted answer and the gold answer. However, as describe in Section 2. 2, some answers are composed from a set of values (e. g. third example in Table  1 ). To handle this, we define the Mean F1 score as follows: For each instance, we compute the F1-score (harmonic mean of precision and recall) as a measure of the degree of overlap between the predicted answer set and the gold set for a given instance. The resulting perinstance F1 scores are then averaged to produce a single dataset-level score. This allows a method to obtain partial credit for an instance when it answers with at least one value from the golden set.
Related Work The goal of automatically extracting structured information from unstructured Wikipedia text was first advanced by  Wu and Weld (2007). As Wikidata did not exist at that time, the authors relied on the structured infoboxes included in some Wikipedia articles for a relational representation of Wikipedia content. Wikidata is a cleaner data source, as the infobox data contains many slight variations in schema related to page formatting. Partially to get around this issue, the authors restrict their prediction model Kylin to 4 specific infobox classes, and only common attributes within each class. A substantial body of work in relation extraction (RE) follows the distant supervision paradigm  (Craven and Kumlien, 1999) , where sentences containing both arguments of a knowledge base (KB) triple are assumed to express the triple's relation.
Conclusion We have demonstrated the complexity of the WIKIREADING task and its suitability as a benchmark to guide future development of DNN models for natural language understanding. After comparing a diverse array of models spanning classification and extraction, we conclude that end-to-end sequence to sequence models are the most promising. These models simultaneously learned to classify documents and copy arbitrary strings from them. In light of this finding, we suggest some focus areas for future research. Our character-level model improved substantially after language model pretraining, suggesting that further training optimizations may yield continued gains. Document length poses a problem for RNN-based models, which might be addressed with convolutional neural networks that are easier to parallelize.
5 dBs with respect to a wild range of deinterlacers. Our framework has been implemented in Matlab (8. 0. 0. 783 (R2012b)) and the tests have been realized on a quadcore Intel-PC@4 GHz. Due to the independent block-based processing, the proposed deinterlacing approach is prone to distributed/parallel implementation, thus highly reducing the computation time obtained with a sequential implementation. Moreover, as the proposed algorithm adapts the motion estimation in function of region's saliency, due to our used threshold T s for motion computation, only ≈ 1/3 of field regions is motion processed (as it can be seen in Fig. 3 ). The parameterization allows, thus, to drastically decrease the complexity attached to motion-compensated schemes, by preserving its advantages where the user attention is focused.
For example, for sodium: Na Net ¼ Na Particle À Na Background H Particle H Background Figure  5  shows sections of the spectra near masses 27 and 40 for three different locations on the line scan depicted in Fig. 1b. In Figs. 5a  and 5b , data from the location X = 445 is shown. This spectrum was chosen It should be noted that particles on COSIMA targets are assigned names for ease of reference, with the first name distinguishing the individual particle and the second name the week of collection. If a target is imaged more than once in a week, a number may be appended. A COSISCOPE image of a section of target around the particle has superimposed on it a blue grid whose intersections are approximately the locations at which spectra were taken. The coordinate axes are shown in green. The particle is the light-colored area near the center of the image.
Acknowledgments-COSIMA was built by a consortium led by the  Max-Planck-Institut f€ ur Extraterrestrische Physik, Garching, Germany in collaboration with Laboratoire de Physique et Chimie de l'Environnement et 8 de l'Espace, Orl eans, France; Institut d'Astrophysique Spatiale ,  CNRS/Universit e Paris Sud, Orsay, France ;  Finnish Meteorological Institute, Helsinki, Finland; Universit€ at Wuppertal, Wuppertal, Germany; von Hoerner und Sulger GmbH, Schwetzingen, Germany; Universit€ at der Bundeswehr, Neubiberg, Germany; Institut f€ ur Physik, Forschungszentrum Seibersdorf, Seibersdorf, Austria; Institut f€ ur Weltraumforschung, € Osterreichische Akademie der Wissenschaften, Graz, Austria ; and is led by the  Max-Planck-Institut f€ ur Sonnensystemforschung, G€ ottingen, Germany. The support of the national funding agencies of Germany (DLR, grant  50 QP 1302 ),  France (CNES) ,  Austria, Finland , and the  ESA Technical Directorate  is gratefully acknowledged.
To avoid being overwhelmed with data, often the simulation outputs the data for a sub-sample of all the computed (x, y, z, t) points. Parametric studies are becoming increasingly popular in industrial environments. For instance a consortium of companies, including EDF, one of the biggest electricity producer in the world, has developed specific software infrastructures for parametric studies like OpenTurns  [8]. But such environments are working with data stored in files and cannot cope with very large amount of data. A solver like Code_Saturne, a CFD code developed at EDF, can run large scale numerical simulations  [10] , dealing with meshes from several million up to a few billion million cells. Code_Saturne already supports in-situ visualisation based on Catalyst  [12] , but for a single run.
To reduce the data size, a first solution commonly adopted in industrial parametric studies consists in eliminating part of the complexity by choosing one variable (eliminate multivariability) and one point in the simulation (eliminate spatial multidimensionality). This solution leads to a reduced dataset where only the multiple values of one parameter for the different simulations and timesteps are kept. Fig. 1  shows 600 Monte Carlo simulations of temperature in function of time in a thermo-hydraulical transient parametrical simulation performed at EDF, simulating a large break loss of primary coolant accident in a power plant. The engineer can study the loss of temperature by exploring this set of curves but they are focusing on a specific variable and on a specific location in the simulation domain.
On the simulation side, the modifications required are minimal. The code needs to embed the logic for establishing the connection with Melissa, the routine to scatter the data to the correct target Melissa processes, and connection closing. Current Melissa implementation relies on the ZEROMQ communication library  [11]  commonly used for distributed applications. ZEROMQ allows several clients to connect to the same server port, and takes care of message transfer, buffering and aggregation in the background. It proved very convenient for development. The main ZEROMQ limitation we identified so far is the lack of direct support for high performance networks like Infiniband. It needs to rely on IP over infiniband instead. However in our context we did not experience any performance issue, neither on the simulation side nor on the server side.
III. EXPERIMENTS We present here a first set of early experiments at modest scale. All the computations presented in this paper were performed on the Froggy machine. Each Froggy node have two Intel E5-2670 (Sandybridge) 2. 6 Ghz 8 cores processors, 64 GB memory, and are interconnected by a FDR InfiniBand network. The batch scheduler used on this machine is OAR  1. The simulation code is Code_Saturne 2  , a parallel CFD code developed by Electricité de France (EDF). The statistical analysis is performed by Melissa. The use case created by EDF for this experiment simulates a purge of a volume of hot water by introducing a flow of cold water (Fig. 3 ). The parametric study consists in varying the initial temperature For this experiment, Melissa computes the mean, variance and min and max of temperature and pressure for each element of the mesh at each time step.
We used a 3D mesh of 6 002 400 elements. Adapting Code_Saturne to support Melissa was straightforward. We wrote a new writer (Code_Saturne plugin that encapsulates the output data management code) based on Melissa. This writer code is about 400 lines of codes including 40 Melissa specific ones. We first ran one simulation writing the outputs to disk (Ensight Gold data format). The output is 24 MB per time step and per output field. If we run 100 simulations with 30 time steps and two output fields each, this leads to write 144 GB on disc. When analysing the results in situ these data are directly processed by Melissa. Disk usage is limited to storing the final statistics (24 MB per time step per field and per statistic, for a total of 5. 76 GB). Statistics for each time step can next be visualized with traditional tools like VTK/Paraview (Fig.
In this work, we present a review of the current landscape of existing tools designed to facilitate the visualization of multidimensional data in translational research platforms. Specifically, we reviewed the biomedical literature for translational platforms allowing the visualization and exploration of clinical and omics data, and identified 11 platforms: cBioPortal, interactive genomics patient stratification explorer, Igloo-Plot, The Georgetown Database of Cancer Plus, tranSMART, an unnamed data-cube-based model supporting heterogeneous data, Papilio, Caleydo Domino, Qlucore Omics, Oracle Health Sciences Translational Research Center and OmicsOffice V R powered by TIBCO Spotfire. In a health sector continuously witnessing an increase in data from multifarious sources, visualization tools used to better grasp these data will grow in their importance, and we believe our work will be useful in guiding investigators in similar situations.
Driving motivation for the review Quality visualization can be applied to any of the numerous domains where big data has recently affected the health-care arena such as, among others, managing cost, improving quality improvement, monitoring patients for clinical deterioration and improving treatment efficiency in emergency care  [10] [11] [12] [13]. In clinical research, multidimensional data can be used to help segment patients or elucidate disease pathway. This has most notably been seen in oncology with large data sets containing various genomics and clinical data for thousands of cancer patients such as The Cancer Genome Atlas (TCGA  [14] ) or the International Cancer Genome Consortium (  [15] ). However, multi-omics research has extended into a wide variety of fields such as dementia and Alzheimer's disease (Alzheimer's Disease Neuroimaging Initiative  [16] ), autism spectrum disorder (National Database for Autism Research  [17] ), psychiatric diseases (Psychiatric Genomics Consortium  [18] ), as well as for rare diseases (RD-Connect  [19] ).
Correlation matrices using multiple scatter plots show an additional insight into the interaction between variables. In addition, heatmaps are commonly used for multidimensional data, especially in genetic research with expression, pathway or molecular abundance data and involve a matrix where each cell is colored according to a gradient and is often clustered by samples  [22]. Heatmaps and other visualizations are available in a wide variety of software such as R, Matlab V R , SAS V R , as well as to users without programming knowledge through programs with intuitive user interfaces (e. g. ClustVis  [23] , HemI  [24] ). Another increasingly common technique for visualizing the relationships between variables in multidimensional data sets is parallel coordinates. Here, vertical axes corresponding to each variable scaled to a common height are placed next to each other and connected with lines representing different samples  [25].
For instance, OmicCircos  [33]  is an R package that produces circular plots capable of integrating expression, copy number variations (CNV) and protein fusions as well as visualizations of statistics that compare data across these sources. This allows researchers a high-level view that may facilitate the understanding of complex diseases such as cancer or psychiatric diseases. Two other interesting R packages that integrate multi-omics with visualizations are coMET  [34] , which incorporates epigenetic results and other types of genomic data such as expression profiles, and caOmicsV  [35] , which also provides several options of viewing various genomic data side-by-side other phenotypic data. The field of data visualization is immense. Dedicated tools and libraries have been developed and exist through a rising number of open-source and fee-based platforms.
For example, many scientists rely on various programming languages or statistics packages with data-visualization capabilities such as R  [36]  or Python Matlibplot  [37]. More and more researchers are turning to JavaScript graphics libraries to enhance visualization with dynamic capabilities. Such libraries include Highcharts  [38] , Chart. js [39] , Dygraphs  [40] , JavaScript InfoVis Toolkit  [41]  and D3. js (Data-Driven Documents  [42] ) (for comprehensive overview and sideby-side comparison of these libraries see  [43] ). In sum, impressive techniques have been developed to answer to the clear need for strong data visualization in health-care research. However, such tools and techniques are not easily accessible to the clinician or biologist end users. R packages or Python library are easy to leverage for a bioinformatician, but the knowledge gap is often too wide for biologists and clinicians without a background in bioinformatics or biostatistics.
The first three platforms identified that respected these inclusion criteria were three platforms from the previous review [cBioPortal, The Georgetown Database of Cancer (G-DOC) Plus and tranSMART]. We then searched scientific literature available through PubMed V R [46] using Medical Subject Headings terms and free-text search, and subsequently identified 367 articles potentially describing visualization for heterogeneous data (PubMed queries and literature search, details are available in Supplementary Table  S1 ). We identified three new platforms through this step, and one from citations tions used are (A) dynamic pivot table (using R 'rpivotTable' package), (B) correlation matrix (using R 'PerformanceAnalytics' package), (C) Heatmap clustered by rows and columns (using R 'gplots' package), (D) 3D scatterplot using color and size (using R 'scatterplot3d' package) and (E) parallel coordinates showing all data (using d3 Javascript library 'd3.