dataesr/scientific-paragraphs-categorization huggingface dataset:
News agencies produce thousands of multimedia stories describing events happening in the world that are either scheduled such as sports competitions, political summits and elections, or breaking events such as military conflicts, terrorist attacks, natural disasters, etc. When writing up those stories, journalists refer to contextual background and to compare with past similar events. However, searching for precise facts described in stories is hard. In this paper, we propose a general method that leverages the Wikidata knowledge base to produce semantic annotations of news articles. Next, we describe a semantic search engine that supports both keyword based search in news articles and structured data search providing filters for properties belonging to specific event schemas that are automatically inferred.
INTRODUCTION Information and communication technologies have provided tools and methods to make the production of information more democratic. In this context, journalists and technologists have developed the notion of "data journalism", which takes advantage of structured and numerical data in the production and distribution of news. It also takes advantage of the growing popularity of Linked and Open Data and the development of structured knowledge bases such as DBpedia  [5] , YAGO  [3]  or Wikidata  [17]  to facilitate information analysis and to access a variety of points of view. However, knowledge is still far from being entirely represented in structured databases, and the most prominent way to convey information to the end user is still the free text, complemented by multimedia content.
Intertwining structured and unstructured data in information systems is still an open research problem. In this paper, we present a system for aggregating unstructured news articles and structured data describing events leveraging on the Wikidata knowledge base. This approach makes use of several Information Retrieval and Information Extraction tasks. In the context of Information Extraction, the knowledge associated with news articles can typically be used for training event extractors in a distant supervision mode  [12]. From the Information Retrieval perspective, the approach makes it possible to retrieve news articles describing events using either keyword-based queries or filters that typically make use of properties available in knowledge bases. It also allows to query Wikidata and then to read an entire annotated article describing the corresponding event.
We implemented a system which is available at http://asrael. eurecom. fr/ and covers the last two tasks. Figure  1  illustrates the architecture of our system. Events described in news articles are mapped to events from Wikidata (Section 4. 1), and attributes from the Wikidata instances are used to annotate the news articles when possible (Section 4. 3). Wikidata events belong to specific classes, but these classes are too finegrained for being used in a search engine. Furthermore, many event classes actually share a similar structure (i. e. sets of attributes). For example, a general election schema is relevant for describing any type of elections regardless of the more specific Wikidata types such as "Bundestag election" (Q1007356), "direct election" (Q1196727), "Elections in Saudi Arabia" (Q4119635).
As part of this architecture, specific controlled vocabularies, such as the IPTC Media Topics or News Codes, are used to categorize news items together with other industry-standard thesauri. In previous work, we designed an OWL ontology for the IPTC News Architecture and we converted the IPTC NewsCodes into a SKOS thesaurus  [15]. IPTC is now publishing itself the IPTC Media Topics in SKOS and has further developed the rNews vocabulary, largely based on Schema. org, for describing news articles. In this work, we re-use the rNews vocabulary to describe the original metadata attached to news articles. Furthermore, we annotate the news articles using properties and entities from Wikidata once events reported in the news have been mapped to existing Wikidata events. While mapping text to knowledge has been the subject of a large body of work, represented in the recent ages with work such as  [7]  or all the work about entity linking  [1] , mapping text to event representations and more particularly news articles to event representations has not been the focus of lots of studies.
DATA 3. 1 AFP News Articles In this work, we make use of a very large corpus of text newswire written in English provided by the French news agency AFP. More precisely, we use over 2 million articles covering the period 2004-2017. The topics are worldwide news ranging from politics, diplomacy, sports to natural disasters or economy and business. Each document is an XML file compliant with the NewsML standard, containing a title, a document creation time (DCT), a dateline where the article was written, one or several IPTC Media Topics and a set of keywords (slugs), as well as a textual content split into paragraphs. The main topic of an article is generally a specific event, and sometimes, other older events are referred to in order to look at the current one from a wider perspective. This is why we consider that it is possible to associate an AFP article with one single event.
Wikidata Occurrences Wikidata is a free and open knowledge base that can be read and edited by both humans and machines  [17]. It acts as a central storage for the structured data of its Wikimedia sister projects including Wikipedia, Wikivoyage, WikiNews, and others. In our work, we focus on newsworthy events, i. e. something that happens within a locality and a temporality, and that could be described in one or more news articles. In regard to this definition, there are many general and specific classes in Wikidata that are related to events, but all these classes have in common the same parent class named "Occurrence" (Q1190554). For example, the event "Cargolux Flight 7933" (Q3107014) is an instance_of (P31) an "aviation accident" (Q744913) which is a subclass_of (P279) an "aviation occurrence" (Q15733640) which is a subclass_of (P279) "occurrence" (Q1190554).
In the remainder of this paper, we call Wikidata Event Type (or WET) the value of the property instance_of (P31) of a Wikidata event. In the previous example, "aviation accident" (Q744913) is a WET. An instance can have several Wikidata Event Types. We lined up on the temporal coverage of the AFP articles corpus and we considered all Wikidata event instances during the period from 2004 to 2017. The event date can be represented by three properties in Wikidata: • P585:point_in_time if the event is a one-off event; • P580:start_time and P582:end_time if the event has a duration. In the second case, we consider the events that have started and ended during the period 2004-2017. The selection of Wikidata events consists of 60k Wikidata instances. As shown in Figure  2 , the distribution over the years is not uniform.
Scoring function. For mapping a Wikidata instance to an AFP article, we define the following criteria: score(article) = t i ∈ar t icl e t f. id f I MT (t i ) × 1 keywor ds (t i ) (1) where 1 keywor ds (t i ) = 1 if t i ∈ W ET 0 otherwise and t f. id f I MT (t i ) = t f I MT (t i ) × loд N d f I MT (t i ) (2) with t i the i t h token in the article, d f I MT (t i ) the number of IMTs associated with documents containing t i , t f I MT (t i ) the number of occurrences of t i in the articles sharing the same IMT, and N the number of IMTs. If the AFP article has more than one IMT, then the highest t f I MT for each word is considered. The weights are based on IPTC Media Topics. Hence, they increase the relevance of a token in a particular context. For instance, "Police" will be more relevant in an event regarding a crime compared to an event regarding an earthquake, even if the token is present in both articles.
An event is considered to be about the same subject as a Wikidata instance if this score is higher than a threshold. Empirically, this threshold is set to 0. 04 and roughly corresponds to two tokens in common between the article and the list of keywords. In order to evaluate the quality of this mapping, we manually built a set of 406 pairs (article, Wikidata instance) based on 88 instances explicitly linked to a WikiNews page  8  in Wikidata. Only a very small part of Wikidata events are linked to a WikiNews page  9 but we used them to ensure that only "media-worthy" Wikidata events are considered in our manual evaluation. We share this gold standard for further comparison  10. We report in Table  1  the precision (correct mapping rate), recall (1-missed mapping rate) and F1-measure, when considering the first 3, the first 5 or all the sentences in the AFP articles.
Schema Clustering As depicted in Table  2 , Wikidata event types are often fine-grained and the subclass hierarchy can vary in terms of quality and depth. We seek more coarse-grained event categories for the relevance and robustness of our news classification and potential filters for our search engine. Indeed, from a human perspective, several WETs (e. g. NATO summits and G20 summits) share the same or a very similar structure and clustering them together will make the classification process easier, as well as simplify the interaction with the user within the search engine interface. To do so, we adopted a hierarchical clustering method based on 3 similarity measures. Each similarity is based on a different representation of the Wikidata Event Type. • Label representation: Even if composed of only a few words (see Table  2 ), the labels of the Wikidata Event Type (WET) can be a good clue for deciding whether two clusters are similar or not.
• Content representation: this representation is based on the content of the articles. A WET document is built by concatenating all the articles mapped to this WET at the previous step (Section 4. 1). The content representation R c (T ) is then a vector of all words t i weighted by their t f. id f W ET , computed as follows: t f. id f W ET (t i ) = t f W ET (t i ) × loд M d f W ET (t i ) ( 4 ) where t f W ET (t i ) is the number of occurrences of the term in the WET document, d f W ET (t i ) is the number of WET documents containing the term, and M is the total number of W ET s in our dataset. • IPTC Media Topic representation: in order to improve and to facilitate the stopping decision of the clustering, we add a feature based on the IMTs (see Section 3. 3). As each AFP article is associated with one or several IMTs, the mapping described in Section 4.
1 provides also a mapping between a WET and a list of IMTs. We interpret these 11 http://spacy. io/ codes as a new vocabulary describing the WET and we use again a tf. idf representation of this new vocabulary. The representation R imt (T ) is a sparse vector of size equal to the total number of IMTs present in the corpus, where, for each IMT imt: t f. id f T (imt) = t f T (imt) × loд M d f T (imt) (5) where t f T (imt) is the number of articles labeled by the IMT imt that have been mapped to a Wikidata event of type T , d f T (imt) is the number of WETs mapped with at least one article with label imt, M is the total number of W ET s in our dataset. We use these three representations to compute the following similarity between two WETs T i and T j : sim(T i ,T j ) = α × cos(R l (T i ), R l (T j )) (6) + β × cos(R c (T i ), R c (T j )) (7) + γ × cos(R imt (T i ), R imt (T j )) (8) where cos is the cosine similarity measure and the weights α, β and γ are empirically set as 0.
Our agglomerative hierarchical clustering procedure is based on the Ward's method. To cut the resulting dendrogram we used a threshold defined by the elbow method, aiming at finding at which number of clusters the marginal gain of variance will start dropping. According to the graph of Figure  4 , the best threshold is 0. 23. This leads from 933 initial Wikidata event types to 119 clusters in total. Some extracts of the dendrogram are available in Figure  7. We observe that this clustering step enables to group together natural disasters, or summits (NATO, G8,. ) into coherent clusters. We also obtained several Election clusters, with the three main ones which seem related to legislative elections, parliamentary elections and general elections. We empirically evaluated the quality of the clusters for choosing the parameters of our model.
Entity Annotation. When Wikidata properties have a textual value, our goal is to search whether this value is present or not in the news article. We also use DBpedia redirections to collect different variations of the mention to search in the text. 4. 3. 2 Quantity Annotation. When Wikidata properties have a numerical quantity, the problem of finding this information in the news article is much harder. The news articles come as a continuous stream and some reported information generally evolves over time, such as the death toll after a deadly accident. Consequently, a proper semantic annotation of a news article should not always consider the exact quantity value indicated in Wikidata. Therefore, for quantitative values, we introduce some flexibility and a quantity in the text (float or integer) is added to a candidate list of annotations if it is in the +/-10% range of a property and if it is in the first five sentences of the article (supposedly, the earlier in the text, the most relevant to the article main subject).
These candidates are then ranked according to the relevance of the semantic context of the numerical value in the text (considering that the context expresses the type of the attribute). Consequently, given two quantities, the more the context (e. g. "9 were killed on Saturday") is similar with the property type (e. g. number_of_killed rather than magnitude_on_Richter_scale), the more probable the quantity is to be linked with this property. The news article is then annotated with the wikidata property and the most relevant quantity.
4. 3. 3 Serializing the Annotations. We represent both the news articles metadata and the semantic annotations in RDF. We first convert the AFP news article metadata encoded in NewsML in RDF using the rNews vocabulary. For example, the metadata associated with the news article described by Listing 1 indicates that this story was created on 24/03/2015 with the English headline 'No survivors' in Germanwings crash: transport minister. < http :// asrael. eurecom. fr / news /71 e6c1b5 -cbfa -3 f85 -8510 -e200652f6735 > a rnews : Article ; rnews : dateCreated "2015 -03 -24 T12 :41:21 Z "^^xsd : dateTime ; rnews : headline "' No survivors ' in Germanwings crash : transport minister " @en ; dc : subject < http :// cv. iptc. org / newscodes / subjectcode /03013000 > , < http :// cv. iptc. org / newscodes / subjectcode /04015000 > , < http :// cv.
Listing 1: Semantic annotation of a news article The event being described in this news article exists in Wikidata as Q19671417. We create an instance of the schema:Event 12  which is about this news article. In this article, the number of dead people (150) is correctly found (Listing 2). The schema S34 is one of the schema output of the clustering phase described in the section 4. 2. < http :// asrael. eurecom. fr / news /71 e6c1b5 -cbfa -3 f85 -8510 -e200652f6735 > rnews : about < http :// asrael. eurecom. fr / event /71 e6c1b5 -cbfa -3 f85 -8510 -e200652f6735 >. < http :// asrael. eurecom. fr / event /71 e6c1b5 -cbfa -3 f85 -8510 -e200652f6735 > a schema : Event , wd : Q750215 , rnews : Concept ; rdfs : label " ' No survivors ' in Germanwings crash : transport minister " ; dc : identifier " urn : newsml : afp.
com :20150324 T124135Z : TX -PAR -ENS90 :5" ; owl : sameAs wd : Q19671417 ; wdt : P1120 "150" ; wdt : schema " S34 ". Listing 2: Semantic annotation of a news article 4. 3. 4 Annotation Dataset. As a result, we created annotations associated with 370 properties extracted from Wikidata. This dataset can easily be used for a relation extraction task, with a distant supervision system. The evaluation of this automatic annotation is part of our future work. Note that this step is not necessary to build the search engine described hereafter.
Search Engine We load all RDF annotations in a triple store using the Openlink Virtuoso software. The full text of the news article is also indexed in the triple store. We then developed a user interface that performs SPARQL queries to provide views on the data. The Figure  5  depicts the view of the news article  13. On the top right, we show an infobox composed of the main named entities extracted in the article using the ADEL system  [11]. The Figure  6  depicts the view of the news article  14. On the left panel, the user has selected the schema S34 corresponding to crash accident. Therefore, a set of additional properties are automatically suggested as new filters, such as the number of victims. The user has entered the value 50 and the search engines retrieves the news articles that describe crash accidents that have yielded at least this number of victims.
CONCLUSION AND FUTURE WORK In this paper, we develop an event-based search engine capable to query both the structure data of knowledge bases and the unstructured textual content of news articles. This facilitates the navigation through events of the same type and aggregate complementary information about the same event. Furthermore, we produced a semi-automatically annotated text dataset. This dataset could be  used as a distant supervision for training an annotation system. This system could then be able to extract the structure of the events from the news article, even if they are not in Wikidata. We also plan to work on a multilingual support for this system.
In Figs. 2c  and 5a, d , embryos were dehydrated in three steps to absolute methanol and cleared with Murray's clear (2:1 benzyl benzoate/benzyl alcohol) on a glass depression slide. Deep sequencing and quality filter. All deep sequencing libraries were quality controlled: The DNA yield and fragment size distribution were determined by fluorometry and chip-based capillary or polyacrylamide gel electrophoresis, respectively. Libraries were sequenced on the Illumina platforms GAIIx or HiSeq by the Advanced Sequencing Facility of the Francis Crick Institute to produce single or paired-end reads of at least 40 bases. Next-generation capture-C libraries were sequenced on MiSeq with a read length of 150 bases. Sequencing samples and read alignment results are summarised in Supplementary Data 1. The metrics of paired-end alignments such as insert size mean and standard deviation were determined by Picard (CollectInsertSizeMetrics) from the Broad Institute (USA).
The threshold cycle (C t ) was derived from the maximum acceleration of SYBR fluorescence (second derivative maximum method). ChIP-Seq library preparation. 0. 5-10 ng ChIP DNA or 5 ng input DNA were used to prepare single (only Tbx6 ChIP-Seq) or indexed paired-end libraries using the KAPA Hyper Prep Kit  62, [64] [65] [66]. Post-sequencing analysis of ChIP-Seq. Single reads of maximal 50 bases were processed using trim_galore v0. 4. 2 (Babraham Institute, UK) to trim off lowquality bases (default Phred score of 20, i. e. error probability was 0. 01) and adapter contamination from the 3′ end. Processed reads were aligned to the X. tropicalis genome assembly v7. 1 (and v9. 1 for some ChIP-Seq data) running Bowtie2 v2. 2. 9  67  with default settings (Supplementary Data 1). Alignments were converted to the HOMER's tag density format  68  with redundant reads being removed (make-TagDirectory -single -tbp 1 -unique -mapq 10 -fragLength 175 -totalReads all).
Only uniquely aligned reads (i. e. , MAPQ ≥10) were processed. We pooled all input alignments from various developmental stages. This created a comprehensive mappability profile that covered ∼400 million unique base pair positions. All chromatin profiles were position-adjusted and normalised to the effective total of 1 million aligned reads including multimappers (counts per million aligned reads, CPM). Agreeing biological replicates according to ENCODE guidelines  60  were subsequently merged. For stage 10 + (early gastrula stage) β-catenin and Smad2 ChIP-Seq, external datasets  26, 69, 70  were used as biological replicates. HOMER's peak caller was used to identify transcription factor binding sites by virtue of ChIPenriched read alignments (hereafter called peaks): findpeaks -style factor -minDist 175 -fragLength 175 -inputFragLength 175 -fdr 0.
2  56  to improve mapping accuracy across splice junctions. The alignments were sorted by read name using the sort function of samtools v1. 3. 1  75. Exon and intron counts (-t 'exon;intron') were extracted from unstranded (-s 0) alignment files using VERSE v0. 1. 5  76 in fea-tureCounts (default) mode (-z 0). Intron coordinates were adjusted to exclude any overlap with exon annotation. For visualisation, genomic BAM files of biological replicates were merged using samtools v1. 3. 1 and converted to the bigWig format. These genome tracks were normalised to the wigsum of 1 billion excluding any reads with mapping quality <10 using the python script bam2wig. py from RSeQC v2. 6. 4  77. Differential gene expression analysis. Differential expression analysis was performed with both raw exon and intron counts excluding those belonging to ribosomal and mitochondrial RNA using the Bioconductor/R package DESeq2 v1.
16. 1  78. In an effort to find genes with consistent fold changes over time, p-values were generated according to a likelihood ratio test reflecting the probability of rejecting the reduced (∼developmental stage) over the full (∼developmental stage + condition) model. Resulting p-values were adjusted to obtain false discovery rates (FDR) according to the Benjamini-Hochburg procedure with thresholds on Cook's distances and independent filtering being switched off. Equally, regional expression datasets  79  without time series were subjected to likelihood ratio tests with reduced (∼1) and full (∼condition) models for statistical analysis. Fold changes of intronic and exonic transcript levels were calculated for each developmental stage and condition using the mean of DESeq2-normalised read counts from biological replicates.
Both intronic and exonic datasets were filtered for ≥10 DESeq2-normalised read counts detected at least at one developmental stage in all uninjected or DMSO-treated samples. Gene-specific fold changes were removed at developmental stages that yielded <10 normalised read counts in corresponding control samples. Next, the means of intronic and exonic fold changes were calculated across developmental stages. The whole dataset was confined to 3687 genes for which at least 50% reductions (FDR ≤10%) in exonic (default) or intronic counts could be detected in α-amanitin-injected embryos. Regional expression was based on exonic read counts by default unless the intronic fold changes were significantly (FDR≤10%) larger than the exonic fold changes (Supplementary Data 9). Analysing ribosome footprinting and mass-spectrometry data.
The ribosome footprinting reads were trimmed 5′ by 8 bases and 3′ by as many bases overlapping with the adapter sequence 5′-TCGTATGCCGTCTTCTGCTTG-3′ from the 5' end. All trimmed reads between 27 and 32 bases in length were aligned first to ribosomal RNA as listed in the SILVA rRNA database 80 using Bowtie v1. 0. 1  81  with the following parameters:--seedlen 25 (seed length) --seedmms 1 (number of mismatches allowed in the seed) --un (unaligned reads were reported). All non-aligned (rRNA-depleted) reads were mapped to the gene model 6. 0 of X. laevis using Tophat v2. 0. 10 82 --no-novel-juncs (spliced reads must match splice junctions of gene model 6. 0) --no-novel-indels (indel detetion disabled) --segment-length 15 (minimal length of read fragment to be aligned) -GTF v6. 0. gff3 --prefilter-multihits (reads aligned first to whole genome to exclude reads aligning >10 times) --max-multihits 10.