dataesr/scientific-paragraphs-categorization huggingface dataset:
Aligned Dataset We built two aligned datasets that consist of:(i) 256850 instances of Wikipedia summaries aligned with 2. 74M DBpedia triples, and (ii) 358908 instances of Wikipedia summaries aligned with the total of 4. 34M Wikidata triples respectively. The size difference of our datasets is explained in a twofold manner. Firstly, there are Wikipedia biographies without an infobox (i. e. and, thus, without any available triples in the Mapping-based Objects and Literals DBpedia datasets). Secondly, even if they do have an infobox, the retrieved triples that are made available in the DBpedia dumps might not meet the requirements of our task (i. e. Section 4. 2). For example in case the objects of all the triples that allocated to a Wikipedia biography are strings other than dates or number then this summary would be excluded from the respective aligned dataset.
Modelling the Generated Summaries We retained only the first two sentences of each summary in order to reduce the computational cost of our task; summaries that consist of only one sentence were included unaltered. Since it would be impossible to learn a unique vector representation for the main discussed entity of each Wikipedia summary due to the lack of occurrences of the majority of those entities in the datasets, we replaced them with the special <item> token. We used a fixed vocabulary of 30000 and 32000 of the most frequent tokens (i. e. either words or entities) of the summaries that are aligned with the respective DBpedia and Wikidata triples. Similarly to the input triples (i. e. Section 4. 2), all occurrences of numbers in the text are replaced with the special token 0, except for year-objects that are mapped to the special <year> token  [10].
Every out-of-vocabulary word is represented by the special <rare> token. Using a single special token for all the rare entities that have not been included in the fixed target vocabulary would substantially limit the model, causing unnecessary repetition of this particular token in the generated summaries. Inspired by the Multi-Placeholder model  [11] , we first attempt to match a rare entity that has been annotated in the text, in the subjects or the objects of the allocated triples. In case it exists in the triples, then it is replaced by a placeholder token, which consists from the predicate of the triple, a descriptor of the component of the triple that was matched (i. e. obj or subj ), and the instance type of the entity. The instance type of an entity is obtained from the Instance Types dataset.
For example, in the case that the subject of the triple: (dbr:The Adventures of Roderick Random dbo:author dbr:Tobias Smollett) is annotated as a rare entity in the corresponding summary, it is replaced with the special token: dbo:author sub dbo:Book. In the case that a rare entity is matched to the object of the triple: (Kevin Carr dbo:birthPlace dbr:Morpeth, Northumberland) it is replaced with the token: dbo:birthPlaceobj dbo:Settlement. We refer to those placeholders as property-type placeholders. In case the entity does not have a type in the Instance Types dataset, the instance type part of the placeholder is filled by <unk> token (e. g. dbo:birthPlace obj <unk>). If the rare entity is not matched to any subject or object of the set of corresponding triples, then it is replaced by the special token of its instance type.
Modelling the Input Triples Similar to the Wikipedia summaries, we represent the occurrence of the main discussed entity of the corresponding summary as either subject or object of a triple with the special <item> token. A shared, fixed dictionary was used for all subjects, predicates and objects. First, we included all the predicates and entities that occur at least 20 times. Triples with rare predicates are discarded. Every out-ofvocabulary entity is replaced by the special token of its instance type, which is retrieved from the Instance Types dataset. For example the rare entity of dbr:Mamma Mia! is replaced by the dbo:Musical token. In case an infrequent entity is not found in the Instance Types dataset, it is replaced with the special <unk> token. We appended to the source vocabulary only the instance type tokens that occur at least 20 times, and, finally, we used the <resource> token for the rare entities with also infrequent instance types.
In order to increase the homogeneity of the dataset in terms of the number of triples that are aligned with each Wikipedia summary, we limit the number of allocated triples per summary E to: E min + 0. 25σ E ≤ E ≤ E + 1. 5σ E. (15) In case a biography is aligned with less triples then it is excluded for the respective dataset. If a summary is aligned with more triples, we first attempt to exclude potential duplicates (e. g. Fiorenzo Magni dbp:proyears 1945 and Fiorenzo Magni dbp:proyears 1944 would result in the same triple: <item> dbp:proyears <year>). In case their number still exceeds the limit, we retain only the first ones until the threshold is reached. Table  3  shows statistics on the initial and the trainingready version of each corpus. An example of the structure of the datasets is displayed in Table  4.
Experiments We use the above datasets of aligned Wikipedia summaries with DBpedia and Wikidata triples in order to train and evaluate the performance of our neural network models. Both datasets are split into training, validation and test with respective portions of 85%, 10%, and 5%. The implemented architectures have been developed using the Torch 11 software package. Any cleaning or restructuring procedure that has been carried out on the datasets has been conducted with Python scripts. The code will be made publicly available with the notification of acceptance. Our proposed neural network architectures learn to generate a textual summary as a sequence of words and entities. In order to infer the verbalisation of the predicted entities in a generated summary, we experiment with two different approaches which are described in detail below.
Generating Words Along With URIs In this setup, all the entities that have been annotated in the text with DBpedia Spotlight are replaced with their respective URIs. The summaries vocabulary is comprised of words and the entities' URIs. The model thus learns to generates words along with the URIs of entities. In order to improve the generated text further, as a postprocessing step we replace:(i) the <item> token, with its corresponding surface form, and (ii) tokens of DBpedia or Wikidata entities in the text, with their most frequently matched surface form, as these are recorded during our data pre-processing (i. e. Section 4. 1). 11 Torch is a scientific computing package for Lua. It is based on the LuaJIT package.
Training Details We train two different models. The first one is the triple encoder coupled with the GRU-based decoder to which we refer as Triples2GRU; the other is the same triple encoder coupled with the LSTM-based decoder (Triples2-LSTM). For each dataset, we train each model on our task of generating a summary once as a combination of words with URIs (/w URIs, 5. 1) and once as mixture of words and surface form tuples (/w Surf. Form Tuples, 5. 2). For the recurrent component of our networks, we use 1 layer of(i) 650 LSTM cells and (ii) 750 GRUs, resulting in 3. 38M and 3. 375M recurrent connections respectively. We found that increasing the number of layers does not improve the performance of our architectures, whereas the dimensionality of the hidden states plays a crucial role in achieving the best possible results.
001 and 0. 001  [30]. The networks were trained with mini-batch RMSProp with an initial learning rate value of 0. 002. Each update is computed using a minibatch of 85 dataset instances. An l 2 regularisation term over the parameters is also included in the cost function. After the 3nd epoch, the learning rate was decayed by 0. 8 every half epoch. We trained all of our models on a single Titan X (Pas- cal). LSTM-based models complete an epoch of training: (i) in around 25 minutes when trained on the Wikidata dataset, and (ii) 17 minutes when trained on the DBpedia one; the GRU-based architectures require (i) around 22 minutes when trained on the Wikidata dataset, and (ii) 15 minutes when trained on the DBpedia one.
Automatic Evaluation We use perplexity  13  , BLEU  14  , and ROUGE  15  on the validation and test set. Perplexity indicates how well the model learns its training object (i. e. Section 3. 4), whereas BLEU and ROUGE measure how close is the generated text to the actual Wikipedia summary. Essentially, BLEU and ROUGE are complimentary to each other. The first computes a modified version of n-grams precision  16  , whereas the latter computes n-grams recall, of the automatically generated sentences with respect to the empirical Wikipedia summaries. We adapt the code from the evaluation package that was released by Peter Anderson 17  , which was originally implemented to score textual descriptions from images. Perplexity, BLEU 1, BLEU 2, BLEU 3, BLEU 4, and ROUGE L (i. e. an alteration of the original ROUGE that is automatically measured on the longest common subsequence) results are reported in Table  6.
To demonstrate the effectiveness of our system, we compare it to two baselines. First, we compute expected lower bounds for BLEU scores by using a random Wikipedia summary generation baseline. We regard the latter as a particularly strong baseline, due to the fact that Wikipedia biographies tend to follow a limited number of textual patterns. For each triples set on the validation and test set, the random system generates a response by randomly selecting a Wikipedia summary from our training set. Secondly, we use the KenLM toolkit  [33]  in order to build a 5-gram Kneser-Ney (KN) language model. During testing, similarly to the case of our neural network approach, for each triple set in the validation and test set, we use beamsearch with a beam of size 10, in order to generate the 10 most probable summaries.
Human Evaluation Given the exploratory nature of our task, human evaluation is necessary in order to objectively assess the performance of our approach. The human evaluation was conducted using seven researchers, all of whom are experts in the field of the Semantic Web. For each corpora, we compiled a list of 15 randomly selected sets of triples along with the textual summaries that have been generated from each one of our proposed models (i. e. (i) GRU with URIs and surface forms, and (ii) LSTM with URIs and surface forms). The sets of triples are sampled from the test set. We conducted two separate experiments, one for each corpora. Our experiment showed that in our dataset, sets with less triples usually lack enough information for our model to generate a summary (i. e. Section 5. 4). Hence, we included only sets that consist of at least 6 triples.
Discussion Two examples of textual summaries that are generated by our models are shown in Table  7. We selected two representative sets of triples from the test set. The examples illustrate our approaches' capability of generating sentences that couple information from several triples from each set. In the first example, all the models are able to capture the main entity's gender for the input triple set. However, only in the case of the models equipped with surface form tuples, we are able to correctly verbalise the entity of dbr:Actor correctly as "actress" in the text. This is due to the fact that in the biographies dataset, the most frequent surface, with which the entity of dbr:Actor has been associated, is "actor". Subsequently, actor is used as the replacement of all the occurrences of the dbr : Actor entity in the summaries that are generated by our w/ URIs models.
The learned embeddings on the decoder side capture information that is both coupled with the embeddings on the encoder side (e. g. the embedding of the pronouns "She" and "her" are coupled implicitly with the existence of the triple: <item> dbo:occupation dbr:Actress, and their own probability of occurring in the context of the sequentially generated text (e. g. word with its first letter capitalised when it is following a full stop). Consequently, items that have similar semantic meaning find themselves close together in the continuous semantic space. Table  9  shows the nearest neighbours of some of the most frequently occurring entities in our datasets which have been learned by our models. This illustrates our models' capability to successfully infer semantic relationships among entities.
Conclusion To the best of our knowledge this work constitutes the first attempt to use neural networks for Natural Language Generation on top of Semantic Web triples. We propose an end-to-end trainable system that is able to generate a textual summary given a set of triples as input. The generated summary discusses various aspects of the information encoded within the input triple set. Our approach does not require any hand-engineered templates and can be applied to a great variety of domains. We propose an method of building a loosely aligned dataset of DBpedia and Wikidata triples with Wikipedia summaries in order to satisfy the training requirements of our system. Using these datasets, we have demonstrated that our technique is capable of scaling to domains with vocabularies of over 400k words.
Species-Specific and QC Probes Design A total of 169 probes were designed on hypervariable region of the 16S rDNA from the inclusion bacterial strains listed in Table  1. The hypervariable regions of the 16S rDNA have been sequenced prior to make probes design. Multiple alignment analysis using ClustalW was applied to the probe design. The uniqueness of the sequence of each probe was queried against sequences in Genbank database with a BLAST search. Probe quality criteria, namely length of the oligonucleotide between 20 and 25 nucleotides long, equal melting temperature, lack of hairpin and dimer formation, were assessed with Primer 3plus  [26, 27]. Two types of artificial probes were further designed for quality control of the process. The probes were purchased from Eurofins (Les Ulis, France) with their 5 end NH2-modified.
2 × SSC), for 3 min at RT and 300 rpm. The slides were then air-dried by centrifugation and scanned at 10 µm resolution and at 100% of photomultiplier tube gain using the confocal laser scanner DendriScan (Innopsys, Carbonne, France). Fluorescence intensities of spots scanned were numbered by the DendriSoft software version 1. 0. 0 (Innopsys, Carbonne, France), were converted into numerical values and then stored in. gpr data sheets. For statistical analysis, as described below, raw fluorescence data for each spot minus its surrounding fluorescence background were used.
Data Analysis and Pattern-Mapping Statistical Analysis For discrimination analysis, a statistical model (profile) specific to each bacterium was constructed based on Random forest (RF) algorithm  [32]. To this end, we used the caret e1071 and random forest packages within the R statistical analysis software  [33]  available at http://www. r-project. org. RF is a versatile machine learning method capable of performing both regression and classification tasks. It is an "ensemble" approach that generates many decision tree models and aggregates their results. Each of the classification trees is built using a bootstrap sample of the data, i. e. , hybridization intensities from bacteria samples, and at each split the candidate set of variables (i. e. , probes) is a random subset of the variables.
Validation of the Bacterial SPECIES-specific Oligonucleotide Probes on the DendriChip ® The use of 16S rDNA to identify bacteria species has several advantages such as internal transcribed spacer hypervariable regions, abundance of nucleic acid sequences of bacterial genomic regions in databases and availability of reported specific capture probes and universal primers. Nonetheless, several studies on 16S rDNA-based detection have shown inevitable non-specific binding because sequences in this region do not show enough sequence diversity which can hamper the species-specific bacteria detection  [34, 35]. Another and too often forgotten problem when dealing with hybridization between complementary DNA strands is the importance of the surface chemistry onto which nucleic acid fragments termed oligoprobes are immobilized.
Indeed, several reports have shown that the chemistry used to retain the nucleic acids on the surface can dramatically affect the specificity and selectivity of the probes with their complementary DNA/RNA targets  [36] [37] [38]. Taking into account these problems, we designed a large set of 25 mer probes from sequenced 16S rDNA region of eleven pathogen bacteria (inclusion bacteria) commonly found in respiratory tract infection with Primer3Plus software (Table  1 ). We retained a set of 169 probes whose specificity for these 11 bacteria were further queried against sequences from GenBank databases NCBI and EMBL-EBI. The first step for the design of the DendrisChips ® was to retain the probes that were the most specific to each bacterium. Technically, we printed several Dendrislides with the 169 probes and hybridized CY5-labeled PCR amplicons from DNA extracted from our "inclusion" bacteria (Table  1 ).
Machine-Learning Method to Identify Discriminant Probes for Bacterial Species The fact that species-specific probes were found for each of the eleven pathogen bacteria was already a good achievement (Figure  2 ). However, we often found probes designed originally for a given species that showed fluorescent signals to many bacteria (see rectangle box in Figure  2 ). Therefore, to assign a specific signature for each bacterium, we built a database that initially contained 166 hybridization patterns and which was obtained from pure culture bacteria strains, known bacterial strains spiked in DNA extracted from respiratory tract of healthy person and clinical.
Machine-Learning Method to Identify Discriminant Probes for Bacterial Species The fact that species-specific probes were found for each of the eleven pathogen bacteria was already a good achievement (Figure  2 ). However, we often found probes designed originally for a given species that showed fluorescent signals to many bacteria (see rectangle box in Figure  2 ). Therefore, to assign a specific signature for each bacterium, we built a database that initially contained 166 hybridization patterns and which was obtained from pure culture bacteria strains, known bacterial strains spiked in DNA extracted from respiratory tract of healthy person and clinical Therefore, to assign a specific signature for each bacterium, we built a database that initially contained 166 hybridization patterns and which was obtained from pure culture bacteria strains, known bacterial strains spiked in DNA extracted from respiratory tract of healthy person and clinical samples from which the bacterial species had been rigorously identified and confirmed by standard microbiology culture or by PCR.
As shown in Figure  3 , a two dimensional projection of the dataset using canonical discriminant analysis  [39]  allowed discriminating some but not all pathogens. Discrimination was noticed with Bordetella pertussis (Bp), Haemophilus influenzae (Hi), Legionnella pneumophila (Lpn), Klebsiella pneumoniae (Kpn), and Moraxella catharralis (Mca) but the other six bacteria were found in the same cluster, which precludes using this statistic linear method for complete discrimination. Diagnostics 2018, 8, x FOR PEER REVIEW 9 of 17 samples from which the bacterial species had been rigorously identified and confirmed by standard microbiology culture or by PCR. As shown in Figure  3 , a two dimensional projection of the dataset using canonical discriminant analysis  [39]  allowed discriminating some but not all pathogens.
Discrimination was noticed with Bordetella pertussis (Bp), Haemophilus influenzae (Hi), Legionnella pneumophila (Lpn), Klebsiella pneumoniae (Kpn), and Moraxella catharralis (Mca) but the other six bacteria were found in the same cluster, which precludes using this statistic linear method for complete discrimination. We therefore employed Random Forests (RF) method  [32]  since this machine (statistical) learning method is more dedicated to prediction problems that are non-linear and that involve complex high-order interaction effects, which is actually the case with microarrays  [40]. With this method, we generated training datasets employing the hybridization data from a 10-fold crossvalidation experiment using inclusion and exclusion bacteria. The training set from this crossvalidation was equally divided into 10 different data subsets.
Nine out of ten were used to train the classifier, while the tenth subset was used as the test set. This procedure was repeated ten times, with a different subset being used as the test set. According to this process, a two-class (present/absent) random forest model was built for each bacterium with t decision trees, generated at random, using a different subset of the training dataset with replacement to train each tree. The remaining training data (Out-Of-Bag (OOB)) were used to estimate prediction error and variable (i. e. , probe) importance for each bacterium. At each node of a tree, a number m of variables was selected at random, where m < M (total number of variables). The node was split using the best variable among the m ones. Each tree was grown to the largest extent possible. Thus, when a new sample entered the prediction We therefore employed Random Forests (RF) method  [32]  since this machine (statistical) learning method is more dedicated to prediction problems that are non-linear and that involve complex high-order interaction effects, which is actually the case with microarrays  [40].
With this method, we generated training datasets employing the hybridization data from a 10-fold cross-validation experiment using inclusion and exclusion bacteria. The training set from this cross-validation was equally divided into 10 different data subsets. Nine out of ten were used to train the classifier, while the tenth subset was used as the test set. This procedure was repeated ten times, with a different subset being used as the test set. According to this process, a two-class (present/absent) random forest model was built for each bacterium with t decision trees, generated at random, using a different subset of the training dataset with replacement to train each tree. The remaining training data (Out-Of-Bag (OOB)) were used to estimate prediction error and variable (i. e. , probe) importance for each bacterium.
At each node of a tree, a number m of variables was selected at random, where m < M (total number of variables). The node was split using the best variable among the m ones. Each tree was grown to the largest extent possible. Thus, when a new sample entered the prediction system, the corresponding hybridization data of this sample run down all of the t trees to yield a result that is obtained by aggregating the predictions of the t trees (i. e. , majority votes for classification). This process led to a probability of the presence or absence of any of the 11 bacteria in the sample. With this reference dataset, the random forest (RF) model showed an average misclassification rate of less than 10% for eight of the 11 bacterial strains. In the case of Legionella pneumophila, Chlamydiae pneumoniae and Streptococcus pneumoniae, this rate reached 20%, which could be explained by a reduced signal intensity of the probe sets due to matrix effects and/or a low number of independent samples in the training dataset.