dataesr/scientific-paragraphs-categorization huggingface dataset:
Therefore, before presenting the performance of SQUALL in the QALD-3 challenge (Section 7. 3), we first assess the grammaticality of QALD questions in SQUALL (Section 7. 1), and the naturalness of their reformulation in SQUALL (Section 7. 2). We do not evaluate the efficiency of our system in detail as this is not an issue: the 100 questions of the challenge are translated in 6s. We do not present either a usability study on the formulation of SQUALL queries because we consider that guided input is necessary to overcome the habitability problem  [12] , which is not the object of this paper and is already addressed in existing work  [13]. In our evaluation, we focus on the 99 English test questions (question 18 does not exists in QALD-3) for the DBpedia dataset. Questions in other languages are out of the scope of this paper.
Grammaticality of QALD questions In this subsection, we assess the grammaticality of QALD questions in SQUALL. In other words, we answer for each question whether it can be parsed as a SQUALL sentence assuming the right lexicon and the right schema have been defined. For example, the first question "Which German cities have more than 250000 inhabitants?" is a correct SQUALL sentence, if we assume that "German" is an adjective, "cities" is a noun, and "inhabitants" is a relational noun. It also assumes that there is a relation from each city to each known inhabitant. This is unlikely in a dataset like DBpedia, but quite possible in other contexts (e. g. , in a civil registry). Therefore, grammaticality evaluates the coverage of natural language by SQUALL, but not its adequation to a particular RDF dataset.
Naturalness of SQUALL questions The cost of using a CNL instead of spontaneous language is the need for reformulation, and the benefits is more precision and less ambiguity. We have reformulated the QALD questions into SQUALL sentences with the double objective to keep them as natural as possible, and to match the RDF schema of DBpedia. From the training phase, we had already learned some of the DBpedia vocabulary, and other URIs were found manually with Google searches and DBpedia browsing. We spent on average a few minutes per question for the reformulation phase. The SQUALL reformulations of the 99 test questions, as well as the reformulations of the 100 training questions, are available on SQUALL's homepage  11. For illustration purposes, Table  4  lists a few original questions along with their SQUALL reformulation.
Alternately, the replacement could be done on the intermediate representation of even SPARQL query, with the benefit that the semantic role of words (e. g. , class, property) would be explicit. The latter approach is analogue to and could reuse the techniques of ontology-based query answering systems (see  Section 8). Some discrepancies are deeper in that they exhibit conceptual differences between natural language and the ontology. We shortly discuss three cases: • "List all episodes of the first season of the HBO television series The Sopranos!" vs "List all TelevisionEpisode-s whose series is res:The Sopranos and whose seasonNumber is 1. ". In natural language, an episode is linked to a season, which in turn is linked to a series. In DBpedia, an episode is linked to a series, on one hand, and to a season number, on the other hand.
Alternatively, assuming linguistic knowledge, the word "location" could be mapped to the graph pattern { ?x dbo:location ?y } UNION { ?x dbo:headquarter ?y } UNION { ?x dbo:locationCity ?y } where ?x and ?y respectively stand for the subject and object of the relation. Such graph patterns could easily be exploited in the translation from the intermediate representation to SPARQL without the need to change the SQUALL language and its parsing. Another problem related to heterogeneity is that some expected domain and range axioms are not verified in some cases. For example, in question 19 "Give me all people that were born in Vienna and died in Berlin. ", 2 out of the 6 expected answers are not instances of the class Person. This is why our reformulation "Give me all Person-s whose birthPlace is res:Vienna and whose deathPlace is res:Berlin.
Therefore, it relies on a reformulation of original questions in the vocabulary of the target RDF dataset, and its score in the challenge (F-measure = 0. 90) is hence not directly comparable to other participants' score. However, it demonstrates that complex natural questions can be reliably translated to SPARQL. SQUALL better handles complex syntactic constructs such as nested relative clauses, and coordinations. It can generate arbitrarily many triples, it supports all features of QALD questions (comparatives, superlatives, counting), and more (disjunction, negation, other aggregations, updates, etc. ). Many CNLs have been defined in the past decades  [17] , but SQUALL is the only one that targets SPARQL queries and updates. Other CNLs for the Semantic Web rather target ontological axioms (facts and rules).
ACE  [19]  is a general purpose CNL that can target various formalisms, and there are a number of more specialized CNLs targeting OWL axioms (e. g. , SOS, Rabbit)  [20]. In Kuhn's survey  [17] , SQUALL is evaluated as having: a high precision (P 5 ), a medium expressiveness compared to NL (E 3 ), natural sentences (N 4 ), and a short description (S 4 ). In comparison, a natural language is evaluated as P 1 E 5 N 5 S 1 , i. e. , maximally expressive and natural, but minimally precise and short-defined; and SPARQL is evaluated as P 5 E 3 N 1 S 5. Guided navigation. Scalewelis  [41]  (F-measure = 0. 33) is the only QALD-3 participant based on guided navigation. Like SQUALL, it is based on a reformulation of original queries, but users are guided step by step, and do not have to actually write the query.
Conclusion and perspectives In the spectrum that goes from full natural language to formal languages like SPARQL, SQUALL (Semantic Query and Update High-Level Language) occupies a unique position. It offers the same expressiveness as SPARQL for querying and updating RDF data, and still qualifies as a controlled natural language (CNL). This means that among the natural language interfaces, SQUALL is the one that is by far the most expressive; and that among the formal languages, SQUALL is the one that is the most natural. The current limit of SQUALL is that end-users have to comply with its controlled syntax, and have to know the RDF vocabulary (i. e. , Which are the classes and properties?). However, the important result is that SQUALL can be used as a substitute for SPARQL because this entails no loss, neither in expressiveness, nor in precision.
• applying mapping techniques on the intermediate representation. SQUALL's intermediate representation is compatible with those of OQA systems, only being more complex combinations of triple patterns. In simple cases, mapping simply replaces words by URIs (e. g. , "mayor" by dbo:leader). In more complex cases, mapping may replace combinations of triples by other combinations of triples (e. g. , a count of "entrances" by the property dbp:entranceCount). • using SQUALL as an intermediate representation. If full natural language is to be accepted as input, a solution is to use mature NLP tools (e. g. , Stanford NLP parser) to parse a spontaneous sentence, and then translate the resulting dependency graph to SQUALL, which is arguably much closer to NL than SPARQL is. • using guided input and dialog.
TASSo -A Layered Model In order to better understand the TASSo framework, we make use of the layered model shown in Figure  2. The framework is divided such that each layer can run isolated from each other. This model, originating from the classical decomposition  13  is extended and reorganised according to the aforementioned challenges. As previously explained, the TASSo platform builds on top of the WoT, meaning that each module is considered as a thing. Following this assumption, every functionality is accessible through a REST API.
This layer also handles the distribution of historical data over multiple databases. We here advocate, when possible, to store data directly on the device producing it. As this is not possible on very constrained hardware, the producing devices will push it on other nodes offering storage services, selected based on their proximity. In order to limit the amount of stored data, we introduce a registration mechanism, where clients such as the Adaptation Level requiring historical data will announce their needs by specifying which resource has to be stored for how long.
Results: Here we present ONCOCNV, a method that includes a multifactor normalization and annotation technique enabling the detection of large copy number changes from amplicon sequencing data. We validated our approach on high and low amplicon density datasets and demonstrated that ONCOCNV can achieve a precision comparable with that of array CGH techniques in detecting copy number aberrations. Thus, ONCOCNV applied on amplicon sequencing data would make the use of additional array CGH or SNP array experiments unnecessary.
However, how to reliably detect copy number changes, in particular gene deletions and amplifications, in amplicon sequencing data is still open to discussion. Here, we focus on the identification of copy number alterations (CNAs) also known as large copy number changes (CNVs) of the actionable genes targeted by amplicon sequencing. Although there are several algorithms to detect CNAs in exome sequencing data  (Amarasinghe et al. , 2013; Boeva et al. , 2011; Li et al. , 2012) , it is questionable as to whether the same approaches can be efficient when applied to amplicon sequencing data. First, amplicon sequencing targets fewer regions and thus provides less information than exome sequencing datasets (510 000 exons versus 4200 000 exons); consequently, data normalization can be less effective on amplicon sequencing data.
Second, because of the different protocols used for library preparation, amplicon sequencing data can have various biases. Importantly, while for exome sequencing experiments an effort has been made to uniform exon coverage, amplicon sequencing technology emphasizes extremely high depth of coverage with less regard to coverage homogeneity. Here we provide a solution to the challenging question of extracting CNAs from amplicon sequencing data by (i) defining a method to normalize read coverage with a small set of normal control samples and (ii) assigning statistical significance to putative CNAs resulting from the segmentation of normalized profiles. We validated the proposed method on (A) a high amplicon density dataset of eight tumor samples for which array comparative genomic hybridization (array CGH) profiles were available, (B) a high amplicon density dataset of 30 ErbB2-positive ovarian cancer samples and (C) a low amplicon density dataset of 30 tumors, coupled with single nucleotide polymorphism (SNP) array data.
Datasets To validate our method, we used three experimental amplicon sequencing datasets: A, B and C. The high amplicon density tumor DNA for datasets A and B was subjected to amplicon sequencing with target selection using the Ion Xpress TM Plus Fragment Library Kit by Life Technologies (panel of 406 genes including 46500 exons covered by 415 000 amplicons). Dataset C is a part of the SHIVA clinical trial (Le  Tourneau et al. , 2012). This dataset was generated using the Ion AmpliSeq TM Cancer Panel V1 in combination with the Ion AmpliSeq TM Library Kit 2. 0 by Life Technologies; for our analysis, we used 11 genes covered by 96 amplicons. We used 15 diploid samples (high amplicon density, control dataset X) and 6 diploid samples (low amplicon density, control dataset Y) to create the baseline for the normalization of datasets A/B and C, respectively.
Read counting In the analysis of high-throughput sequencing data, an appropriate definition of read counts (RCs) is of high importance. In whole-genome sequencing data, RC refers to the number of reads starting in a given window  (Boeva et al. , 2011). In WES and RNA sequencing data analysis, RC corresponds to the number of read mappings overlapping each exon  (Amarasinghe et al. , 2013). When dealing with amplicon sequencing data, one could apply a similar approach and reasoning in terms of exons. However, this may lead to a loss of information. In amplicon sequencing, one exon can be targeted using two or more closely located or overlapping amplicons; averaging this information will decrease the sensitivity of CNA detection. In ONCOCNV, we therefore implemented an intuitive procedure to calculate RCs: each read is assigned to only one amplicon region, the one with which the read alignment has the maximum overlap.
Normalization with respect to CG-content For target sequencing data, as for whole-genome sequencing data  (Boeva et al. , 2011) , we observe a significant GC-content bias (Fig. 2A  and Supplementary Fig. S2A ). The shape of the GC-content dependency may vary even between datasets generated within the same laboratory (Supplementary Fig. S2B ). We correct for the GC-content bias by using local polynomial regression fitting [LOESS  (Cleveland et al. , 1992) , R package stats, degree = 2]. The resulting values NRC GC do not depend on the GC content of the amplicon sequences (Supplementary Fig. S2C ).
S5C ). In our example (datasets A and B), the first three components account for 90. 5% of the total variance in the NRCs of the diploid control samples, whereas the first component explains only 79% of the variance (Supplementary Fig. S6A ). Thus, by default, we keep the first three PCs as baseline. This parameter may, however, be changed by the user after examination of the explained variance in the output of ONCOCNV. Also, if the number of control samples (n) is less than four, only (n -1) PCs will be kept. At least two control samples are necessary to run ONCOCNV. When several tumor/matched normal datasets are present, the baseline should be created using all the control samples available. For the final normalization of the sample RCs, we use a linear regression over the baseline. The resulting profile, NRC Final , shows lower variance than NRC Len and no directional bias (Supplementary Fig.
Correction of control sample RCs for gender A control dataset is often generated from a mixture of male and female samples. To correct the RCs in the control samples for gender, we apply the following strategy. After the first normalization step (normalization with respect to library size), we calculate M i =meanðNRC Len on chrXÞ=meanðNRC Lib Þ for each control sample i. Then we choose the optimal model with one or two components for M i , according to the Bayesian information criterion for expectationmaximization initialized by hierarchical clustering for parameterized Gaussian mixture models  (Fraley and Raftery, 2002). If the best model includes two components, we can conclude that our dataset represents a mixture of male and female samples: samples falling in a cluster with smaller M i are male and samples falling in a cluster with higher M i are female.
This is based on the idea that the mean values of NRCs of segments (segment mean values) should roughly correspond to integer copy number changes (Supplementary Fig. S9 ). In other words, the segment mean values corresponding to the same number of copies should cluster together. A similar approach was used by  Gusnanto et al. (2012)  to detect CNAs in whole-genome sequencing data. Using R package mclust  (Fraley et al. , 2012) , we cluster weighted means of segment log ratio values. To prevent clustering errors, we add Gaussian noise to the data (Supplementary Fig. S9B  and C ). For each segment, the standard deviation of the added random variable is equal to the standard error of the mean, and thus depends on the length of the segment. Our approach is nearly identical to the classic non-parametric density estimation using the Parzen window method  (Parzen, 1962).
Comparison with WES-specific methods: NextGENe and ADTEx We decided to compare our CNA calling method in ultra-deep targeted sequencing data with two WES-specific tools: ADTEx  (Amarasinghe et al. , 2013) , a method based on hidden Markov models (HMMs) for CNA calling in targeted (exome) sequencing data, and NextGENe, commercial software designed by SoftGenetics. Although both ADTEx and NextGENe were tested on WES data only, conceptually there is no reason why these methods should not be applied to a smaller panel of targeted genes ($400 in our case). We applied ONCOCNV, ADTEx and NextGENe to a high amplicon depth sequencing dataset of eight tumor samples. The array CGHs performed for these samples showed different CNA complexity levels (Supplementary Fig. S13 ). Although we also compared our method with CONTRA  (Li et al.
Comparison of normalization results The values resulting from the ONCOCNV normalization procedure correlate highly with the log ratio values of the array CGH experiments performed on the same samples (Figs  2C, D  and 3A , Supplementary Table  S1 ). The correlation was higher for ONCOCNV NRCs than for RCs normalized by ADTEx and NextGENe. For example, for sample A1, the linear correlation between CGH log ratio values and ONCOCNV NRCs was 0. 83, compared with 0. 65 and 0. 62 for ADTEx and NextGENe, respectively (Supplementary Fig. S15 ). This demonstrates that the normalization procedure used in ONCOCNV has clear advantages over those used in CNA calling methods developed for exome sequencing data analysis. We confirmed that correction for the GC content plays a key role in amplicon data normalization (Supplementary Fig.
Validation on 30 ErbB2-positive tumor samples To validate our method further, we applied ONCOCNV to analyze amplicon sequencing data generated for 30 ErbB2-positive ovarian cancer samples. The copy number status of the ErbB2 gene was assessed via fluorescence in situ hybridization using FFPE tissue sections. The gene ErbB2 (also known as HER2) encodes a receptor tyrosine kinase whose activation plays an important role in cell survival and drives cell proliferation  (Tai et al. , 2010). ErbB2 is frequently overexpressed in breast, gastric, ovarian and prostate cancer. ErbB2 overexpression most often occurs because of a gene amplification  (Pauletti et al. , 1996). Several drug compounds are used in clinics to block ErbB2 signaling in the case of overexpression of ErbB2. Thus, a personalized treatment is available for patients with amplification of the ErbB2 gene (approved in clinical practice for breast and gastric cancer patients).
Validation on the SHIVA clinical trial dataset We applied ONCOCNV to a low amplicon density dataset of tumors sequenced in the SHIVA clinical trial (Le  Tourneau et al. , 2012). The data corresponded to a targeted sequencing of 190 amplicons covering the exons of 46 actionable genes. The number of amplicons varied from 1 to 17 per gene. As our algorithm was designed to deal with data containing multiple amplicon regions per gene, for our analysis, we kept 11 genes. Each of them contained at least seven amplicon regions. The 11 genes selected were located on 10 different chromosomes. In the SHIVA trial, a dual approach is used: amplicon sequencing is used to detect single nucleotide variants and indels, whereas SNP arrays provide information about the copy number status of the genes and contamination by normal cells.
We wanted to test whether amplicon sequencing-based detection of CNAs could replace currently used SNP arrays. As when using only 11 genes, it is often impossible to assess the correct zero level, we considered relative gains and losses in the calculation of CNA prediction accuracy (see Discussion). For example, if the SNP array technique detected a one copy loss of the first five genes whereas ONCOCNV predicted a one copy gain of the last six genes, we considered that the latter prediction was correct (the absolute prediction accuracy is shown in Supplementary Fig. S20 ). We observed a high accuracy of CNA detection for samples with low contamination by normal cells: for all but one sample with 518% contamination, the CNA detection accuracy was at least 90% (Fig. 6 ). As we expected, our ability to detect CNAs in low amplicon density datasets strongly depends on the normal contamination (Pearson correlation coefficient 0.
DISCUSSION We proposed a method to analyze amplicon sequencing data to be used to detect CNAs involving at least several exons. Our method, ONCOCNV, demonstrated a high accuracy of CNA predictions for the samples for which we had the results of array CGH analysis as gold standard. ONCOCNV significantly outperformed methods developed for exome sequencing data analysis. This is because of the extensive normalization strategy, which includes normalization for the GC content, amplicon length (via LOESS correction) and other technology-specific biases (via PCA of the control samples). An important advantage of ONCOCNV is the refinement of breakpoints falling initially within a gene (i. e. after the segmentation of NRC profiles). ONCOCNV reanalyzes the NRCs upstream and downstream of the putative breakpoint and decides whether intragenic breakpoint has indeed taken place and that it is not the result of inaccurate segmentation (Supplementary Fig.
S12 ). In our dataset, we observed samples (e. g. controls 3 and 9, Supplementary Fig. S2A ) where a large fraction of amplicons (26% for both) have zero or extremely low RCs. We would like to point out that ONCOCNV is able to detect such artifacts and analyze CNAs correctly in the corresponding samples. The prediction of gains and losses by ONCOCNV includes information on average log ðNRCÞ. In addition, it estimates possible copy numbers. This estimate is based on the assumption that the tumor sample genome is diploid and that the sample is not contaminated by normal cells. We are aware that this assumption might be far from being true. However, we prefer to provide an estimate of the probable copy number to allow the user to distinguish between gains and amplifications, and between one and two allele losses.
The question of predicting the exact copy number cannot be solved by using copy number profiles alone. The assessment of B allele frequencies provides a possible solution  (Popova et al. , 2009). We did not use this solution in ONCOCNV because of the high dispersion of B allele frequencies in the amplicon sequencing data (data not shown). CNAs detected by ONCOCNV (and by other software whose algorithm is based only on the analysis of copy number profile) are relative to the evaluated zero level. For instance, if we measure the copy numbers of only a pair of genes, we will not be able to see a gain or loss if this happens simultaneously in both genes. Thus, to properly estimate the zero level, many genes should be included in the targeted gene panel. As we expected, we detected a fall in the CNA detection accuracy for samples that were highly contaminated by normal cells.
In this situation, the noise in measurements is comparable with the expected difference between measurements in the case of one copy gain or loss. As our approach does not use information about B allele frequencies, certain combinations of normal contamination and tumor ploidy significantly impair the efficiency of the technique. This is especially true when there are a small number of measurements (amplicons) per gene. We would like to point out that ONCOCNV does not detect one-amplicon amplifications/deletions. We believe the noise in current amplicon-based technologies does not allow detection of such CNVs with high certainty. However, there are methods, such as CONTRA  (Li et al. , 2012) , which predict the CNVs of single amplicon regions. We do not position ONCOCNV as a tool for exome sequencing data analysis.