dataesr/scientific-paragraphs-categorization huggingface dataset:
Methods Objectives -This study aimed at predicting the survival of patients affected by various tumor types from their gene expression analysis. This is a classical task with gold-standard datasets that we used to evaluate methodological improvements (Supplementary Table  1 ). We have developed a new format of survival data to train deep learning models, a prior-knowledge based dimension reduction and a transfer learning strategy. We hypothesized that these methods should help model performance and interpretability. Labels -Survival -We used the publicly available survival data of the TCGA database from Liu et al. 38. The top 5% of patients with the highest overall survival were removed because they were considered cured (by surgery, as their overall survival was higher than nine years). Patients with no follow-up were also removed (i.
Labels -Organ - For the pre-training on organ prediction, we have pooled GTEx and TCGA data  8, 26. GTEx concerned the analysis of normal organs and TCGA the analysis of primary tumors classified by organs of origin. A random split (90%, 10%) was performed on the organ data set (no test cohort was required as the organ data were used for the pre-training task). Input -RNA sequencing -Inputs used to feed DeepOS were gene expression values estimated from RNA-seq. RNA-seq was the most frequent analysis commonly performed in both GTEx and TCGA and allowed to gather a maximum of examples matched with the labels described above. RNA-seq is a multistep process. RNA is first extracted from the tissue sample and sequenced. For TCGA, a vast majority of primary tumor samples came from surgical interventions while for GTEx, it came from nondiseased tissue samples from human donors.
Gene expression is then estimated by the number of RNA fragments corresponding to a genome locus from a sequenced sample. TCGA and GTEx gene expressions were analyzed with the same bioinformatic pipeline from raw sequencing data and available in Recount2  21. Gene expression was estimated in TPM (transcripts per million) with the Rail-RNA pipeline  39. TPM followed a Poisson distribution, so we log-transformed and scaled the data matrix using natural logarithm. Input -Dimension reduction on prior-knowledge -RNA-seq gene expression data is usually highly dimensional (~23k protein coding genes plus non-coding regions) which can be a source of overfitting during the learning step of deep neural networks  24. To reduce the dimensions of input data, we selected important cancer-related genes based on prior-knowledge.
MSigDB database  19  provided gene lists related to cancer hallmark and LM22 provided immune cell line specific gene lists  20  that are important mechanisms for cancer evolution. These two sources comprised a total of 4,499 genes also found in GTEx and TCGA RNA-seq data. Inputstudy of the gene selection -For comparison, we trained models with random selections of 4,499 input genes, excluding the ones found from cancer hallmarks and LM22. We trained those models on the same RNA-seq data, using the same workflow (hyper-parameter search and selection of the best model on validation C-index). We replicated the experiment twice, each time with different selections of random genes (#1 and #2). Models' architecture -DeepOS model is a multilayers perceptron (MLP), which consists of at least three types of layers: the input layer, hidden layers and the output layer.
Evaluation of the model -Metric for survival prediction -We used the concordance correlation coefficient (concordance index, or C-index) to evaluate survival models with censored data  15, 16. C-index represents the proportion of concordant pairs divided by the total number of possible evaluation pairs. For example, if a patient A has deceased at time tA and a patient B has been censored at time tB, they can still be compared if tA<tB. If the model gives a prediction pA for patient A and pB for patient B, the pair can be qualified as concordant if pA< pB and non-concordant otherwise. If tA>tB then it is not possible to evaluate this pair and it will not count as a possible evaluation pair. We also computed the survival AUROC using sklearn (sksurv. metrics. cumulative_dynamic_auc), which is a cumulative area under the ROC curve adapted to censored data  43.
Evaluation of the model -Performances by survival time - To further evaluate DeepOS predictions, we have assessed the performance depending on survival time. We have sorted the test cohort by OS and divided the cohort into 10 subgroups, each group composed of 50 patients. We have then computed the Cindex of each subgroup. Evaluation of the model -Learning curves -In order to evaluate the effect of the training set size on the model performances for survival prediction, we have generated learning curves. We used the validation cohort of 500 patients, given by the data split described previously. For the training set, data were iteratively and randomly added, from 500 to 5,529 samples, with steps of 500 samples. Every time a new training and hyper-parameters search was launched. The C-indexes were computed for the training and validation sets with plots for the median, the first and the third quartile.
Acknowledgments The authors are grateful to their colleagues and collaborators for their advices and support for this work and specifically:  Rebecca Clodion ,  Roger Sun ,  Eric Angevin ,  Antoine Hollebecque ,  Daniel Gautheret ,  Stefan Michiels ,  Fabrice André ,  Andrei Zinovyev ,  Laurence Calzone ,  Emmanuel Barillot ,  Eric Deutsch ,  Jean-Yves Blay ,  Jean-Charles Soria , and  Christophe Massard. The results shown here are based upon data generated by the  TCGA Research Network : https://www. cancer. gov/tcga.
3 3 Figure 2: Pan-cancer survival data descriptiona, Flow-chart of the survival cohort. Pan-cancer RNA-seq and survival and clinical data were retrieved from the TCGA dataset. After selection of the 6,529 samples fulfilling the selection criteria, we used an 80%/10%/10% random split rule to create the training, validation and testing datasets. b,c, Kaplan-Meier survival curves of the whole cohort (b) and per tumor types (c). TCGA study abbreviations, median overall survival per tumor type and 95% confidence intervals are detailed in Supplementary Table3.
HIFUN is a high-level query language for expressing analytic queries of big datasets, offering a clear separation between the conceptual layer, where analytic queries are defined independently of the nature and location of data, and the physical layer, where queries are evaluated. In this paper, we present a methodology based on the HIFUN language, and the corresponding algorithms for the incremental evaluation of continuous queries. In essence, our approach is able to process the most recent data batch by exploiting already computed information, without requiring the evaluation of the query over the complete dataset. We present the generic algorithm which we translated to both SQL and MapReduce using SPARK; it implements various query rewriting methods. We demonstrate the effectiveness of our approach in temrs of query answering efficiency.
Introduction Data emanating from high-speed streams are prevalent everywhere in today's data eco-system. Examples of data streams include IoT data  [1] , data series  [2] , network traffic data  [3] , financial tickers  [4] , health care transactions  [5, 6] , the Linked Open Cloud  [7, 8]  and so on. In order to extract knowledge and find useful patterns  [9, 10] , these data need to be rapidly analyzed and processed. However, this is a challenge, as new data batches arrive continuously at high speed, and efficient data processing algorithms are needed. The research community has already provided open-source and commercial distributed batch processing systems such as Hadoop  [11] , MapReduce  [12]  and Oracle Cloud Infrastructure  [13] , that allow query processing over static and historical datasets, enabling scalable parallel analytics.
However, such approaches are untenable in real world problems, and continuous query processing techniques have been developed to overcome this limitation by utilizing the storage space to produce results in an incremental fashion rather than aiming to process all the available data  [24, 25]. Continuous query processing is a major challenge in a streaming context. A continuous query is a query which is evaluated automatically and periodically over a dataset that changes over time  [26] , and allows the user to retrieve new results from a dataset without the need to issue the same query repeatedly. The results of continuous queries are usually fed to dashboards, in large enterprises, to provide support in the decision-making process  [27]. In addition, besides their ever increasing volume, datasets change frequently, and as such, results to continuous queries have to be updated at short intervals.
Motivation and Contributions As new data are constantly arriving at a high rate, datasets are growing rapidly, and the re-evaluations of queries are incurring delays. The problem on which we focus in this work is incremental query evaluation; that is, given the answer to a given query at time t, on dataset D, how can one find the answer to the same query at time t on dataset D , assuming that the answer at time t has been saved andthe results become stale and stagnant over time. Incremental processing is an auspicious approach for refreshing mining results, as it uses previously saved results to avoid the cost of re-computation from scratch. There is an obvious relationship between continuous queries and materialized views  [28] , since a materialized view is a derived database relation whose contents are periodically updated by either a complete or an incremental refresh based on a query.
Incremental view maintenance methods  [29]  exploit differential algorithms to re-evaluate the view expression in order to enable the incremental updating of materialized views. However, in our case, although the target was the same, the methods used were different: we devised an algorithm for incremental processing of continuous queries that processes only the most recent data partition and exploits already computed information, without requiring evaluating the query over the entire dataset. In this paper, we study this problem in the context of HIFUN, a recently proposed high-level functional language of analytic queries  [30, 31]. Two distinctive features of HIFUN are that (a) analytic queries and their answers are defined and studied in the abstract, independently of the structure and location of the data, and (b) each HIFUN query can be mapped either to an SQL group-by query or to a MapReduce job.
HIFUN separates the conceptual and the physical levels so that one can express analysis tasks as queries at the conceptual level independently of how their evaluations are done at the physical level using existing mechanisms (e. g. , SQL engines, Graph engines or MapReduce) which perform the actual evaluation of queries. In  [32] , the authors investigated how HIFUN can be used for easing the formulation of analytic queries over RDF data. In this works we focus on relational and unstructured data. For example, if the data reside in a relational database, then the mapping mechanism can use SQL in order to evaluate the conceptual query, whereas if the data are unstructured, then mapping mechanisms need to be algorithms specialized to the evaluation based on MapReduce programming model. Our approach exploits the Spark Streaming and the Spark Structured Streaming in the physical layer to implement an incremental evaluation algorithm using HIFUN semantics.
• Then we map the generic algorithm to the physical level, implementing the evaluation mechanism both as SQL queries and MapReduce tasks. • Further, we show how HIFUN query rewriting can be implemented in the physical layer. • Finally, we experimentally show that our implementation provides considerable benefits in terms of efficiency. Although an incremental evaluation of queries based on the HIFUN language was already published  [33] , and although similar rewritings have been used extensively on relational databases  [34] , this is the first time that similar rewritings have been proposed for the HIFUN functional language. In addition, they are not trivially implemented either. In this paper, besides the functional layer, we also show how to implement it in two completely different physical setups.
The remainder of the paper is organized as follows. In Section 2, we present related work, and in Section 3 the theoretical framework and the query language model used. Then in Section 4, we describe our algorithms for incremental evaluations of continuous queries at the HIFUN level, and in Section 5, the corresponding implementation at the physical level. In Section 6, we present an evaluation of our system, and finally, in Section 6, we conclude the paper and discuss possible directions for future work.
Related Work To respond to the intensive computational requirements of massive dataset analysis, a number of frameworks are already available in the literature that deal with continuous queries such as Spark Streaming and Flink  [35]. Continuous queries were introduced for the first time in the TQL language in Tapestry  [36] , for content-based filtering of an append-only email and message posting database. Conceptually, a restricted subset of the SQL was used, and it was converted into an incremental query that was defined to retrieve all answers obtained in an interval of t seconds. The incremental query was issued continuously, every t seconds, and the union of answers returned constituted the answer to the continuous query. An incremental evaluation approach was used, to avoid the repetitive computations and to return only the new results to the users.
Besides this, other architectures have been proposed as well to process continuous queries, such as the NiagaraCQ  [37]  and the OpenCQ  [38]. Both systems use continuous queries over changing data, similarly to Tapestry, periodically executing individual queries. However, all aforementioned approaches focus on relational databases and thus do not deal with issues that appear when moving to semi-and unstructured data. STREAM  [26]  is an all-purpose framework that focuses on addressing the demands imposed by data streams on data management. The authors pay attention to memory management in order to enable approximate query answering. In particular, one of the project's goals is to understand how to efficiently run queries given a bounded amount of memory. The system can process streaming data, but it is designed as a centralized system.
Preliminaries-The HIFUN Query Language In this section, we describe briefly the conceptual model of the HIFUN language. For further details on the HIFUN language, the interested reader is referred to some relevant papers  [30, 31]. The basic notion used in defining HIFUN is that of an attribute of a dataset. In HIFUN, each attribute is seen as a function of the dataset for some domain of values. For example, if the dataset D is a set of tweets, then the attribute "character count" (denoted as cc) is seen as a function cc : D → Count, such that, for each tweet t, cc(t) is the number of characters in t. Next we provide an example that we will use throughout the paper. Consider a distribution center (e. g. , Walmart) which delivers products of various types in a number of branches, and suppose that D is the set of all delivery invoices collected over a year.
Analysis Context Now, the attributes d, b, p and q of our running example are "factual" or "direct" attributes of D in the sense that their values appear on the delivery invoices. However, apart from these attributes, analysts might be interested in attributes that are not direct but can be "derived" from the direct attributes. Figure  1  shows the direct attributes together with several derived attributes: attribute r can be derived from attribute b based on geographical information on the location of each branch; and attributes s and c can be derived from a product's master table. The set of all attributes (direct and derived) that are of interest to a group of analysts is called an analysis context (or simply a context). Actually, as we shall see shortly, the context is the interface between the analyst and the dataset, in the sense that the analyst uses attributes of the context in order to formulate queries (in the form of triples, as seen earlier).
Incremental Computation in HIFUN In this section, we show how we can use HIFUN language to incrementally evaluate continuous queries. An important common feature of real-life applications is that the input data continuously grow and old data remain intact. As such, for the rest of this paper we assume that the dataset being processed can only increase in size between t and t. In such a scenario, the idea of incrementally evaluating a continuous query is to use the results of an already performed computation on the old data, and evaluate the query only on the data appended between t and t ; eventually new and previous results should be merged. As we saw earlier, in an HIFUN query Q = (g, m, op), the domain of the definition of an answer coincides with the range of the grouping function. Consider, for instance, the query Q = (b, q, sum) of our running example, asking for the totals by branch.
The domain of the answer is the set of all branches and this set is also the range of b. We call this set the key of the query. Now, in a continuous query, the key might change over time. Indeed, in the case of totals by branch, new branches might start operation between t and t , whereas branches operating at time t might cease to operate between t and t. Figure  3  illustrates our incremental approach for continuous queries-the same query asked two times. We perceive the problem of incremental evaluation as follows: let ans be the answer of continuous query Q at time t, on dataset D; let ans be the answer of Q at time t on dataset D , where D = D + ∆D; and let incr be the answer of Q at time t on dataset ∆D. Moreover, let K be the key of Q at time t and let K be the key of Q at time t. Our algorithm for computing ans supports both distributive and non-distributive operations.
Continuous HIFUN Queries over Micro-Batches In the micro-batching approach, as a dataset continuously grows and as new data become available, we process the tuples in discrete batches. The batches are processed according to a particular sequence. As a high volume of tuples can be processed per microbatch, the aforementioned mechanism uses parallelization to speed up data processing. An initial dataset D i is followed by a continuous stream of incremental batches ∆D i that arrive at consecutive time intervals ∆t. As we already explained, incremental evaluation would produce the query results at time t + ∆t by simply combining the query results at time t with the results from processing the incremental batches ∆D i. Two key observations should be made here. The first is that computations needed are solely performed within the specific batch, following the evaluation scheme described in the previous section.
Continuous HIFUN Queries Using MapReduce The abstract definition of a query evaluation is implemented on the physical level using the MapReduce programming model exploiting Spark Streaming. Spark Streaming is a stream processing framework based on the concept of discretized streams and provides the DStream API, which accepts sequences of data which arrive over time. The API implements the micro-batch stream processing approach with periodic checking of internal state at each batch interval. Internally, each DStream is represented as a sequence of data structure called resilient distributed datasets (RDDs) which keeps the data in memory as they arrive in each batch interval. Generic Evaluation Schema to MapReduce. In this section, we elaborate on the generic query evaluation schema, described in Section 3, by presenting details of its implementation in the physical layer when Spark Streaming and the MapReduce programming model are sed: Rewritings to MapReduce As already described, a set Q can be rewritten according to certain rewriting rules.
In this subsection, we give a detailed description how the evaluation mechanism leverages these rules. Common Grouping and Measuring Rewriting Rule. In this case of rewriting, n different operations are applied to the common grouping and measuring attributes. In the Query Input Preparation step the information is extracted from the initial unstructured dataset using the common grouping and measuring attributes, which appear in the query set Q. A map function is used to apply the given common attributes to each record of the initial DStream and returns a new DStream containing the values of the common grouping and measuring attributes for each record. This is useful for the following evaluation steps. As a next evaluation step, a map function constructs the grouping partition as follows. The mapper receives the tuples created from the previous step and emits key-value pairs < K, V >, where K is the value of the common grouping attribute for each tuple and value V is synthetic and carries a list of measuring attribute values.
In this case n different measuring attributes are reduced to the common grouping attribute, applying n possible different operations. In the Query Input Preparation step, the information is extracted from the initial unstructured dataset using the common grouping and the n different measuring attributes which appear in the query set Q. A mapping function is used to apply the given common grouping attribute and the n measuring attributes to each record of the initial DStream. It then returns a new DStream, containing the values of the common grouping and measuring attributes for each record, useful for the next evaluation steps. As a next evaluation step, a map function constructs the grouping partition as follows: The mapper receives the tuples created from the previous step and emits the key-value pairs < K, V >, where K is the value of the common grouping attribute for each tuple and the value V is synthetic, and carries a list of n measuring attribute values.
Translating Continuous HIFUN Queries to SQL The abstract definition of a query evaluation can be realized when the involved dataset D is stored in an unbounded append-only relation table; and also, mapping this abstract definition to the existing physical-level mechanism using the semantics of the SQL exploiting group-by SQL queries of Spark Structured Streaming. The basic idea in Structured Streaming is treating continuously arriving data as a table that is being continuously appended. Structured Streaming runs in a micro-batch execution model as well. Spark waits for a time interval and combines into batches all events that were received during that interval. The mapping mechanism defines a query on the input table, as if it were a static table, computing a result table that will be updated through the data stream.
Select A, op(B) as ans Q (A) From join(T, S) GroupBy A (4) To this end, next we present examples of mapping analytic queries directly to SQL. We shall use the context of the Figure  5  and we assume that the dataset is stored in the form of a relation using the star schema shown in that figure. In general, a star schema includes one or more fact tables, indexing a number of associated dimension tables. In our example, this star schema consists of the fact table FT and two-dimensional tables: the dimensional table DT Branch of the branch and the dimensional table DT Product of the product. The edges of the context are embedded in these three tables as functional dependencies that the tables must satisfy, and the underlying attribute in each of these three tables in the key of the table. Our implementation handles the above relation schema as follows: The fact table is represented as an unbounded table containing the primary incoming streaming data, and the dimensional tables DT Branch and DT Product are represented as static tables which are connected to the fact table.