dataesr/scientific-paragraphs-categorization huggingface dataset:
However, we would like to evaluate our algorithm on stochastic environments too. To achieve this, we use the same datasets, but we modify the reward using the following formula r t $ rðs t ; a t Þ À k1 frðs t ;a t Þ [ rg B p 0 where r is the classical reward function. r, k are hyperparameters and B p 0 is a Bernoulli distribution of parameter p 0. In our experiments, we choose k ¼ 8 and p 0 ¼ 0:1. Different values of r are used for each dataset such that about the half of the states verify rðs t ; a t Þ [ r. Hyperparameters of each model have been manually tuned based on our experiments and hyperparameters used in previous papers. However, due to the substantial time required to train these models (specifically LODAC and LODAC-O), we were unable to test a wide range of hyperparameter combinations.
Results discussion In this section, we discuss the results of our experiments, which are presented in Table  1  for the stochastic environment and in Table  2  for the deterministic environment. We bold the highest score across all methods. Complete results of all different runs can be found in Appendix A. Stochastic environment The first general observation is that LODAC and LODAC-O generally outperform riskfree algorithms. The only exception is with the expert_replay dataset where LODAC-O provides worse results. This is not really surprising since actors trained with O-RAAC contains an imitation component, and obviously, the imitation agent provides really poor performance on this dataset. LODAC provides really interesting results as it significantly outperforms risk-free algorithm in terms of CVaR 0:7 and return on all datasets.
Moreover, it provides better results than LODAC-O on the medium and the expert_replay dataset, while achieving comparable result on the expert dataset. A final observation is that risk-free We compare the mean and the CVaR 0:7 of the returns. LODAC outperforms other algorithms on the medium and expert_replay datasets while achieving comparable results on the expert dataset. We bold the highest score across all methods RL policies provide generally bad performances on this stochastic environment. Deterministic environment First, a drop of performance between the deterministic and the stochastic environment can be noticed. This is not surprising, as stochastic environments are generally more challenging than deterministic ones. However, this change appears to affect risk-free algorithms more significantly than LODAC-O or LODAC.
Indeed, we get a difference of more than 27% with LOMPO and COMBO on the medium and expert_replay dataset in terms of return between the deterministic and the stochastic environment. This difference even reached 42% for COMBO on the expert_replay dataset. For LODAC-O and for the same tasks, we obtain a deterioration of less than 17%. In contrast to risk-free methods, and in a lesser degree LODAC-O, adding stochasticity to the dataset does not seem to have a significant impact on the performance of LODAC. Indeed, with this algorithm we observe a drop in performance of less than 9%. For the medium dataset, a difference of only 5:23% can even be noticed.
Conclusion While offline RL appears to be an interesting paradigm for real-world applications, many of these real-world applications are high-dimensional and stochastic. However, current high-dimensional offline RL algorithms are trained and tested in deterministic environments. Our empirical results suggest that adding stochasticity to the training dataset significantly decreases the performance of highdimensional risk-free offline RL algorithms. Based on this observation, we develop LODAC. LODAC can be used to minimize various risk measures like Conditional Value-at-Risk. Our theoretical considerations in Sect. 4  show that our algorithm relies on a strong theoretical foundation. Finally, the use of LODAC to minimize CVaR empirically outperforms previous algorithms in term of CVaR and return on stochastic highdimensional environments.
D Script used for simulations Hereinafter the script written in python used to obtain simulated value of β. The number of simulations is chosen in order to reach a 99% confidence interval. print ( "The␣ c o n f i d e n c e ␣ i n t e r v a l ␣ a t ␣99%␣ i s : ␣ [ { } ␣-␣ { } , ␣ {} ␣+␣ { } ] ". format ( mean_Z , 3 * stdev_Z /math. s q r t (T) , mean_Z , 3 * stdev_Z /math. s q r t (T) ) ).
Graphs, and notably RDF graphs, are a prominent way of sharing data. As data usage democratizes, users need help figuring out the useful content of a graph dataset. In particular, journalists with whom we collaborate [4] are interested in identifying, in a graph, the connections between entities, e. g. , people, organizations, emails, etc. We present a novel, interactive method for exploring data graphs through their data paths connecting Named Entities (NEs, in short); each data path leads to a tabular-looking set of results. NEs are extracted from the data through dedicated Information Extraction modules. Our method builds upon the pre-existing ConnectionLens platform  [4, 5]  and follow-up work on dataset abstraction  [8, 9]. The contribution of the present work is in the interactive and efficient approach to enumerate and compute NE paths, based on an algorithm which automatically recommends subpaths to materialize, and rewrites the path queries using these subpaths.
Motivation and Problem Statement Data graphs, including RDF knowledge graphs, as well as Property Graphs (PGs), are often used to represent data. More broadly, any structured or semi-structured dataset can be viewed as a graph, having: (i) an internal node for each structural element of the original dataset, e. g. , relational tuple, XML element or attribute, JSON map or array, URI in an RDF graph; (ii) a leaf node for each value in the dataset, e. g. , attribute value in a relational tuple, text node or attribute value in XML, atomic (leaf) value in a JSON document, or literal in RDF. The connections between the data items in the original dataset lead to edges in the graph, e. g. parent-child relationships between XML or JSON nodes, edges connecting each relational tuple node with their attributes, etc.
In a relational database, primary/foreign keys may lead to more edges, e. g. , the node representing an Employee tuple "points to" the Company tuple representing their employer. This graph view of a dataset has been introduced to support unstructured (keyword-based) search on (semi)structured data, since  [3, 15]  and through many follow-up works  [20]. Entity-rich graphs Building on this idea, the ConnectionLens system  [4, 5]  has been developed to facilitate, for non-IT users such as data journalists, the exploration of datasets of various models, including relational/CSV, XML, JSON, RDF, and PGs. ConnectionLens turns any (set of) datasets into a single graph as outlined above. For instance, the data graph at the top of Fig. 1  features RDF triples about NASA spacecrafts (labeled edges), and an XML document describing presidents who attended spacecraft launches (tree with labeled nodes and unlabeled edges).
ConnectionLens also includes Information Extraction modules, which extract, from any leaf node in the data graph, Named Entities (People, Locations and Organizations)  [5] , as well as other types of entities that journalists find interesting: temporal moments (date, time), Website URIs, email addresses, and hashtags. We designate any of these pieces of information as entities, and we model them as extra nodes, e. g. , in Fig. 1  (top), organizations appear on a pink background, people on yellow and locations on green, respectively. Each entity is extracted from a leaf text node, to which it is connected by a dashed edge. When an entity is extracted from more than one text node, the edges connecting it to those nodes increase the graph connectivity, e. g. , "NASA" extracted from the nodes 15 and 17.
Goal: efficient, interactive exploration of entity connections Journalists are interested in data paths ending in entity pairs of certain kinds in a given dataset, e. g. in Fig. 1 , "how people are connected to places?". When shown the set of corresponding labeled paths, users may pick one to further explore: how many pairs of entities are connected by each path?, which entities are most frequent?, etc. Note that it is important to consider paths irrespective of the edges directions in the data graph. This is because, depending on how the data is modeled, we may encounter x boughtProperty ----------→ y locatedIn ------→ c, or x hasOwner ← ------y locatedIn ------→ c; both paths are interesting. Challenges and contributions The analysis outlined above raises two challenges. First, it requires materializing the entity pairs connected by the paths, which may be very costly, if (i) the graph is large, and/or (ii) there are many paths (the latter is almost always true, if the data is complex/heterogeneous, and/or if we allow paths to traverse edges in both directions).
Second, the large number of paths may overwhelm users. Non-expert users, or users which are not familiar with the dataset structure, cannot be expected to state "only the paths that they would like to see", since they lack technical expertise and/or dataset knowledge. However, if prompted by the system, they can give valuable input on whether certain links (or connections) are worth making, or whether they are just spurious links that would generate uninteresting paths. An example of uninteresting path is: in a dataset of French national assembly members, all addresses are in France, thus the France named entity enables connecting any two people. Our contributions towards addressing these challenges is as follows. (i) From the data, we generate a small set of user questions, which help us leverage the domain knowledge that users may have, to generate interesting paths (Sec.
From datasets to data graphs In order to propose a general approach that works on any data format (e. g. RDF, JSON, XML, etc. ), we start by building a data graph out of each input dataset (Sec. 2. 1). These graphs may be large, thus enumerating paths on them would be inefficient. Therefore, we leverage a more compact structure, namely a collection graph (Sec. 2. 2). The graph representation and the collection graph are based on prior work  [8, 9]. Finally, we explain how we produce a single collection graph out of several datasets (Sec. 2. 3), which journalists may need to exploit together in a particular investigation.
From a dataset to a data graph We transform any dataset as a directed graph G 0 = (N 0 , E 0 , λ 0 ) where N 0 is a set of nodes, E 0 is a set of vertices connecting N 0 nodes and λ 0 is a function assigning a label l to each node and edge (l may be empty). We map each data model on G 0 as follows. RDF graphs are naturally mapped on G 0 : each subject, respectively object, is turned into a node and an edge labelled with the property is connecting them. An XML document can be naturally viewed as a tree, with element nodes having element and attribute children. XML elements may carry #ID attributes whose values uniquely identify them; other XML elements may carry #IDREF attributes, whose values act as "foreign keys" referring to other elements by their #ID value. ID-IDREF information can be supplied in an optional Document Type Description (DTD) or XML Schema (XSD); when these are not available, ID-IDREF links can be found by profiling  [16, 1]  techniques, which we also implemented.
In the graph representation of an XML document, ID-IDREF links lead to more edges between elements (thus, the graph is no longer a tree). JSON documents are also modelled as trees, where each map, array, and leaf value is a node and parent-child edges are connecting them. Named entities (NE) are extracted from each leaf of the data graph (RDF literal, XML text node or attribute value, or JSON value). For instance, in Fig. 1 , the NASA organization (pink background) is extracted from both the RDF and XML datasets. Each extracted NE is materialized by a new node in the graph, connected via an extraction edge (dashed arrows in Fig. 1 ) to each leaf text node from which it has been extracted.
g. , agency and pilot are in the same outgoing clique because they are both property of node 10 in Fig. 1 , and also with descr because node 14 has agency and descr; incoming property cliques are symmetrically defined based on incoming properties; (ii) two nodes without types are equivalent if they have identical outgoing and incoming property cliques. For instance, nodes 10 and 14 (the two spacecrafts) are equivalent, since they have the same outgoing and incoming property cliques. Fig. 1  (bottom) shows the collection graph corresponding to the data graph. We named the collections C 1 , C 2 , etc. Note that some data models have labeled edges, e. g. , RDF, while others have (mostly) unlabeled edges, e. g. , XML. For uniformity, we transform any labeled edge into a node and extend our summarization also to such nodes.
In Fig. 1 , collection C 6 contains the nodes introduced instead of the pilot edges in the RDF dataset. Collection names in our figure are only for ease of explanation (they are not required in our method). Some collections, such as C 12 , C 13 , have nodes with identical names, in which case we use that name. For collections such as C 1 , with RDF nodes each with a different URI, we use an intuitive name, e. g. , "Spacecraft". Entity collection profiles Each leaf collection in the collection graph corresponds to a set of literals (strings), out of which various NEs may have been extracted. These collections' names end with a # to help distinguish them (e. g. C 5 agency#). For each such collection C, we compute an entity profile storing how many entities of each type were extracted from its string nodes.
From multiple datasets to a collection graph Journalists often need to work with several datasets, e. g. a JSON collection of political tweets, the list of French mayors in XML, and an RDF graph of public investment in companies across France. Such datasets often have common values, e. g. , the cities that mayors represent are also the places where companies are situated. Interesting data paths can be found across data sources. To obtain a single collection graph from a set of datasets, we proceed as follows. First, we build a separated collection graph from each dataset (as in Sec. 2. 2). Then, whenever two collections C 1 , C 2 from distinct datasets share values, we replace them by a new collection C 1,2 , which contains the values of C 1 and C 2 , and inherits all the incoming and outgoing edges of C 1 and C 2 in the collection graph they came from.
Paths between entities In this section, we discuss categories of paths that users might be interested in, and how to ask for their input. An important first observation is: by the way we built the collection graph, to each dataset path corresponds a path in the collection graph. For instance, consider the data path 13 ← 12 name ←---11 pilot ← ---10 agency -----→ 15 → 16 in Fig. 1. The collection graph features the corresponding path ← C 9 ← C 8 ← C 7 ← C 6 ← C 1 → C 4 → C 5 →. Further, some paths in the collection graph correspond to no paths in the data graph. For instance, the path C 6 ← C 1 → C 2 does not correspond to any path in the data, because no spacecraft (part of the collection C 1 ) has both the agency and descr properties. Such collection paths, with no support in the data, are introduced by summarization, which compresses the graph structure with some information loss.
A shared-root path is C 9 ← C 8 ← C 7 ← C 6 ← C 1 → C 4 → C 5. -General, i. e. , the edges may be in any direction. Unidirectional paths are quite rare. This is because entity-connecting paths must have at each end a node from which an entity is extracted. Most of the time, these are two literal (string) nodes (as opposed to internal nodes structuring the dataset). Literals have incoming edges, but not outgoing ones (other than those towards extracted entities); thus, there is no unidirectional path from a literal to another. However, in some RDF datasets, NEs are extracted from URIs, e. g. , the triple https://dbpedia. org/Facebook locatedIn ------→ http://dbpedia. org/California is a unidirectional data path from an Organization to a Location. Similarly, shared-sink paths only occur when nodes in C 1 and C 2 have outgoing edges, and NEs appear in their labels; this only happens in RDF URIs.
Low-specificity connections Some edges in the data graph reflect connections that can be seen as weak, or non-specific. In details, let's first consider data edges with nonempty labels, e. g. , RDF triples. Let e be the edge n 1 a -→ n 2 for some URIs n 1 , a, n 2. The specificity of e, denoted e s , is computed as 2/(N 1,a + N 2,a ), where N 1,a , N 2,a are the numbers edges labeled a outgoing, resp. incoming n 1 , resp. n 2  [5]. The highest N 1,a and/or N 2,a , the lowest e s. For instance, the specificity of each agency edge in Fig. 1  is 2/(1 + 2) = 2/3. For our purposes, we extend specificity to unlabeled edges as follows: the specificity of an edge n 1 -→ n 2 is 2/(1 + n 1,2 ) where n 1,2 is the number of (emptylabel) edges outgoing n 1 , towards nodes having the same label as n 2. For instance, the specificity of the edge between nodes 6 and 7 is also 2/3.
Finally, τ Org and τ Person denote the node types of extracted Organization, respectively, extracted Person entity. Similarly, the pink-background collection path translates into: q 2 ( x) :n(x 1 , τ Org , ), e(x 2 , x 1 , _), n(x 2 , _, C 3 ), e(x 3 , x 2 , descr), n(x 3 , _, C 1 ), e(x 3 , x 4 , pilot), n(x 4 , _, C 7 ), e(x 5 , x 4 , _), n(x 5 , _, C 11 ), e(x 6 , x 5 , _), n(x 6 , _, C 12 ), e(x 6 , x 7 , _), n(x 7 , _, C 14 ), e(x 7 , x 8 , _), n(x 8 , _, C 15 ), e(x 8 , x 9 , _), n(x 9 , τ Person , ) Each of these queries can be evaluated through any standard graph database. However, evaluating dozens or hundreds of path queries on large graphs can get very costly. Further, since we do not know which paths may result from the user choices, we cannot establish path indexes beforehand.
These queries are then removed from Q, the benefits of the remaining views are recomputed over the diminished Q, and the process repeats until no profitable view to materialize can be found. Estimating costs Algorithm 1 needs to compute: (i) cost(•), the cost to evaluate a query q or materialize a view v; (ii) r q,v , the rewriting of q using a view v; and (iii) cost(r q,v ), the cost of such a rewriting. All these costs must be estimated before any query or view results are computed. We do this as follows. For (i), we use the cost estimation of the graph data management system (GDBM, in short) storing the graph. Our implementation relies on PostgreSQL, whose explain command returns both the estimated number of results of certain query (or view), denoted size(q), and the cost of computing those results.
For (ii), recall (Sec. 4. 2) that when v is used to rewrite q, v is a subpath of q, thus there exist i, j such that v = q| i, j. The rewriting r q,v is easily obtained by replacing, in the body of q, the atoms from the ith to the jth, with the head of v. Handling (iii) is more complex than (i). This is because the cost of a query (or view) is estimated based on statistics the GDBM has about the stored graph. In contrast, the GDBM cannot estimate cost of r q,v , because v has not been materialized yet, thus the GDBM cannot reason about v like it does about the graph. To compensate, we proceed as follows: we compute the cost of reading the hypothetical view v max from the database, by multiplying size(v max ), the estimation of the view size, with a constant (we used Postgres' own CPU_TUPLE_COST); then, we estimate the cost of r q,v max as this reading cost plus the cost of estimating the parts of q not in v max plus the cost of joining v max with these (one or two) remaining query parts.
We estimate the cost of each such join by adding their input sizes, which we then multiply with another (GDBM) constant. This reflects the fact that modern databases feature efficient join algorithms, such as memory-based hash joins, whose complexity is linear in the size of their inputs. Complexity Algorithm 1 makes at most O(|V| × |Q|) iterations, which can be simplified into O(|Q| 3 ). Forming a rewriting takes O(n Q ), bringing the total to O(|Q| 3 × n Q ). Impact of heuristics As previously discussed, (H1) is universally adopted in the literature: no candidate view features cartesian products. (H2), imposing that views benefit at least two queries, preserves result quality, i. e. , cost savings, under every monotone cost model, ensuring that the cost of evaluating a query q is at least that of evaluating s, when s is a subquery of q.
Experimental evaluation Our approach is fully implemented in Java 11, on top of ConnectionLens  [4, 5]  which builds the data graph (Sec. 2. 1) and Abstra  [8, 9]  which builds the collection graph (Sec. 2. 2); these are stored in PostgreSQL. We experimented on a Linux server with an Intel Xeon Gold 5218 CPU @ 2. 30 GHz and 196GB of RAM. We used PostgreSQL v9. 6. Our system is available at: https://team. inria. fr/cedar/projects/pathways/. Our evaluation seeks to answer the two following questions: (i) how are entities connected in each dataset? (Section 5. 2) and (ii) does Algorithm 1 help to evaluate paths queries over the data graph? (Section 5. 3).
Datasets and settings We present experiments on an XML, an RDF and a JSON dataset. They all come from real-life applications (as opposed to synthetic) to stay close to application needs, and to ensure realistic Named Entities (NEs). Indeed, synthetic datasets are often generated with an interest on structure, while the leaf (text) values lack interesting information. We used an XML PubMed subset describing scientific articles from PubMed, a database of biomedical publications. Each article is described by its title, journal, link, year, DOI, keywords list and authors list. Authors are identified by their name and their affiliation; they may declare their conflicts of interest in the <COIStatement> tag. We used the RDF Nasa dataset, describing NASA flights, spacecrafts involved in launches, related space missions and the participating agencies.
Finally, we used the JSON Yelp-Business dataset where each business has an id, a name, an address, a city, a state, a postal code and coordinates (latitude and longitude). It also has a set of categories (e. g. bakery, shoe store, etc. ) and a set of attributes (e. g. acceptCreditCards), the latter may be deeply nested. They also received reviews from customers modeled as a number of stars (from 0 to 5) and the number of reviews. YelpBusiness4, 4 times larger than YelpBusiness, allows studying the scalability of our algorithm. Tab. 1 shows for each dataset: its number of nodes |N|, edges |E|, numbers of extracted NEs |τ P |, |τ L |, |τ O | and the minimum edge specificity min(e s ). Without loss of generality, we experiment with the NE types Person, Location, Organization, whose types are denoted τ P , τ L , τ O , respectively.
Path enumeration For each dataset and pair of entity types, Fig. 2  and 3  report the number of paths of each directionality (Sec. 3. 1), the minimum and maximum length L p of each path, and the minimum and maximum data path support (number of results when evaluated on the data), this is denoted S p. For the PubMed (XML) and YelpBusiness (JSON) datasets, we obtained only shared-root paths: this is because of the tree structure of these datasets, where text values (leaves) are only connected by going through a common ancestor node. In the RDF Nasa dataset, we also found general-directionality paths. The JSON datasets are more irregular, leading to more paths. In almost every case, a few collection paths had 0 support, due to dataset summarization (Sec. 3). The maximum support may be high, e.
g. These results show that numerous interesting entity paths exist in our datasets, of significant length (up to 9), and some with high support, bringing the need for an efficient evaluation method. L p ) = 5. (τ 1 , τ 2 ) T 0 |Q T O | |Q NV | |V| |M| T R T Q NV T = T R + T Q NV s = T 0 /T PubMed (τ P , τ O ) 250. 36 5.