dataesr/scientific-paragraphs-categorization huggingface dataset:
e. a DOI registered at DataCite. Attributions to organizational data partners are included in the DataCite entries, including Research Organization Registry identifiers when available. Although geospatial extent metadata is not required by DataCite, the latitude and longitude range is deemed necessary by ONC since location is an essential aspect of ocean data discovery. The implementation used at ONC supports fixed-position, mobile and remote sensing instruments. The query store related recommendations were mostly well-aligned with existing infrastructure. The data discovery interfaces within Oceans 2. 0, allow researchers to access subsets of data sets based on their selected criteria of time, variables, formats and data product processing parameters. These query details are stored within a relational database and are assigned a resolvable internal identifier, but were not normalized for uniqueness (this was deemed low priority due to low likelihood of exactly the same queries, and since it can be re-considered in the future due to saving all queries).
Despite this apparent success, repositories can still find it daunting to implement the recommendations. Indeed, most of the pilots received additional funding for implementation. This is understandable. Data stewardship is an unending (underfunded) and increasingly complex process. We have found that certain questions or concerns frequently arise for which we now have answers based on real-world experience. Therefore, the rest of this section is a sort of "FAQ" which may help address the concerns of future adopters. Do the recommendations work for any kind of data? Yes, it appears so. The solutions presented include small-scale textual data, relational databases (MySQL, PostgreSQL), NoSQL Databases (MongoDB), native XML Databases (BaseX, eXist-db), filesystems managed stand-alone or in combination with distributed database systems such as GeoGIS, or multidimensional data cubes such as e.
g. NetCDF files. Queries include dedicated scripts in R or using Java libraries mimicking database functionality over CSV files, SQL, XPATH, dedicated query languages for NoSQL databases, bounding boxes drawn on a map, and specific interfaces for data cubes such as NCDF subset services for NetCDF files. Neither conceptually nor in practice have we found a data type or query structure where the recommendations would not apply. Do all updates need to be versioned? Ideally, yes. In practice, probably not. Settings that see extremely high-frequency updates over massive amounts of data may face a challenge in maintaining ALL states that ever occurred. We propose that in settings where not all states of data that ever existed need to be documented (e. g. for accountability reasons) or where states that were never read (i.
e. updates to the database without any intermittent read operations to that data, data could be overwritten without versioning. Alternatively, versioning at lower frequencies that serves the needs of the respective domains, may be a solution as well. In such cases, it should be made clear that only queries selecting data from certain "stable states" are reproducible, clearly separating data for live-tracking and monitoring from research data serving as a basis for studies and decision making. Similarly, when massive data volumes create economic challenges in maintaining multiple versions, such as re-processing large numbers of satellite image data, respective trade-offs must be considered and some versions must be deprecated. In all cases, a recurrent query should result in a meaningful result even if it is to state that the particular version of the data is no longer available (see next question).
Which PID system should be used?: The one that works best for your situation. The recommendations are neutral to the actual PID system being used. However, it is highly recommended to adopt a system that is widely used within the community, and that allows references to be easily and transparently resolved. Some of the pilots use established systems like DOIs or Handle. net while others have chosen to resolve the query identifiers themselves. Other aspects of PID systems, such as external visibility, use by aggregators, or cost aspects may also influence decision making. This is, for example, demonstrated by the VAMDS implementation pushing the information to Zenodo to issue a DOI to connect to the Scholix initiative. When multiple distributed repositories are queried, do we need complex time synchronization protocols?: No, not if the local repositories maintain time-stamps.
Contrary to the software world, where semantic versioning is widely adopted, it is hard to apply stringent protocols to define the difference between major and minor updates to data. While in the software world, major releases are often those that break certain interfaces or introduce significant new functionality, whereas minor releases comprise mostly of bug fixes and other non-functional improvements, such differences hardly exist with data. Fixing a spelling mistake in a data record will make a difference to data sets extracted from the database with that record suddenly being (no longer) found after the correction. Translating attribute names, changing encodings, or increasing the precision of numeric representation will have an impact on subsequent computations and may thus lead to artifacts.
rather than referring to small and large differences in data it thus seems recommended to refer to "the state of the world at a given point in time", which -as a concept -works across all disciplines and data types. How complex is it to implement the recommendations?: It depends on the setting. The difficulty of implementation obviously depends on the complexity of the data infrastructure, the type and volume of data and changes of the data, as well as the query processing load, among other aspects. Most pilot adopters have reported effort in the order of a few person months spread over a period of about six months to to a year. The effort was greater for the complex distributed environment in VAMDC. The ONC effort was quite significant, but one could argue that the recommendations may have saved effort by providing a guiding framework for their system upgrade.
Acknowledgments This work benefited from the contribution of numerous experts participating in discussions during the WGDC Working Group meetings at the RDA plenaries, as well as the input and feedback from developers to policy makers in the various implementation projects. Their feedback was extremely valuable in initially shaping the recommendations and providing guidance to repositories embarking on the project to adopt them as well. The framework provided by  RDA  for these discussions was invaluable. RDA also helped secure pilot funding from the  European Commission  and the  US National Science Foundation  to support institutions via adopters' grants. The work reported in this paper was further supported by the  Austrian COMET K1 program  and publicly funded by the  Austrian Research Promotion Agency (FFG)  and the  Vienna Business Agency (WAW)  and  CANARIE's Research Data Management programme.
Intra-object Metadata Definition and Generation Intra-object metadata are atomic or more complex information associated with a specific object. We classify them in two categories. Metadata properties are information that describe an object. They often take the form of simple key-value pairs, e. g. , author name, file path, creation date, etc. However, they may sometimes be more complex. Particularly, the description of the columns of a table can be viewed as a complex form of metadata properties. Metadata properties are mostly provided by the file system. However, especially when dealing with textual documents, we use Apache Tika  [36]  to automatically extract metadata such as the author, language, creation timestamp, mime-type and even the program used to edit the document. Refined representations are more complex.
To generate bag-of-word representations, we perform for each document a classical process: tokenizing, stopword removal, lemmatization and finally word count. To generate embedding representations, we project documents in an embedding space with the help of the Doc2Vec model  [21]. Each document is thus transformed into a reduced vector of only a few tens of coordinates. Eventually, we use a custom process to generate refined representations from tables. Each tabular document is read in a Python dataframe and then transformed into a relational table. Modeling and Storage Still using MEDAL  [33] , we follow a graph approach to model the interactions between data and metadata. Therefore, AUDAL's metadata system is centered on Neo4J  [29]. We exploit four types of nodes to manage intra-object metadata.
Thus, each type of user can access the lake with respect to its needs, which makes AUDAL "inclusive", unlike the common vision of data lakes that excludes business users  [12, 20, 24]. Overall, AUDAL's architecture looks a lot like a multistore system, i. e. , a collection of heterogeneous storage systems with a uniform query language  [22]. AUDAL indeed offers a single REST API to query data and metadata across different systems (Neo4J, MongoDB, ElasticSearch, and SQLite). However, AU-DAL also features an extensive metadata management layer that goes beyond what multistore systems do, i. e. , multistores handle only intra-object metadata.
AUDAL's Analysis Features Data Retrieval Data retrieval consists in filtering data from the lake. The techniques we propose for data retrieval are suitable for both textual and tabular documents. Term-based querying allows to filter data with respect to a set of keywords It includes a fuzzy search feature that allows to expand queries with syntactically similar terms. Navigation exploits groupings, i. e. , organized sets of tags that allow data filtering by intersecting several groups. For example, we can retrieve documents edited by a given author on a specific year, who is associated with a department via the intersection of three groups, e. g. , "Scott", 2010 and "Human resources". Finding related data consists retrieving the objects that are the closest of a given object. Closeness is obtained from similarity links.
Datasets and Query Workload The first dataset we use comes from the AURA-PMI project. It is composed of 8,122 textual documents and 6 tabular documents, for a total size of 6. 2 GB. As the AURA-PMI dataset is quite small, we also create an artificial dataset by extracting 50,000 scientific articles from the French open archive HAL. To these textual documents, we add 5,000 tabular documents coming from an existing benchmark  [28] , for a total volume of 62. 7 GB. To compare how AUDAL works on our two datasets, we define a set of 15 queries that reflect AUDAL's main features (Table  1 ). Then, we measure the response time of our workload to assess whether our approach is realistic. In Table  1 , the terms document, table and object refer to textual document, relational table and one or the other indistinctly, respectively.
Experimental Setup and Results Both instances of AUDAL are implemented on a cluster of three VMware virtual machines (VMs). The first VM has a 7-core Intel-Xeon 2. 20 GHz processor and 24 GB of RAM. It runs the API. Both other VMs have a mono-core Intel-Xeon 2. 20 GHz processor and 24 GB of RAM. Each hosts a Neo4J instance, an ElasticSearch instance and a MongoDB instance to store AUDAL's metadata. The execution times we report in Table  2  are the average of ten runs of each query, expressed in milliseconds. Our experimental results show that AUDAL does support almost all its query and analysis features in a reasonable time. We also see that AUDAL scales quite well with respect to data volume. All data retrieval and tabular content analyses indeed run very fast on both the AURA-PMI dataset (174 ms on average) and the larger, artificial dataset (183 ms on average).
Admittedly, half of textual content queries, i. e. , queries #9, #10 and #11, take longer to complete: 5, 2 and 2 seconds on average, respectively, on the AURA-PMI dataset; and 188, 27 and 27 seconds on average, respectively, on the artificial dataset. However, we note that without our approach, such tasks would be definitely impossible for business users. Moreover, the situation can certainly be improved by increasing CPU resources. Thus, we consider our results promising. However, AUDAL's features are achieved at the cost of an extensive metadata system. Table  4. 2 indeed shows that the size of metadata represents up to half of raw data. Yet we deem this acceptable given the benefits. Moreover, it is acknowledged that metadata can be larger than the original data, especially in the context of data lakes, where metadata are so important  [18].
Discussion As stated above, most data lake approaches focus either on data retrieval or data content analyses. Therefore, they present a partial vision of data lakes, in our opinion. In contrast, there exists a system that frees itself from this cleavage  [3]. However, it does not support unstructured data. More generally, unstructured data are very rarely supported in data lakes. Our own CODAL data lake  [32]  does manage textual documents management, but only textual documents. It is therefore limited. In contrast, AUDAL goes beyond these limitations by featuring both data retrieval as well as content analyses. In addition, AUDAL supports both tabular documents and, above all, textual documents whose inclusion in data lakes still challenging.
Conclusion and Future Works In this paper, we present AUDAL, presumably the first methodological approach to manage both textual and tabular documents in a data lake. AUDAL includes an extensive metadata system to allow querying and analyzing the data lake and supports more features than state-of-the-art data lake implementations. In terms of queries, AUDAL indeed supports both data retrieval and data content analyses, including Text-OLAP and SQL querying. Moreover, AUDAL also allows the exploitation of a data lake not only by data scientists, but also by business users. All these makes AUDAL an "inclusive" data lake. In our near-future research, we plan a deeper validation of AUDAL on two aspects. First, we will work on that complexity and time cost of metadata generation algorithms. Second, we will study how AUDAL's analysis interface is useful to and usable by business users, e.
g. , using the widely used SUS (System Usability Scale) protocol  [6]. Another perspective is data lineage tracking to allow AUDAL support version management. This is particularly important for tabular documents that are often merged or altered. Such a lineage could be implemented by extending AUDAL's refined representations. Finally, we envisage to include more unstructured data types into a lake, i. e. , images, videos and/or sounds, and manage their particular metadata for retrieval and analysis.
The intensive knowledge-based computation at inference time is managed with in-memory KG indices (as is common in RDF stores), and any-time algorithms for both C-NN computation and inference. This allows us to scale up to standard link prediction benchmarks, which count up to 100K entities. Scaling further to millions of entities like in DBpedia or Wikidata is left for future work. The contribution of this work is a novel approach to knowledge graph completion that has the following properties: (1) it is a form of instance-based learning, i. e. it can perform inferences without training, and hence accomodate new data without re-training; (2) it is a symbolic approach, i. e. it can provide explanations for each inference; (3) it shows competitive performance on standard link prediction benchmarks.
The existing methods vary by how they learn the embeddings, and how they combine them. Those methods are based on a range of techniques including: matrix factorization, tensor factorization, neural networks, and gradient descent. For example, one of the first method for KGs, TransE  [7] , models a relation as a translation in the embedding space of entities, and scores a candidate triple according to the distance between the translated head and the tail. The authors of TransE, Bordes et al, introduced two datasets, FB15k and WN18, respectively derived from Freebase and WordNet, which became becnhmarks in the evaluation of link prediction methods. Toutanova and Chen  [8]  however showed that a very simple method was able to outperform previous methods because of a flaw in the datasets: many test triples have their inverse among the training triples.
It is noteworthy here that Kate is predicted to be a parent although she never appears as a parent in the incomplete graph. This is not possible with inference by copy. Comparison with rules in other approaches. The main difference with other rule-based approaches is that we only generate rules that are specific to the entity e i and relation r k for which predictions are to be made. Looking in more detail to the structure of rules, each approach has its own language bias. In AMIE+  [15] , only by-analogy rules are considered, and equality filters are excluded in most experiments because they make the number of rules explodes with the length of rules. They only retain closed rules, i. e. rules where each variable occurs at least twice. However, non-closed rules still have to be generated in order to reach all closed rules.
Experiments We here report on experiments comparing our approach to other approaches on several standard benchmarks for link prediction. We first present the methodology, and study the impact on performance of the different strategies and hyper-parameters of our approach. Then we report performance results on benchmarks with an in-depth analysis. Finally, we discuss example inferences and explanations, and present a fine-grained comparison with AnyBURL. The companion page 2  provides access to the source code, the datasets, and the experiment main results and logs.
Methodology Datasets. We use four datasets that are used in many work to evaluate the link prediction task (WN18, WN18RR, FB15k, FB15k-237), plus an additional dataset that we derived from the Mondial dataset  [25]. Table  2  provides statistics about datasets (numbers of: entities, relations, train edges, validation edges, and test edges). WN18 and FB15k were introduced by Bordes et al  [7]  for link prediction evaluation. WN18 is derived from a subset of WordNet, and FB15k from FreeBase. In later work, it was observed that many test triples had their inverse in the training set (e. g. , hypernym is the inverse of hyponym in WordNet), and could be solved with very simple rules (e. g. , (y, hypernym, x) → (x, hyponym, y)). More challenging datasets were introduced by removing from the validation and test sets all pairs of entities  that occur in the training set.
FB15k-237 was introduced by Toutanova and Chen  [8] , and WN18RR was introduced by Dettmers et al. [9]. We introduce Mondial as a subset of the Mondial database  [25] , which contains facts about world geography. We simplified it to the task of link prediction by removing labelling edges and edges containing dates and numbers, and by unreifying n-ary relations into binary relations. Triples whose relation is rdf:type were also removed from the validation and test sets because they do not represent proper links between entities. The dataset is available from the companion page. It is smaller than other datasets but it makes it easier to interpret results, and we use it to study the impact of the different strategies and hyper-parameters without introducing a bias in our evaluation on the classical benchmarks.
Task. We follow the same protocol as introduced in  [7] , and followed by subsequent work. The task is to infer, for each test triple, the tail entity from the head and the relation, and also the head entity from the tail and the relation. We call test entity the known entity, and missing entity the entity to be infered. We evaluate the performance of our approach by using the same three measures as in  [17] : MRR and Hits@{1,10} (given in percents in this paper). Like in previous work, we use filtered versions of those measures to reflect the fact that, for instance, there may be several correct tail entities for a 1-n relation (e. g. , the relation from awards to nominees). For example, if the correct entity is at rank 7 but 2 out of the first 6 entities form triples that belong to the dataset (and are therefore considered as valid), then it is considered to be at rank 5.
Method. Because our approach has no training phase we can use both train and validation datasets as examples for our instance-based inference. We consider a few alternative strategies: • Random vs Ordered choice of a query element in the partitionning algorithm (see Section 5. 2); • by-copy vs by-analogy vs both kinds of inference rules (see Section 6. 1); • MC vs DS score for ranking infered entities (see Section 6. 2). Our approach has only three hyper-parameters (and no parameter to learn): the maximum depth of the description of the test entity, and two timeouts: one for the computation of C-NNs, and another one for the inference of an entity ranking. For the inference of a ranking of entities, we set λ = 2 in the definition of rule confidence, and α 0 = 0. 95 in the DS-score. The implementation of our approach has been integrated to SEWELIS as an improvement of previous work on the guided edition of RDF graphs  [26].
We do so by choosing the same tasks and measures as in previous work because it was not possible for us to run them ourselves (no access to a GPU), and also because it allows for a fairer comparison (e. g. , choice of hyper-parameters by authors). We also compare our approach to rule-based approaches: AMIE+  [15] , RuleN  [16] , and AnyBURL  [17]. Most performance results of the above approaches are reproduced from Meilicke et al. [17]  (see Section 7. 3 for details). We add yet another baseline Freq that simply consists in ranking entities e j according to their decreasing frequency of usage in r k over the train+valid dataset, as defined by f req j = |ans(x ← (x, r k , y), (y = e j ))|. It is independent of the test entity, and therefore acts as a default ranking.
Impact of Strategies and Hyper-parameters and Consistency of Results Impact of strategies. Table  3  compares the obtained MRR (in percents) on the Mondial dataset with different strategies. The timeout was set to 0. 1s for both C-NN computation and inference, and the maximum depth was set to 3. For recall, there are three axes composing the strategy: • choice of a spliting element: Random vs Ordered; • kinds of inference rules: by-copy vs by-analogy vs both (by-copy+analogy); • aggregation of inferences: MC (max. confidence) vs DS (Dempster-Shafer). This results in 12 different strategies, and Table  3  shows that the best performing combines Random, by-copy+analogy, and MC. Looking more in detail, we observe that combining both kinds of rules is always beneficial, as well as aggregating inferences with maximum confidence.
Consistency of results. To evaluate the consistency of the results of our approach, we perform two analyses on the Mondial dataset (with timeout=0. 1s). First, we analyze the variability of the performance measures by spliting the test set 10-fold. We observe that the standard error accross the 10 folds is very small for all measures: e. g. , 0. 5% for MRR, around 1% for Hits@1 and Hits@10, and 0. 4% for maximum confidence. Second, we analyze the relevance of the maximum confidence by measuring how predictive it is about the correctness of predictions. Figure  4  shows the ROC curve for the classification task that consists in predicting whether the top entity in the filtered ranking is correct based on the hypothesis that its associated maximum confidence is above a threshold varying from 0 to 1.