dataesr/scientific-paragraphs-categorization huggingface dataset:
Unfortunately, Git is not an ideal platform for versioning such relational databases and it certainly does not aim to replace them. Instead, tabular data must be kept in text-formats such as CSV, or XML, and this has issues for scalability, cohesive editability and problems standardisation of data for which relational databases were precisely designed to overcome. A partial solution to this limitation is to continue to use relational databases for live editing while depositing the data periodically into a Git repository via export to CSV or similar. Alternatively (or perhaps even in parallel), another solution is to use the automated 'continuous integration' features of platforms like GitLab CE (which was made available after our pilot and which we are currently testing), designed originally to continually test code, to instead check the referential integrity of data stored in CSV files by import to a relational database whenever changes are made to the data.
Sustainability of Data For long-term sustainability, it is good to ask what would happen with the Panormos data in 100 years when the world will probably have forgotten about ODK, Git, GitLab CE and R. Of course, if the repositories are not migrated periodically into up-to-date formats, then there is a potential for information loss: tracing the authorship of commits may be difficult (Git), the web interface will no longer work (GitLab CE) and the code for the analysis (R) may not be possible to run on current hardware. However, one of the main advantages of a working Git repository is that the history is stored in a different, hidden folder (/. git) and the latest version of the data is preserved in its latest state in a standard file-system. This means that as long as good archival formats are selected and the file system used is robust, the data has the best chance of survival.
Additionally, in the shorter-term, GitLab CE relies on very widely supported Free Software platform (Unix, Ruby and Git) and has its own API, which means that the content of the Git repositories can be easily transferred and duplicated. From the point of view of the R code, the benefit of the prose-oriented 'literate programming' that RMarkdown documents encourage is the production of a script designed to be understandable to computers AND humans, such that even if the reproducible analysis is broken or the code interpreter obsolete, the commentary which accompanies the code should be sufficiently self-explanatory to be translated into a new language. The design of the Project Panormos Survey data management therefore strengthens its long-term preservation and scope for reuse. The minutiae of the author-tracking system may be lost forever at some point, but this will no doubt be less important when the data have already been checked, analysed and re-used.
To our mind, the main problem is the lack of evidence for either view, which means the in-built assumptions of both are very difficult to test. Unsurprisingly, many project groups, including ourselves, are disinclined to be a test case! It is fair, we believe, to publish the general coordinates of large and protected sites when they are already known to locals or the general public. Indeed, attempting to stop this would be futile, given the number of geographic tags relating to ancient sites to be found in the publicly accessible Google Earth or Open Street Map databases. We must simply encourage and support state institutions to protect the sites sufficiently. 5 But for small-scale survey scatters or tiny, remote or difficult to protect sites, the ethics of open online access to their location is complex, and state institutions view their publication with considerable reserve.
g. Zenodo, GitHub, ADS, an institutional repository or a private web page, etc. , see Table  3 ) when or soon after the final 'results' are published. The advantage is that it provides confidence that the data is "clean, valid and meaningful"  (Marwick, pers comm. )  and to protect authors from "the worst case of scooping, where an unaffiliated scholar is the first to publish substantial findings from our data. " Though providing protection to the primary scholar, we feel this fails to address one of the major problems in archaeological fieldwork, namely the frequency with which archaeological progress is held back by the unwillingness or difficulty with which many finds are published in citable form before the final publication (which because of the difficulty of pulling together such a monumental document, too often never makes it to print).
There are nascent forays into raising the prestige of data publications: the Journal of Open Archaeological Data provides a peer-reviewed outlet and implementation of this idea in which the analysis of the datasets is kept to a minimum. But undeniably the limited number of contributed papers so far (22 data papers since foundation in 2012, as of April 2017) compared to the countless archaeological projects being undertaken across the world reflects the fact that prestige still adheres to older formats, or at least established journals, in a market in which there are a confusing plethora of specialist publications. The model of such journals is also inherently static: credit accrues to the compiler of the database as a static entity, the 'paper', but what happens when new finds demand an update? An addition of one find may be important, but would it justify an entirely new 'data paper'? In the absence of a fundamental shift in systems of academic progression, licensing (i.
e. legal protection of intellectual property) remains the main avenue by which the risk of scooping can be managed. This is how the Free Software movement functions (The Free Software Foundation 2016), and there are equivalent licenses such as Creative Commons, Open Data Commons (see table 4) which are appropriate for scientific data (Open Definition 2016). This involves attaching meta-data on the reuse of published data and analysis for a certain duration: for example, by specifying a license allowing access and browsing but no 'derivative work' or re-publication during a defined embargo. In concrete terms, this would involve publishing first under a Creative Commons CC BY-ND license, followed by re-release under a CC BY license after a specified time period. If a five-year barrier on reusage is placed upon initial primary data publication, this would provide a reasonable time for the primary researchers to establish a secondary publication trail, with the additional bonus that outside scrutiny might provide contributions that improve the secondary analysis.
Our original plan had been to make all of our Git repositories available for public access, but we did not fully realise the consequences of the fact that ODK embeds GPS data within the exported field form data files. As such, it would be impossible to open our original raw Git archive without breaking the imperative to protect spatial data. For our purposes, we are therefore forced to create a 'public' version of surveydata with the spatial data redacted. This is clearly unsatisfactory within the spirit of full transparency. We would advise future projects who use version control to plan for these kinds of eventualities very carefully when committing data to data histories. Projects should also pay attention to discussion around the similar security issues that have been noted in the code development world, e.
OpenDataKit (ODK) OpenDataKit is a suite of software and related protocols designed to facilitate a complete flow of form design, data entry and data aggregation primarily organised around the semi-Open Source Android platform. Form templates are stored in XForm syntax. The forms can include multiple choice questions, free text input, dates and media such as photographs. We used XLSForm to design our forms, which relies on a simple Excel-table structure to manage questions and expected answers, and kept versioned copies of this as it evolved in the Project's GitLab CE-hosted survey-design repository. The latest version of the form was converted into an XForm and uploaded to our central ODK Aggregate server for the project. In our case we used a Google-based service called App Engine, to run ODK Aggregate but the software can be installed on many other types of server.
GitLab CE GitLab as a web-based platform can be used via a pre-existing central service such as GitLab. com, or installed on any kind of network-connected private server. There are two editions of the software: GitLab EE (Enterprise Edition), which has a few extra features for larger organisations, and is the version run on the GitLab. com service (for free as of 2015-2017), but cannot be installed on a private server without a license. GitLab CE (Community Edition), in contrast is released with an open license and can be easily installed on any private server. We opted for the GitLab CE private server model for a number of reasons: 1. more control over the storage location of the data; 2. more flexibility as to the number of users who might access the service during the pilot. An instance of GitLab CE was set up on a private server managed by the IT department of the German Archaeological Institute that has subsequently been migrated to a server at the University of Cambridge.
R R is a mature, 20± year-old language dedicated to statistical computing  (Ihaka & Gentleman 1996). It is actively maintained open and Free Software: as of May 2017, the last release (version 3. 3. 3 -'Another Canoe') was only two months old. From the beginning, R was designed for statistical modelling and data analysis, but also for outputting well-designed publication-quality 'plots' (i. e. graphs). Perhaps for this reason it is increasingly becoming a dominant analytical language for a number of scientific fields (from biostatistics to finance and sociology). One strength of R is its versatility in working in multiple platforms. This makes it much easier to reproduce analytical results with different operating systems on different computers. For our project, this was particularly beneficial as participants were variously using Linux, Mac and Windows and could download and test the code when necessary.
Acknowledgements: Thanks to:  Anja Slawisch  (DAI),  Hasibe Akat  ( Milet Müzesi ), as directors of the Project Panormos field projects from 2012 to 2015;  Sebastian Weber  ( DAI ) for technical help; the whole 2015 Panormos Survey team for their enthusiastic and witty contributions to the experiment, including daily log entries. The survey project was funded in 2015 by the  German Archaeological Institute (DAI). In the spirit of archaeological collaboration,  Anja Slawisch  (DAI) and  Ben Marwick  (Washington) offered invaluable advice and suggestions to early drafts of this article, which considerably improved to the manuscript. Additional valuable suggests were made by two anonymous referees to whom we are very grateful. Remaining errors naturally remain at the authors' doors.
Introduction RiboNucleic Acids (RNAs) are fascinating biopolymers. Beyond their coding capacities, they can serve as a medium for the transmission of genetic information, as in the case of highly structured RNA viruses such as Ebola or HIV  (Wilkinson et al. , 2008). They can also perform a large diversity of catalytic and regulatory functions, as demonstrated by the 2474 functional families found in the current release of the RFAM database  (Nawrocki et al. , 2015). This versatility is such that RNA is currently considered by a whole scientific community as the most parsimonious explanation for the molecular basis of the origin of life  (Cech, 2015). This versatility, coupled with the combinatorial specificity of its interactions with other nucleic acids, makes RNA a tool of choice for designing nanoarchitectures through programmable self-assembly  (Li et al.
Approach We introduce the concept of non-redundant sampling to study RNA kinetics, using locally optimal secondary structures as representative structures. Working with a reduced conformation space allows to mitigate the limitations of existing (redundant) sampling approaches. The problem of constructing all locally optimal secondary structures was addressed in  Saffarian et al. (2012). Here, we describe an alternative generation algorithm which is efficient, and allows the specification of comprehensive structural restrictions (Section 2. 2). We then define the first non-redundant sampling algorithm for locally optimal RNA structures (Section 2. 3), allowing for the exploration of RNA folding landscape. The two algorithms are implemented within the standalone software RNANR.
Materials and Methods All experiments were run on a laptop with Intel Core i7 5600 CPU equipped with a quad core at 2. 6 GHz with 16 GB of RAM under Ubuntu 16. 04 LTS. RNANR was implemented in C, and is freely available. It interfaces the RNALib, using the C þþ API provided in the Vienna package  (Lorenz et al. , 2011) , to access the individual contributions of the 2004 version of the Turner energy model. The tests were done on a version compiled with gcc using GNU99 standard. Gradient walks were performed used the move_gradient function of the ViennaRNA package  (Lorenz et al. , 2011) , which performs a gradient descent in the Turner energy landscape and return one of the closest local minima.
Datasets Three datasets were considered in our validation effort. The uniform dataset consists in random, uniformly distributed, RNA sequences of length from 10 to 140 nt increasing by 10 nt, with 50 samples per length. In order to assess the characteristics of real RNAs, we also gathered the RNAStrand dataset, which consists of the 154 RNA sequences of length between 120 and 170 nt downloaded from RNAStrand database  (Andronescu et al. , 2008) , filtering out undefined symbols. Finally, since currently available kinetics data is too scarce to allow for a quantitative comparison of tools, we created a dataset of 250 bistable sequences of length 100 nt. A bistable RNA sequence presents two stable conformations differing by a sufficient number of base pairs. We generated random uniform sequences of length 100 nt, and retained only those whose most stable LOSS (MFE) had freeenergy lower than -30 kcal.
Program parameters Unless noted otherwise, the settings of the different programs used in our comparisons are those described in this section. For RNANR, the default mode is that of non-redundant sampling, with 20 samples. The structural restrictions include a minimum helix length a ¼ 3, a minimal base pair distance h ¼ 3, a maximal unpaired region length d ¼ 7, and a maximal number of helices in multiloop b ¼ 4. For RNALocopt the number of returned samples is the same and the temperature was set to 310. 15K. For RNASLOpt, the suboptimality percentage was set to 100%, meaning that all LOSSes with energy ranging between the MFE and 0 are returned. Since we are interested in LOSSes independently of their stability, we set the barrier_cutoff to an arbitrarily large value of 30 kcal. mol -1 to speed up the computations by avoiding the computation of the energy barriers.
Likewise, the number of top stable LOSSes can be chosen arbitrarily, here its value is set to 3. RNALocmin was run using the second version of the built-in adaptative search script, referred to as asearch and coded in Python  (Kucharik et al. , 2014). The parameters used are those by default, i. e. 10 000, 10 and 0. 1 respectively for the number of samples per iteration, number of iterations and convergence parameter.
Comparison of folding landscape analysis efficiency To compare the quality of sampled sets of structures, we considered an artificial bistable dataset described in Section 3. 1, and produced representative sets of structures using the four programs mentioned in Section 3. 2. We analyzed sampled sets using a standard kinetic analysis pipeline based on an estimation of energy barriers for each pair of structures, followed by a numerical integration using treekin. For each sequence in the bistable dataset, each program was used to generate n sam ¼ 50, 75 and 100 samples (output truncated if necessary). Gradient walks were performed to each sample set, and duplicated Turner LOSSes were removed. As a control, we also included the results of RNAsubopt -e, adjusting DE to return at least n sam LOSSes.
Nussinov LOSSes are less numerous than Turner LOSSes We compared the numbers of LOSSes returned by RNANR, RNALocopt and RNASLOpt. For each sequence in the uniform dataset, these sequences were subjected to runs of each software. For RNANR, we performed three runs, each with different value of c (no limit, 4 and 3 respectively). For RNALocopt and RNASLOpt we performed one run for each with parameters as specified in 3. 2. The values of each run were then aggregated by the sequence length. The results are shown on Figure  5. Figure  5A  shows the evolution of number of LOSSes for different programs in function of sequence length. We observe that RNASLOpt returns the smallest number of results. This is consistent with the primary objective of  Li and Zhang (2011)  to reduce the number of structures as a way to reduce the complexity.
Nussinov LOSSes are very close to turner LOSSes We used gradient walks to determine how distant are Nussinov LOSSes, output by RNANR, to their counterpart in the Turner energy model. For each sequence in the RNAStrand dataset, a nonredundant sampling of 1000 distinct structures was performed by RNANR. These structures were then subject to a gradient descent, resulting in a Turner LOSS. We tracked the modifications, both in term of energy and base pairs, induced by the walk. Our results are summarized in Table  1. Overall, 52. 49% of the Nussinov LOSSes were already local minima with respect to the Turner energy model. Moreover, our analysis shows that, from a Nussinov LOSS, it is sufficient to add or remove an average of 0. 703 base pairs, contributing an average 0. 547 kcal. mol -  , to reach a Turner LOSS.
Efficiency of sampling methods for LOSSes A first time benchmark, whose results are given on Figure  5C  was performed on our uniform dataset. We observe that, without limiting c, RNANR is fastest for sequences of length under 80 nt. Around this length, it is briefly matched by RNASLOpt, whose execution time increases spectacularly around 110 nt. RNALocopt is faster than RNASLOpt or RNANR (no limitation on c) for longer sequences (more than 120 nt) due to the polynomial nature of the underlying algorithm. However, for values of c set to 3 or 4, the execution time of RNANR becomes polynomial, and RNANR becomes considerably faster in practice than its competitors. Of course, one needs to exercise caution before setting restrictions that could lead to the omission of important conformations, and it would probably not be wise to set c ¼ 3 for RNAs beyond 80 nt.
However, setting c ¼ 4 makes RNANR faster than any other tested software, while only missing a negligible proportion of existing conformations (cf Fig. 6 , for multiloop branch number equal to 6) for sequences beyond 300 nt. A second basic time benchmark, described in Section 3. 4 and in Figure  1 , measures the rate of production of distinct Turner LOSSes generated using different software. We observe that RNANR returns more unique LOSSes in a given time, even when including the time for precomputations and gradient descents, than both versions of RNALocmin and RNALocopt (note that RNASLOpt does not perform sampling). This is mainly due to the redundancy within the returned samples, as indicated by the diminishing production speeds for both methods. The second version of asearch uses an alternative strategy for n-scheduling which increases its number of samples linearly between iterations, and seems to eventually perform better than its initial version.
Non-redundant sampling allows a deeper exploration of kinetic landscapes The main new contribution of RNANR is its non-redundant sampling algorithm, which allows to obtain a set of locally optimal A B Fig. 6. Rationale for our restricted search space. Effect of structural limitations on structure counts obtained from RNAStrand database  (Andronescu et al. , 2008). (A) Proportion of structures having maximal multi loop branches below a given threshold. A branch number within multi loop is the number of stems sorting from a multiloop and is equal to c þ 1. (B) Proportion of helices having at least given length For our RNAStrand dataset, 1000 Nussinov LOSSes were generated. For each structure, one of the closest Turner LOSS was determined using a gradient descent. On average, a Nussinov LOSS is distant by 0:547 kcal.
This means that higher values of c will be more difficult to attain for some sequences than for others, and for these cases the usage of non-redundant sampling might prove more advantageous. Our second test consisted in creating the samples using redundant generation, and comparing these results with the nonredundant sampler of RNANR. For this purpose, we used RNALocopt to generate 1000 LOSSes for each sequence of our uniform dataset. For each set of structures, the number of unique samples and the coverage c were computed. RNANR was then used to generate a set of structures achieving the same coverage c. The averaged number of structures for each length was reported, leading to the values shown in Figure  7C  and D. While the number of unique structures returned by RNALocopt increases the sequence length, the coverage value c diminishes.
Therefore, several approaches  [4, 21, 25, 27]  have been developed to automatically discover keys from RDF data. However, these works have also shown that for several datasets, there are no or only few keys that are valid in the entire dataset. This is the reason why we aim to mine conditional keys in this paper, i. e. , keys that are valid in only a part of the data. A conditional key is an axiom saying that under particular conditions, no two distinct entities can have the same values on a particular set of properties. For example, we can say that at a German university, no two professors can advise the same doctoral student. The situation might be different at a French or American university -hence the key is "conditional" to German universities. In this paper, we distinguish conditional keys from classical keys, which hold for an atomic class (or for every tuple of a table in a relational database).
Related Work Relational Databases. Two types of key discovery approaches have been proposed for relational databases  [16, 24] : data-driven  [24] , where keys are discovered from the tuples in a table, and schema-based  [16] , where property combinations of a certain size are generated and then checked on the tuples. Such approaches cannot be applied directly to KBs, because they are geared towards relations that contain one single value for each subject. In KBs, in contrast, a relation can contain several objects for the same subject. Knowledge Bases. Approaches for KBs  [3, 4, 21, 25, 27]  can be roughly classified into two groups, depending on how they deal with multivalued properties  [2] : The forall-key approaches  [4, 25]  discover keys that fire when two entities share all values for each property.
[3]  discovers discriminative combinations of corresponding properties that can be used to link two datasets with different schemas. KD2R  [21]  extends the relational data-driven approach of  [24]  in order to exploit ontology axioms (such as the subsumption relation) and considers multivalued properties. SAKey  [27]  introduces additional filtering and pruning techniques to discover efficiently some-keys with exceptions. To avoid scanning the entire dataset, both KD2R and SAKey discover first the maximal non-keys and then derive the keys from this set. Yet, none of these approaches is able to mine the conditional keys that we aim at in this paper. Conditional Functional Dependency Mining. A conditional functional dependency (CFD) expresses a functional dependency between two sets of attributes that holds on a subset of tuples  [6].