dataesr/scientific-paragraphs-categorization huggingface dataset:
Finally, proprietary clouds are pay-per-use, which slowed us down for a first proof of concept. However, for future work on larger use cases or with different needs, they might be reconsidered. Hadoop ecosystem. A third option explored is about using an ecosystem from the world of big data open technologies, such as Apache Hadoop 5. Like pro-prietary clouds, Hadoop comes with its own ecosystem, and a lot of services are available to ingest data, store data (especially with the Hadoop Distributed File System, HDFS), manage the metadata system and run different types of data analysis. In particular, the metadata system Apache Atlas is very interesting, and seems to be complete enough for us to implement our metadata system. Nevertheless, following preliminary tests, this solution was finally not retained.
On the one hand, HDFS is suitable for large files, but not for small files, which we have in large quantities in our use case. On the other hand, installing an entire Apache Hadoop ecosystem is a very long, complex process, and requires specific skills, in addition to the need for hardware resources to run the cluster properly. Furthermore, these open source technologies evolve very quickly, and it is necessary to keep a watchful eye on new developments, but also to keep the cluster up to date, which takes time and can sometimes be a complicated task. Finally, in our vision of the data lake, we consider that an Apache ecosystem is welcome when there is a need to manage large amounts of data, but this was not an issue in our use case. Integrating an HDFS storage area into the data lake, as well as some associated services, may however be a very interesting topic for future work.
Technical architecture HOUDAL (public HOUsing DAta Lake), our implementation of the data for public housing, is based on a web application. It has two major parts: the frontend, or client part, which we can refer to as the user interface, and the back-end, or server part, which is broken down into various services, namely the API, the metadata system, the data storage, and the user management service. User interface. HOUDAL is operated through a user interface. Users can connect to the interface and then interact with the data lake. They can consult and explore existing metadata, with a search bar among others. Files stored in the lake can be downloaded. In addition, it is also possible for users to create new metadata, but also to add new files to the data lake. For this, a form is available, with which the user uploads the file.
During this process, metadata are entered semi-automatically : some information, such as file name, size or format are extracted automatically, while others have to be entered manually by the user, such as a description. Once the file upload has been validated, data are saved in the storage area, and metadata are created. From a technical standpoint, the user interface has been developed in ReactJS. API. Through requests methods, the user interface communicates with an API. For any action made by the user on the interface, it generates a request that is sent to the API. The API then queries the application's various services, such as the metadata system, and returns the results of this query to the interface in response to the user's request. This API has been developed in NodeJS. Moreover, HOUDAL works with an identification system: the user must log in to access the data lake.
Besides, some pages of the interface are reserved for users with an administrator role. The information about the users is saved in a Mon-goDB collection. Note that a MongoDB database is not a necessity, and this may change in the future. Metadata system. Since our metadata system is based on goldMEDAL, which has a graph-based representation, we store the data lake's metadata in a Neo4J 6  graph database. The database is accessed only by the API, whether to consult existing metadata or to create new ones. Data entities are nodes, and are linked to each other through processes, or connected to groups. Having all the metadata in a single database is an advantage because it simplifies API calls. Storage area. In our case of use, we only deal with files so there is no database to manage (although database data can also be considered as files).
This implies that for the storage area of HOUDAL, we did not need anything other than a directory of a conventional file system to store our data. Note however, that this practice, although sufficient in our specific case, is limited and could be problematic in other use cases. This is discussed in more detail in the following subsection. A screenshot of HOUDAL's management interface is presented in Figure  2. The interface allows users to create different types of metadata. To this end, the different buttons at the top of the screen redirect users to pages containing forms to create metadata. Moreover, the interface provides a search bar, where the user can perform a keyword search in data entities. On the screenshot, the user has searched for the keyword "solicitation". The interface then provides the user with all data entities containing this keyword in their file name or description.
HOUDAL applied to the use case In Section 3, we presented how the use case generates issues when using traditional BI and BA tools. We also showed that, from a theoretical standpoint, using a data lake would be a good solution to tackle these problems. In order to validate our proposal, we have tried to replicate in HOUDAL the project carried out without the use of a data lake. Here we discuss what HOUDAL's metadata system looks like when applied to this use case, and how it assists data specialists. As presented before, HOUDAL's metadata system is based on three main concepts. The different data files that populate the data lake are data entities. They can be either raw data files sent by landlords (often in comma separated value files) or reworked data, sometimes stored in various formats such as.
pkl or. RData, for Python and R analyses, respectively. In Neo4J, each data entity has its node labeled :ENTITY and the entity's properties, such as file name or description, are stored in the node's attributes. Groupings are use for categorizing data entities. With HOUDAL, users can create as many groupings as necessary, and several groups for each grouping. Data entities can be linked to zero, one or several groups for each grouping. In the physical model, groupings are modeled by nodes carrying a :GROUP-ING label. Groups are also nodes, carrying both a :GROUP label and grouping's name as a second label, in order to facilitate querying. A data entity node can be linked to several group nodes. A data entity node (resp. group node) is linked to a group node (resp. grouping node) with an edge labeled with the grouping's name (resp.
Limitations and future work HOUDAL is functional and is currently in advanced testing phase. Several social landlords have shown interest in our work and, although much work remains to be done, discussions are underway to deploy this application to other social landlords over the long term. However, while being functional, it is still a work in progress, and thus has limitations. The first limitation of HOUDAL is its data storage system. Indeed, considering the use case we have treated, in which there are only files, having a storage system other than a directory was not necessary. On the other hand, in the long term, we would like to be  able to offer users the possibility of carrying out BI analyses, which often implies the presence of a data warehouse, and therefore a relational database.
To overcome this problem, we wish to add, at least initially, a relational database in the storage space of the data lake. In order to make it easy for the user to use it, we also wish to propose through the interface a page allowing the user to enter SQL queries, in order to create and fill his databases and tables. The second limitation arises at the level of filling the data lake with source data. Although this process is currently simple, it remains tedious in the case of a large number of files, and if the data source is not a native file, it forces the user to generate a file from the source. If we take the example of a source being a relational database, the user must extract data from the tables as CSV files and then insert them in the data lake. This adds one more step in the process of storing data into the lake, and we could argue that it "denatures" the source in a certain way.
For example, in the case of data concerning Social Housing, if a dataset contains details about the income of tenants or their professional situation, it must be considered sensitive and therefore only accessible by users who have been validated beforehand. In the case of the use we dealt with in the data lake, there was no notion of sensitive data. For these reasons, we wish to work on this aspect. To this end, we want to introduce the notion of roles and restrictions in the data lake management application. We want the roles to be modular, i. e. to be able to define the roles for a given project, and to associate restrictions to these roles. It is also possible to restrict access to metadata according to a profile : to be able to only consult, to be able to create new metadata, to be able to modify it, and so on.
CONCLUSION For achieving Data Intelligence in the context of big data, some challenges must be addressed first. One of or proposals is the use of a data lake, in which all types of data are stored, and from which various kinds of analyses can be performed. This is a trendy research topic with several possibilities being explored. Given the constraints of our real-life use case, we drew inspiration from our metadata system based on data semantics, namely MEDAL, to which we made a few modifications to create goldMEDAL. It is based on three main concepts: data entity, grouping and process. This metadata system allows us to define a frame-work that doesn't give a predefined shape to the data lake. Conventional data lakes operate in ELT; we prefer to add two steps to get an EDLT(L) operating mode.
Step D, for Describe, indicates that metadata are extracted before storing the data in the lake. The optional step L, for (Re)Load, is for analyses conducted on the lake's data, which may generate new reworked data; when this is the case, these data are back-fed into the lake, and go through the Describe step again. In the context of public housing, we have implemented our framework and we have created HOUDAL, a data lake for public housing. After a review of the different possibilities to carry out this development, we chose to develop a web application from scratch, in order to best fit the requirements of our use case. HOUDAL is composed of a user interface and of an API that queries the various services necessary for the proper functioning of the lake, in particular the data lake storage area and the metadata system.
HOUDAL is showing satisfactory results, but we have many areas for improvement to make it more complete. In particular, we would like to improve the feeding of the data lake by being able to query various sources, but also to simplify the data extraction from the lake, whether through ETL or scripting languages. Finally, we also want to make HOUDAL more robust against GDPR-related problems, which we did not have to deal with in our case of use, but which are nevertheless very common. One of HOUDAL's long-term objectives is to allow the study of a social landlord's patrimony, and more precisely the attractiveness of its dwellings. This is indeed a key issue for social landlords. In the past, qualitative studies have been carried out with the help of business experts to obtain strategic information on their patrimony and the actions to be taken for the future.
To meet these constraints, we believe that using HOUDAL as presented in this article can be successful. Indeed, it is a good solution to store both internal data from social landlords and external data gathered from the Internet, while describing them properly with the appropriate metadata. Moreover, the study of housing attractiveness is a good example of a Data Intelligence project and the creation of advanced indicators. The external data will be reworked and then cross-referenced with internal data, with the final objective of providing an attractiveness score for each dwelling.
Type of study An accurate level of evidence is required depending on the type of study. For randomized clinical trials, calculation of the sample size, a robust randomization method, and knowledge of the primary and secondary outcome measures and loss to follow-up are important. For matched cohort studies, the definition of the cohort and matching is important. The authors should clearly inform the readers if the study groups are comparable and any potentially confounding variables between the study groups at baseline. For registry studies, it is important to include where/what is the registry, and how complete and comprehensive the database is. Meta-analyses are overdone; for the meta-analysis to be interesting, it should include important study questions. If the answer to these questions is negative, then the meta-analysis is probably not necessary.
of the mechanisms of action are uncertain, this reflects on uncertainty. As model complexity increases, the uncertainty tied to the model framework is reduced, but a more complex model uses more parameters and the data uncertainty increases  (Gaber et al. 2009 ). For ERA Acute there are uncertainties of the model that belong to both the structure (model uncertainty) and the numerical parameters used (epistemic uncertainties). On the condition that the functions and calculation sequences in the model are correct, if the input is not changed, the model output stays the same. In this case, uncertainty in the model output is solely affected by variation in the input parameter. This is called Epistemic, or reducible uncertainty  (Helton et al. 2006) , related to lack of knowledge of the true value of a constant parameter  (Marino et al.
2008). For the individual parameters used in a model it is important to distinguish clearly between: Variability: How spread out or clustered a data set is, e. g. the (natural) variation in the measured values found in nature and Uncertainty-The lack of certainty or knowledge about what the value of the parameter/data truly is. Such data uncertainty is specific to the individual parameter. As mentioned above, a more complex model uses more parameters and data uncertainty therefore increases  (Gaber et al. 2009). ERA Acute uses many parameters. Sensitivity Analysis (SA) tells us how the model's response can be apportioned to changes in model inputs. It is algorithm specific. For models with a high number of parameters, sensitivity analyses are useful to rank the relative importance of the factors and processes involved  (Saltelli 2004 ).
ERA Acute is a new method and testing the sensitivity of the model to variation in the input parameters is an important part of uncertainty handling, with the goal of ensuring that ERA Acute does not under-estimate environmental risk. All data sets and parameter values have inherent uncertainties and a model consisting of a series of calculations will need some method of handling uncertainty. In the process of developing ERA Acute, the following activities were carried out: 1. Sensitivity testing of the risk functions to the variation in input parameters 2. A pilot study to score the parameters and propose feasible uncertainty handling The functions of ERA Acute are built so that individual parameters representing biological or environmental characteristics can be improved as knowledge increases, thereby reducing uncertainty by a continuous improvement process.
Sensitivity testing provides knowledge of which of the parameters that contribute most to the final endpoint values, and therefore the testing provides information on which parameters that would be most important to improve by further research if they have high uncertainty. ERA Acute covers four compartments and uses a large number of functions. The input parameters (values and datasets) that are used are based on knowledge from few and highly diverse incidents. Validating the results of the method and applying the results with an acceptable level of (un)certainty is therefore challenging (see  Chap. 4 "Testing and Validating Against Historic Spills"). In such a case of applying a complex model and multitude of uncertain parameters, it is important to realize and accept that we do not know the "true risk" as a number as such but need to ensure that the model does not underestimate the risk and can be used to compare risks.
Use Best Practice for ODS set-up to ensure comparable results. Use a concentration grid (with many layers) that cover the same water column where the fish egg/larva are distributed Frackilled A Strong. Estimated in OSCAR during the ODS B Moderate C High/Moderate/Low (depending on setting) Comments/recommendations: Estimated by OSCAR during the ODS (potential acute mortality in a cell). Standard deviation (SD) of the SSD and the species sensitivity may be adjusted before one run the ODS. The species sensitivity is a safety factor. The OSCAR database LC50 values will be divided by this factor, accounting for more (factor >1) or less (factor <1) sensitive fish larva/egg N per cell A Strong. Depends on the quality of the data received from the data provider. Compared to e. g. birds the distribution is to a large degree dependent on sea currents B Moderate C Moderate/high Comments/recommendations: Use the best available data to reduce uncertainty and increase the quality of the predictions.
Formally, models generated by such algorithms are defined by a corresponding data-dependent partition P n of R d. Each element of the partition is named a cell. Let us define for any r ⊆ R d such that Q(r) > 0 and for any h measurable E[h(Y ) | X ∈ r] := E[h(Y )1 X∈r ] P(X ∈ r) and V(Y | X ∈ r) := E Y 2 | X ∈ r -E [Y | X ∈ r] 2 and their empirical counterparts for any r ⊆ R d such that Q n (r) > 0: E n [h(Y ) | X ∈ r] := n i=1 h(Y i )1 Xi∈r n i=1 1 Xi∈r and V n (Y | X ∈ r) := E n Y 2 | X ∈ r -E n [Y | X ∈ r] 2. Any such algorithm following the ERM principle associates to P n an estimator such that g n 1 {A∈Pn:Qn(A)>0} = A∈Pn Qn(A)>0 E n [Y | X ∈ A]1 A (1. 3) which minimizes the empirical risk among the class of piecewise constant functions over P n. Those algorithms use the dataset D n twice: The partition P n = P n (D n ) is constructed according to the dataset, then the dataset is used again to calculate the value of g n on each element of P n as in (1.
4. Select a minimum set of rules C n using Algorithm 1 given in the appendix. If at any step the set of selected rules forms a covering, the selection process is stopped. A rule is added to the currently selected set of rules if and only if it has at least a rate 1 -γ ∈ (0, 1) of points not covered by the current set of rules 2. The set S n , sorted by decreasing empirical coverage rate, is browsed first. Then, if necessary, the set I n , sorted by increasing empirical variance, is browsed. As explained in Definition 2. 1 the set C n is the union of the significant set C s n = C n ∩ S n and the insignificant set C i n = C n ∩ I n. In practice the set C s n is the most interesting one because it identifies the rules where the conditional mean is prominent. The datasets and the code for the illustrations are available on GitHub.
Real data For this application, we consider seven public datasets presented in Table  2. with different dimension and number of observations. For the dataset Student we have removed the variables G1 and G2 that are the first and the second grade respectively because the target attribute G3 has a strong correlation with G2 and G1. In  Cortez and Silva (2008)  the authors state that despite the complexity of the regression model, the prediction of G3 without G2 and G1 is much more useful in practice.
Execution For each dataset we run 20 executions. For each execution, the data are randomly split into a training set and a test set, with 70% and 30% ratios respectively. As a baseline of accuracy we consider RF  (Breiman, 2001)  and as a baseline of interpretability we consider the CART algorithm  (Breiman et al. , 1984). We compare Covering Algorithm with RF as rule generator (CA RF), GB trees as rule generator (CA GB), SGB trees as rule generator (CA SGB) with RuleFit  (Friedman and Popescu, 2008)  (with rules only), Node harvest  (Meinshausen, 2010)  (Nh) and SIRUS  (Bénard et al. , 2020). These algorithms are the main existing interpretable rule-based algorithms for regression settings that are based on tree ensembles. Node harvest uses also a tree ensemble as rule generator. The algorithm considers all nodes and leaves of RF as rules and solves a linear quadratic problem to fit a weight to each rule.
Results Experimental results are gathered in Table  4. For each dataset and algorithm we present an average over the 20 executions of the number of selected rules (Nb rules), the empirical coverage rate on the training set (Cov), the interpretability index (1. 4) (Int) and the mean squared error on the test set divided by the empirical variance of the target (MSE). These results emphasize that suitable data-dependent quasi-coverings (see Definition 2. 1) are very efficient to generate an interpretable rule-based model, see also Section 4. 3. 3. They generate very simple models with an interpretability index (1. 4) much lower than the other algorithms. Moreover the sets of selected rules C n form in most cases a covering of the training set. CA achieves a good interpretability-accuracy trade-off with emphasis on interpretability except for the dataset Student that might be too complex to be interpretable.
Examples of interpretation Any model generated by a CA can be summarized in a table of rules. We present one set of rules selected by CA RF for the dataset Ozone in Table  5. The measure ∆ n is the empirical mean deviation ratio of the prediction from the mean of Y. Table  5  is a description of the model. Together with Table  6  we are able to translate in natural language the significant rules in Table  5 , according to the definition of interpretability of  Biran and Cotton (2017). • Rule R1 has a length equal to 1 and suggests that if the temp variable is reasonably low then the daily maximum of the hourly average ozone concentrations in Upland tends to be low. • Rule R2 has a length equal to 1 and suggests that if the ibt variable is not too high then the target tends to be low. • Rules R3 and R4 have a length equal to 2 and suggest that CA has detected an interaction between high ibt and the humidity variable.
Comments In Figure  7  and Figure  8  we present boxplots of the MSE over the 20 executions for each algorithm for the Machine and Ozone datasets. These two datasets has been chosen to show that CA can have both a very variable accuracy (Figure  7 ) and a stable and a good accuracy (Figure  8 ) for two datasets with the same dimension. One reason could be that rule generators of CA are random generators of rules that may not capture the good features at every run. In particular we have identified the problem of predicting the empty cells of the partition generated by the quasi-coverings. Then, by convention, g n (x) = 0 for x ∈ A with Q n (A) = 0. Thus, the accuracy of CA strongly depends on the coverage Q n (A) of the sets A ∈ P(C n ) which can be too small to be stable. In Definition 2. 1 of suitable datadependent quasi-coverings only the coverage of the elements of C n is controlled.