dataesr/scientific-paragraphs-categorization huggingface dataset:
Linking data of heterogeneous Knowledge Bases In this set of experiments, we compare the linking capability of each of the three semantics for integrating two heterogeneous knowledge bases conforming to different ontologies. As before, we also analyse the impact of allowed exceptions on the data linking performance. Datasets. (DBpedia 2016 VS. YAGO 2016) Each experiment takes as input the complete et of triples of a given class found in two real-world knowledge bases, DBpedia and YAGO 8 , both released in 2016. The data for this experiment have been provided initially in  [24]  containing data for ten different classes. These classes describe different domains such as organisations, actors, locations etc. Data linking challenge. While both DBpedia and YAGO contain information extracted from Wikipedia, they remain highly heterogeneous.
Since the two knowledge bases conform to different ontologies the vocabulary used to describe identical properties may differ. Note that DBpedia usually contains more properties than YAGO to describe one class. Additionally, a significant part of YAGO is manually verified before being released. As a result, the use of different strategies and methodologies used for the creation of each knowledge base lead to the construction of highly heterogeneous datasets. The authors of  [24]  have executed a step of property alignment where different properties referring to the same information have been matched. Then, the properties of YAGO have been renamed using its DBpedia counterparts. In the final datasets released, only triples containing properties found in both datasets have been kept. Table  11  shows for each of the provided classes the number of common properties, the number of triples for both YAGO and DBpedia and finally the number of instances for each of them.
Data linking using 0-almost keys In this experiment, we compare the linking results of the three key semantics when 0 exceptions are authorised. For each class and each semantic, the keys have been discovered in YAGO and then used to provide links towards DBpedia. This choice has been made since YAGO tends to be cleaner and contains less errors. The first three columns of Table  12  contain the FMeasure scores that show the linking capability of each key semantic. As it can be easily noticed, similarly to the previous experiments, for all classes, F-keys obtain an FMeasure score of 0 showing that F-keys fail to provide any links when 0 exceptions are allowed. This is due to the strict nature of this semantic. On the contrary, S-keys and SFkeys provide much better results, reaching an FMeasure score of 56,33 for the class Book when using SF-keys and an FMeasure score of 79,51 for the class City when using S-keys.
, located in the two cities found in each dataset (described by the property Islocatedin-inv), have to be the same. As we can easily observe, this is not the case in this example since the two knowledge bases do not contain exactly the same information. Therefore, as suggests the definition of SF-keys, comparing the complete set of values can be an obstacle when data come from different sources that do not always contain all the information. Moreover, the different representation of the data in different datasets is another reason why SF-keys do not provide good linking results. For example, the property Preflabel is used in both knowledge bases to provide a label for each instance in the data. While for YAGO this property contains always only one label per instance, this is not the case for DBpedia.
For example, the actor Johan Paulik has two labels in DBpedia, "paulik, johan" and "johan paulik" while only the second one is found in YAGO. Since the property Preflabel appears often in both S-keys and SF-keys, it is easy to see that Skeys containing this property can provide high scores of FMeasure, unlike SF-keys that are not able to deal with this heterogeneity. Therefore, depending on the nature of the data and the structure of each knowledge base, Skeys or SF-keys are more appropriate. When are SF-keys more suitable? In this experiment, the only case where SF-keys bring better linking results than S-keys is when 30 exceptions are authorised for the class Museum. In this example, all keys found by both semantics are the same except for the SF-key(Islocatedin, Preflabel) that brings additional links, leading to an increase of the FMeasure.
Conclusion In this paper we present BECKEY, a generic approach for the discovery of keys in large RDF knowledge bases. BECKEY is able to discover keys under three different semantics introduced in the context of the Semantic Web. The different key semantics are presented and compared theoretically. In order to discover keys of different semantics, a structure able to capture these semantics is introduced. We propose a semantic agnostic algorithm for the efficient discovery of almost keys in large knowledge bases. The extrinsic experimental evaluation has allowed a comparison of the linking capacity of the three semantics. It is demonstrated that F-keys are not useful for data linking when data are incomplete, which is often the case for the data on the Web. Overall, S-keys and SF-keys tend to bring similar results of a significant linking quality, proving their relevance for the data linking task.
table, 42,22% of the triples in the initial version (db2013) are removed from the newer ) while more than 50% of the triples of the version db2016 are new. Similarly, 19,57% of the initial instances are not appearing in the newer version while around 40% of the instances in the db2016 are new. Finally, this evolution is not only in terms of triples and instances but also in terms of properties. Therefore, 29,13% of the properties used in the older version are not present in the new version while 24,32% of the properties in the db2016 are new. Even if the task of linking two different versions of DBpedia would seem straightforward the heterogeneity and the significant changes at different levels, as shown in Table7, make this task a challenging problem. In order to verify the quality of the discovered keys, a goldstandard containing the instances of both datasets has been created.
Searching and Selecting the Relevant Literature Six electronic databases (CINAHL, Google Scholar, PsycINFO, PubMed, Scopus, and Social Science Journals) were searched on 12 November 2019, using "resilience engineering" as the keyword-i. e. , TITLE-ABS-KEY ("resilience engineering,"), in line with a recent systematic review  [9]  and meta-analysis  [11] , to identify articles published or in press from January 2003 to November 2019. The search was limited to full-text articles and conference proceedings published in English. "Grey literature" was also searched by reviewing reference lists to identify any articles that may have been missed. The titles and abstracts were screened by two independent research assistants (RA) and included if they: (i) used groups, teams or organisations as a unit of analysis; (ii) were journal articles or conference proceedings; (iii) focused on safety.
Data Extraction The key information required for analysis and synthesis was exported into a data extraction sheet. Information regarding research aims, theory/model, research design (methods, participants), factors and variables measured, instruments used and approaches to statistical analysis were extracted. There is no specific published criteria for evaluating psychometric properties for RE studies; however, reliability and validity are two of most common used for evaluating the rigour of in quantitative studies  [41]. Reliability is most assessed through Cronbach's alpha coefficient, with a minimum value of 0. 70 suggested to be generally acceptable for new measures, constructs and/or scales  [42]. Validity is the degree to which the research phenomenon being investigated is accurately measured, and is generally assessed for content, construct and criterion  [42].
Results Figure  1  presents the findings of our search and selection strategy. The search of six databases and grey literature generated 3884 articles, from which 3731 duplicates were removed, leaving 153 articles for title and abstract screening. A further 78 were screened out at this stage, resulting in 75 articles for full-text review. An additional 58 were deemed not eligible at this stage, resulting in 17 studies for the final review and synthesis, these are summarized in Table  1. Safety 2020, 7, x FOR PEER REVIEW 5 of 26 Each question was assessed using a YES, NO, LIMITED response by the first researcher and cross-checked by the second. While this process was used for appraising the quality of published works selected for review, no articles were filtered out at this stage.
Results Figure  1  presents the findings of our search and selection strategy. The search of six databases and grey literature generated 3884 articles, from which 3731 duplicates were removed, leaving 153 articles for title and abstract screening. A further 78 were screened out at this stage, resulting in 75 articles for full-text review. An additional 58 were deemed not eligible at this stage, resulting in 17 studies for the final review and synthesis, these are summarized in Table  1.
Strengths, Limitations and Implications There are several strengths with this review. It utilized a positivist paradigm to investigate objective instruments and measures  [10, 35]  and applied a pragmatic theoretical perspective to focus on practical outcomes  [36]. The researchers used a structured approach for searching and selecting articles. The title, abstract and full text article searches and selection were carried out by two independent Ras, and the final set of articles was also subjected to a process of critical appraisal. Collectively, these demonstrate a high degree of rigor in the review process, and confidence in the results and discussions. There are also some limitations with this review. The exclusion of other bibliographic databases (such as Ei Compedex of Inspec), could potentially lead to overlooking articles that may have been published-for example, from a system engineering perspective.
Conclusions This review was aimed at reviewing the conceptualization, definition and measurement of RE-identifying any instruments used, and the psychometric measures tested, in order to inform a theoretical framework and measurement instrument that can be used to advance research and practice in the field. A positivist paradigm and the theoretical framework of pragmatism, together with a structured integrative review method, were used to enhance the rigour of the research process. The search across six comprehensive databased generated over 3900 articles, of which 17 were included in the final synthesis. In total, 15 survey instruments were identified, but only four of these were supported with theory. In total, 11 of these were from Iran. A minimum of three and a maximum of 13 dimensions were used in the surveys; however, there was a wide diversity in the number of questions asked for each dimension.
Living monitoring of methodological quality of trials Asking systematic reviewers to collect a limited number of additional data when they are performing their review should be doable. The incremental cost of collecting additional data at this stage represents a limited and manageable effort with significant potential long-term benefits. Some of the data necessary for monitoring the methodological quality of trials are already extracted for most reviews (e. g. , items to assess the Cochrane Risk of Bias tool [RoB]). For example, systematic reviewers could check whether a Core Outcome Set is proposed for the disease and recorded in the trial report or could identify major methodological issues. Providing access to this information will help trialists improve the planning of their future trial by avoiding reproducing some methodological errors or forgetting essential outcomes and therefore decrease the waste in future research.
-Living monitoring of transparent reporting of trials In the same way, we could perform a living monitoring of the transparent and complete reporting of trials by assessing the quality of reporting for the most important items. We could perform a permanent audit of the rate of publications and posting of trials, their compliance with the 2007 US Food and Drug Administration Amendments Act (FDAAA) that requires sponsors of trials in the United States to post results on ClinicalTrials. gov and the European Union requirement to post results on the European Clinical Trials Database (EudraCT). Furthermore, we could use the RoB assessment as a marker of the completeness of reporting. Indeed, with the new version of the RoB tool (RoB 2. 0), all signaling questions rated by systematic reviewers as "No information" are considered poorly reported.
-Living reporting system of the quality, transparency and accessibility of data All these data would then be used for a living disclosure of the quality and transparency of reporting. The mapping and disclosure by medical specialties, journals, funders or universities of research transparency would likely have a positive effect on various stakeholders and would help raise awareness about the waste related to poor reporting, thereby motivating actors to develop quality improvement programs. In the same way, a mapping of the stakeholders who make their data publicly available would be useful  [13]. We can also have a more proactive and incentive approach to improve transparency by identifying on ClinicalTrials. gov and EudraCT all trials as soon as they are terminated and systematically encouraging principal investigators via automatically generated emails to post their results on registries within 1 year after the end of the trial.
Bridging the gaps between evidence synthesis communities, clinical practice guidelines developers, agencies for health technology assessment, health policy authorities and primary researchers A new evidence ecosystem should allow for bridging the gap between not only systematic reviewers and trialists but also with developers of clinical practice guidelines. Living NMA provides a network of current evidence for the clinical question of interest representing all treatments assessed and trials performed in this setting. It offers constant access to updated global evidence synthesis and may help improve real-time knowledge transfer from systematic reviews and meta-analyses to clinical practice guidelines. This living NMA repository would be the marketplace for evidence synthesis, whereby groups of guideline developers will find the best evidence and access to the data that were extracted.
Nowadays, the cloud computing industry is enjoying an exponential growth, where several cloud service providers compete to be one of the market leaders. Usually, providers offering similar services use different non-functional attributes to describe them. Thus, given the heterogeneity and diversity of services descriptions, the selection process of the appropriate cloud service becomes challenging. Architects no longer know what criteria to use to make the suitable cloud services selection. In this paper, we highlight the challenge of identifying key elements of comparisons and their relationship for selecting cloud services. Further, we propose a methodology to solve this issue based on real data available from service providers and benchmark work. Our methodology is validated based on two case studies of cloud relational databases and cloud queuing services.
A database service for instance may include supplementary functions, including monitoring of certain features or auditing of functions; (3) Service attributes provide information on service level agreements (SLAs) such as SLA on support availability, SLA on service availability and protocols supported by the service; (4) Technical attributes include the attributes that describe how the service is deployed or operated. They have different impact on NFR of the service. To determine the impact of NFAs on NFRs, we first discussed with architects about their experiences with service deployment. Also, we reviewed the technical documentation of cloud services to understand why a NFA was advertised by cloud providers. By this way, we understand its influence on the NFR of the service. Determining the rate of influence of NFAs on NFRs is difficult.
It depends on factors whose information is not available from the service providers. This includes the deployment of the provider's architecture: the number of nodes on which the service is deployed and the technologies used to develop the service. In addition, the influence of NFAs on NFRs depends on the nature of the applications for which the service is intended. Due to these difficulties, we only indicate the positive, neutral or negative impact on the technical attribute (+, =, -). Table  1  gives an example of a database service's technical attributes and their impact on NFRs. + = = = = IOPS + = = = = Backup - + + = = Rollback = + = = = Replication - + + = =.
Selection of NFAs and NFRs from an Empirical Study To ensure the completeness and the relevance of the collected technical attributes, we propose to conduct an empirical study. The aim is to understand how architects decide the types of components to develop a new application, and the criteria that guide their choices. We are particularly interested in core components (database, messaging, caching, indexing, monitoring. ) that can be deployed as servers or consumed as services in the cloud. We conducted a semi structured interview, as done in  [1] , on a general perception of services with seven software architects. All of them have more than four years of experience in cloud computing environments and have participated in several projects covering different application areas such as data engineering, software/systems development, engineering systems and test infrastructure.
Individual interviews were performed. Each lasted around one hour. The interviews are recorded, transcribed and synthesized. The architects' responses are assessed on the basis of their technical knowledge and experience. The survey for this study is built on 37 questions divided into three parts: questions on the architect's experience, questions on the overall application and questions on each service/component 2. For example, when asked: "Why do you want to move from IBM's provider database service to Amazon for your application?" Architects A1, A2, A3, A4 and A6 replied: "IBM does not provide Virtual Private Cloud (VPC) for its services and this motivates us to switch to Amazon services" and architects A5 and A7 replied: "We consider VPC important but not important enough to migrate our service to another cloud service provider".
Empirical Validation of the Method To validate the approach described in Sect. 2, we conducted two case studies: SQL database services and queue services. We examine these services on the basis of actual configuration data available from service providers and benchmarking works. Due to space limitation, only the first use case (SQL database) is presented. The second use case is shared here 3. In the rest of this section, we apply our methodology to identify the key elements of the comparison and their relationships to the SQL cloud service. -Identification of Architects' Requirements: For this step, we focus on questions raised in Subsect. 2. 1. The questions are grouped according to their influence on the two NFRs reliability and availability. Regarding availability, architects raised a set of questions such as: What are the regions available for service? Are they adequate in terms of legal and regulatory requirements? Do service plans provide replication and fail over support? For reliability, the architects raised a set of questions such as: What is the frequency/severity of failures for the service? What is the mean time between failures (MTBF)? What is the mean time to repair the system (MTTR)? Note that the findings of this study are generic and can be reused for all types of services.
-Result of Identification of NFAs and their influence on NFRs: Many SQL database service solutions are available and are deployed by different cloud service providers. The interesting attributes of these services are the following: (1) The capacity attributes: the only ones that can be identified are: (i) Storage capacity in Gb; (ii) IOPS that corresponds to the underlying disk performance; (iii) The maximum simultaneous connection to the database; and (iv) The row limit. (2) The functional attributes: include the monitoring and the audit. (3) The technical attributes: are more diverse and heterogeneous and they may have different impact on NFA. As explained previously, with the exception of performance, for which additional benchmarking may be required, there are only two ways to determine the impact of NFAs on cloud services NFRs: use the expertise of the architects who deployed the service or consult the technical documentation 5,6,7.
To complete our study, we reviewed previous benchmarking work that has been done directly on cloud SQL services  [2, 6]. We collected the following attributes in addition to those we identified on the performance of cloud SQL services: (1) user connections are a resource that can limit performance; (2) replication and failover significantly reduce service performance; (3) the location of the data center influences performance; (4) number of read replicas improves performance of SQL cloud database services. -Result of selection of NFAs and NFRs from an empirical study We carried out an empirical study with the architects on cloud SQL database services. A set of responses are collected. For example, to the question: "Why did you choose the High Availability (HA) option for your service when it costs about twice as much as a regular instance? architects A1, A2, A5 and A6 replied: "The MySQL service is a business service in our application.
It should be very available. This is the critical part of the application. A failure of this service will cause the failure of the entire application. A fail over is necessary to maintain this service available when the instance is "blocked". It has happened that the primary instance is "blocked" and that a restart of the instance takes up to 30 min (supplier ticket). To avoid these downtimes, we have opted for the HA option" and architects A4, A3 and A7 replied: "A complete failure of a zone is probably very rare, so HA is an important option, but not so important as to be indispensable. " We asked several questions about the selection of cloud SQL database services and ensured that the list of NFAs previously collected was relevant to our study. Based on previous studies and the architects answers, a relevant list of NFAs and their influences on NFRs was developed (as shown in Table  2 ) to select SQL cloud services based on the application requirements of the architects.
Conclusion In this paper, we proposed a methodology for identifying key elements of comparisons and their relationship for cloud service selection. It is based on actual data available from service providers and benchmarks. First, we conducted a survey among architects to understand their requirements for selecting cloud services. Further, we identified issues that interest architects and we ranked them according to their influence on NFRs. Then, we identified the attributes of the service provider plans that answer the architects' questions and their influence on the NFRs from the technical documentation. Moreover, we reviewed the work on cloud benchmarking to collect the attributes that influence performance. Finally, we conducted an empirical study to ensure that the list of attributes we identified was complete and to add those that were missing.