dataesr/scientific-paragraphs-categorization huggingface dataset:
In addition the current performance of ServOMap (described below) does not depend on any domain specific background knowledge. The work presented in this paper is an extension introduced partly in  [35]  of the approach implemented within the ServOMap system  [36, 37] , a highly configurable large scale ontology matching system able to process large ontologies associated with multilingual terminologies. ServOMap takes as input ontologies described in standards languages RDF(S)  [38] , OWL  [39] , OBO  [40]  and SKOS  [41]  and provides equivalence mappings between their entities. It relies on an Ontology Repository (OR) system, ServO  [42, 43] , a system able to manage multiple KOS while providing indexing and retrieving features. It is based on the Lucene full text search engine API  [44].
The rest of the paper is structured as follows. In Evaluation and results we give an overview of the method that we propose for the matching of large ontologies. In particular we detail the new contextual similarity computing strategy for the retrieval of candidate mappings based on the structure of the input ontologies. In Results we present the results obtained by our approach on an official dataset dedicated to evaluating ontology matching systems. We discuss the obtained results and the limitations of the approach then offer new perspectives on our work in Discussion before concluding. From now on we use the generic term ontology (formally defined in the following section) to denote any KOS, ranging from simple thesauri to very formal ontologies.
The dataset used The dataset used is the LargeBiomed dataset of the OAEI campaign. The LargeBiomed dataset is one of the official datasets which has been provided since 2012 within the context of the OAEI campaign. It is currently the most challenging task in terms of scalability and complexity. It is dedicated to the evaluation of automated large scale matching systems. The ontologies in this dataset are semantically rich and contain tens of thousands of entities. The track consists of finding alignments between the FMA containing 78,989 concepts  [31] , the SNOMED-CT containing 306,591 concepts  [7]  and the NCI Thesaurus (NCI) containing 66,724 concepts  [32]. For this evaluation, the 2009AA version of the UMLS Metathesaurus is used as the gold standard for the track reference alignments  [34].
The Metathesaurus is a very large, multi-purpose, and multi-lingual vocabulary database that contains information about biomedical and health related concepts, their various names and the relationships among them. It is built from the electronic versions of more than 160 biomedical resources including thesauri, classifications, code sets and lists of controlled terms. It is worth noting that as UMLS may contain some incoherencies and is not complete, the performance of an automated matching system could be affected when using a reference alignment from this resource. In order to measure the behaviour of the matching system according to the size of the input ontologies three matching problems are identified: the FMA-NCI matching problem, the FMA-SNOMED matching problem, and the SNOMED-NCI matching problem as indicated in Table  2 , with each problem divided into two subtasks: small and large.
Results The evaluation reported in this section for the three first versions of ServOMap was performed in a server with 16 CPUs allocating 15 GB RAM in the context of the OAEI campaign. The latest version was evaluated using a laptop (Intel Core CPU 2. 8 GHz) running under Ubuntu Linux with 6 GB RAM. The computation times are expressed in seconds. The precision, recall and Fmeasure were computed according to the formulas described in Related work. We summarise, in the following section, the performance of the different versions on the dataset described above.
Results of ServOMap_V4 Table  6  describes the results achieved by the ServO-Map_V4 system. For the FMA-NCI matching problem, the system obtained 89. 4% (2,725 mappings) and 79. 3% (3,163 mappings) respectively for the small and the large task. For the FMA-SNOMED task there was a significant increase in terms of recall compared to the previous system which led to better F-measures of 83. 4% and 78. 1% respectively for the small and the large tasks. Regarding the SNOMED-NCI matching task the F-measure was 74. 4% for the small task (13,047 mappings) and 68. 4% (15,525 mappings) for the large task. We noted that the computation times of ServOMap_V4 and ServOMap_2013 were roughly similar when the repair facility time is not taken into account. Finally, Table  7  presents the performance achieved by ServOMap_V4 on the small fragment of the input ontologies of the LargeBiomed dataset when coupled with the LogMap-Repair logical consistency facility check.
Second, the introduction of several string similarity metrics, computed for each candidate pair, as well as the new contextual similarity strategy based on ML, impacted the computation times of ServOMap. Regarding the behaviour of ServOMap_2012 and ServOMap-lt, the performance of the latter was better in terms of F-measure for all the tasks described above. This could be explained by the fact that the recall of ServOMap-lt is a step ahead due to its ability to compare the labels of the concepts simply in the different tasks. Using the tf. idf measure, completed by a Levenshtein distance-based selection of best candidates, could be sufficient for the kind of resources within the Large-Biomed dataset. This finding is in line with the results obtained by  [61]  after comparing different string   similarity metrics of ontology matching.
Therefore, it is not possible to check whether the performance is similar for other resources. In contrast, ServOMap is a generic approach which has been evaluated using standard benchmarks provided by the OAEI campaign, with various datasets. Zhou and colleagues used Natural Language Processing techniques to map a drug dictionary to RxNorm  [65]. They mapped about 6,000 terms from Partners Master Drug Dictionary and 99 of the top prescribed medications to RxNorm. The mapping was performed at two levels: term level (by performing string-based matching using specific pre-processing techniques)  [66]  and concept level (relying on routes group). The evaluation showed an F-measure of 84. 2% for the concept level mapping. For the closest task in terms of scalability, our system achieved a performance ranging from 85.
Effect of logical assessment on mapping repair According to Jiménez-Ruiz et al. [57]  the application of mapping repair techniques has a significant impact on the quality of the mappings with respect to their logical coherence. They conducted an empirical evaluation using two state-of-the-art mapping repair systems, Alcomo  [69]  and LogMap-Repair  [57]. The evaluation was conducted using the results provided by the best systems from the OAEI 2012 LargeBiomed track, including ServOMap. We then experimented by reusing the LogMap-Repair logical consistency repair facility in ServOMap_2013 and on the results provided by ServOMap_V4 with the small task of the LargeBiomed dataset. Even though our goal was to improve the performance of ServOMap, we noticed that in some cases the logical assessment is too aggressive as it discards some correct candidate mappings and negatively impacts the F-measure, while reducing the number of incoherence mappings.
Oriented and cross-lingual mappings The current version does not take into account the matching of two input ontologies described in two different languages. For instance, comparing an ontology with terms in English to an ontology with terms in German. Therefore, we plan to investigate an approach for cross-lingual ontology matching. In addition, ServO-Map can only provide equivalence mappings. The idea is to complete our matching strategy by providing users the ability to compute subsumption or disjoint relationships between entities of two input ontologies. Regarding these oriented mappings, at a large scale, an important challenge will be the evaluation of the provided mappings because of the availability of suitable reference alignments. Finally, it is worth conducting an extensive evaluation of the impact of the different parameters, the type of matching problems and the characteristics of the input ontologies on the performance of the ServOMap system.
Acknowledgements We would like to thank  Mouhamadou Ba  and  Amal Kammoun  who participated in the technical implementation of the ServOMap system. The research work in its first unrevised form was presented at SWAT4LS'2012, Paris, France. We would also like to thank the anonymous reviewers for their valuable comments. We acknowledge the support from the  French Agence Nationale de la Recherche (ANR)  and the  DGA  under the  TecSan  grant number  ANR-11-TECS-012. Received: 23 May 2013 Accepted: 12 September 2014 Published: 28 October 2014.
INA/GRM. Identification of files that have contributed to the final version of a work. Log every DAW operation of a composer during the composition of a jingle. -Ensure that the file called "Final-Mixdown" is actually the one that produced the latest audio files of the work; identify possible format changes (stereo, 8-channel, mp3); identify the intermediate versions; detect missing data and check information integrity are key features. EMI Music. Recovery and edit of past productions; Contributors listing. Test the replacement of the drum from a recording traced by Gamelan. -Accurately identify which tracks to replay; substitute an identified track to another; replay the final mix session with the replaced tracks; identify contributors of the project.
Architecture Overview The technical goal of the Gamelan research project is to create a software "metaenvironment" (also called Gamelan), in the sense that it aims at producing knowledge over production environment utilization. It integrates some music production softwares and is able to describe the production workflow, from source to final product, at an abstraction level which is higher than the data level. For this purpose, methods from several fields were combined: trace engineering for the tracking system, and knowledge modeling and engineering for ontology design and the querying and visualizing system, as shown on Fig. 1. Gamelan promotes two categories of users: users of the tracking system, who generate traces while they interact with the digital tools of the studio during the production process, and users of these traces, at the other end of the whole system.
Production Tracking System The first part of the meta-environment deals with raw traces of production, through automatic logging of user interaction events and contextual data asking. It involves the tracking system user, at data level, to collecting production traces. We define trace as the recordings of computer-mediated activity from program execution events. In a digital music production context, traces are both user interaction event logs, from applications and operating system, and production artifacts themselves, typically imported or exported files. The underlying hypothesis of Gamelan on production process tracking is that the digital realm allows to track a production activity without disturbing it too much, which is often admitted 7.
Tracking System User Interface The first development result is the production tracking system, which combines the GamelanTracker software and "gamelanized" production softwares, mainly Audacity. The development affects three layers: software and file system events tracking, based on messaging; -production file movement monitoring and back-up recording; -manual entry information collecting, via ontology-compliant user interface. This part integrates musical and sound production softwares and has its own non-invasive user interface: instead of the common pop-up windows, we designed an unobtrusive menu, accessible from a small "Track on/off" checkbox icon, as shown on Fig. 2 , top left.
Software Activity Tracking Implementation Traces are to be mobilized in contexts that are never totally predictable and these inscriptions will report a reality that has evolved by itself. That is the reason why we designed a software activity tracking that is as agnostic as possible, through raw messaging, listening and logging. The messaging part relies on an open-source standard commonly used in the computer music community: OSC (Open Sound Control 8  ) developed at UC Berkeley  [7] , which is a communication protocol for modern networking technology, with a client/server architecture (UDP and TCP). In order to produce usage data  [8, 9] , we modified open-source domain production softwares, mainly Audacity  9  , an open-source software for recording and editing sounds, written in C++. We added specific OSC messaging functions into several functions of interest, such as open, save, save as, play, stop, cut, copy, paste, etc.
This way, each time the user performs an action through a user-level function call of the production software, our modified software sends a complete OSC message, built with the following: application name, application version number, time stamp, function name, plus specific function parameters if needed. GamelanTracker, a corresponding tracking application, has been developed. It receives and logs every message broadcasted during production time from three possible sources: -"gamelanized" applications, for action logs (OSCMessages. txt) -File System, for file movement logs (FolderState. txt) -Operating System, for application change logs (CurrentApplication. txt).
This taxonomy has been performed iteratively, since it is tightly dependent on our participation in successive productions. Thus, at each new integration to the creation or the updating of a work, we flatten and question our taxonomy and term normalization, in order to verify that the semantic commitment is respected. For common features, we import standard ontologies, among which vCard 11 for standard identity information. In order to ease incremental development and testing, we divided the ontology in two parts. The first part represents the model, with classes and properties, uploaded on a dedicated server icm. ircam. fr 12. The second part contains modelcompliant data sets, with "DiMPO individuals", uploaded on an OWL server gsemantic. ircam. fr 13 (an OpenRDF Sesame repository with the OWLIM-Lite engine).
The Differential Approach. The differential approach for ontology elaboration systematically investigates the similarity and difference relations between each concept, its parent concept and its sibling concepts. So, while developing this structure, we tried to respect a strong ontological commitment by applying a semantic normalization, i. e. by asking, for each concept c, the four differential questions of Table  1. To carry out this semantic normalization task from a practical point of view, we used DOE 14  [14]  and Protégé 15 softwares, for both taxonomy building, refining and exporting (RDFS, OWL, etc. ). Fig. 4  shows an excerpt from DiMPO.
A Sesame OpenRDF semantic repository has been installed from an Ontotext Owlim-Lite version. It handles structured data storage and management, reasoning and querying. Data Storage and Management. OWL/RDF data ingestion on the semantic repository is triggered by a short Java program integrated into GamelanTracker and using Sesame API. An online graphical interface allows repositories management at http://gsemantic. ircam. fr. Reasoning. The OWLIM inference engine performs completion of facts through "total materialization" at load time. This reasoning strategy slows down upload but speeds up retrieval and querying, which is what we have chosen. Querying. The semantic repository embeds a query engine accessible through HTTP. We use the SPARQL 18  language to write RDF queries against the semantic repository, with triple patterns syntax.
Production Patterns When DiMPO ontology reached a decent and stabilized level, we entered a second phase of the ontological research: production patterns design. Production patterns define audio creation acts, such as editing, shown on Fig. 11  (in UML). The use of these patterns allows to represent a set of actions with a musical meaning, incorporating the vocabulary developed in the ontology. Our query patterns are grounded on these production patterns. Reuse of ontology vocabulary in production patterns eases their translation into query patterns, especially when using RDF-compliant query languages such as SPARQL, as we do onto our Sesame repository containing OWL individuals. Here, knowledge can be viewed as bilocalized: on the semantic repository side for interrelated individuals of the semantic trace database, and on the query manager side for the formalized relations of the query patterns base.
Acknowledgments. The  Gamelan  project is funded by  French National Research Agency. It started in 2009 and will end in 2013. The partners are: - IRCAM (Institut de Recherche et Coordination Acoustique/ Musique), -Heudiasyc -UMR 7253 CNRS UTC, -INA (Institut National de l'Audiovisuel ), -EMI Music France.
It is also responsible of getting the runId and propagate it to all the application nodes. The second component is the Node Agent which is responsible for the collection, upload, backup and cleanup in each application node. On each agent, its control logic and Helper functionality (i. e. the calculation of the thresholds in the Node Agent), are independent of the target expert system and load testing tool. On the contrary, the logic that interfaces with the tools needs to be customized. To minimize the code changes, this logic is encapsulated in Wrapper packages which are only accessed through their interfaces. This scenario is presented in Figure  4  which shows the structure of the package Expert System Wrapper. It contains a main interface IExpertSystem to expose all required actions and an abstract class for all the common functionality.
EXPERIMENTAL EVALUATION Two experiments were performed. The first one evaluated the overhead introduced by our proposed approach. Meanwhile, the second one assessed the productivity benefits that a tester can gain by using the proposed approach. Firstly, the overhead was measured in a single-node environment using three combinations of WAIT: The applications alone to get a baseline (BL), the applications with manual WAIT data collection, and the applications with the automated WAIT (AWAIT). For each combination using WAIT, the Sampling Interval (SI) was configured to 480 seconds (a commonly used value) and 30 seconds (minimum value recommended for WAIT). The remaining test configurations were set to reflect real-world conditions: A workload of 2,000 concurrent users; a duration of 1 hour; a HD Threshold of 100MB; and a Time Threshold of 10 minutes.
Then AWAIT monitored the application to assess how well it was able to identify the injected bugs and estimate the corresponding time savings in performance analysis. The set-up was identical to the multi-node test previously described except the duration which was one hour. The 1st ranked issue was not an injected bug but an issue related to the clustering set-up of Tomcat. The 2nd ranked issue was the lock contention. It is worth noting that both issues were detected since the early versions of the incremental report from the automated tool. Their high frequency (above 96% of the samples) could have led a tester to pass this information to the development team so that the diagnosis could start far ahead of the test completion. The final report reinforced the presence of these issues. After identifying an issue, a tester can see more details, including the type of problem, involved class, method and source code line.
RELATED WORK Performance testing. Multiple research works have focused on improving its involved processes. For example, the authors of  [14]  introduced a cloud-based testing harness which uses an execution engine to parallelise the run of test scripts on large shared-clusters of nodes. Meanwhile, the work on  [4]  presents an approach to automate the deployment of the major open-source IaaS cloud kits (such as OpenNebula 11  ) on Grid 5000 environments. Other efforts have concentrated on evaluating the benefits of cloud testing in practice. For example, the work on  [15]  presents an industrial study case of the new methodologies and tools that have developed to streamline the testing of their cloudbased applications. Finally, other works have focused on automation. However, most of this research has focused on automating the generation of load test suites  [1, 13].
Substantial research on the use of search engines as a tool for L2 learners has been carried out suggesting that the web plays an important role in fostering language awareness and learner autonomy (e. g. Shei 2008a(e. g. Shei  , 2008b;; Conroy 2010). According to Bathia and Richie (2009: 547), "the application of Google for language learning has just begun to be tapped. " Within the framework of this study it was assumed that the students, conversant with digital technologies and using Google and the web on a regular basis, could use various search options and the search results to self-correct their errors instead of relying on their teacher to provide direct feedback. After receiving some in-class training on how to formulate Google queries, the students were asked to use a customized Google search engine limiting searches to 28 information websites to correct up to ten "untreatable" errors occurring in two essays completed in class.
Google-driven language learning According to  Rundell (2000: n. pag. ) , the web "is not a corpus at all according to any standard definitions: what it is is a huge rag-bag of digital text, whose context and balance are largely unknown. " Berg (2005: 2), for his part, argues that "the Web turns out to be a somewhat intractable collection of textual material, […] a rather haphazard accumulation of digital text. " The acronym GALL (Google-assisted language learning) was first coined by  Chinnery (2005)  who described Google as an informative, productive, collaborative, communicative, and aggregative tool with lots of pedagogical uses. Substantial research on Google as a tool for second-language learners has since then been carried out (e. g. Guo & Zhang 2007; Milton 2006; Shei 2008a Shei , 2008b;; Wu, Franken, & Witten 2009)  suggesting that it plays an important role in fostering language awareness and learner autonomy.
According to  Bathia and Richie (2009: 547) , "the application of Google for language learning has just begun to be tapped. " A number of studies, however, point to problems associated with the use of Google and the web for language learning, namely the abundance of potentially unreliable data and the daunting task of scouring huge amounts of language  (Berg 2005; Kilgarriff 2001; Renouf 2003; Fletcher 2004; Robb 2003a Robb , 2003b;; Rundell 2000). Robb (2003a)  calls it "a quick ʻn dirty corpus tool," he warns about its use in class (2003b), explaining that queries are limited to specific words only, that there is no way of assessing the reliability of the language featured in the search results, and that these are not presented in a user-friendly format. Several attempts at harnessing and systematizing web output have been made though.