IR Scientific Data: How to Semantically Represent and Enrich Them
Authors: Toine Bogers, Georgeta Bordea, Paul Buitelaar, Nicola Ferro, Gianmaria Silvello (Aalborg University Copenhagen, National University of Ireland, Università di Padova, 2016)
Topic: Lexical and Semantic Resources and Analysis

Abstract
Experimental evaluation carried out in international large-scale campaigns is a fundamental pillar of the scientific and technological advancement of IR systems. Such evaluation activities produce a large quantity of scientific and experimental data, which are the foundation for all the subsequent scientific production and development of new systems. We discuss how to annotate and interlink this data, by proposing a method for exposing experimental data as LOD on the Web and as a basis for enriching and automatically connecting this data with expertise topics and expert profiles. In this context, a topic-centric approach for expert search is proposed, addressing the extraction of expertise topics, their semantic grounding with the LOD cloud, and their connection to IR experimental data.

Introduction
The importance of research data is widely recognized across all scientific fields as this data constitutes a fundamental building block of science. Recently, a great deal of attention was dedicated to the nature of research data AUTHOR and how to describe, share, cite, and re-use them in order to enable reproducibility in science and to ease the creation of advanced services based on them AUTHOR. Nevertheless, in the field of IR, where experimental evaluation based on shared data collections and experiments has always been central to the advancement of the field AUTHOR, the LOD paradigm has not been adopted yet and no models or common ontologies for data sharing have been proposed. So despite the importance of data to IR, the field does not share any clear ways of exposing, enriching, and re-using experimental data as LOD with the research community. Therefore, the main contributions of this paper are to: \item define an RDF model of the scientific IR data with the aim of enhancing their discoverability and easing their connections with the scientific production related to and based on them; \item provide a methodology for automatically enriching the data by exploiting relevant external entities from the LOD cloud. Experimental Data}

Conclusion
In this paper we discussed the data modelling and the semantic enrichment of IR experimental data, as produced by large-scale evaluation campaigns. In particular, the main results of the paper are: \item an accurate RDF data model for describing IR experimental data in detail, available at ; \item a dataset about CLEF contributions, extracted expertise topics and related expert profiles; \item the online accessible LOD DIRECT system, available at , to access the above data in different serialization formats, RDF+XML, Turtle, N3, XML and JSON. Future work will concern the application of these semantic modeling and automatic enrichment techniques to other areas of the evaluation workflow. For example, expert profiling and topic extraction could be used to automatically improve and enhance the descriptions of the single experiments submitted to an evaluation campaign, which are typically not very rich and often cryptic---for example ``second iteration with tuned parameters'' as description---and to automatically link experiments to external resources, e.g., describing the used components, such as stemmers or stop lists, and systems. Finally, the RDF model defined within DIRECT opens up the possibility of integrating established DL methodologies for data access and management which increasingly exploit the LOD paradigm AUTHOR. This would enable broadening the scope and the connections between IR evaluation and other related fields, providing new paths for semantic enrichment of the experimental data.