Perspectives on Evaluating Diverse Open Web Search Applications with TIREx
Channel: IIIA Hub
Date: 2024-03-22

Perspectives on Evaluating Diverse Open Web Search Applications with TIREx
Maik Fröbe, Friedrich-Schiller-University Jena, Germany

Talk at the Search Engines course, A.Y. 2023/2024 (https://iiia.dei.unipd.it/education/search-engines/)
MD in Computer Engineering (https://degrees.dei.unipd.it/master-degrees/computer-engineering/) 
MD in Data Science (https://datascience.math.unipd.it/)
University of Padua, Italy

----------------------------------------------------------------------
Perspectives on Evaluating Diverse Open Web Search Applications with TIREx

Abstract: 
The OpenWebSearch.eu project (https://openwebsearch.eu/) aims to enable a diverse and vibrant search engine ecosystem in Europe. Given that multiple players with potentially divergent goals collaborate in this ecosystem, evaluation plays a central role in ensuring that many diverse use cases can be implemented. The core components of the OpenWebSearch.eu project are evaluated with TIRA/TIREx with more than 30 available IR test collections. TIREx simplifies the construction and reproducible evaluation of search engines. The talk will cover a hands-on session on Docker, ir_datasets, PyTerrier, and TIRA so that students can submit their solutions for LongEval 2024 via TIREx.

Bio: 
Maik Fröbe is a PhD student at the Webis group with research interests in information retrieval. He has co-organized the Touché shared task since 2020 and the SCAI shared task since 2021 and was the main organizer of the clickbait spoiling task at SemEval-2023. He is an active developer of TIRA/TIREx, which improved the reproducibility of a number of shared tasks and has an archive of more than 500 research prototypes. Maik participates in the OpenWebSearch.eu project, focusing on the reproducible and continuous evaluation of the software artifacts underlying the Open Search Foundation.