INEX REPORT

Relevance Feedback at INEX 2005
Carolyn J. Crouchcontact details
ccrouch@d.umn.edu

Relevance feedback in the INEX environment is addressed by several papers in the proceedings ofthe INEX 2005 Workshop [1]. The most extensive discussion is provided by Schenkel and Theobald[2]. Mihajlovic, et. al. [3], and Sauvagnat, et. al. [4], describe approaches to relevance feedback in
specific sections of papers devoted to discussions of multiple INEX tasks. For detailed descriptions of
the systems and methods used and results obtained in each case, see the papers cited.
Schenkel and Theobald [2] describe two approaches to what they refer to as structural feedback,which focuses on the incorporation of structural aspects in the feedback process. Their first approachreranks results returned by an initial, keyword-based query using structural features derived fromresults with known relevance. Their second approach involves expanding traditional keyword queries
into content-and-structure queries. Official results, evaluated using the INEX 2005 assessment
method based on rank-freezing, show that reranking outperforms the query expansion method on thisdata.
Mihajlovic, et. al. [3], also investigate the potential use of structural information from relevant
elements for feedback purposes. The structural information used in this case is twofold: the title ofthe journal containing the element and the element type. Two experiments are described. One
analyzes the effects of assigning different weights to the structural information found in the top 20.
The second seeks to determine which of the two types of structural information is more useful in thiscontext. Official results show little gain in either case for 2005 data.
Sauvagnat, et. al.[4], describe their experiments in relevance feedback as follows. Their “structureoriented” approach first seeks to identify the generic structure shared by the largest number of
relevant elements and then use this information to modify the query. A second method, termed“content-oriented”, utilizes terms from relevant elements for feedback. A third method involves a
combination of the two. Official results show improvement in some case but are not consistent acrossquery types.
Interested readers may also find descriptions of relevance feedback experiments in other papers (i.e.,papers not focusing on relevance feedback per se) in [1]; those reported here were submitted by theirauthors as part of the relevance feedback track at INEX 2005. As is noted by many researchers, this is
only the second year for relevance feedback research at INEX, and the work is still in the early stagesof investigation. We look forward to interesting research in this area as it evolves over time.

ACM SIGIR Forum58

Vol. 40 No. 1 June 2006

References[1] Norbert Fuhr, Mounia Lalmas, Saadia Malik and Gabriella Kazai (eds): Advances in XML
Information Retrieval and Evaluation: Fourth Workshop of the Initiative for the Evaluation of XMLRetrieval (INEX 2005), Dagstuhl 28-30 November 2005, Lecture Notes in Computer Science, Vol3977, Springer-Verlag, 2006.
[2] Schenkel, R., and M. Theobald. Relevance Feedback for Structural Query Expansion, in: Norbert
Fuhr, Mounia Lalmas, Saadia Malik and Gabriella Kazai (eds): Advances in XML Information
Retrieval and Evaluation: Fourth Workshop of the Initiative for the Evaluation of XML Retrieval(INEX 2005), Dagstuhl 28-30 November 2005, Lecture Notes in Computer Science, Vol 3977,Springer-Verlag, 2006. [to appear][3] Mihajlovic, V., Ramirez, G., Westerveld, T., Hiemstra, D., Blok, H., and A. de Vries, TIJAH
Scratches INEX 2005: Vague Element Sections, Image Search, Overlap, and Relevance Feedback, in:Norbert Fuhr, Mounia Lalmas, Saadia Malik and Gabriella Kazai (eds): Advances in XML
Information Retrieval and Evaluation: Fourth Workshop of the Initiative for the Evaluation of XMLRetrieval (INEX 2005), Dagstuhl 28-30 November 2005, Lecture Notes in Computer Science, Vol3977, Springer-Verlag, 2006. [to appear][4] Sauvagnat, K., Hlaoua, L., and M. Boughanem, XFIRM at INEX 2005: Ad-hoc and RelevanceFeedback Tracks, in: Norbert Fuhr, Mounia Lalmas, Saadia Malik and Gabriella Kazai (eds):
Advances in XML Information Retrieval and Evaluation: Fourth Workshop of the Initiative for theEvaluation of XML Retrieval (INEX 2005), Dagstuhl 28-30 November 2005, Lecture Notes inComputer Science, Vol 3977, Springer-Verlag, 2006. [to appear]

ACM SIGIR Forum59

Vol. 40 No. 1 June 2006