  Relevance plays a central role in information retrieval (IR), which has
received extensive studies starting from the 20th century. The definition and
the modeling of relevance has always been critical challenges in both
information science and computer science research areas. Along with the debate
and exploration on relevance, IR has already become a core task in many
real-world applications, such as Web search engines, question answering
systems, conversational bots, and so on. While relevance acts as a unified
concept in all these retrieval tasks, the inherent definitions are quite
different due to the heterogeneity of these tasks. This raises a question to
us: Do these different forms of relevance really lead to different modeling
focuses? To answer this question, in this work, we conduct an empirical study
on relevance modeling in three representative IR tasks, i.e., document
retrieval, answer retrieval, and response retrieval. Specifically, we attempt
to study the following two questions: 1) Does relevance modeling in these tasks
really show differences in terms of natural language understanding (NLU)? We
employ 16 linguistic tasks to probe a unified retrieval model over these three
retrieval tasks to answer this question. 2) If there do exist differences, how
can we leverage the findings to enhance the relevance modeling? We proposed
three intervention methods to investigate how to leverage different modeling
focuses of relevance to improve these IR tasks. We believe the way we study the
problem as well as our findings would be beneficial to the IR community.
