Neural features of video topical relevance

51 For over 50 years, Information Retrieval Systems (IRSs) have been helping users to find relevant information. However, IRSs often retrieve non-relevant information causing user dissatisfaction. Relevance therefore has become a key concept for IRS building and evaluation. Research in this area used to be centred in a system view of relevance, but since it has become widely accepted that relevance is a human perception (Saracevic, 1996), the research on relevance has moved to a user view of relevance (Mizzaro, 1997), focusing on relevance in the world of the user such as the way users assess relevance depending on the problem at hand, context and criteria. However, as any other human perception, relevance is an elusive one (Saracevic, 1996). Researchers therefore continue to investigate the human nature of relevance. For instance, advances in Neuroimaging have allowed researchers to look at relevance from a neuroscience perspective. Moshfeghi et al. (2013) have conducted a seminal fMRI study to investigate the neural correlations of relevance. They used a block design to find the brain regions that activate during the task of image topical relevance assessments. Their results have shown three regions (i.e. inferior parietal lobe, inferior temporal gyrus and superior frontal gyrus) where activity was significantly greater during the assessment of the relevant images. These three regions were located in the right hemisphere of the brain. However, it has been suggested that if text was used instead of images, brain activations could possibly be located in the left hemisphere of the brain. Subsequently, Gwizdka (2013) have used event-related fMRI to investigate the brain activity during assessments of relevance of individual words to news articles. Based on preliminary results, it has been suggested that different levels of brain activity could be related to different levels of relevance. However, it is unclear where such activations occurred. Despite the importance of prior studies, more research is needed in order to deeply understand the neural bases of relevance. In addition, fMRI studies design and stimuli choices can have significant impact on the measured fMRI signals (Amaro et al., 2006). Therefore, our idea is to use video stimuli in a stochastic rapid-event related fMRI study to find regions where activity patterns relate to video relevance. We designed our study carefully to allow a more realistic scenario without losing statistical power (Dale, 1999). In addition, we aim to investigate the extent to which it is possible to predict the relevance of individual stimulus from brain features. In contrast, prior studies have looked at the average response to the whole set of stimulus. The following are our two major research questions. (RQ1) What are the brain mechanisms of video relevance judgement? (RQ2) Can we use brain features to predict relevance? To answer our RQs, we performed a controlled user study where 24 subjects judged the relevance of videos during 3 predefined search tasks that were selected from the TREC 2002 Video Track (Topic IDs: 79, 80, 82). The experiment, which followed a within-subjects design, has one independent variable that is the relevance or nonrelevance of videos. It has also two dependent variables: (1) the level of activation when answering RQ1; and (2) the video relevance prediction accuracy when answering RQ2. A schematic representation of the optimised design is provided in Figure 1.