DOCTORAL ABSTRACT

Engaged or Frustrated? Disambiguating
engagement and frustration in search
Ashlee Edwards
ashlee.a.edwards@gmail.com
Abstract
One of the primary ways researchers have characterized engagement is by an increase
in search actions. Another possibility is that instead of experiencing increased engagement,people who click and query frequently are actually frustrated; several studies have shownthat frustration is also characterized by increases in clicking and querying behaviors. This
research seeks to illuminate the differences in search behavior between participants who areengaged and frustrated, as well as investigate the effect of task interest on engagement andfrustration. To accomplish this, a laboratory experiment was conducted with 40 participants.
Participants completed four tasks, and responded to questionnaires that measured theirengagement, frustration, and stress. Participants were asked to rank eight topics based oninterest, and were given their two most interesting and two least interesting tasks. Poor
search result quality was introduced to induce frustration during their most interesting andleast interesting tasks.
This study found that physiological signals hold some promise for disambiguating engagement and frustration, but this depends on the time frame and manner in which they areexamined. Frustrated participants had significantly more skin conductance responses during
the task, while engaged participants had greater increases in skin conductance during the first
60 seconds of the task. Significant main and interaction effects for interest and frustration
were found for heart rate in the window analysis, indicating that heart rate fluctuations overtime can be most effective in distinguishing engagement from frustration. The multilevel
modeling of engagement and frustration confirmed this, showing that interest contributed
significantly to the model of skin conductance, while frustration contributed significantly tothe model of heart rate.
This study also found that interest had a significant effect on engagement, while thefrustrator effectively created frustration. Frustration also had a significant effect on selfreported stress. Participants exhibited increases in search actions such as clicks and scrolls
during periods of both engagement and frustration, but a regression analyses showed thatscrolls, clicks on documents, and SERP clicks were most predictive of a frustrating episode.
A significant main effect for interest was found for time between queries, indicating that
this could be a useful signal of engagement. A model including the physiological signals andsearch behaviors showed that physiological signals aided in the prediction of engagement andfrustration.
Findings of this research have provided insight into the utility of physiological signals in
distinguishing emotional states as well as provided evidence about the relationship among

ACM SIGIR Forum88

Vol. 50 No. 1 June 2016search actions, engagement and frustration. These findings have also increased our understanding of the role emotions play in search behavior and how information about a searcher’semotional state can be used to improve the search experience.

Acknowledgements
I’d like to acknowledge the help and guidance I received from my dissertation committee: Diane
Kelly (chair), Ioannis Arapakis, Jaime Arguello, Heather O’Brien, and Robert Capra.
Available online at: https://aedwards.squarespace.com/s/Edwards-Dissertation.pdf

ACM SIGIR Forum89

Vol. 50 No. 1 June 2016