BCI detection of deliberately hidden user states – or : Can we detect bluffing in a game ?

Previous studies suggest that human-machine interaction could be enhanced by providing information about the actual user's state, thus allowing for automated adaption of the system. While some user states might be inferred from the user’s behaviors, others like e.g. the perception of a selfor machine-induced error are hard to access externally. But such covert user states are reflected in physiological parameters like EEG and can be detected with a passive brain-computer interface (pBCI). Aim of this study was to expand the complexity of covert user states, which can be detected by a pBCI. An inherently covert and therefore complex state is the deliberate attempt to mislead an opponent in a game (e.g. poker) – namely bluffing. We recorded high-density EEG from 6 pairs of subjects while they were playing a bluffing dice game against each other. The game included dedicated states in which the players had to bluff or to give up. When classifying whether a player would bluff or not, we achieved a cross-validated single-trial accuracy of 81.4% (± 6.5%) over all subjects. Based on that, further investigations of the detection of covert user states show promise to enhance the performance of human-machine systems. Enhancement of HMS using knowledge about user states With the proceeding development and increasing deployment of technical systems, the technical part of those systems becomes more and more complex and sophisticated. But every technical system has a user who has to cope with it. Although much research focuses on the technical part of such human-machine systems (HMS), rather little attention has been paid to ensure the user's capability to interact with the system. User-friendly design of HMS has therefore become an important part of current research. New approaches evolve such as adaptive or interpretative HMS heading for optimal support of the user (Chen & Vertegaal, 2004; Rötting et al., 2009). One possible scenario is the adaptive reallocation of actions between operator and machine according to e.g. fatigue or attention. Or a computer could show the user the information he was looking for if it knew what it was. An application could also exit automatically in case the user notices that he opened it erroneously. The key information for the design of such context-sensitive systems is knowledge about current mental or cognitive states implicitly generated by the user like arousal, fatigue, workload, or intentions. Furthermore, emotional user states like surprise, satisfaction, or frustration provide interesting information. This could lead to a more human way of interaction between human and machine as well as to a new kind of adaptive and interpretative, and above all, to more holistic HMS (Rötting et al., 2009; Cutrell & Tan, 2008). Detection of covert user states by passive BCI How could these user states be detected? Access would be possible by questionnaires or ratings of the users – but those would suffer from subjectiveness, interruption of the user’s interaction Jessika Reissland, Christian Kothe, Sabine Jatzev, Matti Gärtner, Sebastian Welke and Thorsten O. Zander 178 with the system, and retrospectivity. Furthermore, users could deliberately hide these user states. Overt measures like behavioral data (e.g. mouse-click behavior, reaction times, or error rates) or psychophysiological measures are more objective without being influenced by the user’s memory or social desirability and provide continuous data without interruption of the interaction with the system. Using haptic data (Park et al., 2005) or eye gaze (Asteriadis et al., 2009; Rötting et al., 2009), it has been shown that providing the system with this kind of psychophysiological data leads to improved human-machine interaction (HMI). Nevertheless, although these measures might correlate to cognitive processes, they can only indirectly give insight into the actual cognitive state of the user (Müller et al., 2008). User states have a covert component, which cannot be observed from the outside. This gives the basis to define those user states of interest as covert user states, analogously to covert attention (Posner & Cohen, 1984). One psychophysiological measure, which is continuously available and can provide objective real-time information about user states, which are hardly inferable from exogenous factors, is the electroencephalogram (EEG). It has indeed been shown that covert user states like mental effort, workload, fatigue and attention can be identified in EEG (Gevins et al., 2007). Furthermore, Musha et al. (1997) extracted different EEG features for emotions. Very interesting EEG features for HMI design are error potentials (Falkenstein, 2000) or loss of control (Jatzev et al., 2008).