# Measuring Immersion and Affect in a Brain-Computer Interface Game

Gido Hakvoort, Hayrettin Gürkök, Danny Plass-Oude Bos, Michel Obbink, Mannes Poel

► To cite this version:

Gido Hakvoort, Hayrettin Gürkök, Danny Plass-Oude Bos, Michel Obbink, Mannes Poel. Measuring Immersion and Affect in a Brain-Computer Interface Game. 13th International Conference on Human-Computer Interaction (INTERACT), Sep 2011, Lisbon, Portugal. pp.115-128, 10.1007/978-3-642-23774-4_12. [https://doi.org/10.1007/978-3-642-23774-4_12](https://doi.org/10.1007/978-3-642-23774-4_12)

HAL Id: hal-01590583

https://inria.hal.science/hal-01590583

Submitted on 19 Sep 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Measuring immersion and affect in a brain-computer interface game

Gido Hakvoort, Hayrettin Gürkök, Danny Plass-Oude Bos, Michel Obbink, and
Mannes Poel

University of Twente, Faculty EEMCS,
P.O. Box 217, 7500 AE, Enschede,
The Netherlands

gido.hakvoort@gmail.com, h.gurkok@utwente.nl, d.oudebos@cs.utwente.nl,
mobbink@gmail.com, m-poel@utwente.nl

**Abstract.** Brain-computer interfaces (BCIs) have widely been used in medical applications, to facilitate making selections. However, whether they are suitable for recreational applications is unclear as they have rarely been evaluated for user experience. As the scope of the BCI applications is expanding from medical to recreational use, the expectations of BCIs are also changing. Although the performance of BCIs is still important, finding suitable BCI modalities and investigating their influence on user experience demand more and more attention. In this study a BCI selection method and a comparable non-BCI selection method were integrated into a computer game to evaluate user experience in terms of immersion and affect. An experiment with seventeen participants showed that the BCI selection method was more immersive and positively affective than the non-BCI selection method. Participants also seemed to be more indulgent towards the BCI selection method.

**Keywords:** Brain-computer interfaces, affective computing, immersion, games

## 1 Introduction

A brain-computer interface (BCI) can be described as a communication link between the brain and the machine. In a BCI system, signals from the brain are analyzed to determine the user's state of mind or intentions, which in turn can be translated into actions [9]. BCI systems have been applied for medical use to help disabled users by giving back mobility [8] and breaking the isolation of people with physiological disorders [7,11].

As successful applications of BCIs become news and commercial BCI input devices become publicly available, BCIs are finding their way into recreational use. However, as the scope of BCI applications is expanding from medical to recreational use, the expectations for BCIs are also changing. Currently they are unable to meet the high performance and accuracy of existing input modalities such as mouse and keyboard, and are therefore unfit as replacement. Instead they should be seen as separate modalities which can be used beside, or in combination with, existing input
modalities [17]. However, using BCI as input modality still comes with many challenges. Where increasing the performance of BCIs has always been an important goal for medical studies, the way they are applied as modalities and the influence they have on the user experience is becoming more and more important for recreational BCI applications.

Whether BCIs are suitable for recreational applications is as yet unclear as they have rarely been evaluated for user experience. They may turn out to be valuable additions for recreational applications such as games which are developed to be challenging and enjoyable. Moreover the inaccuracy of BCIs can become a challenging factor in games. As gamers love working with new technologies, are capable of adapting quickly to a new environment and are used to the concept that games have to be mastered [16], they may be more indulgent towards BCI modalities.

The purpose of this study was to evaluate a BCI system for user experience in the terms of affect and immersion. As making selections is an important aspect in many games and BCIs are frequently used to make selections, a BCI selection method was used in this study. One of the most frequently used brain signals in BCI systems to make selections is the steady-state visually evoked potential (SSVEP) [1]. In most cases SSVEPs are triggered by presenting a modulated visual stimulus with a periodic signal, usually at frequencies above 5 Hz, to a user. The periodic signal of the stimulus can then be traced back in the measured brain signals which are mostly recorded from the occipital region of the scalp [23]. The power of an SSVEP only covers a narrow bandwidth matching that of the stimulus [15]. This makes them relatively easy to detect, which is why the BCI selection method used in this study was based on SSVEP. The BCI selection method and a comparable non-BCI selection method were integrated into a computer game where both introduced a challenge factor. As the BCI selection method would be able to directly translate the user intentions into in-game actions, it was expected that it would enrich the user experience in terms of immersion and affect.

How immersion and affect can be influenced by input modalities will be explained in the background in section 2. The BCI and non-BCI selection methods and how they were integrated in a game will be described in section 3. In section 4 the experimental setup and how both selection methods were evaluated will be explained. After this, the results of the experiment will be reported, followed by the discussion and the conclusion.

# 2 Background

## 2.1 Immersion

Immersion has meaning in various contexts, such as while reading a book, watching a movie or playing games. Whether the term is used consistently in these contexts is unclear. However, for playing games there seems to be a shared concept of immersion among gamers [4]. Immersion in games is often accompanied by high levels of concentration, losing track of time and empathy.
In a study by Brown *et al.* [4] an attempt was made to define immersion within games. In their study they examined the concept of immersion experienced by gamers. Their results indicate that immersion is not just a static experience, but has different levels of involvement. They defined three levels of involvement: engagement, engrossment and total immersion. They also state that to reach a certain level of immersion, a number of barriers must be crossed. Some of these barriers are related to human characteristics, such as personal preferences, empathy and the will to invest time and attention. Others are related to the construction of the game such as the graphics, a plot and atmosphere. However, to reach a state of total immersion an important barrier to take is related to the controls. Through them gamers translate their intentions into in-game actions [19] and virtually controls should become invisible to the gamer. As intentions originate from the brain, users first need to translate their intentions into real world actions to handle the controls. Even if these real world actions become virtually invisible for the user, they still need to be performed. Using a BCI to detect intentions may allow them to be translated directly into in-game actions, making the real world actions redundant.

In a study by Jennett et al. [12] immersion in games was further investigated. They identified five factors of immersion: cognitive involvement, real world dissociation, emotional involvement, challenge and control. As in the study of Brown et al., some factors were related to human characteristics (cognitive involvement, real world dissociation and emotional involvement) and others were related to the construction of the game (challenge and control). To measure these factors, as well as the total immersion, they developed a questionnaire which was also used in this study to measure the immersion while using the BCI and non-BCI selection methods.

## 2.2 Affect

Affect can be referred to as experiencing emotions and has some overlap with immersion [12]. It has a large impact on how well users are able to perform tasks and how they respond to possible usability problems. According to Norman [18], a more positive affect causes users to be more indulgent towards minor usability problems. Although there are many dimensions associated with affect, according to Picard [20] the three most commonly used dimensions of emotion are valence, arousal and dominance. Picard also notes that the valence and arousal dimensions are critical in recreational applications.

Bradley et al. [3] developed a questionnaire, the self-assessment manikin (SAM), to measure emotional responses in these three dimensions. In Fig. 1 some emotions associated with valence and arousal can be seen. Integrating the selection methods into an enjoyable, challenging and task oriented environment, such as a game, should result in a more positive affect in terms of valence and arousal which will aid users to overcome the inaccuracy of the selection methods.
<figure><img src="image_2.png" /><figcaption>Fig. 1. Emotions in the valence and arousal space</figcaption></figure>

## 3 Selection Methods

To measure the effects of the BCI and non-BCI selection methods on the user experience in terms of immersion and affect, the selection methods were integrated into a simple game. The game, called Mind the Sheep!, offered the players an enjoyable, challenging and task oriented environment.

### 3.1 Mind the Sheep!

Mind the Sheep! consists of a playground representing a meadow on which a few obstacles, such as fences, gates or trees are placed. There are three dogs in the playground which are controlled by the player. Depending on the objectives of a specific level, the playground can be populated with a number of sheep or collectibles. A screenshot of the game can be seen in Fig. 2 with ten white sheep and three black dogs. The goal of the game is to gather all sheep into a pen or gather all collectibles as quickly as possible using the three dogs. Players can use one of the selection methods to select one of the three dogs.

To move a dog, players point at any location on the playground with the mouse and start the current selection method by pressing the left mouse button. As long as they hold down the mouse button the selection method continues to be operative thereby increasing the accuracy of the selection method. Releasing the mouse button stops the selection method and one of the dogs is moved to the location indicated by the player. When players indicate a position unreachable for the dogs, the instruction is ignored. As the accuracy of the selection methods increase over time, there is a trade-off between accuracy and speed. It also requires users to multi-task as they need to concentrate on a dog and keep an eye on the sheep at the same time.
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Screenshot of Mind the Sheep! with three black dogs and ten white sheep. The pen is located in the center.</figcaption></figure>

The dogs use a simple A* search based path finding algorithm to move to a specific location on the map. By default, sheep will walk around and graze randomly. However, when a dog approaches, they will tend to flock and move away from the dog allowing them to be herded in a desired direction. The flocking behavior is introduced by using the boids algorithm [22]. By positioning the dogs at strategic locations on the map a flock of sheep can be directed into the pen.

## 3.2 BCI Selection

To use SSVEPs for making selections, stimuli emitting unique periodic signals should be presented to a user simultaneously. When a user focuses on one of the stimuli, the periodic signal of that stimulus can be traced back in the user's brain signals. As each stimulus represents a single target, the corresponding target can then be determined as the selected one. As users would not want to continuously make selections, they were able to start and stop this selection method thereby controlling the presence of the stimuli. The size of the stimuli and the frequency of their periodic signals have an influence on how well they are reflected in user's brain signals. Therefore, these properties were determined in a pre-experiment study.

In the pre-experiment study a small white cross was placed at the center of an LCD monitor on a black background (Fig. 3). During the study participants were exposed to different types of trials, in which the presented stimulus varied in size and frequency. The detailed setup of the pre-experiment study was described in [10].
<figure><img src="image_4.png" /><figcaption>Fig. 3. Screen layout of the monitor during the pre-experiment study.</figcaption></figure>

Based on the work of Volosyak *et al.* [24], 7 different frequencies (6 Hz, 6.67 Hz, 7.50 Hz, 8.57 Hz, 10.00 Hz, 12.00 Hz and 15.00 Hz) were used. Their study showed that frequencies which are integer factors of the refresh rate of an LCD monitor are more suitable for presenting stimuli. Using the factor of the refresh rate produces a more stable stimulus frequency because only whole frames are visible on the LCD screen. The diameters of the stimuli were set to 2 and 3 centimeters, which is consistent with the work in the literature [5,14].

In each trial a simple stimulus, a blinking white circle, appeared at the location of the cross. The participant focussed for 4 seconds on the stimulus. Between trials, participants had 6 seconds rest to relax their vision. All trials were presented 25 times and were placed in a random order prior to the study, thus for each participant, 25 segments of 4 seconds of data were recorded for each different trial.

The results of the pre-experiment study with 7 participants indicated that a good set of frequencies is 10 Hz, 12 Hz and 15 Hz with a diameter of 3 centimeters. Without any training, the frequencies in this set were classified correctly with an average recall of 78.1 % ($\sigma$ = 18.5) using a CCA-based detection method which will be further explained in section 4.2.

## 3.3 Non-BCI Selection

The BCI selection method introduces a challenge factor when a selection is being made as users need to concentrate on a stimulus to make an accurate selection. The main challenge is to gather enough data to be able to make a good detection, which is directly related to the time the selection method is operative. Although it would be much easier for users to make a selection by pointing and clicking with a mouse, this would offer little challenge and give the non-BCI selection method a great advantage when compared with the BCI selection method. To pose a similar challenge in the non-BCI selection method, it was modelled to be similar to the BCI selection method in terms of the time required for an accurate selection.

Similar to the BCI selection method users were able to start and stop the non-BCI selection method. When operative, the dogs were highlighted one at a time with an
increasing highlight period. The diameter of the stimuli were kept the same as in the BCI selection method, 3 centimeters. When users stopped the selection method, the current highlighted dog was selected.

To make an accurate selection, users had to react in time when the dog they wanted to select was highlighted. With a highlight period of 250 ms users should have enough time to react as this is close to the average human reaction time [13]. The highlight period should reach 250 ms only by the time that is equal to the time needed to make a selection using the BCI selection method, which is around 2.5 seconds [14]. Therefore, the highlight period started with 100 ms and was increased 5% after every highlight, with a maximum highlight period of 500 ms.

# 4 Methods

## 4.1 Experimental Setup

The experiment held in this study consisted of two different sessions. In each session a participant used one of the selection methods (BCI or non-BCI) while playing Mind the Sheep!. Each participant used both selection methods. However, a counterbalanced measures design was used to avoid confounding variables such as learning a strategy to play the game.

Each session was divided into three trials, a familiarity trial, an easy trial and a difficult trial. In the familiarity trial participants could get used to the selection method by selecting and moving the dogs. During this trial, participants had to collect 10 objects which were placed across the playground. Next, in the easy trial, participants had to pen a small flock of 5 sheep using the dogs. During this trial two pens were placed on the playground, one on the left and one on the right of the screen to make the task easier for the participants. Finally, in the difficult trial, participants had to gather 10 sheep, which were more scattered across the playground, into one pen that was placed in the center of the playground. Between the two sessions, participants were given a break of ten minutes.

The layout of the playgrounds across the trials were kept the same to ensure no playground was more difficult for one of the selection methods. However to ensure that participants did not create an optimal strategy for a specific trial, the positions of the dogs, collectible objects and sheep were altered for the different selection methods.

A timeout was set for each trial for the participants to finish the level by collecting all objects or gathering all sheep into a pen. Participants had 3 minutes, 5 minutes and 10 minutes for the familiarity, easy and difficult trials respectively. Since immersion in games is often accompanied by losing track of time, the time left was not visible for the participants. Otherwise it could have influenced their perception of the elapsed time.

The game ran on a PC which was connected to a projector<fnref n="1" />. The projector was mounted on the ceiling and projected the game on a screen approximately 3 meters away from a participant. The sizes of the stimuli were scaled proportionally with the

<footnote n="1">¹ Mitsubishi WD510U, 93.3", 60Hz, 1360x768</footnote>
increased distance to the screen. To make sure the frequencies for the SSVEP based BCI selection method were correctly presented by the projector they were checked with a light sensor.

Participants sat on a chair behind a table and by using the mouse on the table they were able to start and stop the selection methods. The data acquisition ran on a separate PC and sent the raw electroencephalography (EEG) data to the game PC.

## 4.2 SSVEP Detection

Prior to the experiment, 32 electrodes were placed according to the international 10-20 system [21]. For the EEG data acquisition a BioSemi ActiveTwo system<fnref n="2" /> was used. During each SSVEP selection EEG data was used from eight parietal-occipital electrodes (i.e. Pz, P3, P4, PO3, PO4, Oz, O1, O2). The EEG data was re-referenced to the common average reference (CAR) [6] of all 32 electrodes, after which a detection method was used to determine which frequency the participant was focusing on.

There are various methods to detect the presence of SSVEP. One of the most popular and widely used detection methods is power spectral density analysis (PSDA) where a fast Fourier transform (FFT) is used to estimate the power spectral density (PSD) of a time window of the EEG signal. The magnitude of each stimulation frequency can then be used for classification. A relatively new approach is using canonical correlation analysis (CCA) where sets of reference signals are constructed for each one of the stimulation frequencies. Each set contains the *sine* and *cosine* for the first, second and third harmonic of the stimulation frequency. The re-referenced EEG data and each set with reference signals are used as input for the CCA. CCA tries to find pairs of linear transformations for the two sets such that when the transformations are applied the resulting sets have a maximal correlation. The stimulation frequency with the highest maximum correlation is classified as the frequency the participant was focusing on.

As CCA-based detection methods have some improvements and advantages compared to PSDA-based detection methods, such as better signal-to-noise ratio (SNR), lower inter-subject variability and the possibility of using harmonic frequencies [2], a CCA-based detection method was used in this study.

## 4.3 Questionnaires and Data Acquisition

To measure the affective reaction of the participants while playing the game, they were requested to fill in a SAM [3] after each trial. It was expected that if participants became more frustrated by using a selection method, this would result in higher arousal and lower valence scores. The dominance would also be higher if participants had the feeling that a selection method was working properly and that they were in control.

<footnote n="2">² BioSemi, Amsterdam, The Netherlands</footnote>
After each session participants filled in a questionnaire on immersion. The questionnaire by Jennett *et al.* [12] was used for this. It contains 31 questions and is designed to measure the total immersion as well as five different factors of immersion (cognitive involvement, emotional involvement, real world dissociation, challenge and control). Although valence, arousal and dominance would probably differ between selection methods for which participants have an aversion, this does not necessarily mean participants would not become immersed in the game.

Furthermore some game statistics, such as the number of times a dog was selected, the total number of selections, the average selection duration and the time participants needed to finish a trial were collected while participants played the game. Ideally participants would use all the dogs as this would make it easier to gather the sheep. The number of times a particular dog was selected would be a good indication of how well the selection methods performed for the participants.

At the end of the experiment participants were asked which selection method they would like to use if they were given the opportunity to play the game again. This should have given a good indication of which selection method the participant preferred.

After the experiments, scores were obtained for the total immersion, the five immersion factors, the SAM questionnaire and the game statistics. The results from the SAM questionnaire were averaged over the three trials to get an average score for the valence, arousal and dominance scales. As each participant used both selection methods, the scores were compared using a Wilcoxon signed-rank test to determine if there was a significant difference between the two selection methods.

## 4.4 Participants

Seventeen participants (7 female and 10 male), aged between 17 and 37 (μ = 22.00, σ = 4.74) participated in the experiment. All participants except for one had normal or corrected-to-normal vision and described themselves as daily computer users. Although eight participants had at least one-time experience with EEG, fourteen participants had no experience with BCIs. Before the experiment, all participants signed an informed consent form and they were paid according to institution's regulations.

## 5 Results

When the 17 participants were asked which selection method they would choose if they were given the opportunity to play the game again, 5 of them chose the non-BCI selection method and 12 of them chose the BCI selection method. This is a first indication that participants prefer the BCI selection method over the non-BCI selection method. A detailed insight might be provided by the scores for immersion, the SAM questionnaire and the game statistics, which are described below.
## 5.1 Immersion

Based on the immersion questionnaire the total immersion score was calculated for both selection methods. On average the participants rated the BCI selection method (μ = 160, σ = 14.55) higher than the non-BCI selection method (μ = 144, σ = 21.89). This difference was significant (Z = -2.155, p = 0.031).

The five immersion factors were also analyzed and the scores, averaged over participants, are shown in Table 1. The scores for all factors, except for the challenge factor, are higher for the BCI selection method. The five immersion factors were examined for significant differences between the BCI and non-BCI selection methods. There are significant differences for the cognitive involvement factor (Z = -2.219, p = 0.026), the real world dissociation factor (Z = -1.992, p = 0.046), the emotional involvement factor (Z = -2.013, p = 0.044) and the control factor (Z = -2.310, p = 0.021). For the challenge factor no significant difference (Z = -0.476, p = 0.634) was found.

<table><caption>**Table 1.** Scores of the five immersion factors, averaged over participants. Values are represented as μ(σ) with * indicating a significant difference with p < 0.05.</caption><thead><tr><th></th><th>non-BCI</th><th>BCI</th></tr></thead><tbody><tr><td>Cognitive*</td><td>53.12 (9.01)</td><td>58.47 (5.36)</td></tr><tr><td>Dissociation*</td><td>25.18 (5.64)</td><td>28.06 (4.79)</td></tr><tr><td>Emotional*</td><td>54.35 (8.54)</td><td>59.88 (5.95)</td></tr><tr><td>Challenge</td><td>20.88 (2.89)</td><td>20.71 (2.34)</td></tr><tr><td>Control*</td><td>32.06 (5.77)</td><td>37.06 (5.08)</td></tr></tbody></table>

## 5.2 Affect

Based on the SAM questionnaire the total SAM scores were calculated for both selection methods and the average results are shown in Table 2.

<table><caption>**Table 2.** Average SAM scores, the values are represented as μ(σ) with * indicating a significant difference of p < 0.05.</caption><thead><tr><th></th><th>non-BCI</th><th>BCI</th></tr></thead><tbody><tr><td>Valence*</td><td>6.37 (1.55)</td><td>7.08 (1.31)</td></tr><tr><td>Arousal</td><td>4.82 (2.31)</td><td>4.53 (2.37)</td></tr><tr><td>Dominance</td><td>5.65 (2.46)</td><td>6.08 (1.64)</td></tr></tbody></table>

For the valence scale the difference was significant (Z = -2.012, p = 0.044), however, no significant difference was found for the arousal (Z = -0.315, p = 0.752) or dominance (Z = -0.403, p = 0.687) scores.
## 5.3 Game Statistics

The game statistics which were collected during the experiments are shown in Tables 3 and 4. They show the number of selections and the average selection time. For the number of selections (Table 3) there are significant differences for the easy task trial (Z = -2.202, p = 0.028) and the difficult task trial (Z = -2.107, p = 0.035). However, no significant difference was found for the familiarity trial (Z = -1.866, p = 0.062).

<table><caption>**Table 3.** Average number of selections for each trial, presented as μ(σ) with * indicating a significant difference of p < 0.05.</caption><thead><tr><th></th><th>non-BCI</th><th>BCI</th></tr></thead><tbody><tr><th>Familiarity</th><td>12.53 (4.87)</td><td>10.47 (2.67)</td></tr><tr><th>Easy*</th><td>43.00 (26.27)</td><td>27.88 (7.86)</td></tr><tr><th>Difficult*</th><td>98.88 (65.90)</td><td>74.12 (30.28)</td></tr></tbody></table>

<table><caption>**Table 4.** Average selection time (in seconds) for each trial, presented as μ(σ).</caption><thead><tr><th></th><th>non-BCI</th><th>BCI</th></tr></thead><tbody><tr><th>Familiarity</th><td>1.92 (0.86)</td><td>2.49 (1.71)</td></tr><tr><th>Easy</th><td>1.96 (1.04)</td><td>2.64 (2.63)</td></tr><tr><th>Difficult</th><td>1.95 (1.31)</td><td>1.71 (0.85)</td></tr></tbody></table>

Although no significant differences were found for the average selection time (Table 4) between the BCI and non-BCI selection methods for any of the three trials, it can be seen that for the familiarity and easy task trials the average selection times for the BCI selection were around 0.5 seconds higher. However, for the difficult task trial the average selection time was lower.

## 6 Discussion

Most participants indicated that they preferred playing the game with the BCI selection method. This seems to be supported by the results of the immersion questionnaire. The total immersion score was significantly higher for the BCI selection method, indicating that participants were more immersed than when using the non-BCI selection method. For the BCI selection method participants only had to make their intentions clear to the BCI system by concentrating on the dog they wished to select. Besides starting and stopping the selection method, there were no other actions required. However, for the non-BCI selection method, participants still had to translate their intentions into an action, stopping the selection method at the correct time. As participants were able to translate their intentions directly into in-game actions while using the BCI selection method, they might have become more easily immersed.

Further inspection of the five immersion factors showed that all factors, except for the challenge factor, were significantly higher for the BCI selection method. The questions related to the challenge factor were about the game itself. As the averages
of the challenge factor were almost equal for the two selection methods, they indicate that participants did not find the game more challenging using one of the two selection methods. For using the selection methods, the control factor might be a better indicator. As the control factor was significantly lower for the non-BCI selection method, it indicates that participants had more trouble using the non-BCI selection.

The results of the SAM questionnaire indicate that participants were more content using the BCI selection method. The results of the cognitive involvement and real world dissociation factors were also significantly higher for the BCI selection method. This could have been caused by the fact that participants had to concentrate on a dog while using the BCI selection method and did not have to translate their intentions into real world actions.

Some participants developed a strategy to deal with the trade-off between accuracy and speed. When they wanted to be accurate in their selections, they waited long enough to make an accurate selection. However, when they wanted to be quick, they moved all three dogs as one by pressing rapidly at a location on the playground. Although this behavior was observed for both selection methods, for the BCI selection method participants appeared to switch between precise and quick whenever they wished. However, for the non-BCI selection method they appeared to prefer only making quick selections, explaining the significantly higher number of selection for the non-BCI selection method.

For the BCI selection method the number of selections was lower and, although not significant, the average selection time was higher. Participants were also more immersed and content. Apparently, participants accepted that for a good SSVEP detection they had to wait a couple of seconds. However, for the non-BCI selection method they understood that it was related to their own reaction speed and suddenly they appeared to be in a hurry. Although the non-BCI selection method was modeled to be similar to the BCI selection method, it might have had an effect on the preference of the participants, as they did not want to wait to make a selection. The non-BCI selection method did introduce the same challenge and required participants to multi-task similarly to the BCI selection method. However, participants seemed to be more indulgent towards the BCI selection method than towards the non-BCI selection method. This could be caused by the curiosity of participants for the BCI selection method or the self overestimation by participants while using the non-BCI selection method.

# 7 Conclusion

In this study a BCI system was compared to a non-BCI system to evaluate the user experience in terms of immersion and affect. For the BCI system a selection method based on SSVEP was integrated into a game, introducing a challenge factor. A comparable non-BCI selection method based on time was also implemented into the game, introducing an equal challenge. Seventeen participants played the game with both selection methods in three trials. They rated each selection method on immersion and affect.
The results show that the BCI selection method was found to be more immersive and positively affective than the non-BCI selection method. While using the BCI selection method participants were able to directly translate their intentions into in-game actions, which made it easier for them to become more immersed. In this case, the user experience in terms of immersion and affect seemed to be improved while using the BCI selection method. Furthermore, participants appeared to have more patience when using the BCI selection method than when using the non-BCI selection method, which could have been caused by the curiosity of participants for the BCI selection method or the self overestimation by participants while using the non-BCI selection method.

For future studies it would be interesting to look at long term effects on immersion and affect. Is the BCI selection method still significantly more immersive and affective when participants are used to it? Another question is whether the indulgence towards the BCI selection method is permanent or only temporary. It would also be interesting to add a selection method based on pointing and clicking which was left out of this research. Thereby testing for the cognitive load as the selection methods used in this study required participants to multi-task. Given the intended purpose of this research, it would also be interesting to use commercial BCI input devices (e.g. Emotiv's EPOC). This would also require a proper study comparing the equipment used in this research and commercial BCIs to determine the differences and if they are feasible to use within the setup of this experiment. Other (non-)BCI selection methods, such as speech and gestures could also prove to increase immersion and affect.

**Acknowledgments.** The authors gratefully acknowledge the support of the BrainGain Smart Mix Programme of the Netherlands Ministry of Economic Affairs and the Netherlands Ministry of Education, Culture and Science. The authors would also like to thank Michiel Hakvoort for his technical support on the game and Lynn Packwood for improving the language of this paper.

## References

1. Beverina, F., Palmas, G., Silvoni, S., Piccione, F., Giove, S.: User adaptive BCIs: SSVEP and P300 based interfaces. PsychNology Journal 1(4), 331–354 (2003)

2. Bin, G., Gao, X., Yan, Z., Hong, B., ShangkaiGao: An online multi-channel SSVEP-based brain–computer interface using a canonical correlation analysis method. Journal of Neural Engineering 6(4), 046002 (2009)

3. Bradley, M.M., Lang, P.J.: Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry 25(1), 49–59 (1994)

4. Brown, E., Cairns, P.: A grounded investigation of game immersion. In: CHI ’04 extended abstracts on Human factors in computing systems. pp. 1297–1300. ACM (2004)

5. Cheng, M., Gao, X., Gao, S., Xu, D.: Design and implementation of a brain-computer interface with high transfer rates. IEEE Transactions on Biomedical Engineering 49(10), 1181–1186 (2002)

6. Cooper, R., Osselton, J., Shaw, J.: EEG technology. Butterworths (1969)
7. Farwell, L., Donchin, E.: Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology 70(6), 510–523 (1988)

8. Galán, F., Nuttin, M., Lew, E., Ferrez, P., Vanacker, G., Philips, J., Van Brussel, H., Millán, J.: An asynchronous and non-invasive brain-actuated wheelchair. In: 13th International Symposium on Robotics Research (2007)

9. van Gerven, M., Farquhar, J., Schaefer, R., Vlek, R., Geuze, J., Nijholt, A., Ramsey, N., Haselager, P., Vuurpijl, L., Gielen, S., Desain, P.: The brain–computer interface cycle. Journal of Neural Engineering 6(4), 041001 (2009)

10.Hakvoort, G., Reuderink, B., Obbink, M.: Comparison of PSDA and CCA detection methods in a SSVEP-based BCI-system. Technical Report TR-CTIT-11-03, Centre for Telematics and Information Technology, University of Twente,(2011)

11.Hoffmann, U., Vesin, J., Ebrahimi, T., Diserens, K.: An efficient P300-based brain-computer interface for disabled subjects. Journal of Neuroscience Methods 167(1), 115–125 (2008)

12.Jennett, C., Cox, A., Cairns, P., Dhoparee, S., Epps, A., Tijs, T., Walton, A.: Measuring and defining the experience of immersion in games. International Journal of Human-Computer Studies 66(9), 641–661 (2008)

13.Lansing, R., Schwartz, E., Lindsley, D.: Reaction time and EEG activation under alerted and nonalerted conditions. Journal of Experimental Psychology 58(1), 1–7 (1959)

14.Lin, Z., Zhang, C., Wu, W., Gao, X.: Frequency Recognition Based on Canonical Correlation Analysis for SSVEP-Based BCIs. IEEE Transactions on Biomedical Engineering 53(12), 2610–2614 (2006)

15.Lopez, M., Pelayo, F., Madrid, E., Prieto, A.: Statistical characterization of steady-state visual evoked potentials and their use in brain–computer interfaces. Neural Processing Letters 29(3), 179–187 (2009)

16.Nijholt, A., Plass-Oude Bos, D., Reuderink, B.: Turning shortcomings into challenges: Brain-computer interfaces for games. Entertainment Computing 1(2), 85–94 (2009)

17.Nijholt, A., Tan, D., Allison, B., et al.: Brain-Computer Interfaces for HCI and Games. In: CHI ’08 extended abstracts on Human factors in computing systems. pp. 3925–3928. ACM (2008)

18.Norman, D.: Emotion & design: attractive things work better. Interactions 9(4), 36–42 (2002)

19.Pagulayan, R., Keeker, K., Wixon, D., Romero, R., Fuller, T.: User-centered design in games. In: The human-computer interaction handbook. pp. 883–906 (2002)

20.Picard, R.: Affective computing. The MIT press (2000)

21.Reilly, E.L.: EEG Recording and Operation of the Apparatus. In: Electroencephalography: Basic Principles, Clinical Applications and Related Fields, pp. 139–160. Lippincott Williams & Wilkins (1999)

22.Reynolds, C.W.: Flocks, herds and schools: A distributed behavioral model. In: SIGGRAPH ’87 Proceedings of the 14th annual conference on Computer graphics and interactive techniques. pp. 25–34. ACM (1987)

23.Ruen Shan, L., Ibrahim, F., Moghavvemi, M.: Assessment of Steady-State Visual Evoked Potential for Brain Computer Communication. In: 3rd Kuala Lumpur International Conference on Biomedical Engineering 2006. pp. 352–354. Springer (2006)

24.Volosyak, I., Cecotti, H., Gräser, A.: Impact of Frequency Selection on LCD Screens for SSVEP Based Brain-Computer Interfaces. Bio-Inspired Systems: Computational and Ambient Intelligence pp. 706–713 (2009)