# What can we learn from user interviews in BCI sessions?

Aline Roc, Fabien Lotte

## ► To cite this version:

Aline Roc, Fabien Lotte. What can we learn from user interviews in BCI sessions?. BCI 2023 - 10th International BCI Meeting Balancing Innovation and Translation, Jun 2023, Bruxelles (BE), Belgium. Proceedings of the 10th International BCI Meeting 2023. hal-04125216

HAL Id: hal-04125216

https://inria.hal.science/hal-04125216

Submitted on 12 Jun 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# What can we learn from user interviews in BCI sessions?

Aline Roc¹,², Fabien Lotte¹,²,*

¹Inria, France — ²LaBRI (CNRS / Bordeaux INP / Univ. Bordeaux), France

*E-mail: rocaline@proton.me

**Introduction:** What factors contribute to the success of motor imagery training for novice users of EEG-based Brain-Computer Interfaces (BCI)? How do novice users experience and adapt to Kinesthetic Motor Imagery (KMI) training? One way to find out more is to ask the users themselves and to collect qualitative data with interviews during BCI sessions. However, the practice is not common, although it tends to be more democratized in recent literature, mostly with patients [1—10].

**Methods:** Semi-structured interviews were conducted at the end of each of three BCI training sessions for 24 individuals, for a total of 72 interviews. The interview questions covered performances, difficulties and strategies, i.e. movements that users chose to imagine for KMI. The questions were designed to clarify users' choices, the reasoning behind them, and identify any barriers they encountered during the training. Interviews were analysed to identify any training parameters that users found difficult or distracting, which is a lever to better understand users’ needs and to ultimately design better exercises for the future of BCI user training [11].

**Results:** Thematic analysis showed important inter-individual discrepancies. While some user struggled to find strategies, others found it easy. Similarly, KMI was difficult for some users, but many others explained that the KMI was not difficult per se. Still, users tend to report quick improvements in their KMI auto-evaluation within and between sessions.

Overall, this series of interviews highlighted various issues with common training protocols, some well-known, but that may not receive enough attention within the community. This includes:

- Users tend to be prone to overthinking, and they struggle to go or stay in a relaxed state.

- BCI requires focus, fighting fatigue, and coping with bad performances.

- Handling artefacts and avoiding movements is difficult and consumes attention.

- Eyes-open imagination is hard, especially with continuous feedback.

- Inter-trial duration can feel too short, and MI can be hard to maintain over the trial duration.

- Some MI tasks may implicitly involve other body parts and interfere with right/left separation.

- Users do not always contain their curiosity and sometimes can show unexpected behaviours such as disobeying instructions.

- Understanding feedback is hard, notably identifying min/max and assimilating sham feedback.

- Contextual factors can strongly affect the KMI experience, e.g. room temperature.

**Discussion/Significance:** Interviews were conducted with healthy users over their three first BCI sessions. Although results might not apply well to all populations, they suggest overall that BCI training is highly demanding and causes users to struggle despite high motivation. This emphasizes the need for the development of more engaging and effective training approaches that help novice BCI users explore the interaction, understand instructions and feedback, and maintain focus during training.

**Acknowledgements:** This work is supported by the grant ERC–2016–STG–714567.

<footnote>[1] Onose et al. (2012) On the feasibility of using motor imagery EEG-based BCI in chronic tetraplegics for assistive robotic arm control: a clinical test and long-term post-trial follow-up. Spinal cord, 50(8):599–608.</footnote>

<footnote>[2] Holz et al. (2013) BCI controlled gaming: Evaluation of usability by severely motor restricted end-users. AI in medicine, 59(2):111–120.</footnote>

<footnote>[3] van de Laar et al. (2013) Experiencing BCI control in a popular computer game. IEEE TCIAIG, 5(2):176–184.</footnote>

<footnote>[4] Peters et al. (2015) BCI users speak up: The virtual users’ forum at the 2013 international BCI meeting. Arch. Phys. M., 96(3):S33–S37.</footnote>

<footnote>[5] Schicktanz et al. (2015) Qualitative assessment of patients’ attitudes and expectations toward BCIs and implications for future technology development. Frontiers in systems neuroscience, 9:64.</footnote>

<footnote>[6] Klein et al. (2016) BCI-based control of closed-loop Brain stimulation: attitudes and ethical considerations. BCIs, 3(3):140–148.</footnote>

<footnote>[7] Kögel et al. (2019) Using BCIs: a scoping review of studies employing social research methods. BMC medical ethics, 20(1):1–17.</footnote>

<footnote>[8] Kögel et al. (2020) What it takes to be a pioneer: Ability expectations from BCI users. NanoEthics, 14(3):227–239.</footnote>

<footnote>[9] Benaroch et al. (2021) Long-term BCI training of a tetraplegic user: Adaptive Riemannian classifiers and user training. Frontiers in Human Neuroscience, 15:635653.</footnote>

<footnote>[10] Schmid & Jox. (2021) The power of thoughts: a qualitative interview study with healthy users of BCIs. Clinical Neurotechnology meets Artificial Intelligence, 117–126. Springer.</footnote>

<footnote>[11] Roc et al. (2021) A review of user training methods in BCIs based on mental tasks. Journal of Neural Engineering, 18(1):011002.</footnote>
# What can we learn from user interviews in BCI sessions?

Aline ROC*, Fabien LOTTE* Inria, France (Potioc team) & LaBRI (CNRS / Bordeaux INP / Univ. Bordeaux), France

<figure><img src="image_2.png" /></figure>

Electroencephalography (EEG-)based BCI

Kinesthetic Motor Imagery (KMI) training

24 healthy users

3 sessions

72 interviews

## Research questions

* How do novice users experience and adapt to KMI / EEG-BCI?

* What factors contribute to success for novice users?

## Method

Post-session interviews were conducted by experimenters, covering performances, difficulties encountered, and movements that users chose to imagine for KMI. Recordings were transcribed for a systematic thematic analysis.

## Dataset

24 participants took part in three training sessions on different days with a common protocol (Graz feedback, 2-class CSP/LDA classification).

Semi-structured interviews were conducted at the end of each session, for a total of 72 interviews.

## Main results

* dominant themes, i.e. widely reported trends common in 5+ users.

* lesser-common themes, i.e. valuable insights but mentioned by < 5 users.

* High **motivation** and shared perception of **skill** improvement.

* Long and repetitive sessions cause boredom, an increasing feeling of **fatigue**, and a struggle to **focus** over time (intrusive thoughts).

* Users are prone to **overthinking** the task and struggle to be or stay in a relaxed state, especially when facing low performances.

* Difficulty to **stand still** during KMI worsened by prolonged immobility.

* Eyes-open imagination is hard, especially with continuous visual feedback.

* Limited grasp of feedback, notably in **identifying min/max**, understanding there is **no idle class**, and assimilating what **sham feedback** is.

* Inter-trial duration can feel too short, as users need time to feel ready.

* KMI can be **hard to maintain over time** for the trial duration.

* Some MI tasks may implicitly involve **other body parts** and interfere with right/left separation. KMI is easier with the **dominant hand**.

* Users do not always **contain their curiosity** and sometimes can show **unexpected behaviours** such as disobeying instructions.

* Contextual factors strongly affect KMI experience, e.g. **room temperature**.

* **Tensions and artefacts** can be caused by important imagined muscular effort.

* Trials switch too often and too randomly. Users attempt to **anticipate** the next cue and are disturbed by wrong prediction.

* Users struggle with **inter-trial consistency** (stability within task).

* Some users felt not enough support to find the **right strategy**.

## Sample quotes

As soon as I see the blue bar going up and down, I lose my focus.

We're not used to imagining a movement without actually doing it. And so, to maintain it, that feeling of maintaining the movement, I think that's what's hard, because there's no haptic feedback, it's just... it's hard to keep imagining, actually.

What is difficult is to stay focused when you see that it doesn't work. It takes me out of my routine, I have parasitic thoughts, and it distracts me, so I get frustrated.

I did a test where... maybe I'm a bad subject but... when it told me to do right, I did left! But I didn't do that often!!! Just that one time. Besides, I knew it wasn't right, but I told myself: I'll try anyway.

As the runs go on, it's more and more tiring, it's more difficult because it's hard to keep your attention. Two or three times, I think I forgot to be really focused.

I think that maybe they are trying to frustrate me and that the blue bar is not moving forward on purpose, aiming to frustrate me. Maybe they don't tell me and make me believe that it is not moving forward, and in fact it is moving forward very well.

I was like "uuugh" and I felt like even my breathing was changing at the same time, and I was trying not to make any movements and stuff, but I couldn't stop holding my breath like I was lifting something real!

## Related work

Working with qualitative data is not a common practice in the BCI field, although it tends to be more democratized in recent literature, mostly with patients [1--10].

## References

[1] Onose et al. (2012) On the feasibility of using motor imagery EEG-based BCI in chronic tetraplegics for assistive robotic arm control: a clinical test and long-term post-trial follow-up. Spinal cord, 50(8):599-608.

[6] Klein et al. (2016) BCI-based control of closed-loop Brain stimulation: attitudes and ethical considerations. BCIs, 3(3):140-148.

[2] Holz et al. (2013) BCI controlled gaming: Evaluation of usability by severely motor restricted end-users. AI in medicine, 59(2):111-120.

[7] Kögel et al. (2019) Using BCIs: a scoping review of studies employing social research methods. BMC medical ethics, 20(1):1-17.

[3] van de Laar et al. (2013) Experiencing BCI control in a popular computer game. IEEE TCIAIG, 5(2):176-184.

[8] Kögel et al. (2020) What it takes to be a pioneer: Ability expectations from BCI users. NanoEthics, 14(3):227-239.

[4] Peters et al. (2015) BCI users speak up: The virtual users' forum at the 2013 international BCI meeting. Arch. Phys. M., 96(3):S33-S37.

[9] Benaroch et al. (2021) Long-term BCI training of a tetraplegic user: Adaptive Riemannian classifiers and user training. Frontiers in Human Neuroscience, 15:635653.

[5] Schicktanz et al. (2015) Qualitative assessment of patients' attitudes and expectations toward BCIs and implications for future technology development. Frontiers in systems neuroscience, 9:64.

[10] Schmid & Jox (2021) The power of thoughts: a qualitative interview study with healthy users of BCIs. Clinical Neurotechnology meets Artificial Intelligence, 117-126. Springer.

[11] Roc et al. (2021) A review of user training methods in BCIs based on mental tasks. Journal of Neural Engineering, 18(1):011002.

## Conclusion

Everyone knows that BCI training is highly demanding and causes users to struggle despite high motivation. And still, this may not receive enough attention within the community. Exploring the training experience with real interview data can help to identify appropriate solutions connected to actual user needs, and ultimately improve user training protocols. [11]

## Practical takeaways

* Help users understand KMI and feedback with tutorials and demonstrations.

* Offer real breaks and longer inter-trial duration to help maintain energy and focus.

* Present some exercises with less information density for beginners (e.g. open loop exercises, only post-trial feedback, eye-close KMI or non-visual feedback).

* Provide engaging approaches and opportunities to experiment without cues during training or at the end of the session.

## Contact info

rocaline@proton.me

https://team.inria.fr/potioc

## Special thanks

Dreyer, P., Kolodzienski, L., Appriou, A., Monseigne, T.

Research supported by the grant

ERC-2016-STG-714567