# Sense-It: Towards new types of BCI-based sensorimotor neurofeedback for motor rehabilitation

Mathilde Yousefi, Jérémy Frey, Sébastien Rimbert, Altamira Gabriela

Herrera, Stéphanie Fleck

► To cite this version:

Mathilde Yousefi, Jérémy Frey, Sébastien Rimbert, Altamira Gabriela Herrera, Stéphanie Fleck. Sense-It: Towards new types of BCI-based sensorimotor neurofeedback for motor rehabilitation. SMC 2025 - IEEE International Conference on Systems, Man, and Cybernetics, Oct 2025, Vienna, Austria. hal-05268313

HAL Id: hal-05268313

https://inria.hal.science/hal-05268313v1

Submitted on 19 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Sense-It: Towards new types of BCI-based sensorimotor neurofeedback for motor rehabilitation

Mathilde Yousefi<fnref n="1" />, Jérémy Frey<fnref n="2" />, Sébastien Rimbert<fnref n="3" />, Gabriela Herrera Altamira<fnref n="4" /> and Stéphanie Fleck<fnref n="1" />

**Abstract**— Combined with conventional rehabilitation of the upper limbs, kinesthetic motor imagery (KMI) is a promising direction in post-stroke neurorehabilitation. Unfortunately, KMI is a complex mental task for people, as it provides no feedback, making it impossible to self-assess, correct, and improve their performance and, consequently, their rehabilitation. To overcome this difficulty, we have designed Sense-IT, a deformable interface coupled with a gamified BCI that provides tangible multisensory feedback (visual and kinesthetic) to allow for a supportive and differentiable rehabilitation program according to patients' needs and preferences. This original double-blind and mixed study (N=36) compares the effects of three feedback modalities (visual, kinesthetic via Sense-IT, bimodal) on the user experience and motor area stimulation. Although neurophysiological results did not reveal significant differences in contralateral motor cortex activation across the three feedback modalities, subjective measures highlighted the added value of tangible and multimodal feedback. Participants reported that the kinesthetic feedback provided by Sense-IT enhanced their engagement and task comprehension. Overall, these findings support the relevance of combining visual and kinesthetic neurofeedback to improve user experience and emphasize the importance of delivering reliable and meaningful feedback in BCI-based motor rehabilitation.

## I. INTRODUCTION

Support for innovations in upper limb motor rehabilitation is crucial, given the rising number of stroke cases and their long-term effects. Approximately 40% of stroke patients suffer from upper limb impairment, which increases the need for rehabilitation services [1]. Combined with traditional therapies, kinesthetic motor imagery (KMI) shows promise in enhancing neuronal plasticity in individuals with motor disorders [2], [3]. KMI involves imagining movement without actually executing it while focusing and reactivating the haptic sensations felt during the real movement, such as, tactile, proprioceptive, and kinesthetic sensations. By doing so, KMI stimulates the relevant motor areas of the brain in a similar way to actually performing the movement [4]. Unfortunately, KMI tasks are unfamiliar and do not generate proprioceptive feedback. Thus, even with clear explanations from an expert, KMIs remain abstract, intangible, and difficult to understand. The lack of feedback on mental performance is one of the

main limitations. In addition, these tasks generate fatigue and significant mental workload [5], [6].

In this context, KMI-based brain-computer interfaces (KMI-BCI) open novel approaches for neurorehabilitation by using brain activity signals from electroencephalography (EEG). KMI-based BCI allows users with upper limb disabilities to communicate with machines without physical exertion. It provides real-time neurofeedback (NFB), which can foster training or self-learning strategies [7]. Current research often prioritizes technical aspects such as classifier optimization, yet it is important to investigate user-centered elements that affect the control of KMI-BCI. It is crucial to focus on providing comfortable and acceptable BCI solutions and helping users interact by knowing if their mental task is correct, assessing their mastery, repeating consistently, and even improving their performance [5].

While studies on BCIs have examined various feedback modalities —including visual and haptic (e.g., vibrotactile, tendon vibration, or robotic) [8], [9] — there has been little exploration of kinesthetic haptic neurofeedback through deformable tangible interfaces. Some approaches include soft gloves [10], [11], however, they must be carefully adapted to individual users. For instance, a glove designed for the left hand may not be suitable for the right hand, and standard-size gloves often fail to accommodate individuals with exceptionally small or large hands. Moreover, 25% to 48% of stroke survivors experience spasticity [12], which can significantly complicate glove placement. Spasticity may either impede proper glove positioning or be so severe that it completely prevents the glove from being worn. Given these challenges, it is crucial to develop solutions that are generalizable across a diverse population with varying physical characteristics and designed to maximize patient comfort and usability.

This study aims to enhance the understanding of kinesthetic feedback provided by Sense-IT within a BCI (see figure 1 below), focusing on user experiences, compared to visual-only and bimodal (kinesthetic + visual) NFB. It addresses two key questions:

* Q1: Does the sensory modality of NFB influence the user experience?

* Q2: Does the sensory modality of NFB differentially engage motor cortex activity during or after KMI?

Ultimately, the study explores the potential for NFB interchangeability in rehabilitation programs, allowing them to be tailored to meet patients' specific needs or preferences.

<footnote>*This work was financially supported by ANR-19-CE33-0007</footnote>

<footnote n="1">¹Mathilde Yousefi and Stéphanie Fleck are with PErSEUs laboratory, University of Lorraine, France. mathilde.yousefi@univ-lorraine.fr ; stephanie.fleck@univ-lorraine.fr</footnote>

<footnote n="2">²Jérémy Frey is with Qualya, France jfrey@qualya.io</footnote>

<footnote n="3">³Sébastien Rimbert is at Inria Centre Université de Bordeaux, France . sebastien.rimbert@inria.fr</footnote>

<footnote n="4">⁴Gabriela Herrera Altamira is with Capgemini Engineering, France Gabriela.herrera@capgemini.com</footnote>
<figure><img src="image_2.png" /><figcaption>Fig. 1. Sense-IT is a deformable TUI used as a BCI output device to help the user perceive their motor brain activity. Deforming enables the opening and/or closing of the hand, raising and closing of the thumb, to reproduce grasp, pinch, and hand-opening movements. It aims to aid the neurostimulation of motor cerebral areas damaged by stroke, resulting in hemiplegia or hemiparesis of the upper limbs.</figcaption></figure>

## II. MEANS AND METHODS

### A. System description

The BCI (see Figure 1) used in this study was designed using an iterative participatory approach involving a multi-disciplinary group of researchers, therapists and patients in the subacute post-stroke phase [13]. The solutions' ease of use is not the only critical dimension highlighted. Thanks to its human-centered approach, the design of the BCI seeks to address different areas of need in the context of motor rehabilitation and KMI-based BCIs: to optimize compatibility with standard rehabilitation gestures and targeted motor stimulation, while minimizing fatigue and spasticity. It also aims to lighten the cognitive load of novice patients, by individualizing training to encourage engagement and learning.

1) *The gamified BCI:* The BCI relies on the distinction between two distinct mental tasks: KMI and resting state. These two mental tasks are known to generate, respectively, ERD and ERS modulations [14] that can be easily detected by a BCI using machine learning algorithms [15]. The system offers a non-immersive VR environment, enabling users to choose from 12 mini-games. Thus, sessions and movements can vary according to the patient's needs and interests. The imaginary movement in this study corresponds to the hand's closing (Figure 1 - 1). The related mini-game is called "Grasp-It", which gave its name to this game-based BCI program. The mini-games present a first-person perspective, showing a hand holding a deformable object. In our specific scenario, a ketchup bottle is positioned above a hot dog, prompting the user to imagine firmly squeezing the bottle to put a maximum amount of sauce (Figure 1 - A). This principle is the same for all the other mini-games, only the object to be grasped, pinched, or released changes. Whatever the selected game, the VR environment provides visual cues via a traffic light, allowing the subject to quickly identify when to perform the BCI (green light) and rest (red light) tasks (Figure 2). In addition, a red and orange light indicates

to the user to prepare themselves before performing the KMI to avoid any motor preparation during the rest state. Optionally, neurofeedback may be provided either:

* Visually (visual NFB) through the virtual first-person scene. In this study, the virtual hand closes and the sauce flows out according to 4 levels (none, low, moderate, high) related to KMI performance, i.e., the input data, the user's brain signal for controlling the BCI, captured by the EEG. These levels were linked to 0-25%; 25-50%; 50-75%; 75-100% BCI performance. In addition, a dynamic performance gauge is displayed, displaying a numerical score between 0 and 100%, encouraging the user to surpass their previous session record.

* Kinesthetically (kinesthetic NFB) directly on the participant's proper hand mobilized thanks to Sense-IT (see below) according to the four performance levels (too low, moderate, high, very high). There is no virtual hand or object, just the traffic light. The performance gauge was not visible in this modality.

* Or bimodally, combining the two previous NFB modalities. The object and the hand, visible on the screen and in the physical world, are activated synchronously. Both virtual and physical feedback respond dynamically to the quality of the executed KMI. The performance gauge and the virtual sauce are also activated.

Across all three modalities, feedback is delivered after the KMI phase and preceding the resting phase. This sequential approach allows users to focus entirely in the KMI without any distraction and then dedicate their entire attention to interpreting the feedback, as patients previously asked for. To evaluate the effects of the feedback modality, participants were presented with randomized feedback indicators that were independent of their actual KMI performance (same likelihood for each level of feedback).

2) *The Sense-It output device:* Sense-IT (see Figure 1) is a deformable and graspable interface designed to render the effect of a specific hand gesture (grasping, pinching, or releasing) on virtual objects tangible. The device resembles a joystick with two openings: one at the top for the thumb and another on the side for the remaining four fingers. Two balloons, one at each opening, are inserted into the joystick from the base to generate the artifact deformations. A hose connects each balloon to a tensiometer pump, enabling in-
flation and deflation, to induce finger movements. A custom-
made silicone membrane covers the entire device, controlling
deformation properties, resistance, and tactile comfort. This
thin bladder provides a smooth surface that is easy to clean
and disinfect, ensuring hygiene and longevity. An Arduino
Nano board controls the pumps, housed in a separate box
connected via hoses. The device is easy to grasp or to
position in the user's hand by a therapist if needed (e.g.,
for a person with spasticity). It is non-invasive and safe for
users, with a reversible design, allowing identical use for
right and left-handed. Notably, Sense-It has a low production
cost (around one hundred euros), particularly compared to
exoskeletons. It is suitable and portable for stroke victims,
even those with severe spasticity.

Sense-It is designed to be combined with the Grasp-
It virtual gaming environment. It simulates the mechanical
deformation of the object visible in the virtual first-person
scene (here, the ketchup bottle). Inflation/deflation durations
are set to synchronize with the visual environment. Sense-
It aims to promote the embodiment of the KMI, helping
the user to feel, recognize, and reinforce task execution
through direct kinesthetic stimuli in their hand, offering a
more immersive and realistic experience.

B. Participants

Ethical frameworks in France for studies destined for ther-
apeutic applications state that preliminary studies (user tests,
clinical pre-studies, etc.) cannot involve patients to protect
their vulnerability until the relevance and quality of the
systems are fully assured. Consequently, this study involved
36 (19 women) neurotypical participants aged between 18
and 70 years old (76.5% were aged between 18 to 29). 34
were kept for the UX part (removal of marginal values)
and 29 for EEG analysis (missing recordings and only
selected right-handed participants to better compare motor
cortex activity). All were novices in KMI and BCIs before
starting the sessions, as would patients at the start of their
rehabilitation protocol. Protocols have received favorable
opinions from Inria's ethics committee (COERLE n°188, n°
211).

C. Experimental set-up

A trained researcher performed the experiment in a quiet
room using a 64-electrode Biosemi EEG cap. Participants sat
comfortably, with their non-dominant arm on an armrest and
their hand placed on an ergonomic pillow, free from muscular
tension. It is important to note that this is a double-blind
study.

a) *Calibration step:* After a short KMI training session, all participants began the BCI calibration phase without feedback. During the calibration, the participants were constantly provided with traffic light visual cues according to this chronology:

* For the first run, the light turns red for 30 seconds, where the participants must rest. They must not move or perform any motor imagery.

* The light changes from red to orange for 2 seconds. This phase warns the participants to prepare to perform KMI.

* Once the light turns green, they must perform the KMI task of grasping for 4 seconds.

* The light turns back to red. Participants rest again for 5 to 7 seconds. Then, the cycle of a new trial starts: preparation (orange), KMI (green), rest (red), and start again (Figure 2).

For all participants, the gesture to imagine was the grasping movement associated with squeezing a ketchup bottle (see gesture one and set up A in Figure 1).

b) *Neurofeedback modality tests:* To determine the influence of the feedback nature alone, the 36 subjects were divided into six groups following six different NFB sequences. Six groups of six subjects received a sensory NFB (visual, kinesthetic, or bimodal) following a Latin square, limiting the primacy bias effect. All subjects received feedback uncorrelated with their actual BCI - KMI performance, eliminating learning and performance bias.

The NFB tests' runs followed a sequence similar to the
calibration. However, after performing the KMI (during the
green light), the participant receives uncorrelated feedback,
for 2 seconds (low performance feedback) to 6 seconds (very
high performance feedback), during which the orange light is
displayed. The participants carried out two runs of 20 trials
for the calibration phase, and 2 runs of 15 trials per feedback
condition (visual, kinesthetic, and bimodal) (see Figure 2).
Participants were able to rest between runs. The visual,
kinesthetic, and bimodal feedback consisted of the animation
described in paragraph II-A.1. In the kinesthetic and bimodal
modalities, the Sense-It deflated and inflated during the
orange (feedback) and red (rest) lights, respectively.

D. User experience assessment

UX questionnaires adapted to BCI, whether multisensory
or not, are lacking. Therefore, we have chosen an inferential
mixed-methods approach by questionnaires and verbatim
cross-matching. Questionnaires, distributed throughout the
experiment and for all feedback methods (see Figure 2), aim
to describe and understand:

* The subjective perception of the user experience, i.e., the view of the tested system and its different feedback methods.

* The evolution of the general state of the subjects, both in terms of perceived fatigue and the evolution of the general emotional state.

* The subjective perception of the NFB's scaffolding qualities, to estimate the support that the nature of the feedback can provide for performing the KMI tasks.

To do this, the results of the questionnaires NExT-Q [16], the AttrakDiff [17], the meCUE [18], the Four Items Fatigue Screen (FIFS) [19], the Medical Emotion Scale (MES) [20] and the Nasa-TLX [21], as well as the Motivational Activity Values (MAV) [22] and the K-Uses [23] were analyzed and compared. This paper focuses on the most interesting results. These questionnaires are commonly used in Human-computer interaction (HCI), learning, and healthcare contexts. They have been validated and chosen for their simplicity and ease of use. In addition, participants' oral feedback was also collected throughout the NFB runs to better understand the influence of each feedback modality and how the system could help them perform the KMI task. The statistical tests were carried out using JASP (n=34).
<figure><img src="image_3.png" /><figcaption>Fig. 2. Experimental procedure timeline and traffic light duration for each step.</figcaption></figure>

Bonferroni-type corrections were applied when taking re-
peated measurements. Correlations were carried out using
Spearman's rho.

E. BCI performances and EEG offline analysis

1) *Pre-processing:* Offline analyses were performed using EEGLAB2024.2 [24]. EEG data was resampled at 128 Hz, re-referenced using common average, band-pass filtered between 0.5Hz and 40Hz using a FIR filter, and divided into 17-second epochs corresponding to 7 seconds before and 10 seconds after the start of the KMI for each run.

2) *Online BCI performances:* During this experiment, we computed online classification performance with the OpenViBE software. After the calibration step, the recorded EEG signals were band-passed using a 5th-order Butterworth filter between 8 and 30 Hz. Each trial was segmented into 3.5-second windows during the KMI task time and a rest time starting respectively 0.5 seconds after the green light signal and 2.5 seconds after the red light signal. For the calibration, we collected 40 trials for both mental states (KMI and resting state). The training was performed using a CSP filter and a linear discriminant analysis (LDA). The classifier was trained to discriminate the two classes from 1s-windows with a sliding window of 0.0625 s during the 3.5 s of the EEG segments. Thus, for each trial and each class, there are 40 patterns leading to 40 predictions.

3) *Event-related spectral perturbation and topographic maps:* To analyze the differences between each session, we performed an event-related spectral perturbation (ERSP) analysis between 8-35 Hz (n=29). We compute the ERSPs using the gain model approach equivalent to the *band power method* [14]. We used wavelet DFT method (3 cycles), and we computed the ERSP for the whole period, with a baseline between -5.5s and -2.5s per feedback condition. We chose this time window, in the middle of the "rest" period, to avoid as much as possible activity arising post-movement (e.g. beta rebound) or by movement anticipation. We also computed the topographic ERSPs (which are equivalent to ERD and ERS) in the mu (8-13 Hz), beta (13-30 Hz), and mu+beta (8-30 Hz) bands during and after the KMI task (Figure 4). Brain

topography maps allow us to display the possible changes
over different electrodes on the scalp to localize which part of
the brain was involved when the subject performed KMI/rest.
ICLabel was used to remove components with artifacts, with
1.5 removed on average. Autorej was used to filter epochs
containing artifacts, with no hit among the whole dataset.

III. RESULTS

A. User experiences

a) *User experience quality:* Overall, opinions were positive to neutral about the whole system and the experience (e.g., P10 "Very good experience, I enjoyed doing it. I learned a lot, and I hope this interface can be developed further.") Based on the AttrakDiff questionnaire, participants found the global system's attractiveness desirable to neutral (figure 3 - (A)). The unpredictable aspects of the system were perceived more unfavorably, significantly impacting the sense of being frustrated at the end of the session (Paired sample T-test, Wilcoxon signed-ranked, MES before vs. after, p<0.001), which is not surprising since the subjects received random feedback. Only three participants found the aesthetic aspect of Sense-IT “weird.”

b) *Influence on the user's general state:* Before the session, participants reported low to moderate tiredness (FIFS mean score (MS=4.35, std=2.30) and stress (MS=3.03, std=1.82). Their emotional state (MES) was primarily positive (MS=3.19 for positive emotions and MS=1.22 for negative emotions on a scale of 1 to 5). As often observed in KMI-BCI sessions [6], [5], the fatigue scores and declared mental workload (measured by Next-Q and NASA-TLX) increase progressively during the test, regardless of the NFB provided. Surprisingly, within the NASA-TLX scores, the difference regarding physical effort is significant (p=0.002 Friedman's Anova - Bonferroni correction - the more the runs accumulated, the more the physical demand increased). According to oral reports from the participants, this seems related to the difficulty of remaining still and relaxed for a long time. No dimension is judged negatively from the Me-CUe. On the contrary, respondents find the system's
use relatively "soothing" and "relaxing" (resp. MS=4.21
std=1.62; MS=4.25 std=1.50; max score =7). Regarding the
emotional state, no significant difference is noted, except
feelings related to the pleasure of accomplishing something
(Figure 3).

c) NFB subjective preference: According to the results of the NEXt-Q questionnaire completed after the different NFB sessions, no significant differences were found between the NFB modalities for any of the dimensions evaluated by the questionnaire. Nevertheless, participants receiving a bimodal NFB initially showed more confusion (Mean score = 2.36, std = 1.36) compared to Sense-It alone (3.42, std = 1.10) or visual alone (3.00, std = 1.19). When asked about their preferred feedback modality, participants indicated an equal preference for visual and kinesthetic feedback. However, 24 out of 34 participants declared at the end of the session that they preferred bimodal NFB. Of the 24 people who preferred bimodal NFB, 17 chose tangible NFB and 11 visual NFB as a second choice. The consistency of the bimodal rendering is recognized. The participants indicated that the deformation of the Sense-It was proportional to the visual feedback and, above all, authentic (see below). The participants did not declare any discomfort or disagreement on this point.

1) *Scaffolding properties of the NFB:*

a) Mental charge support: Participants highlighted the challenging dimension of the KMI task (MS=3.51, std=0.56; P13 "Seen as a game, it's stimulating in the short term. But it requires effort, it's tiring.", P33 "Using this system requires little practice. The feedback is interesting".) Whatever the NFB condition, the more time passes, the more difficult the task seems to participants, and the more they lose engagement (NEXt-Q - p=0.043 and 0.002 resp. Friedman's Anova - Bonferroni correction). Furthermore, the NASA-TLX revealed a significant increase in the frustration dimension (Friedman's Anova p= 0.018, t-test Wilcoxon signed-rank p<0.001 resp.) between the calibration phase (without any feedback) and the phases involving visual and bimodal NFB modalities (Durbin-Conover comparisons p= 0.003). A consensus emerged from the participants' verbatim suggesting that the absence of feedback is detrimental (P7 "The lack of feedback can be confusing when trying to imagine an object to hold.", P2 "Without feedback, I didn't know if I was doing the mental task correctly" P5 "Without feedback it was difficult to imagine the sensations", P11 "I don't know what's happening and how I'm performing").

Nevertheless, the NFB modality does not significantly
change the cognitive load associated with repeated KMI
execution over a long session. Interestingly, however, the
kinesthetic NFB provided by Sense-IT seems to make the
KMI task slightly less demanding regarding NEXt-Q scores.
However, the differences are not statistically significant (see
Figure 3-C).

b) *Pedagogical Value:* Participants generally consid-
ered that the system has good pedagogical qualities to
support the KMI task (K-Uses, see Figure 3-B), even though
they do not have feedback on their actual performance.

Participants particularly appreciated the support provided to understand the task to perform (MS=4.25, std =0.72, on a 1 to 5 scale), the certainty of success thanks to the system (MS=3.9, std =0.56), the feeling of being efficient (MS=3.84, std = 0.4) and to quickly know how to complete the task (MS= 3.62, std=0.87). The participants evaluated the motivational value of the activities (MVA) they had to do throughout their session positively, although the tasks were only moderately related to their interests in life (MS= 2.59, std=0. 95; not at all = 1, totally = 4). They felt they had enough time to do the tasks (MS=3.25, std=0.67), confirming the NASA-TLX scores on low charge due to time pressure. They perceived that the KMI tasks were explicit enough to feel capable (MS=3.75, std=0.50) and required little prior knowledge reinvestment (MS=2.09, std=0.96). The sensorial perceptions of performance are motivational levers. Still, they can also be frustrating for some (P2 "It was satisfying to sense the handle deflate completely", P29 "Visually, there was a little frustration when nothing came out"). In this case, the display of numerical scores is more valued (P35 "It's more meaningful when it's numerical", P29 "The visual allows you to see the result." P25 "The visual gave me the motivation I needed to stay focused on the task"). At the end of the 60-minutes test session, the subjects felt significantly more proud (Wilcoxon signed-ranked, – MES – before vs. after, p=0.007) and interested (p=0.019), which is encouraging for the motivational dynamic necessary for repetitive sessions over extended periods in motor rehabilitation.

c) NFB supportive qualities: Most subjects identified different affordances and supportive properties depending on the type of unimodal NFB they received. The kinesthetic modality appeared to enhance the embodiment of the sensations related to the motor task. Participants emphasized the tactile and physical aspects of this modality: P11 "With the tangible aspect, I felt feedback in terms of tactile sensation, and in addition, I was holding an object. For me, I could visualize the object that I could hold. I can see what it represents without visually seeing what it might look like. It is easier to imagine the sensations when you have an object and a feedback that render the pressure with which you have to imagine the movement", P13 "It was more fun because you could feel your own results", P23 "The kinesthetic feedback was perhaps the most concrete, because the response was also concrete", P15 "This feedback provided a kinesthetic sensation that helped me to call upon my muscle memory", P26 "The tangible allowed me to concentrate better on the sensations" P27 "I could imagine crushing what I had in my hands."

The visual feedback helped participants situate themselves within the task by enabling mental imagery and association with culturally familiar scenarios: P3 “The visual aspect allowed me to project myself into a story”, P5 “It made it possible to imagine an object and thus facilitate the movement”, P10 “Being able to see an actual situation helped me concentrate more on squeezing the ketchup bottle. However, it also introduced some confusion that may disorient the subject in the long term” P11 “I can see what my performance
<figure><img src="image_4.png" /><figcaption>Fig. 3. (A) AttrakDiff portfolio, (B) K-Uses mean scores; (C) Mental charge rated via NExT-Q according to the three types of NFB tested (V=Visual, K=Kinesthetic, B=Bimodal)</figcaption></figure>

represents without knowing exactly what I did.”, P27 “I imagined a visual action more often than a sensory action”.

The majority therefore emphasize their complementarity, which explains their significant preference for bimodality (P21 “The kinesthetic allowed me to visualize the movement better since my hand was already in the right position; the visual allowed me to imagine myself pressing on the ketchup bottle and there was perhaps an additional feeling of challenge”, P29 “The visual aspect helps you to visualize the task, while the kinesthetic aspect gives a tactile, physical sensation and is more motivating”). On the other hand, three subjects (P40, P15, and P19) indicate that they do not want to receive feedback to be able to concentrate and “clear their minds”, as it disturbs them.

## B. Motor Cortex Activity, BCI performances and EEG Power Modulation Across Feedback Conditions

1) **BCI performances:** Average actual performances (true positive rate) were of 72.1 (sd=17.7). More specifically, performances for bimodal feedback averaged 71.7 (sd=25), for visual NFB 72.7 (sd=21) and for kinesthetic NFB 71.2 (sd=23.2). There were no significant differences across the NFB (F=0.183, p=0.833).

2) **Topographic ERSP Analysis:** We analyzed topographic maps of event-related spectral perturbations (ERSPs) in the 8–30 Hz frequency band across the three feedback conditions (visual, kinesthetic, and bimodal), focusing on two time windows: during the kinesthetic motor imagery (KMI) task (1.5–4s) and the post-KMI feedback period (4.5–6s).

During the KMI phase, all conditions elicited a clear desynchronization (ERD) over the contralateral motor cortex, particularly around electrode C4, consistent with left-hand KMI. This pattern confirms that participants successfully engaged in the task and that the motor cortex was appropriately activated. No significant differences were observed between feedback conditions during this activation period (p<0.05 with FDR correction), suggesting that feedback modality did not influence task-related motor activation.

<figure><img src="image_5.png" /><figcaption>Fig. 4. ERSP during (above) and after (below) KMI, in the 8-30 Hz frequency bands, for bimodal, kinesthetic and visual NFB, and attached statistics.</figcaption></figure>

To further explore the effects of feedback, we examined ERSPs during the feedback period following the motor imagery task (4.5–6 s). Again, no significant differences were found between the visual, kinesthetic, and bimodal conditions. Complementary analyses in the mu (8–13 Hz) and beta (13–30 Hz) bands also revealed no statistically significant differences between modalities for both time-window periods.

3) **Time-Frequency ERSP Analysis:** Time-frequency representations from electrode C4 showed slightly stronger desynchronization in the mu band during the visual feedback condition (Figure 5). This effect is likely related to higher baseline mu activity in this condition. For the kinesthetic and bimodal feedback conditions, ERSP profiles were similar, and no notable differences were observed in the post-KMI period.

Overall, these findings confirm that KMI effectively activated the expected motor regions in all feedback conditions, but the nature of the feedback itself did not significantly modulate sensorimotor rhythm dynamics, either during or after the task.
<figure><img src="image_6.png" /><figcaption>Fig. 5. Time-frequencies of C4, C3 and all electrodes, for bimodal, kinesthetic and visual NFB, and attached statistics</figcaption></figure>

4) *Baseline Power Analysis:* To ensure that the observed ERSP modulations were not confounded by baseline differences across feedback conditions, we conducted an additional analysis of absolute EEG power (8–30 Hz) during three key time windows: (1) the pre-trial rest period used for baseline correction in the ERSP computation, (2) the motor imagery (KMI) period, and (3) the post-KMI feedback period. A one-way ANOVA revealed no significant differences in signal power across the three feedback modalities (visual, kinesthetic, bimodal) during the baseline window. Furthermore, no significant differences were found during either the KMI or the feedback periods. These results confirm that the ERSP effects reported above are unlikely to be artifacts of baseline shifts induced by the feedback modality.

Importantly, prior studies have shown that the choice of baseline can significantly influence the interpretation of ERD, especially when the baseline period itself contains task-related desynchronization or anticipatory activity, potentially leading to misleading conclusions [25]. In our case, the absence of power differences across baseline periods strengthens the validity of our ERSP comparisons and confirms that the feedback condition did not systematically alter the underlying EEG power during rest, task, or feedback phases.

### C. Unexpected results

We observed significant correlations between the actual performance of KMI, as measured by EEG – not the randomly displayed feedback –, and the feeling of estimated performance (r=0.407, p=0.021), satisfaction (r=0.502, p=0.003) and ease of use (r= -0.502, p=0.003) reported via the Kuses. Surprisingly, actual performance positively correlated with how participants estimated their performance when they received visual feedback (V: r=0.670, p<0.001) and satisfaction (V: r=0.636, p<0.001; B: r=0.467, p=0.007), despite not receiving a score proportional to their actual performance.

On the other hand, the better a subject performed, the less they found the system easy to use, regardless of the NFB modality (V: r= -0.528, p=0.002; K: r= -0.400, p=0.023; B: r= -0.516, p=0.002). This suggests, that either the strength of gamification generates assessment bias and/or the subjects can intuitively situate themselves and self-evaluate their performance. The latter possibility seems probable. Notably, participants observed the unpredictability of the NFB, which was seen as a shortcoming within the AttrakDiff evaluation. This observation is confirmed by NEXt-Q scores across all NFB modalities and may be attributed to random NFBs, irrespective of their actual performance. This could explain the frustration of not really “figuring out how to make it happen every time”. Indeed, some subjects reported inconsistent feedback during debriefing (P24 "It's strange, I always did [the KMI] in the same way and yet I never got the same scores.", P25 - “I didn't always understand why I sometimes succeeded very well and sometimes not at all, even though I seemed to be very focused on the task: I don't know if it was a bug or just me doing the exercise badly”). Interestingly, these subjects performed consistently well in KMI across all NFB modalities (mean EEG scores >80%). These results confirm preliminary work indicating that in KMI-based BCIs, subjects attempt to perform via inferences [6], [5]. This inferential process can sometimes mislead subjects, as evidenced by their comments: (e.g., P4 "When I imagine all the way the green light it works better than just occasionally. [...] I'm torn, on the one hand it can help, on the other hand I feel like I've been less successful", P27 “I knew why I was failing, I was imagining things too much visually rather than by feeling, but it was still difficult to correct myself"). Our subsequent study, involving subjects who receive signals related to their effective KMI performance, will address this assumption. These aspects also reinforce the importance of providing accurate, reliable, and trustworthy NFB, which is at the heart of the issues regarding BCI accuracy [26]. Regarding some studies' results [27], it would be interesting and useful to deeper investigate the effects of biased – positive or negative – feedback on the KMI task, related to the participants' mental states.

## IV. CONCLUSION

The kinesthetic NFB, provided by Sense-It, appeared as comfortable and relevant as visual NFB in our KMI-based BCI. Moreover, due to the qualitative fusion between visual and kinesthetic NFB, multimodality opens a promising way to enhance user experience and more effectively engage the motor cortex. Thanks to these HCI cross-modalities, the proposed BCI setup seems to make KMI more accessible and intuitive, rendering it more tangible through object-related affordances, culturally and physically situated, embodiment-based sensory stimulations, and motivational challenges through gamification. These findings validate the design choices and reinforce the importance of human-centered design-based approaches in BCI development.

Despite the absence of significant differences in motor cortex activation or in online performance across feedback
conditions, the overall findings highlight the potential of combining sensory modalities to enhance user engagement and task comprehension. This proof of concept supports the development of modular and user-centered rehabilitation strategies, where the adaptability and perceived quality of neurofeedback play a central role in fostering motivation and sustained use. Future works will study comparisons with other feedback modalities more often used in neurorehabilitation (e.g., functional electric stimulation).

## ACKNOWLEDGMENTS

This study was financially supported by ANR-19-CE33-0007. The authors would like to sincerely thank all the study's participants. We would also like to thank Nathalie Skiba, Michel Guillaume, and Kevin Renaudot from Alchime-Open Edge, who were involved in the design of Sense-It; Octarina society and Isabelle Ciré, who was involved in the design of the BCI games, and Laurent Bougrain for his support and the loan of EEG equipment.

## REFERENCES

[1] L. Rafsten, C. Meirelles, A. Danielsson, and K. S. Sunnerhagen, "Impaired motor function in the affected arm predicts impaired postural balance after stroke: A cross sectional study," *Frontiers in Neurology*, vol. 10, 2019.

[2] C. D. Guerrero-Mendez, H. Rivera-Flor, A. C. Villa-Parra, and T. F. Bastos-Filho, "Exploring novel practical approach to post-stroke upper-limb neurorehabilitation based on complex motor imagery tasks," in *46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)*, pp. 1-6, IEEE, 2024.

[3] J. O'Brien, R. Martyn Bracewell, and J. A. Castillo-Martínez, "The role of kinaesthetic motor imagery in promoting activation of the lumbrical muscles in the hemiparetic hand: a randomised controlled study," *Revista ciencias de la salud*, vol. 9, no. 1, pp. 5-16, 2011.

[4] A. Guillot, C. Collet, V. A. Nguyen, F. Malouin, C. Richards, and J. Doyon, "Brain activity during visual versus kinesthetic imagery: an fMRI study," *Human Brain Mapping*, vol. 30, pp. 2157-2172, July 2009.

[5] S. Rimbert and S. Fleck, "Long-term kinesthetic motor imagery practice with a BCI: Impacts on user experience, motor cortex oscillations and BCI performances," *Computers in Human Behavior*, vol. 146, p. 107789, Sept. 2023.

[6] S. Rimbert, L. Bougrain, and S. Fleck, "Learning How to Generate Kinesthetic Motor Imagery Using a BCI-based Learning Environment: a Comparative Study Based on Guided or Trial-and-Error Approaches," in *SMC 2020 - International IEEE Conference on Systems, Man and Cybernetics*, Oct. 2020.

[7] S. Le Franc, G. Herrera Altamira, M. Guillen, S. Butet, S. Fleck, A. Lécuyer, L. Bougrain, and I. Bonan, "Toward an adapted neurofeedback for post-stroke motor rehabilitation: State of the art and perspectives," *Frontiers in Human Neuroscience*, vol. 16, 2022.

[8] M. Fleury, G. Lioi, C. Barillot, and A. Lécuyer, "A survey on the use of haptic feedback for brain-computer interfaces and neurofeedback," *Frontiers in Neuroscience*, vol. 14, 2020.

[9] W. Wider, J. A. Mutang, B. S. Chua, N. T. P. Pang, L. Jiang, M. A. Fauzi, and L. N. Udang, "Mapping the evolution of neurofeedback research: a bibliometric analysis of trends and future directions," *Frontiers in Human Neuroscience*, vol. 18, 2024.

[10] N. Guo, X. Wang, D. Duanmu, X. Huang, X. Li, Y. Fan, H. Li, Y. Liu, E. H. K. Yeung, M. K. T. To, J. Gu, F. Wan, and Y. Hu, "SSVEP-based brain computer interface controlled soft robotic glove for post-stroke hand function rehabilitation," *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, vol. 30, pp. 1737-1744, 2022. Publisher: Institute of Electrical and Electronics Engineers (IEEE).

[11] N. Cheng, K. S. Phua, H. S. Lai, P. K. Tam, K. Y. Tang, K. K. Cheng, R. C.-H. Yeow, K. K. Ang, C. Guan, and J. H. Lim, "Brain-computer interface-based soft robotic glove rehabilitation for stroke," *IEEE Transactions on Biomedical Engineering*, vol. 67, no. 12, pp. 3339-3351, 2020. Publisher: Institute of Electrical and Electronics Engineers (IEEE).

[12] A. S. Association, "Spasticity." Available: https://www.stroke.org/en/about-stroke/effects-of-stroke/physical-effects/spasticity, 2023. [Accessed: 2025-07-18].

[13] G. Herrera Altamira, N. Skiba, A. Lécuyer, L. Bougrain, and S. Fleck, "Multisensory neurofeedback design for KMI embodiment," in *Proceedings of the 2023 ACM International Conference on Interactive Media Experiences Workshops*, IMXw '23, (New York, NY, USA), pp. 56-58, Association for Computing Machinery, Oct. 2023.

[14] G. Ffurtscheller and F. H. Lopes da Silva, "Event-related EEG/MEG synchronization and desynchronization: basic principles," *Clin Neurophysiol*, vol. 110, pp. 1842-57, Nov 1999.

[15] F. Lotte and M. Congedo, *Brain-computer interfaces. Vol 1, foundations and methods*, ch. EEG Feature Extraction, pp. 130-131. London ISTE Ltd Hoboken, 2016.

[16] J. Bismuth, F. Vialatte, and J.-P. Lefaucheur, "Relieving peripheral neuropathic pain by increasing the power-ratio of low-β over high-β activities in the central cortical region with EEG-based neurofeedback: Study protocol for a controlled pilot trial (SMRPain study)," *Neurophysiologie Clinique = Clinical Neurophysiology*, vol. 50, pp. 5-20, Feb. 2020.

[17] C. Lallemand, V. Koenig, G. Gronier, and R. Martin, "Création et validation d'une version française du questionnaire AttrakDiff pour l'évaluation de l'expérience utilisateur des systèmes interactifs," *European Review of Applied Psychology*, vol. 65, pp. 239-252, Sept. 2015.

[18] M. Minge, M. Thüring, I. Wagner, and C. V. Kuhr, "The mecue questionnaire: A modular tool for measuring user experience," in *Advances in Ergonomics Modeling, Usability & Special Populations (M. Soares, C. Falcão, and T. Z. Ahram, eds.), (Cham)*, pp. 115-128, Springer International Publishing, 2017.

[19] M. P. Davis, D. Khoshknabi, D. Walsh, R. Lagman, M. T. Karafa, A. Aktas, and A. Platt, "Four-item fatigue screen: replacing the Brief Fatigue Index," *The American Journal of Hospice & Palliative Care*, vol. 30, pp. 652-656, Nov. 2013.

[20] M. Duffy, S. Lajoie, R. Pekrun, and K. Lachapelle, "Emotions in medical education: Examining the validity of the Medical Emotion Scale (MES) across authentic medical learning environments," *Learning and Instruction*, Sept. 2018.

[21] S. G. Hart and L. E. Staveland, "Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research," in *Advances in Psychology (P. A. Hancock and N. Meshkati, eds.), vol. 52 of Human Mental Workload*, pp. 139-183, North-Holland, Jan. 1988.

[22] R. Viau, *La motivation dans l'apprentissage du français*. L' Ecole en Mouvement Series, ERPI, 1999.

[23] C. Baraudon, J.-B. Lanfranchi, J. M. C. Bastien, and S. Fleck, "Conception d'une échelle française d'évaluation de l'utilisabilité des nouvelles technologies éducatives par l'enfant," *Médiations et médiatisations*, pp. 44-67, Jan. 2021. Number: 5.

[24] A. Delorme and S. Makeig, "Eeglab: an open source toolbox for analysis of single-trial eeg dynamics including independent component analysis," *Journal of Neuroscience Methods*, vol. 134, no. 1, pp. 9-21, 2004.

[25] S. Rimbert, D. Trocellier, and F. Lotte, "Impact of the baseline temporal selection on the ERD/ERS analysis for Motor Imagery-based BCI," in *EMBC 2023 - 45th Annual International Conference of the IEEE Engineering in Medicine and Biology Society*, (Sydney, Australia), July 2023.

[26] C. A. Stefano Filho, R. Attux, and G. Castellano, "Actual, sham and no-feedback effects in motor imagery practice," *Biomedical Signal Processing and Control*, vol. 71, p. 103262, 2022.

[27] J. Mladenovic, J. Frey, S. Pramij, J. Mattout, and F. Lotte, "Towards Identifying Optimal Biased Feedback for Various User States and Traits in Motor Imagery BCI," *IEEE Transactions on bio-medical engineering*, vol. 69, pp. 1101-1110, Mar. 2022.