# Open design of a reproducible videogame controller for MRI and MEG

Yann Harel, André Cyr, Julie Boyle, Basile Pinsard, Jeremy Bernard, Marie-France Fourcade, Himanshu Aggarwal, Ana Fernanda Ponce, Bertrand Thirion, Karim Jerbi, et al.

<figure><img src="image_1.png" /></figure>

To cite this version:

Yann Harel, André Cyr, Julie Boyle, Basile Pinsard, Jeremy Bernard, et al. Open design of a reproducible videogame controller for MRI and MEG. PLoS ONE, 2023, 18 (11), pp.e0290158. 10.1371/journal.pone.0290158. hal-04850878

HAL Id: hal-04850878

https://inria.hal.science/hal-04850878v1

Submitted on 20 Dec 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
RESEARCH ARTICLE

# Open design of a reproducible videogame controller for MRI and MEG

Yann Harel<sup>1,2,*</sup>, André Cyr<sup>1,3&</sup>, Julie Boyle<sup>1,2,3</sup>, Basile Pinsard<sup>1,3</sup>, Jeremy Bernard<sup>4</sup>, Marie-France Fourcade<sup>4</sup>, Himanshu Aggarwal<sup>5</sup>, Ana Fernanda Ponce<sup>5</sup>, Bertrand Thirion<sup>1,5</sup>, Karim Jerbi<sup>2,3,6,7</sup>, Pierre Bellec<sup>1,2,3</sup>

1 Centre de Recherche de l'Institut Universitaire de Gériatrie de Montréal, Montréal, Canada, 2 Psychology department, University of Montréal, Montréal, Canada, 3 Unité de Neuroimagerie Fonctionnelle, Centre de Recherche de l'Institut Universitaire de Gériatrie de Montréal, Montréal, Canada, 4 Neurospin, CEA, Gif-sur-Yvette, France, 5 Inria, CEA, Université Paris-Saclay, Palaiseau, France, 6 MILA, Montréal, Canada, 7 MEG Imaging Center, University of Montréal, Montréal, Canada

© These authors contributed equally to this work.

* yann.harel@umontreal.ca

<figure><img src="image_3.png" /></figure>

## Abstract

Videogames are emerging as a promising experimental paradigm in neuroimaging. Acquiring gameplay in a scanner remains challenging due to the lack of a scanner-compatible videogame controller that provides a similar experience to standard, commercial devices. In this paper, we introduce a videogame controller designed for use in the functional magnetic resonance imaging as well as magnetoencephalography. The controller is made exclusively of 3D-printed and commercially available parts. We evaluated the quality of our controller by comparing it to a non-MRI compatible controller that was kept outside the scanner. The comparison of response latencies showed reliable button press accuracies of adequate precision. Comparison of the subjects' motion during fMRI recordings of various tasks showed that the use of our controller did not increase the amount of motion produced compared to a regular MR compatible button press box. Motion levels during an ecological videogame task were of moderate amplitude. In addition, we found that the controller only had marginal effect on temporal SNR in fMRI, as well as on covariance between sensors in MEG, as expected due to the use of non-magnetic building materials. Finally, the reproducibility of the controller was demonstrated by having team members who were not involved in the design build a reproduction using only the documentation. This new videogame controller opens new avenues for ecological tasks in fMRI, including challenging videogames and more generally tasks with complex responses. The detailed controller documentation and build instructions are released under an Open Source Hardware license to increase accessibility, and reproducibility and enable the neuroimaging research community to improve or modify the controller for future experiments.

<figure><img src="image_4.png" /></figure>

### OPEN ACCESS

**Citation:** Harel Y, Cyr A, Boyle J, Pinsard B, Bernard J, Fourcade M-F, et al. (2023) Open design of a reproducible videogame controller for MRI and MEG. PLoS ONE 18(11): e0290158. https://doi.org/10.1371/journal.pone.0290158

**Editor:** Bradley R. King, University of Utah, UNITED STATES

**Received:** May 12, 2023

**Accepted:** August 3, 2023

**Published:** November 1, 2023

**Peer Review History:** PLOS recognizes the benefits of transparency in the peer review process; therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. The editorial history of this article is available here: https://doi.org/10.1371/journal.pone.0290158

**Copyright:** © 2023 Harel et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

**Data Availability Statement:** All the data used in this study are made available as part of the Courtois-Neuromod project. Access can be obtained by filling a short form at https://www.cneuromod.ca/access/access/. The preprocessed

## Introduction

Videogames offer a wide range of rich and dynamic environments which open exciting avenues for research in cognitive neuroscience [1]. Major neuroimaging techniques such as
data used to generate the figure is available in the
Supplementary Information.

**Funding:** This project was supported by the Courtois Fondation (https://www.charitydata.ca/charity/fondation-courtois/850271289RR0001/), grant awarded to PB for the Courtois NeuroMod Project 2018-2024 (https://www.cneuromod.ca/). PB is a senior fellow ("chercheur boursier senior") of the "Fonds de recherche du Québec - Santé" (https://frq.gouv.qc.ca/). KJ is supported by funding from the Canada Research Chairs (950-232368; https://www.chairs-chaires.gc.ca/home-accueil-eng.aspx) program and a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada (2021-03426; https://www.nserc-crsg gc.gc.ca/index_eng.asp). The reproduction of the controller by the MIND team, Inria, France was supported by funds from the European Union's Horizon 2020 Framework Programme for Research and Innovation (https://research-and-innovation.ec.europa.eu/funding/funding-opportunities/funding-programmes-and-open-calls/horizon-2020_en) under the Specific Grant Agreement No. 945539 (Human Brain Project SGA3) awarded to BT and through the joint Inria "NeuroMind" team grant to PB, KJ, BT and Alexandre Gramfort. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

**Competing interests:** The authors have declared
that no competing interests exist.

functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) adversely interact with any electronic equipment, which precludes the use of traditional videogame controllers during neuroimaging sessions. This paper reports on a videogame controller compatible with fMRI and MEG systems, developed by the team of the Courtois project on neuronal modeling (CNeuroMod). We evaluated the performance of this new device by comparing it to a regular, commercially available, videogame controller. We also assessed possible interactions between the controller and the quality of fMRI and MEG data acquisitions. In the spirit of open science, we released publicly the plans and documentation for building this controller, and confirmed that an independent team was able to build a reproduction based on the plans and documentation.

Neuroimaging videogames studies have examined how brain networks interact to produce a variety of complex behaviors such as aggression [2–5], learning [6, 7], expertise [8, 9] and flow [10]. Videogames often recruit many complementary abilities at once, e.g. spatial navigation and decision making in first-person shooters, thus probing the integration of diverse cognitive domains within a single environment [11]. Brain activity during videogame play is actively being studied with EEG (e.g. [12]) and fNIRS (e.g. [13]), two neuroimaging techniques that can be used with a traditional videogame setup, but only a handful of studies have attempted to use fMRI or MEG (e.g. [14]) to acquire high-quality functional brain data. Experiments conducted in MRI and MEG scanners need compatible response devices that do not create major acquisition artifacts or safety concerns [15]. These requirements prevent the use of commercially available controllers in research conducted using fMRI and MEG. Although possible to shield electronics [16], such a procedure only partially alleviates concerns on signal quality and possible safety concerns in an MRI environment. Videogames collected during fMRI sessions have relied on non-ecological MRI-compatible response devices such as button pads [2, 4–6, 17, 18], joysticks [19, 20], trackwheel mouse [18], and data gloves [6, 8]. These devices do not necessarily match those normally used to play the corresponding videogames, for example a button pad instead of a gamepad [17], or a trackball [18] instead of a mouse. Atypical devices may negatively impact the experience and performance of players in a brain scanner, and severely limit the type of games which can effectively be played within a scanning environment.

A commercial, fully MRI-compatible handheld videogame controller was recently proposed by Current Designs Inc., Philadelphia, PA, USA, requiring the separate acquisition of an electronics interface and a fiber optics cord extension, resulting in significant costs. An alternative approach to commercial solutions for research has however emerged: Open Source Hardware (OSH), made possible by the increasing accessibility of 3D printing, engineering parts, and modular micro-electronics [21]. Inspired by the Free and Open Source Software philosophy, OSH refers to technological artifacts for which the design files, documentation, and required software are freely available under an open-source license specifically dedicated to sharing hardware products. This approach provides many benefits, in line with the principles of open science: promoting accessibility and reproducibility, significantly reducing construction costs, and allowing for iterative design and customization.

The aim of this study was to develop and validate a videogame controller that is both MRI- and MEG-compatible and can be reproduced by other neuroimaging research teams. This was achieved by making detailed documentation and design files accessible to allow for 3D printing and assembly. The controller was evaluated by comparing its responsiveness to a reference controller used in a mock scanner, in terms of button press and release latencies in a cued-response experiment. We also checked the impact of the CNeuroMod controller usage on subject motion, using frame displacement metrics derived from fMRI acquisitions. Finally, we acquired phantom MRI scans as well as MEG empty room data to determine if any change in
background noise could be attributed to the controller's presence in the scanning environment.

# Methods

## Controller

The documentation for building the controller is provided in the Appendix of this article and can also be found at https://controller-doc.readthedocs.io/en/latest/. It contains a Bill of Materials (BOM) of the commercial parts necessary to build the controller, detailed assembly instructions, source code for the microcontroller, and a download section for the 3D models and Gerber files. The 3D model files are available either as .STL files, which can be readily sent to a 3D printer, or as. STEP files for manipulation and modification using Computer-Aided Design (CAD) software. Gerber files describe the different layers of the Printed Circuit Board (PCB) that need to be sent to a manufacturer for production. The buttons layout of the controller is presented in Fig 1.

**Design choices.** The design of handheld videogame controllers has been heavily influenced by the early model of the Super Nintendo Entertainment System (SNES™, Nintendo). This controller features two sets of buttons: one for the left hand which consists of four arrow-like keys intended for spatial navigation, and another set of four buttons for the right hand offering various types of game-specific actions. The combination of these eight buttons is sufficient for players to explore rich action spaces and covers the basic requirements of a vast array of videogame titles, especially retro videogames (that is, games released on consoles of the generations preceding the PlayStation™ and XBox™ systems).

<figure><img src="image_5.png" /><figcaption>**Fig 1.** Models and pictures of the different controller parts. A Split view of the various 3D-printed parts B 3D models of the assembled switch part. C 3D model of the PCB. D 3D model of the assembled controller. E-H Real pictures of the corresponding elements as assembled at our facility.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0290158.g001
The controller was designed with the six following specific criteria in mind. Accordingly, the handheld part of the controller must:

1. be built using a 3D printer and commercially available parts in order for it to be easily reproducible,

2. be entirely void of electronics and ferro-magnetic components to ensure MRI- and MEG-compatibility,

3. reliably register all button presses with a temporal precision comparable to that of the reference controller,

4. mimic the feeling and comfort of a reference commercial videogame controller, the Super NES controller, a device held with both hands each manipulating a set of 4 buttons using the thumbs,

5. allow users to comfortably play while lying down (position as imposed by MRI scanner and possible in some MEG scanners),

6. be comfortable to use for individuals with different hand sizes.

**Controller system.** The device is made up of two parts linked by a bundle of fiber optics (Fig 2): the first is the actual handheld controller, and the second is a printed circuit board (PCB) that allows communication between the controller and a stimulation computer. The handheld controller and fiber optics bundle are MRI and MEG-compatible, and as such can be used in the MRI room or MEG chamber while the PCB contains unshielded electronics and must remain in the control room. Our pilot controller featured a 10m long fiber optics cable, which was sufficient to connect the controller in the MRI room with the PCB in the control room.

**Software.** A Teensy 3.5 (Teensy®) microcontroller board was programmed to parse the optical signals and manage communication between the PCB and a stimulation computer. Programming was done in the Arduino Integrated Development Environment (IDE; Arduino) using the Teensyduino add-on (https://www.pjrc.com/teensy/teensyduino.html), which allows programming Teensy family boards using most of the Arduino libraries. This software must be uploaded to the Teensy board located atop the PCB.

## Task design

The datasets used in this study come from the Courtois NeuroMod databank (https://www.cneuromod.ca/gallery/datasets/). For the purpose of this project, we only utilized data from 4 subjects of 6 participants available in the databank, specifically the 4 subjects that performed the gamepad task. The code used for data analysis is available at https://doi.org/10.5281/zenodo.7847577 (or on GitHub https://github.com/courtois-neuromod/controller_validation).

**Subjects.** Four participants (2 female) were scanned while they performed a variety of tasks in the MEG and MRI scanners. All four subjects were right-handed and had normal hearing for their age, ages ranged between 31 and 47 at first data set acquisitions in 2018 (i.e. *hcptrt* dataset), and the timespan between first and last dataset acquisition (i.e. *gamepad* dataset, see *Control datasets* section) was approximately 2 years and 7 months. Subjects were recruited as part of the Courtois NeuroMod project (CNeuroMod), a dense fMRI database that is being collected on 6 subjects using both naturalistic and controlled stimuli from a broad range of cognitive domains and the main motivator for building the controller. Although the
<figure><img src="image_6.png" /><figcaption>**Fig 2.** Schematics of the two main parts of the controller. The PCB is installed in the control room, while the handheld part is MRI-compatible and can be used in the MRI scanning room.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0290158.g002

participants of the *gamepad* task do not properly reflect the population of videogame players, we expected their behavior to be illustrative of the controllers capability as the task didn't include videogame components per se.

**Ethics statement.** The study was approved by the "Comité d'éthique de la recherche vieillissement neuroimagerie" of the CIUSSS du Centre-Sud-de-l'île-de-Montréal (CIUSSS-CSMTL) under the number CER VN 18-19-22 and all participants provided written consent.

**Utility ratings.** A subjective evaluation of the controller in accordance with ISO-9241 on device comfort was assessed via a questionnaire of 8 items answered on a 5-points Likert scale, adapted from [22]. For the items smoothness during operation, general comfort and overall useability, higher scores indicate better fitness. For the items finger and wrist fatigue, lower scores indicate better fitness. For all other items, better fitness is indicated by scores around 3. All subjects of the CNeuroMod sample answered the questionnaire after having completed the totality of the validation task.

**MRI acquisition.** fMRI data used in this paper were collected as part of the CNeuroMod project using a 3T Siemens Prima Fit MR scanner, a 64-channel head coil, and an accelerated
simultaneous multi-slice imaging sequence [23]. The spatial resolution was 2mm isotropic, and the TR was 1.49s. All subjects wore custom CaseForge [24] head cases to minimize motion. Visual stimuli were projected onto a screen via a waveguide and presented to participants on a mirror mounted on the head coil, and sound was delivered using S15 Sensimetrics headphone inserts. Additionally an anatomical scan was collected using a T1-weighted MPRAGE 3D sagittal sequence, (TR = 2.4 s, TE = 2.2 ms, flip angle = 8 deg, voxel size = 0.8 mm). fMRI and anatomical data were preprocessed using the fMRIprep pipeline [25], using the “long term support” version 20.2.3. For a more detailed description of the MRI or fMRI sequences and set up visit the CNeuroMod’s documentation page: (www/docs.cneuromod.ca/en).

**Gamepad task.** We set out to evaluate the reliability of button responses of our custom controller by comparing it to that of a commercially available, non-MRI-compatible controller. Thus, this experiment consisted of a basic cued-response task implemented in PsychoPy [21] that was administered in two setups in a pseudo-randomized fashion: either in the MRI scanner using our homemade MRI-compatible controller, or in a mock scanner (i.e. a replica of a MRI scanner that does not contain a magnet) equipped with a generic commercial controller modeled after the classic SNES™ controller.

In the MRI setup our MRI-compatible controller was installed in the MRI scanner and connected through a waveguide to the controller interface, located in the control room. The controller interface was in turn connected to the stimulation computer via USB to register and log key presses during the task. The remaining set up was exactly as described above in the section on fMRI acquisition (Fig 3).

In the mock setup we used an MRI simulator made by Psychology Software Tools (https://pstnet.com/products/mri-simulator/), and used an in-house 3D printed replica of the Siemens 64 channel head coil used in the MRI. An in-house built mirror mount replicating the Siemens' mirror attached to the head coil was used to present images to the participants. Instead of a projection, visual stimuli were presented using a small monitor placed within the mock bore, closely replicating the manner in which visual stimuli are presented in the MR scanner. A replica of the SNES™ controller (iNNEXT Retro USB Super Classic Controller, iNNEXT) was installed inside the mock scanner, and directly connected to the stimulation computer via USB (Fig 3).

Participants were instructed they would have to perform a cued-response task, in which, using visual cues, they would have to press one of the 8 buttons present on either the MRI or SNES controllers, for a short or long period of time. Short presses were indicated by the

<figure><img src="image_7.png" /><figcaption>**Fig 3.** Schematics of the MRI and Mock setups. The mock setup closely replicated the MRI setup while allowing the use of a commercial controller instead of our MRI-compatible controller. Icons obtained under CC license at https://thenounproject.com/.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0290158.g003
Project [29] and each task was repeated up to 15 times. The resting state task was repeated 4 or 5 times per subject and consisted of staring at a cross (no response). The gambling task was repeated between 15 and 17 times per subject; using a standard MRI response box subjects were instructed to guess if they thought a hidden number (represented by a “?”) was above or below 5.

The motor task was repeated between 15 and 18 times per subject. In this task, participants were presented a visual cue and were asked move a body part (ie. tongue, fingers, toes, hand, feet) in response to that cue. We expected these three datasets to reflect relatively low, moderate and high levels of motion for the resting, gambling and motor tasks respectively.

Lastly we used fMRI data from 3 of the participants in the *shinobi* dataset, which consists of about 10h (approximately 56 runs) of fMRI data per subject. During the shinobi task, subjects played three selected levels of Shinobi III: Return Of The Ninja Master [26], a retro videogame released on Sega Genesis™, using our custom videogame controller. We expected this last dataset to reflect the amount of motion typically observed during fMRI acquisitions involving videogame play with the CNeuroMod controller.

For more information on each of the control datasets, see the Courtois NeuroMod's Project documentation page: (https://docs.cneuromod.ca/en/latest/).

**fMRI phantom recordings.** Thirty runs of 60 volumes were acquired using a fBIRN gel phantom with the custom controller in the scanner, and 30 more without the controller, all following the imaging sequence described in the *MRI acquisition* section. Each run lasted about 2 minutes, and a total of 60 runs were acquired on a single day.

**MEG empty room recordings.** Empty room noise recordings were acquired as part of the routine procedure for MEG acquisitions. The data was collected on separate days with a MEG CTF-275 (with 270 functioning channels) while no participant was inside the MEG shielded room. In each recording, 35 seconds of signal were acquired at a sampling frequency of 1200Hz. No offset removal nor filters were used during these acquisitions. We used the empty room recordings acquired between 02/04/2019 and 09/06/2021 (N = 80), and split them into 4 subsets of 20 consecutive recordings. These subsets, named T1, T2, T3 and T4 include data acquired before the installation of our custom controller (T1, T2 and T3), and data acquired after the installation of our controller at the MEG facility (T4). Each of these subsets contained recordings spanning over between 1 and 3 and a half months.

## Analyses

**Response latencies (custom controller VS SNES controller).** In order to verify that our controller had the same performance capacity as the commercial controller, we tested the hypothesis of a greater response latency in the MRI (using our controller) than in the mock scanner (using the commercial reference controller). We removed: (a) trials for which no button press of the instructed key was recorded; (b) trials with negative reaction times and durations, which correspond to trials that were anticipated by the subject due to the task's pseudorandom sequence; and (c) trials for which the press timing was recorded but no timing for the release was recorded due to an overlap with the subsequent trial. Additionally, we removed (d) trials from the “long-press” condition for which the press duration was below 1s, as these trials might have been mistakenly performed as “short-press” trials. In total, we removed 71 trials of the mock condition and 50 trials of the MRI condition (Table 1). Statistical analyses were conducted on pooled reaction times, and on press duration for long and short trials separately. One-tailed Kolmogorov-Smirnov tests for independent samples with FDR (Benjamini-Hochberg) correction for multiple comparisons were conducted between the two setups. Effects size were estimated using Cohen's D.
<table><caption>Table 1. Number of removed trials at each step of the preprocessing pipeline.</caption><thead><tr><th></th><th></th><th>A</th><th>B</th><th>X</th><th>Y</th><th>Up</th><th>Down</th><th>Left</th><th>Right</th><th>Total</th></tr></thead><tbody><tr><td rowspan="2">Mistakes</td><td>Mock</td><td>3</td><td>3</td><td>8</td><td>4</td><td>9</td><td>7</td><td>6</td><td>7</td><td>47</td></tr><tr><td>MRI</td><td>6</td><td>3</td><td>0</td><td>3</td><td>6</td><td>3</td><td>3</td><td>2</td><td>26</td></tr><tr><td rowspan="2">Negative RTs</td><td>Mock</td><td>0</td><td>2</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>3</td></tr><tr><td>MRI</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td><td>1</td><td>0</td><td>0</td><td>2</td></tr><tr><td rowspan="2">Negative durations</td><td>Mock</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>MRI</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><td rowspan="2">Delayed trials</td><td>Mock</td><td>0</td><td>1</td><td>2</td><td>0</td><td>2</td><td>1</td><td>0</td><td>1</td><td>7</td></tr><tr><td>MRI</td><td>0</td><td>0</td><td>0</td><td>0</td><td>7</td><td>3</td><td>0</td><td>1</td><td>11</td></tr><tr><td rowspan="2">Missed trials</td><td>Mock</td><td>3</td><td>2</td><td>1</td><td>2</td><td>3</td><td>2</td><td>0</td><td>1</td><td>14</td></tr><tr><td>MRI</td><td>1</td><td>3</td><td>1</td><td>1</td><td>3</td><td>0</td><td>1</td><td>0</td><td>10</td></tr><tr><td rowspan="2">Retained trials</td><td>Mock</td><td>234</td><td>232</td><td>228</td><td>234</td><td>226</td><td>230</td><td>234</td><td>231</td><td>1849</td></tr><tr><td>MRI</td><td>213</td><td>213</td><td>219</td><td>216</td><td>203</td><td>213</td><td>216</td><td>217</td><td>1710</td></tr><tr><td rowspan="2">Total removed</td><td>Mock</td><td>6</td><td>8</td><td>12</td><td>6</td><td>14</td><td>10</td><td>6</td><td>9</td><td>71</td></tr><tr><td>MRI</td><td>7</td><td>7</td><td>1</td><td>4</td><td>17</td><td>7</td><td>4</td><td>3</td><td>50</td></tr></tbody></table>

https://doi.org/10.1371/journal.pone.0290158.t001

**Motion assessment (custom controller VS control tasks).** Framewise Displacement (FD) values were obtained from the confounds file generated by the fMRIprep preprocessing pipeline [25] applied to each of the datasets listed above. For each run of every dataset, the median of FD values were computed, then compared across datasets separately for each subject using two-tailed Kolmogorov-Smirnov tests for independent samples with FDR (Benjamini-Hochberg) correction for multiple comparisons. Specifically, we tested the differences between the *gamepad* dataset and the *friends*, *hcp-rest*, *hcp-gamble* and *hcp-motor* datasets for all 4 subjects, and the differences between the *shinobi* dataset and all others datasets for the 3 subjects that performed the *shinobi* task.

fMRI temporal SNR (with versus without controller). Temporal Signal-to-Noise Ratio (SNR) maps were computed on each run then averaged per condition (data from the tasks was divided into one of two conditions; with or without the controller present in the scanner) using Nipype [30]. For each voxel, the mean and standard deviation across the distribution of runs were computed and displayed separately for the two conditions. Additionally, we extracted the voxels located within an EPI mask (i.e. voxels located in the gel phantom) and compared the distributions of mean and standard deviation using independent T-Tests.

**MEG noise covariance (with versus without controller).** Noise covariance matrices were computed on each empty room acquisition run using the MNE-python toolbox [31]. Because covariance matrices are symmetric, only the upper part of the matrices were used for subsequent analysis. The covariance matrices of the 20 recordings of each subset (T1, T2, T3 and T4) were averaged separately and then Z-scored. The difference between these two distributions of covariances was assessed using a Wilcoxon signed-rank test. The Pearson correlation between the two distributions of values was computed to estimate the effect size of that difference. Finally, pairwise Pearson correlations were computed between individual recordings of each pair of subsets to compare the distributions of correlation coefficients across subsets.

# Results

## Utility ratings

The answers (N = 4) to each question of the utility ratings survey were averaged and inspected separately. All scales ranged from 1 to 5, although the interpretation of each score varies as
summarized below and detailed in the methods section. For smoothness during operation, the average rating was 4.75 (±0.5 std, higher is better). For the force required for actuation and the operation speed, the average rating was 3.25 (±0.5 std, a score of 3 is best). The mental effort required for operation was rated at 3 (±0 std, a score of 3 is best). Both finger and wrist fatigue were rated below 2 (2±1.15 std and 1.25±0.5 std respectively, lower is better). Finally, both general comfort and overall useability were rated above 4 (4.25±0.5 std and 4.75±0.5 std respectively, higher is better).

## Response latencies (custom controller versus reference controller)

Analysis of reaction times (Fig 5) showed no differences (all *p* > 0.05, see Table 2) between our MRI-compatible controller (MRI condition) and the commercial controller (Mock condition) when comparing each key separately. Analysis of release reaction times in long trials revealed significant (*p* < 0.05) differences for 6 of the 8 keys, showing that the MRI-compatible controller led to longer release reaction times (-0.417 < Cohen's *D* < -0.040 for all significant comparisons) than the reference controller. Analysis of press durations in short trials also showed significant differences (*p* < 0.05) for all keys, showing that the duration of short presses was longer using our MRI-compatible controller (-1.332 < Cohen's *D* < -0.300 for all comparisons).

For reference, the naturalistic key durations observed during play in the *shinobi*_training dataset (Table 3) ranged from an average of 164ms (for phasic actions that aren't impacted by keypress duration, such as hitting an enemy) to 1153ms for tonic actions that can be maintained over time, such as moving to the right.

## Motion assessment (custom controller versus control tasks)

Analysis of run-median FD values (Fig 6 and Table 4) showed no significant differences of motion between the *gamepad* dataset and the other datasets that didn't use our MRI-compatible controller, namely the *hcp-rest*, *hcp-motor* and *hcp-gamble* datasets and the *friends* dataset (all *p* > 0.05), with the exception of 2 subjects that showed significant differences between the *gamepad* and *friends* datasets (*p* < 0.01). For all 3 subjects, the *shinobi* dataset had significantly higher run-median FD values than all the other datasets (all *p* < 0.05), except for one subject that showed no significant difference between the *shinobi* dataset and the *hcptrt* resting-state dataset.

## fMRI temporal SNR

The temporal SNR maps showed similar distributions of means and standard deviations across the runs acquired with and without the presence of our controller in the MRI scanner (Fig 7). Comparing the average of voxels located within the EPI mask, the two distributions showed significant differences (both *p* < 1.00e-04; $M_{controller}$ = 80.88; $STD_{controller}$ = 0.88; $M_{nocontroller}$ = 78.11; $STD_{nocontroller}$ = 0.72) suggesting a slight decrease in noise levels after the controller was introduced in the MRI scanning room.

## MEG noise covariance

Analysis of the difference between averaged covariance matrices acquired before (T3) and after (T4) the controller installation (Fig 8) showed no significant difference (Wilcoxon signed-rank test, *p* = 0.139). In contrast, although the controller wasn't present in the scanner during recordings of both T1 and T2, the comparison between these two subsets led to a significant difference (Wilcoxon signed-rank test, *p* = 0.004), showing that the impact of
<figure><img src="image_8.png" /><figcaption>**Fig 5. Distributions of response latencies (pooled across subjects) in the MRI and Mock conditions.** ns: p < = 1.00e+00; \*: 1.00e-02 < p < = 5.00e-02; \*:\*: 1.00e-03 < p < = 1.00e-02; \*:\*:\*: 1.00e-04 < p < = 1.00e-03; \*:\*:\*:\*: p < = 1.00e-04. A Comparison of reaction times (ms). B Comparison of release reaction times (ms) for long trials only. C Comparison of total key press durations (ms) for short trials only.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0290158.g005

environmental fluctuations on noise levels could be detected by this method, while the controller's effect could not. In both cases (T1 vs T2 and T3 vs T4), the Pearson correlation between the two averaged matrices was fairly high (r = 0.901 and r = 0.953, respectively). Finally, the similarity of the distributions of pairwise correlations between individual recordings in the different subsets further suggests that the noise patterns observed after the
<table><caption>Table 2. Statistics table showing the comparisons between the MRI and Mock conditions.</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="3">Mock</th><th colspan="3">MRI</th><th colspan="3">Difference</th></tr><tr><th>N</th><th>Mean</th><th>STD</th><th>N</th><th>Mean</th><th>STD</th><th>KS</th><th>D</th><th>Mean</th></tr></thead><tbody><tr><td rowspan="7">Press reaction times—All trials (ms)</td><td>A</td><td>234</td><td>460</td><td>83</td><td>213</td><td>466</td><td>92</td><td>0.071 (ns)</td><td>-0.071</td><td>6.251</td></tr><tr><td>B</td><td>232</td><td>444</td><td>96</td><td>213</td><td>436</td><td>95</td><td>0.022 (ns)</td><td>0.084</td><td>8.012</td></tr><tr><td>Down</td><td>230</td><td>501</td><td>155</td><td>213</td><td>461</td><td>121</td><td>0.000 (ns)</td><td>0.293</td><td>40.675</td></tr><tr><td>Left</td><td>234</td><td>514</td><td>170</td><td>216</td><td>457</td><td>108</td><td>0.000 (ns)</td><td>0.397</td><td>56.612</td></tr><tr><td>Right</td><td>231</td><td>488</td><td>132</td><td>217</td><td>455</td><td>113</td><td>0.026 (ns)</td><td>0.266</td><td>32.773</td></tr><tr><td>Up</td><td>226</td><td>508</td><td>141</td><td>203</td><td>477</td><td>156</td><td>0.015 (ns)</td><td>0.207</td><td>30.625</td></tr><tr><td>X</td><td>228</td><td>473</td><td>142</td><td>219</td><td>452</td><td>109</td><td>0.020 (ns)</td><td>0.164</td><td>20.772</td></tr><tr><td rowspan="7">Release reaction times—Long trials (ms)</td><td>A</td><td>116</td><td>329</td><td>162</td><td>104</td><td>338</td><td>153</td><td>0.100 (ns)</td><td>-0.058</td><td>9.118</td></tr><tr><td>B</td><td>117</td><td>325</td><td>101</td><td>106</td><td>344</td><td>182</td><td>0.283***</td><td>-0.129</td><td>19.089</td></tr><tr><td>Down</td><td>113</td><td>330</td><td>113</td><td>106</td><td>360</td><td>158</td><td>0.312****</td><td>-0.215</td><td>29.518</td></tr><tr><td>Left</td><td>114</td><td>307</td><td>117</td><td>107</td><td>357</td><td>87</td><td>0.449****</td><td>-0.49</td><td>50.538</td></tr><tr><td>Right</td><td>112</td><td>295</td><td>152</td><td>107</td><td>325</td><td>127</td><td>0.284***</td><td>-0.22</td><td>30.775</td></tr><tr><td>Up</td><td>109</td><td>309</td><td>107</td><td>97</td><td>331</td><td>154</td><td>0.266***</td><td>-0.17</td><td>22.609</td></tr><tr><td>X</td><td>110</td><td>338</td><td>97</td><td>110</td><td>343</td><td>128</td><td>0.136 (ns)</td><td>-0.047</td><td>5.318</td></tr><tr><td rowspan="7">Press durations—Short trials (ms)</td><td>A</td><td>118</td><td>192</td><td>40</td><td>109</td><td>211</td><td>49</td><td>0.245***</td><td>-0.439</td><td>19.633</td></tr><tr><td>B</td><td>115</td><td>195</td><td>36</td><td>107</td><td>222</td><td>74</td><td>0.198*</td><td>-0.457</td><td>26.578</td></tr><tr><td>Down</td><td>117</td><td>189</td><td>46</td><td>107</td><td>213</td><td>57</td><td>0.293****</td><td>-0.47</td><td>24.418</td></tr><tr><td>Left</td><td>120</td><td>180</td><td>41</td><td>109</td><td>208</td><td>61</td><td>0.287****</td><td>-0.536</td><td>27.904</td></tr><tr><td>Right</td><td>119</td><td>176</td><td>48</td><td>110</td><td>236</td><td>62</td><td>0.580****</td><td>-1.078</td><td>59.652</td></tr><tr><td>Up</td><td>117</td><td>186</td><td>41</td><td>106</td><td>231</td><td>72</td><td>0.321****</td><td>-0.762</td><td>44.517</td></tr><tr><td>X</td><td>118</td><td>201</td><td>57</td><td>109</td><td>209</td><td>52</td><td>0.149 (ns)</td><td>-0.152</td><td>8.34</td></tr><tr><td>Y</td><td>118</td><td>202</td><td>50</td><td>109</td><td>222</td><td>45</td><td>0.341****</td><td>-0.412</td><td>19.595</td></tr></tbody></table>

One-tailed (MRI > Mock) Kolmogorov-Smirnov tests for independent samples, with FDR (Benjamini-Hocheberg) correction. KS: Kolmogorov-Smirnov statistics. D: Cohen's D. ns: $p < = 1.00e+00$;
*: $1.00e-02 < p < = 5.00e-02$;
**: $1.00e-03 < p < = 1.00e-02$;
***: $1.00e-04 < p < = 1.00e-03$;
****: $p < = 1.00e-04$.

https://doi.org/10.1371/journal.pone.0290158.t002

addition of our custom controller weren't altered more than expected by normal environmental fluctuations.

## Reproduction

A reproduction of the CNeuroMod controller was built by the engineering team at Neurospin and members of the MIND team at Inria (Paris-Saclay, France; Fig 9). The MIND team was not involved in the design process. The reproduced controller was built according to the design plans and documentation made available online and was fitted to their local MRI system. Minor modifications were implemented by the MIND team, for example the shutter and the cap were printed together instead of assembled after printing and the fiber optic cable was 12m long instead of 10m. Other minor assembly issues (buttons getting stuck) were reported and troubleshooted directly with AC, although other teams will also be able to obtain support via GitHub. The controller was tested by 9 different players during gameplay on Super Mario
<table><caption>**Table 3.** Descriptive statistics of keypress durations observed in the training phase of the *shinobi* dataset. These values reflect typical keypress durations when playing an action platformer videogame on a regular controller.</caption><thead><tr><th rowspan="2">Subject</th><th rowspan="2">Key</th><th colspan="8">Duration (ms)</th></tr><tr><th>Count</th><th>Mean</th><th>STD</th><th>Min</th><th>25%</th><th>50%</th><th>75%</th><th>Max</th></tr></thead><tbody><tr><td rowspan="6">sub-01</td><td>B (Hit)</td><td>27939</td><td>134</td><td>45</td><td>16</td><td>116</td><td>133</td><td>150</td><td>1400</td></tr><tr><td>Y (Jump)</td><td>29645</td><td>350</td><td>187</td><td>16</td><td>200</td><td>317</td><td>467</td><td>1484</td></tr><tr><td>Down</td><td>24710</td><td>469</td><td>423</td><td>16</td><td>233</td><td>350</td><td>567</td><td>9367</td></tr><tr><td>Left</td><td>10683</td><td>669</td><td>670</td><td>16</td><td>200</td><td>450</td><td>883</td><td>15134</td></tr><tr><td>Right</td><td>29014</td><td>1320</td><td>2123</td><td>16</td><td>133</td><td>567</td><td>1600</td><td>38650</td></tr><tr><td>Up</td><td>6478</td><td>428</td><td>1055</td><td>16</td><td>33</td><td>66</td><td>250</td><td>16217</td></tr><tr><td rowspan="6">sub-02</td><td>B (Hit)</td><td>49597</td><td>235</td><td>169</td><td>16</td><td>117</td><td>183</td><td>316</td><td>3800</td></tr><tr><td>Y (Jump)</td><td>41653</td><td>426</td><td>263</td><td>16</td><td>250</td><td>366</td><td>533</td><td>2800</td></tr><tr><td>Down</td><td>17697</td><td>791</td><td>730</td><td>16</td><td>316</td><td>634</td><td>1000</td><td>8967</td></tr><tr><td>Left</td><td>11559</td><td>616</td><td>613</td><td>16</td><td>200</td><td>434</td><td>784</td><td>13950</td></tr><tr><td>Right</td><td>48203</td><td>1228</td><td>1606</td><td>16</td><td>100</td><td>750</td><td>1766</td><td>56317</td></tr><tr><td>Up</td><td>15443</td><td>490</td><td>511</td><td>16</td><td>150</td><td>367</td><td>584</td><td>7316</td></tr><tr><td rowspan="6">sub-03</td><td>B (Hit)</td><td>2155</td><td>180</td><td>214</td><td>16</td><td>133</td><td>150</td><td>183</td><td>4834</td></tr><tr><td>Y (Jump)</td><td>904</td><td>508</td><td>384</td><td>17</td><td>217</td><td>384</td><td>704</td><td>4700</td></tr><tr><td>Down</td><td>747</td><td>824</td><td>858</td><td>16</td><td>250</td><td>667</td><td>1075</td><td>13533</td></tr><tr><td>Left</td><td>398</td><td>575</td><td>517</td><td>16</td><td>233</td><td>434</td><td>817</td><td>4617</td></tr><tr><td>Right</td><td>1627</td><td>1159</td><td>1666</td><td>16</td><td>300</td><td>666</td><td>1358</td><td>22384</td></tr><tr><td>Up</td><td>328</td><td>446</td><td>533</td><td>16</td><td>83</td><td>316</td><td>621</td><td>6317</td></tr><tr><td rowspan="6">sub-06</td><td>B (Hit)</td><td>25961</td><td>61</td><td>53</td><td>16</td><td>50</td><td>50</td><td>67</td><td>5516</td></tr><tr><td>Y (Jump)</td><td>14786</td><td>266</td><td>260</td><td>16</td><td>66</td><td>150</td><td>433</td><td>1483</td></tr><tr><td>Down</td><td>6461</td><td>473</td><td>423</td><td>16</td><td>200</td><td>416</td><td>584</td><td>7833</td></tr><tr><td>Left</td><td>4699</td><td>369</td><td>370</td><td>16</td><td>117</td><td>250</td><td>508</td><td>3883</td></tr><tr><td>Right</td><td>17100</td><td>656</td><td>897</td><td>16</td><td>116</td><td>334</td><td>888</td><td>20017</td></tr><tr><td>Up</td><td>7052</td><td>361</td><td>427</td><td>16</td><td>96</td><td>250</td><td>483</td><td>6600</td></tr><tr><td rowspan="6">Total</td><td>B (Hit)</td><td>105652</td><td>164</td><td>144</td><td>16</td><td>83</td><td>133</td><td>183</td><td>5516</td></tr><tr><td>Y (Jump)</td><td>86988</td><td>374</td><td>248</td><td>16</td><td>200</td><td>333</td><td>500</td><td>4700</td></tr><tr><td>Down</td><td>49615</td><td>589</td><td>581</td><td>16</td><td>234</td><td>433</td><td>766</td><td>13533</td></tr><tr><td>Left</td><td>27339</td><td>593</td><td>610</td><td>16</td><td>183</td><td>400</td><td>766</td><td>15134</td></tr><tr><td>Right</td><td>95944</td><td>1153</td><td>1704</td><td>16</td><td>117</td><td>567</td><td>1550</td><td>56317</td></tr><tr><td>Up</td><td>29301</td><td>445</td><td>658</td><td>16</td><td>67</td><td>283</td><td>534</td><td>16217</td></tr></tbody></table>

https://doi.org/10.1371/journal.pone.0290158.t003

Bros [32] within the MRI scanner and player reported satisfying device useability as measured by the utility ratings. More specifically, the players reported a value of 3 (±0.70 std, a score of 3 is best) for the force required for button activation, 2.55 (±0.88 std, higher is better) for the smoothness during operation, 2.66 (±0.71 std, a score of 3 is best) for the amount of mental effort, 2.88 (±0.33 std, a score of 3 is best) for operation speed, 1.88 (±0.78 std, lower is better) for finger fatigue, 1.33 (±0.71 std, lower is better) for wrist fatigue, 3.33 (±1.22 std, higher is better) for general comfort, and 2.44 (±1.01 std, higher is better) for overall useability.

Discussion

We built the CNeuroMod videogame controller to be compatible with both fMRI and MEG. We wrote the documentation, plans needed by other teams to reproduce the controller and its
<figure><img src="image_9.png" /><figcaption>**Fig 6.** Distributions of the median of FD values across runs in the datasets included in this study. Data were analyzed separately for each subject, which showed similar effects (A-D). E shows the pooled data of all subjects. ns: $p < = 1.00e+00$; \*: $1.00e-02 < p < = 5.00e-02$; \*: $1.00e-03 < p < = 1.00e-02$; \*: $1.00e-04 < p < = 1.00e-03$; \*: \*: $p < = 1.00e-04$.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0290158.g006

software and made it publically available under an Open Source Hardware license. A feasibility study showed that our controller consistently registers button presses and releases with only small differences in timing compared with a commercial controller. We showed that the CNeuroMod controller can be used without inducing abnormal motion levels. The presence of the CNeuroMod controller in the scanning environment had small effects on tSNR maps and noise covariance matrices with fMRI and MEG, respectively.

## Open design and reusability

The design files, software and documentation required to build the controller are all released under a CERN-OHL-P license, an Open Hardware License that allows users to reproduce,

<table><caption>**Table 4.** Descriptive statistics of the run-median Framewise Displacement (FD) values for the different datasets used in the motion analysis.</caption><thead><tr><th rowspan="2"></th><th colspan="4">sub-01</th><th colspan="4">sub-02</th><th colspan="4">sub-03</th><th colspan="4">sub-06</th><th colspan="4">Total</th></tr><tr><th>N</th><th>Mean</th><th>STD</th><th>Dur. (hours)</th><th>N</th><th>Mean</th><th>STD</th><th>Dur. (hours)</th><th>N</th><th>Mean</th><th>STD</th><th>Dur. (hours)</th><th>N</th><th>Mean</th><th>STD</th><th>Dur. (hours)</th><th>N</th><th>Mean</th><th>STD</th><th>Dur. (hours)</th></tr></thead><tbody><tr><td>friends</td><td>194</td><td>0.098</td><td>0.027</td><td>37.89</td><td>194</td><td>0.139</td><td>0.043</td><td>37.89</td><td>196</td><td>0.039</td><td>0.022</td><td>38.308</td><td>194</td><td>0.143</td><td>0.033</td><td>37.89</td><td>778</td><td>0.105</td><td>0.031</td><td>151.978</td></tr><tr><td>gamepad</td><td>8</td><td>0.063</td><td>0.028</td><td>0.685</td><td>8</td><td>0.099</td><td>0.022</td><td>0.685</td><td>8</td><td>0.025</td><td>0.014</td><td>0.685</td><td>8</td><td>0.13</td><td>0.022</td><td>0.685</td><td>32</td><td>0.079</td><td>0.022</td><td>2.74</td></tr><tr><td>hcp-gamble</td><td>15</td><td>0.055</td><td>0.021</td><td>0.801</td><td>15</td><td>0.097</td><td>0.05</td><td>0.801</td><td>15</td><td>0.01</td><td>0.005</td><td>0.801</td><td>17</td><td>0.103</td><td>0.035</td><td>0.908</td><td>62</td><td>0.066</td><td>0.028</td><td>3.311</td></tr><tr><td>hcp-motor</td><td>15</td><td>0.053</td><td>0.019</td><td>0.901</td><td>15</td><td>0.138</td><td>0.041</td><td>0.894</td><td>15</td><td>0.027</td><td>0.029</td><td>0.894</td><td>18</td><td>0.156</td><td>0.051</td><td>1.073</td><td>63</td><td>0.094</td><td>0.035</td><td>3.762</td></tr><tr><td>hcp-rest</td><td>5</td><td>0.102</td><td>0.024</td><td>1.242</td><td>5</td><td>0.141</td><td>0.041</td><td>1.242</td><td>5</td><td>0.025</td><td>0.009</td><td>1.242</td><td>6</td><td>0.148</td><td>0.035</td><td>1.49</td><td>21</td><td>0.104</td><td>0.027</td><td>5.216</td></tr><tr><td>shinobi</td><td>57</td><td>0.12</td><td>0.018</td><td>10.572</td><td>56</td><td>0.158</td><td>0.029</td><td>10.735</td><td>0</td><td>NaN</td><td>NaN</td><td>0</td><td>56</td><td>0.206</td><td>0.031</td><td>11.343</td><td>169</td><td>0.161</td><td>0.026</td><td>32.65</td></tr></tbody></table>

https://doi.org/10.1371/journal.pone.0290158.t004
<figure><img src="image_10.png" /><figcaption>**Fig 7.** Temporal SNR maps on runs acquired with and without the controller present in the scanner. A and B show the mean SNR across runs recorded with and without the controller respectively. C shows the distributions of the averaged SNR values for voxels within the EPI mask. D, E, and F show the same visualizations but for STD values instead. \*\*\*\*: p < 1.00e-04.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0290158.g007

modify and redistribute the covered material. This license has been chosen as it is the most permissive version of the CERN-OHL flavors, with the goal of allowing maximal freedom of reuse to anyone. The presented device requires moderate engineering and technical resources to be built, and the materials used were chosen on the basis of their availability and low costs. All the necessary parts can be obtained for an estimated price around 1000 CAD (Table 5). In comparison, the device proposed by Current Design Inc. (including cables and interface) are sold for a total price of 8390 USD at the time of writing. The provided documentation covers all the steps involved in the preparation, assembly and use of the controller. Through the release of this hardware under a permissive OHL, we anticipate that other research teams or commercial entities will be able to easily reuse and/or modify the controller design beyond purely financial considerations.

## Reactivity of the CNeuroMod videogame controller

The study showed that the CNeuroMod controller consistently registers button presses as well as releases with a temporal precision comparable to a commercial controller. Although no differences were observed in button press reaction times, small significant differences were detected in release reaction times (for long presses) and in press duration (for short presses), showing longer release times with the prototype than with the reference controller. This result might indicate the presence of mechanical friction during the button release, an aspect that could be improved in further design iterations.

This difference in release reaction time could impact game play performance, particularly in games that require dense repetitions of button presses, but is likely negligible when playing most other games, as suggested by the successful collection of the *shinobi* dataset as well as the pilot implemented by the MIND team with *Super Mario Bros* [32]. As a point of reference, the
<figure><img src="image_11.png" /><figcaption>**Fig 8.** Comparison of MEG sensors noise covariance over time. A—D Normalized average covariance across T1 and T2 subsets, and correlation between these two distributions. T1 and T2 both were acquired without the CNeuroMod controller present in the MEG scanner. E—H Normalized average covariance across T3 and T4 subsets, and correlation between these two distributions. T3 was acquired without the controller present in the scanner, while T4 was acquired with the controller. I—L Distributions of the pairwise correlations between each pair of individual recordings, for each pair of subsets.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0290158.g008

difference in short button press duration observed during the *gamepad* task (about 30ms on average) is relatively small compared to the duration of button presses observed while participants played Shinobi using the commercial controller (164ms in average), with very few button presses lasting less than 50ms. In most cases, players can adapt to the controller and learn to release the buttons slightly earlier than they would on another controller. This interpretation of our results was confirmed qualitatively by the participants who all reported that it was necessary to adapt to the controller, but then reported they could play comfortably following this adaptation phase. This effect could also contribute to the utility ratings observed across both instances of the prototype (CNeuroMod and MIND teams). Users from both groups reported great to acceptable usability for all items, although a slightly high finger fatigue (2 for the CNeuroMod sample, 1.88 for the Inria/Neurospin sample) was detected. If the delay at release observed during the cued-response task is caused by mechanical friction, attempts from the subjects at compensating this effect might induce unnecessary finger fatigue, which could be improved in further iterations by adjusting the button and shutter mechanism.
<figure><img src="image_12.png" /><figcaption>**Fig 9.** Pictures of the reproduced controller by team members at Inria/NeuroSpin. A shows the assembled PCB. B shows the bottom part of the controller, with the switch mechanism in the process of being assembled. C shows the optics fibers installed in the handheld part. D shows the fully assembled controller.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0290158.g009

## Motion comparison across various data sets

The analyses conducted on fMRI runs collected on a wide range of tasks showed similar amounts of motion during our *gamepad* task compared to control tasks that did not involve the use of a videogame controller. This suggests that the CNeuroMod controller does not increase the amount of motion in the scanner. Conversely, the comparison between the motion measured during the *shinobi* dataset to that of other control datasets indicated significantly higher levels of motion for the gaming task (*shinobi*), likely to be attributable to the videogame content and not to the controller itself. Even in the case of the *shinobi* dataset, many runs had a fairly low level of motion, with median FD levels below 0.2 or even 0.1. Overall, the usage of the Courtois NeuroMod controller seems compatible with the collection of usable fMRI data, and in some instances even high-quality low-motion data.

## fMRI and MEG compatibility

Analyses performed on fMRI data acquired on a gel phantom head showed a small impact of the CNeuroMod controller on temporal SNR. These differences are statistically significant but are relatively small compared to the amount of noise typically observed during such recordings. Because fMRI recordings are known to be sensitive to thermal noise [33], which can
<table><caption>**Table 5.** Bill of material with estimated prices at the time of writing of this manuscript. More details about the suppliers and item numbers can be found in the documentation.</caption><thead><tr><th>Part</th><th>Supplier</th><th>Qty</th><th>Price (CAD)</th></tr></thead><tbody><tr><td>Fiber optic cable duplex 100m</td><td>Digikey</td><td>1</td><td>279.71</td></tr><tr><td>Fiber optic transmitter</td><td>Digikey</td><td>8</td><td>163.2</td></tr><tr><td>Fiber optic receiver</td><td>Digikey</td><td>8</td><td>198.72</td></tr><tr><td>Teensy 3.5 microcontroller board</td><td>Digikey</td><td>1</td><td>37.76</td></tr><tr><td>Gate driver</td><td>Digikey</td><td>5</td><td>7</td></tr><tr><td>Hose 13mm ID, black PE 1</td><td>Digikey</td><td>1</td><td>111.55</td></tr><tr><td>Hose 19.48mm ID, black PE 1</td><td>Digikey</td><td>1</td><td>160.38</td></tr><tr><td>Voltage regulator</td><td>Digikey</td><td>1</td><td>2.93</td></tr><tr><td>Resistor 330 Ω</td><td>Digikey</td><td>8</td><td>3.67</td></tr><tr><td>Resistor 65 Ω</td><td>Digikey</td><td>8</td><td>2.39</td></tr><tr><td>Capacitor ceramic 0.1 μF</td><td>Digikey</td><td>8</td><td>2.49</td></tr><tr><td>Capacitor ceramic 10 μF</td><td>Digikey</td><td>2</td><td>2.3</td></tr><tr><td>Connector header pin breakaway 24 pos</td><td>Digikey</td><td>4</td><td>14.28</td></tr><tr><td>Connector receptacle 24 pos</td><td>Digikey</td><td>2</td><td>9.18</td></tr><tr><td>Polycarbonate Pan Head Philips Screw 8–32 x 1” 2</td><td>McMaster-Carr</td><td>2</td><td>2</td></tr><tr><td>Polycarbonate Pan Head Philips Screw 8–32 x 1/2” 2</td><td>McMaster-Carr</td><td>3</td><td>3</td></tr><tr><td>Polycarbonate Pan Head Philips Screw 4–40 x 3/16” 3</td><td>McMaster-Carr</td><td>8</td><td>3.49</td></tr><tr><td>Nylon 6/6 Female Threaded Round Standoff 8–32 x 1/4” 2</td><td>McMaster-Carr</td><td>5</td><td>8.35</td></tr><tr><td>Nylon 6/6 Female Threaded Round Standoff 4–40 x 1/8” 3</td><td>McMaster-Carr</td><td>8</td><td>13.12</td></tr><tr><td><strong>Total</strong></td><td></td><td></td><td><strong>1025.52</strong></td></tr></tbody></table>

https://doi.org/10.1371/journal.pone.0290158.t005

fluctuate across the time of the day, the observed differences might be attributable to the fact that these two series of recordings were acquired within the same day but at different time and thus, at slightly different room temperatures. Replicating these results with other datasets would further confirm that the amount of noise induced by the controller stays well within acceptable range.

The MEG noise covariance matrices showed no significant impact of the CNeuroMod controller. Further analysis of empty room recordings collected at various periods (from April 2019 to June 2021) showed that the variation of noise levels induced by the controller stayed well within the range of changes that could be expected from normal environmental fluctuations.

## Implications and future work

The CNeuroMod MRI- and MEG-compatible videogame controller provides a versatile device to play most videogames. This versatility extends to any experimental task that requires the use of 8 or less buttons, representing a very large action space. For example, one can use this controller to answer complex questionnaires, e.g. featuring Likert scale, or perform neuropsychological tasks where users navigate many questions. The CNeuroMod controller is specifically geared towards retro gaming from the generation of the Super Nintendo Entertainment System (SNES™, Nintendo). Although some modern titles require basic input sets that can be covered by our controller, most console games published after the release of the XBox™ (Microsoft) and PlayStation™ (Sony) use joysticks and trigger buttons under the index.

Future design iterations could reduce the latency of button releases, and enable the inclusion of additional buttons and analog joysticks that would expand the scope of playable videogames to modern titles.
## Conclusions

The CNeuroMod controller is designed to be built using commonly accessible tools and materials, and released under an Open Hardware License for maximal reusability. A feasibility study showed response latencies comparable to a standard commercial controller. MRI and MEG experiments showed that the controller does not increase subject motion and has marginal effects on acquisition noise, if any. We hope that the CNeuroMod controller will assist researchers in leveraging the richness of videogame tasks in cognitive neuroscience. Our open design can also contribute to a growing ecosystem of open hardware for reproducible neuroimaging research.

## Supporting information

**S1 Appendix. Build instructions and documentation.**
(PDF)

**S1 Dataset. Cued-response task results.**
(CSV)

**S2 Dataset. Shinobi training behavioral data.**
(GZ)

**S3 Dataset. FD values of CNeuroMod datasets.**
(CSV)

**S4 Dataset. fMRI phantom data.**
(CSV)

**S5 Dataset. Noise covariances of MEG empty room data.**
(GZ)

**S6 Dataset. Utility ratings of the controller in both sites.**
(ODS)

**S1 Graphical abstract.**
(TIF)

## Acknowledgments

The authors would like to thank the MEG Imaging Center, the Functional Neuroimaging Unit at CRIUGM and the Courtois NeuroMod scanning crews for acquiring the data.

## Author Contributions

**Conceptualization:** Yann Harel, André Cyr, Julie Boyle, Basile Pinsard, Karim Jerbi, Pierre Bellec.

**Data curation:** Basile Pinsard.

**Formal analysis:** Yann Harel.

**Funding acquisition:** Bertrand Thirion, Karim Jerbi, Pierre Bellec.

**Investigation:** Yann Harel, Pierre Bellec.

**Methodology:** Yann Harel, Julie Boyle, Basile Pinsard.

**Project administration:** Julie Boyle.
**Resources:** André Cyr.

**Software:** Basile Pinsard.

**Supervision:** Karim Jerbi, Pierre Bellec.

**Validation:** Jeremy Bernard, Marie-France Fourcade, Himanshu Aggarwal, Ana Fernanda Ponce, Bertrand Thirion.

**Visualization:** Yann Harel.

**Writing – original draft:** Yann Harel.

**Writing – review & editing:** Yann Harel, André Cyr, Julie Boyle, Basile Pinsard, Jeremy Bernard, Marie-France Fourcade, Himanshu Aggarwal, Ana Fernanda Ponce, Bertrand Thirion, Karim Jerbi, Pierre Bellec.

## References

1. Dale G, Joessel A, Bavelier D, Green CS. A new look at the cognitive neuroscience of video game play. Ann N Y Acad Sci. 2020 Mar; 1464(1):192–203. https://doi.org/10.1111/nyas.14295 PMID: 31943260

2. Klasen M, Wolf D, Eisner PD, Eggermann T, Zerres K, Zepf FD, et al. Serotonergic Contributions to Human Brain Aggression Networks. Front Neurosci. 2019 Feb 22; 13:42. https://doi.org/10.3389/fnins. 2019.00042 PMID: 30853880

3. Klasen M, Mathiak KA, Zvyagintsev M, Sarkheil P, Weber R, Mathiak K. Selective reward responses to violent success events during video games. Brain Struct Funct. 2020 Jan; 225(1):57–69. https://doi.org/ 10.1007/s00429-019-01986-7 PMID: 31754792

4. Wolf D, Klasen M, Eisner P, Zepf FD, Zvyagintsev M, Palomero-Gallagher N, et al. Central serotonin modulates neural responses to virtual violent actions in emotion regulation networks. Brain Struct Funct. 2018 Sep; 223(7):3327–45. https://doi.org/10.1007/s00429-018-1693-2 PMID: 29948188

5. Zvyagintsev M, Klasen M, Weber R, Sarkheil P, Esposito F, Mathiak KA, et al. Violence-related content in video game may lead to functional connectivity changes in brain networks as revealed by fMRI-ICA in young men. Neuroscience. 2016 Apr; 320:247–58. https://doi.org/10.1016/j.neuroscience.2016.01.056 PMID: 26855192

6. Lim SJ, Fiez JA, Holt LL. Role of the striatum in incidental learning of sound categories. Proc Natl Acad Sci. 2019 Mar 5; 116(10):4671–80. https://doi.org/10.1073/pnas.1811992116 PMID: 30782817

7. Takeuchi N, Mori T, Suzukamo Y, Izumi SI. Integration of Teaching Processes and Learning Assess- ment in the Prefrontal Cortex during a Video Game Teaching–learning Task. Front Psychol [Internet]. 2017 Jan 9 [cited 2021 Jul 20]; 7. Available from: http://journal.frontiersin.org/article/10.3389/fpsyg. 2016.02052/full PMID: 28119650

8. Anderson JR, Bothell D, Fincham JM, Moon J. The sequential structure of brain activation predicts skill. Neuropsychologia. 2016 Jan; 81:94–106. https://doi.org/10.1016/j.neuropsychologia.2015.12.014 PMID: 26707716

9. Wang M, Dong G, Wang L, Zheng H, Potenza MN. Brain responses during strategic online gaming of varying proficiencies: Implications for better gaming. Brain Behav. 2018 Aug; 8(8):e01076. https://doi. org/10.1002/brb3.1076 PMID: 30020566

10. Huskey R, Craighead B, Miller MB, Weber R. Does intrinsic reward motivate cognitive control? a natu- ralistic-fMRI study based on the synchronization theory of flow. Cogn Affect Behav Neurosci. 2018 Oct; 18(5):902–24. https://doi.org/10.3758/s13415-018-0612-6 PMID: 29923098

11. Gray WD. Game-XP: Action Games as Experimental Paradigms for Cognitive Science. Top Cogn Sci. 2017 Apr; 9(2):289–307. https://doi.org/10.1111/tops.12260 PMID: 28296290

12. Sheikholeslami C, Yuan H, He EJ, Bai X, Yang L, He B. A High Resolution EEG Study of Dynamic Brain Activity during Video Game Play. In: 2007 29th Annual International Conference of the IEEE Engineer- ing in Medicine and Biology Society. 2007. p. 2489–91.

13. Klarkowski M, Causse M, Duprès A, del Campo N, Vella K, Johnson D. Using fNIRS to Assess Cogni- tive Activity During Gameplay. Proc ACM Hum-Comput Interact. 2022 Oct 31; 6(CHI PLAY):256:1– 256:23.

14. Cheyne D, Jobst C, Tesan G, Crain S, Johnson B. Movement-related neuromagnetic fields in preschool age children: Movement-Related Brain Activity in Preschool Children. Hum Brain Mapp. 2014 Sep; 35 (9):4858–75.
15. Gross-Weege N, Dey T, Gebhardt P, Schug D, Weissler B, Schulz V. Characterization methods for comprehensive evaluations of shielding materials used in an MRI. Med Phys. 2018; 45(4):1415-24. https://doi.org/10.1002/mp.12762 PMID: 29363769

16. Trees J, Snider J, Falahpour M, Guo N, Lu K, Johnson DC, et al. Game controller modification for fMRI hyperscanning experiments in a cooperative virtual reality environment. MethodsX. 2014; 1:292-9. https://doi.org/10.1016/j.mex.2014.10.009 PMID: 26150964

17. Cross L, Cockburn J, Yue Y, O'Doherty JP. Using deep reinforcement learning to reveal how the brain encodes abstract state-space representations in high-dimensional environments. Neuron. 2021 Feb; 109(4):724-738.e7. https://doi.org/10.1016/j.neuron.2020.11.021 PMID: 33326755

18. Zhang T, Gao JS, Çukur T, Gallant JL. Voxel-Based State Space Modeling Recovers Task-Related Cognitive States in Naturalistic fMRI Experiments. Front Neurosci. 2021 May 6; 14:565976. https://doi.org/10.3389/fnins.2020.565976 PMID: 34045937

19. Kätsyri J, Hari R, Ravaja N, Nummenmaa L. The Opponent Matters: Elevated fMRI Reward Responses to Winning Against a Human Versus a Computer Opponent During Interactive Video Game Playing. Cereb Cortex. 2013 Dec; 23(12):2829-39. https://doi.org/10.1093/cercor/bhs259 PMID: 22952277

20. Gentile DA, Swing EL, Anderson CA, Rinker D, Thomas KM. Differential neural recruitment during violent video game play in violent- and nonviolent-game players. Psychol Pop Media Cult. 2016 Jan; 5 (1):39-51.

21. Peirce J, Gray JR, Simpson S, MacAskill M, Höchenberger R, Sogo H, et al. PsychoPy2: Experiments in behavior made easy. Behav Res Methods. 2019 Feb 1; 51(1):195-203. https://doi.org/10.3758/s13428-018-01193-y PMID: 30734206

22. Natapov D, Castellucci SJ, MacKenzie IS. ISO 9241-9 Evaluation of Video Game Controllers.

23. Setsompop K, Gagoski BA, Polimeni JR, Witzel T, Wedeen VJ, Wald LL. Blipped-controlled aliasing in parallel imaging for simultaneous multislice echo planar imaging with reduced g-factor penalty. Magn Reson Med. 2012; 67(5):1210-24. https://doi.org/10.1002/mrm.23097 PMID: 21858868

24. Power JD, Silver BM, Silverman MR, Ajodan EL, Bos DJ, Jones RM. Customized head molds reduce motion during resting state fMRI scans. NeuroImage. 2019 Apr; 189:141-9. https://doi.org/10.1016/j.neuroimage.2019.01.016 PMID: 30639840

25. Esteban O, Ciric R, Finc K, Blair RW, Markiewicz CJ, Moodie CA, et al. Analysis of task-based functional MRI data preprocessed with fMRIPrep. Nat Protoc. 2020 Jul; 15(7):2186-202. https://doi.org/10.1038/s41596-020-0327-3 PMID: 32514178

26. Sega. Shinobi III: Return of the Ninja Master. Sega; 1993.

27. Friends [TV series]. 1994. Crane, D & Kauffman, M (Executive Producers). National Broadcasting Company

28. Vanderwal T, Kelly C, Eilbott J, Mayes LC, Castellanos FX. Inscapes: A movie paradigm to improve compliance in functional magnetic resonance imaging. NeurolImage. 2015 Nov 15; 122:222-32. https://doi.org/10.1016/j.neuroimage.2015.07.069 PMID: 26241683

29. Van Essen DC, Smith SM, Barch DM, Behrens TE, J, Yacoub E, Ugurbil K. The WU-Minn Human Connectome Project: An overview. NeuroImage. 2013 Oct 15; 80:62-79. https://doi.org/10.1016/j.neuroimage.2013.05.041 PMID: 23684880

30. Gorgolewski K, Burns C, Madison C, Clark D, Halchenko Y, Waskom M, et al. Nipype: A Flexible, Light-weight and Extensible Neuroimaging Data Processing Framework in Python. Front Neuroinformatics [Internet]. 2011 [cited 2023 May 9]; 5. Available from: https://www.frontiersin.org/articles/10.3389/fninf.2011.00013 PMID: 21897815

31. Gramfort A, Luessi M, Larson E, Engemann D, Strohmeier D, Brodbeck C, et al. MEG and EEG data analysis with MNE-Python. Front Neurosci [Internet]. 2013 [cited 2023 May 9]; 7. Available from: https://www.frontiersin.org/articles/10.3389/fnins.2013.00267 PMID: 24431986

32. Nintendo. Super Mario Bros. Nintendo; 1983.

33. Krüger G, Glover GH. Physiological noise in oxygenation-sensitive magnetic resonance imaging. Magn Reson Med. 2001; 46(4):631-7. https://doi.org/10.1002/mrm.1240 PMID: 11590638