Video Game that Heal: Treating Dyslexia with Music
By: Carlos Valcarcel
Tags: Cognitive Science, Videogames, Neuroscience, Music Therapy, Dyslexia

Digital Music Therapy

Reading disabilities are most effectively treated early in a child’s life, preferably while they are still learning to read. The promise of music therapy delivered through digital games is to frequently engage children in therapeutic activities unrestricted by standard treatment settings.

Ultimately, the elements that make digital cognitive training effective are the same ones that make neurologic music therapy effective. The Handbook of Neurologic Music Therapy written by Dr. Michael Thaut set the elementary rules for learning-oriented motor-therapy: ⁵²

Repetition

Feedback

Cueing

Task orientation (instruction)

Active learning (requires effort and interaction)

Ecological validity (functional relevance)

Shaping (adaptive difficulty)

Motivation

Target Cognitive Skill - Rhythmic Perception & Production

To combine these two technologies into a therapeutic solution that supports the treatment of reading disabilities, rehabilitative practice should target players’ perception of the temporal aspects of sound. The objective in rhythmic training is for a player achieve synchronization (entrainment) with a beat or its period (time between beats).

A player’s ability to entrain with complex rhythmic patterns reflects a higher ability to perceive things in time (temporal perception). However, to achieve long-term improvements in temporal perception the training must be capable of activating plastic changes in the brain. The earlier discussion around the impact of music and neurologic music therapy supports the use of multi-modal interaction to activate brain plasticity.⁵³

In one direction, repetitive physical movement helps refine anticipation by tying spatial distance to the period of a beat (this is why walking is a common way to teach children about rhythm). In the opposite direction, anticipation of audible cues primes motor neurons for more effective movement.⁵⁴ This approach leverages the reflexive link between auditory and motor neurons to gradually strengthen cortical connections between the relevant areas of the brain.⁵⁵

[Multi-modal interaction] leverages the reflexive link between auditory and motor neurons to gradually strengthen cortical connections between the relevant areas of the brain.

Thus, it is important that a therapeutic game elicit a physical response from players using audible cues. Simultaneously, they should allow for players to adjust or otherwise correct their physical interactions using audible feedback. This feedback process, otherwise known as sonification, is critical for the effective reinforcement of audio-motor skills⁵⁶ and should be facilitated by the careful cueing, feedback and instruction mechanisms that characterize high quality games and personalized therapy sessions.

Create a Diverse & Engaging Experience

The next step would be to design an experience that meets the criteria necessary for skill transfer in digital games. Digital platforms cannot engage the players’ senses as directly as instrumental play with another human being. Hence, the importance placed on context diversity by the cognitive consequence gaming literature.

In this regard, “context” in gaming is akin to functional relevance in Dr. Thaut’s handbook and should be understood to include everything from visual and audible aesthetics (or game aesthetics) to rules of play and interaction diversity (or game mechanics).⁵⁷

Photo by: cinuse — Taiko No Tatsujin provides a suite of diverse rhythmic games that challenge players in a number of creative ways

Diversity is important when considering game aesthetics and interactions. Engaging the senses through different visual, audible and tactile elements (i.e. vibration, device tilt, positioning, or movement) should avoid specialization to a limited set of gaming environments and interactions. However, aesthetic designs should also avoid providing players with alternative cognitive strategies to succeed.

For example, a 2017 study that looked at existing musical games and examined their potential for rhythmic training found most games relied heavily on reactions to visual cues rather than prediction from synchronization.⁵⁸ This afforded the players the opportunity to cheat themselves by avoiding strict exercise of auditory-temporal perception and audio-motor coordination.

Games that relied heavily on reactions to visual cues rather than prediction from synchronization afforded players the opportunity to cheat themselves

The use of diverse game mechanics is an opportunity to engage players in active learning and expose them to different applications of the target cognitive skills. Nevertheless, it is important that repetition remain a priority and diversity be appropriately sequenced with difficulty and mastery.

Prioritizing repetition simply means targeting the same cognitive skill over and over again (in this case temporal perception). This does not exclude changing the activities or rules of play. In fact, achieving entrainment while splitting focus with another cognitive task, or shifting focus to apply the target ability in different ways, reflects greater ability because it reduces the mental resources available to entrain to a rhythm.

Additional cognitive tasks may be considered part of a “diverse context” and contribute to functional relevance by allowing the player to synchronize in situations where they previously would not have been able to. The key is to make sure players are always figuring out better ways to perceive rhythm by challenging them with measurable and enjoyable tasks.

Adaptive Learning

Without getting into too much detail, there are important considerations in machine learning for any who intend to make this type of solution truly accessible. Diagnostics are an important first step. Plenty of digital reading tests exist online. For dyslexia please visit our friends at the AppRISE Project who are working with scientists all over the U.S. to develop a digital diagnostic test for Dyslexia.

A diagnostic baseline and a cognitive benchmark for the target perceptual skill allows developers to measure the training effect of each element discussed. Deploying them in diverse arrays that allow for intensive training and a consolidation period may prove superior to more gradual increases in difficulty. Or vice-versa. Or it may be the case that people respond differently to each. Machine learning affords neuroscientists and developers the opportunity to work together and maximize the well-being of each and every player.