Comparing a Finger Dexterity Assessment in Virtual, Video-Mediated, and Unmediated Reality

IntroductionIs a virtualised Nine Hole Peg Test as usable as the real version, or a video-mediated version? This is the primary question investigated in this study. The Nine Hole Peg Test is a tool for the therapeutic assessment of finger function and is commonly used with people who suffer from impairments after stroke (1). Various versions are commercially available and consist of either wooden elements (like the original version), or are made from plastic (2). With a virtual reality version of the NHPT, a broader range of therapeutic applications as well as a more patient-based adaptation than the traditional test could be possible. For example, the difficulty could be adjusted based on the patients' performance and frustration tolerance as well as their motivation. This also allows patients with severe impairments to be treated or assessed who otherwise would not be able to perform the test.The development of the virtual Nine Hole Peg Test (vNHPT) requires new hardware as well as software components. The general concept is based on Augmented Reflection Technology (ART) introduced by Regenbrecht et al. (3) and used for a number of studies with healthy participants (4-7) as well as with clinical participants (8, 9). However, for the specific implementation of the vNHPT, developing tracking and rendering approaches which are more sophisticated than what was used in ART, is necessary.In current rehabilitation, there are several approaches to help the patients gain back some of their motor functions. Among the most common is physiotherapy following the Bobath concept (10), which often includes the use of external devices to support the patients in their execution of movement tasks. Another approach is Constraint-Induced Movement Therapy (11). This involves restraining the healthy limb of the patient, and having them perform actions with their impaired limb. Doing so for extensive periods of time (i.e., up to 90% of waking hours) has been shown to improve motor deficits of patients suffering from impairments after stroke (12). A less restraining approach is one which takes advantage of the manipulability of human perceptions, beliefs and even sensations. It was in fact shown that psychotherapies such as Cognitive Behaviour Therapy, have effects on the brain (13). Similar changes in the brain were also shown in a stroke patient treated with Mirror Visual Illusions (14). This phenomenon is commonly referred to as "neuroplasticity" and is described as the brain's ability "to respond to intrinsic and extrinsic stimuli by reorganizing its structure, function and connections" (15). In order to make use of it, therapy approaches should focus on providing environments that allow meaningful therapeutic movements, with adequate intensity and repetitions, as well as motivating the patient and providing appropriate feedback (16). Virtual and Augmented Reality Environments have the potential to be used in this context.In this paper an implementation of such an environment is presented and compared with its real life and video-mediated counterparts.SystemThere are three main technical components plus the physical apparatus itself that contribute to the system: (i) an off-the-shelf webcam with a built in 3D depth sensor with a resolution of 320x240, and an HD 720p RGB image sensor (Interactive Gesture Camera, Creative Technology Ltd) mounted on a custom-built frame (see Figure 1); (ii) a tailor-made plugin to process the data from the webcam for delivery to the application; and (iii) a virtual reality application created using the Unity3D game engine (version 4.2, unity3d.com) which provides the final environment in which the users perform their tasks.The webcam's functions are accessed from the plugin using the Intel Perceptual Computing SDK 2013 (Intel SDK). This allows for access to the raw data from both the depth and the colour sensors and provides features such as basic finger tracking. …