Measurement of Neuro-Mechanical Parameters for the Biomechanical Analysis and a Mixed Reality Data Visualization to Augment the Clinical Eye
Creators: name: Covre, Nicola
Publisher: name: Università degli studi di Trento
Year: 2022

description: This thesis research proposes a set of technical systems which support the biomechanical analysis and it is structured as four connected sub-projects. First, it proposes a markerless motion capture system which estimates the human pose via Time of Flight Cameras. Secondly, it is illustrated the design, development and validation of a cutting-edge RGB-D camera-based system which autonomously estimates the volume, weight and the Inertia of the human anatomical segments. Subsequently, it reports the development of custom-made electromyographic sensors which perform real-time monitoring and data processing of the surface muscle activation. Finally, it proposes a mixed reality application for returning real-time visual feedback both for assisting the test subjects and for augmenting the physician’s eye. In particular, the first part of this work proposes a comparison between two motion capture systems. The study compares a customised breakthrough markerless multi-camera system, Azure Kinect based, and a gold standard Marker-Based optical tracking System, Vicon Nexus. On one hand, a Kalman Filter sensor fusion collects each joint information provided by each Azure Kinect and merges the measures considering the uncertainty related to each camera. On the other hand, the Vicon motion capture uses a plug-in gait model with a standard subject marketization to reconstruct the position of each joint. A further innovative aspect is a peculiar focus on subjects wearing exoskeletons for lower limbs, allowing to highlight the performances and limitations of both systems in presence of noisy environments and occlusions. The computation of volume, weight, and inertia of each anatomical segment has been conducted within the Bullet project as a part of the Eurobench European framework. The goal of this project aims to establish the reaction forces generated within the articulations when a paraplegic subject walks with a lower limb’s exoskeleton and uses the crutches for balancing and support. The anatomical parameters are therefore required to perform biomechanical analysis and run simulations. With respect to the tools and methods reported by the state of the art, here is proposed a novel approach which autonomously estimates the volumes, weight, and inertia from the RGB and depth images of the subjects laying on a hospital bed. x The monitoring of the muscle activation aims to develop and optimize a custom device for offline and real-time EMG applications, such as the visual feedback of muscular activation in mixed reality or the remote actuation of a robotic prosthesis. For this reason, the analysis has been pursued both by using commercial devices, such as Delsys Trigno, and also designing, developing, and validating a custom device for electromyography. Finally, this work reports the development of mixed reality environments through Hololens 2 devices for the visualization and rendering of the measurements. This information is supposed to help both the therapist and the patient address the rehabilitative exercises more intuitively and objectively. In fact, on one hand, this allows the therapist to have a fast real-time evaluation of the patient’s parameters without the need to consult paper documents or nearby monitors. On the other hand, it helps the patient by exploiting ghost avatars or geometrical guideline indicators that compare the subject’s movements with the correct ones