Study of biomechamics in violin performances with kinect and its relationship with Sound
Creators: name: Fernandez, Pablo
Publisher: name: Zenodo
Year: 2020
Subjects: Violin, kinematics, dynamics, measurements, RGBD camera

description: Playing a musical instrument is a highly complex activity, which requires a combination<br> of mental and sensorimotor skills acquired during a long learning trajectory.<br> Traditional music performance pedagogy is mostly based on imitation and feedback<br> on the performance through semantic metaphors and imagery that attempt<br> to explain an acoustic quality or a physiological aspect of the performance such as<br> the posture or the feeling of movement. Such an approach is based on subjective<br> and vague perception that, in many cases, result in frustrating lack of progress to<br> chronic problems, forced breaks from performing during extended periods or even<br> career-ending injury.<br> The long-term aim of this research is to study the scientific and technological challenges<br> around the analysis and modeling of human motion in musical practice as a<br> sensory-motor activity and involves the acquisition, description, storage and analysis<br> of datasets of motion and audio data captured during real performances. More<br> specifically, this work deals with the measurement and description of the kinematics<br> and dynamics of the human upper-body motion during violin playing. A model of<br> expert performers is built as a reference to compare with and automatically evaluate<br> amateurs.<br> Human body motion is typically measured with camera based systems or with wearable<br> inertial sensors technology (e.g. accelerometer, gyroscope) attached to the body.<br> Both techniques, along with bio-mechanical models, can be used to extract features<br> such as joint angle, posture, and joint rotation speed as well as many others which<br> could be used for improved performance coordination to reduce the risk of injury<br> through the measurement and feedback of prolonged joint extension and angular<br> rotation intensity. In this work, we aim to develop low-cost implementations, using<br> off-the-shelf technology, namely RGB-D cameras, to make the system available to<br> a large potential user group. In general, the accuracy of low-cost systems is low.<br> However with the advent of recent consumer level 3D cameras such as the Microsoft<br> Kinect, body motion tracking becomes feasible.<br> Motion of the Violinist Upper-Body is acquired by means of a Microsoft Kinect<br> Camera programmed using the provided C# SDK, which is able to track the position<br> of the human body articulations and bones.<br> Based on the position of these joints, we estimate two type of features. First, kinematic<br> features such as angles between bones, joint positions, displacement, speed<br> and acceleration and second dynamic features (i.e. muscle forces in motion). Dynamic<br> features are estimated by fitting the previously computed kinematic features<br> against a bio-mechanical model. Kinematics allows to analyze motion and pose of<br> violin performers and dynamic features are important in order to detect over-use<br> and help prevent possible muscular injuries. The fitting with the bio-mechanical<br> model is done with the OpenSim software.<br> Statistical analysis and machine learning techniques are used to build a reference<br> model of expert performers that can be used to compare with amateurs and to give<br> feedback and automatic evaluation of any performance.<br> In this work we aim to characterize violin performances in terms of kinematic and<br> dynamic features of the human body and to build a reference expert model that<br> allows to automatically classify between experts and amateurs. By means of the<br> reference model, we can additionally measure how close to the reference a new<br> performance is and give as feedback an automatic rating of its quality.<br> We have recorded a small database of experts, amateurs and non-violinists and our<br> model is able to correctly classify 90% of the cases.
description: Master Thesis