 |
| Human Motion Capture | ||||||
| Participants Martin Côté M.A.Sc. student 2005-2007 Silvain Bériault M.A.Sc. student 2006-2008 Ashwani Panwar International internship 2007 Dr. Pierre Payeur SITE, University of Ottawa Collaborators Dr. Gilles Comeau Dept. of Music, University of Ottawa  Piano Pedagogy Research Laboratory University of Ottawa  Natural Sciences and Engineering Research Council of Canada  University of Ottawa - Interdisciplinary Initiatives Development Fund  Yamaha Canada Music |
This project develops new computer vision technologies for monitoring the complex movements executed
by piano players. Piano teachers still widely use subjective and incomplete visual observation of the
pianist's posture and gestures to help their students improve their play. This does not provide a
quantitative method for evaluating and comparing pianists habits and performance and to detect problematic
situations. It has been demonstrated that bad habits can lead to severe piano-playing related health
problems (PRHP). This reveals a need for technologies that would contribute in the prevention of such
injuries and to the improvement of pedagogical methods, without being too complex to operate. Existing motion capture technologies used in medicine and movie production remain invasive and cumbersome as they involve massive cabling and/or reflective targets to be installed on the body of the performer. Other approaches rely on contrasting backgrounds or on assumptions about the motion and complexity of the scene. These impositions yield an environment that is foreign to a performer, leading him to behave differently than he would in a more comfortable environment. The limitations of such techniques may also obfuscate key performance markers.
In order to alleviate these constraints, advanced computer vision and image processing approaches are investigated as sensing techniques in the context of piano playing. A multi-camera video monitoring network has been designed along with a complete calibration procedure. An innovative image segmentation technique able to adapt to unconstrained environments has also been developed. The system can perform live motion capture of a piano player without using any marker nor imposing a specific dress code to the pianist. The objective of the first phase of research is to achieve markerless motion capture of a performer that can operate without constraints on the surrounding environment. Early experimental results demonstrated the feasibility of the approach, resulting in accurate colored 3D reconstructions of a subject performing some movement.
Future phases are going to investigate the recognition and classification of patterns of movement from the quantitative information extracted from the motion capture system. This will open the door to an exploration of biomechanical impacts of the repetition of some gestures and to the prevention of PRHPs. It will also provide the necessary background to support the development of computerized piano teaching resources that will automatically analyze the movement of the performer's body as well as the depression of keys on the keyboard. |
Related Publications
|
© SMART Research Group, 2008