Key detection for a virtual piano teacher

Type: conference paper

Authors: Adam Goodwin, Richard Green

Abstract
We propose a method for identifying a piano keyboard present in the video footage of a standard webcam with the goal of teaching chords, scales and suggested finger positions to a beginner pianist. Our keyboard identification method makes use of binary thresholding, Sobel operators and Hough transforms, as well as proposed algorithms specific to this application, to first find an area resembling a piano keyboard before narrowing the search to detect individual keys. Through the use of our method the keys of a piano keyboard were successfully identified from webcam video footage, with a tolerance to camera movement and occluded keys demonstrated. This result allowed the augmented reality style highlighting of individual keys, and the display of suggested fingering, for various chords and scales – which demonstrates the potential for our piano teacher program as a learning tool. The demo application achieved an average frame rate of 25.1 frames per second when run on a 2.20GHz dual-core laptop with 4GB RAM; a suitable rate for real-time use.

URL: http://ieeexplore.ieee.org/document/6727030/

Associated tags
metadata ▸ material ▸ Virtual Piano teacher
metadata ▸ contribution ▸ Presentation
metadata ▸ year ▸ 2013
technological dimension ▸ applications ▸ VR, AR
technological dimension ▸ input technologies ▸ Camera
technological dimension ▸ system outputs ▸ Visual
musical dimension ▸ activities ▸ Performing
pedagogical dimension ▸ learning theories ▸ Cognitivism
pedagogical dimension ▸ users ▸ Not defined
pedagogical dimension ▸ venues ▸ Lab