Robust Gesture Recognition For Non-Contact Augmentative Communication Using a Dynamic Binary Frame of Reference

Abstract

A key challenge in gesture recognition for augmentative communication is the ability to characterize a given gesture by its fundamental and invariant properties. This paper proposes a new approach to this problem, based on characterizing gestures with primitive labels such as “left-up to right-down”. It uses movement as a recognition cue and the relative positions between key motion descriptors to characterize the gesture. This approach is robust to rotation and changes in scale and is performer independent. Further, the method can tolerate cluttered backgrounds and does not require the user to wear any accessories. A single representative example is sufficient to characterize a given gesture and no training is required. Experiments with a set of gestures performed by different individuals in a cluttered environment demonstrate the robustness of the approach. Implications for gesture-based augmentative communication are briefly discussed.