Using Local Binary Patterns for Non-Contact Optical Tongue Tracking

Abstract

In this study, a real-time tongue tracking system is developed. The goal is to track a user’s tongue in a safe, non-contact manner using a webcam and image processing algorithms. This system functions in a two- level architecture. First, it detects the approximate location of the mouth. Then, exact mouth state and tongue direction are determined. A rapid object detection algorithm which makes use of Multi-scale Block Local Binary Patterns (MB-LBP) is applied in this system. Instead of pixel intensities, this algorithm employs MB-LBP features for computations in processing digital images which significantly reduces computational requirements and increases the frame rate. The Gentle Adaptive Boosting meta-algorithm (AdaBoost) is used to obtain a classifier for each mouth/tongue state. Six-state classification accuracy is measured using a hold-out test with six subjects of varying ethnicities. Accuracy is comparable with that of our previous prototype, but is more robust to ambient lighting and head pose. Accuracy is expected to further increase with the collection of more training data.