Fast head tracking and applications
I have been developing fast, simple computer vision techniques to track a
user's head and upper body (in 2D), and also building applications which use this
capability in a desktop environment. The tracking technique, called
"draping", calculates a 1D silhouette-like "drape" which covers the
user from the top of the image. Based on this drape, the head is located and simple
poses can be recognized.
The drape is a simulation of point masses at the top of each column connected to their
immediate neighbors by springs. The drape is forced down by gravity, and forced up
by the image foreground (the segmentation of the user). It settles on the user's
head and shoulders (even in the presense of some segmentation noise) as shown below.
This figure shows the live video with head location, a segmentation of the user (based
on a fixed background model), and the "drape":
Demos that use this as a starting point:
- Yes/no detection - Recognize a head shake or nod, and interpret as "Yes/Okay"
or "No/Cancel".
- Pose matching - Recognize from a set of stored poses (not there, user present, saluting
with right or left hand, etc.) and take appropriate action (e.g., a screen saver that
stops when it sees the user in the standard position).
- Aware video player - Plays the video when the user is present, and pauses when the user
leaves the camera's field of view or puts his hands up to say "Stop!"
- Window scaling - Automatically scales a screen window depending on the distance of the
user to the monitor.
- Tic tac toe - Play a game of Tic Tac Toe by estimating the direction which the user's
head is pointing.
