Principal Investigator Louis
Project Website http://groups.csail.mit.edu/vision/vip/watson/index.htm
The real-time object tracker uses range and appearance information from a stereo camera to recover the 3D rotation and translation of objects, or of the camera itself. The system can be connected to a face detector and used as an accurate head tracker. Additional supporting algorithms can improve the accuracy of the tracker.
Watson can track rigid objects in real-time with 6 degrees of freedom using a tracking framework called Adaptive View-Based Appearance Model. The tracking library can estimate the pose of the object for a long period of time with bounded drift. The main application is head pose estimation and gesture recognition using a USB camera or a stereo camera.
The approach combines an Adaptive View-based Appearance Model (AVAM) with a robust 3D view registration algorithm. AVAM is a compact and flexible representation of the object that can be used during the tracking to reduce the drift in the pose estimates. The model is acquired online during the tracking and can be adjusted according to the new pose estimates. Relative poses between frames are computed using a hybrid registration technique which combine the robustness of ICP (Iterative Closest Point) for large movement and the precision of the normal flow constraint. The complete system runs at 25Hz on a Pentium 4 3.2GHz.