Attribute-Based People Search in Surveillance Environments

IEEE Workshop on Applications of Computer Vision (WACV'09)
Snowbird, Utah, 2009

Query results for "bald" and "red shirts".

Abstract

We propose a novel framework for searching for people in surveillance environments. Rather than relying on face recognition technology, which is known to be sensitive to typical surveillance conditions such as lighting changes, face pose variation, and low-resolution imagery, we approach the problem in a different way: we search for people based on a parsing of human parts and their attributes, including facial hair, eyewear, clothing color, etc. These attributes can be extracted using detectors learned from large amounts of training data. A complete system that implements our framework is presented. At the interface, the user can specify a set of personal characteristics, and the system then retrieves events that match the provided description. For example, a possible query is show me the bald people who entered a given building last Saturday wearing a red shirt and sunglasses. This capability is useful in several applications, such as finding suspects or missing people. To evaluate the performance of our approach, we present extensive experiments on a set of images collected from the Internet, on infrared imagery, and on two-and-a-half months of video from a real surveillance environment. We are not aware of any similar surveillance system capable of automatically finding people in video based on their fine-grained body parts and attributes.

Paper

Download the paper in PDF format.

Video

Download a video demo of our system (cool!). Some of the faces in the video were blurred for privacy reasons.

Citation

Daniel Vaquero, Rogerio Feris, Duan Tran, Lisa Brown, Arun Hampapur, and Matthew Turk. Attribute-Based People Search in Surveillance Environments. In IEEE Workshop on Applications of Computer Vision (WACV'09), Snowbird, Utah, December 2009.

BibTeX Entry

@InProceedings{VaqueroWACV2009,
  author    = {Daniel Vaquero and Rogerio Feris and Duan Tran and Lisa Brown and Arun Hampapur and Matthew Turk},
  title     = {Attribute-Based People Search in Surveillance Environments},
  booktitle = {IEEE Workshop on Applications of Computer Vision (WACV'09)},
  address   = {Snowbird, Utah},
  month     = {December},
  year      = {2009}
}

Related Publications

Ankur Datta, Rogerio Feris, and Daniel Vaquero. Hierarchical Ranking of Facial Attributes. IEEE Conference on Automatic Face and Gesture Recognition (FG 2011), Santa Barbara, California, March 22-24, 2011.
Daniel Vaquero, Rogerio Feris, Lisa Brown, Arun Hampapur, and Matthew Turk. Attribute-Based People Search. In Yunqian Ma and Gang Qian, editors. Intelligent Video Surveillance: Systems and Technology, Taylor and Francis Group, LLC, 2009. (book chapter)
Rogerio Feris, Arun Hampapur, Yun Zhai, Russell Bobbitt, Lisa Brown, Daniel Vaquero, Ying-Li Tian, Haowei Liu, and Ming-Ting Sun. Case Study: IBM Smart Surveillance System. In Yunqian Ma and Gang Qian, editors. Intelligent Video Surveillance: Systems and Technology, Taylor and Francis Group, LLC, 2009. (book chapter)
Daniel Vaquero, Rogerio Feris, Lisa Brown, Arun Hampapur, and Matthew Turk. People Search in Surveillance Videos. In Fourth Graduate Student Workshop on Computing (GSWC'09), Santa Barbara, California, October 2009. (extended abstract)