Bhanu B., Han J. Human Recognition at a Distance in Video

Springer, 2011. — 277 p.Biometrics is the study of methods for uniquely recognizing humans based upon one or more intrinsic physical or behavioral traits, including fingerprint, face, voice, gait, iris, signature, hand geometry, palm, ear, etc. Each biometric has its own relative merits, and the choice of a biometric trait for a particular application depends on a variety of issues. The inherent limitation of a single biometric can be alleviated by fusing the information presented by multiple sources. For example, the face and gait traits, or multiple images of face, or the fingerprints of the right and left index fingers of an individual may be used together to resolve the identity of an individual. A system that consolidates the evidence presented by multiple biometric sources is expected to be more reliable.
Most of the current biometric systems for human recognition generally require a cooperative subject, views from certain aspects and physical contact or close proximity. These systems alone cannot reliably recognize non-cooperating individuals at a distance since it has been difficult to recognize a person from arbitrary views when one is walking at a distance in real-world changing environmental conditions. For optimal performance, a system should make use of as much information as can possibly be obtained from the available observations. Gait and face are the two available biometrics which can be easily captured on a video that is acquired from a distance.
Gait is defined as the manner in which a person walks, and is one of the few biometric traits that can be used to identify non-cooperating humans at a distance. Recent advances in human gait analysis introduced a new application area of recognizing and identifying individuals, their gender, age and ethnicity (soft biometrics) and activities in a surveillance environment for security purposes. Most gait recognition algorithms attempt to extract the human silhouette in order to derive the spatio-temporal attributes of a moving individual. Hence, the selection of a good model to represent the human body is pivotal to the efficient functioning of a gait recognition system. Gait-based systems also offer the possibility of tracking an individual over an extended period of time and performing gait-based recognition in 3D with data available from multiple video streams in a surveillance network. However, gait can be affected by clothing, shoes, or environmental context. Special physical conditions such as injury can also change a person’s walking style. The large gait variation of the same person under different conditions (intentionally or unintentionally) reduces the discriminating power of gait as a biometric and it may not be as unique as fingerprint or iris, but the inherent gait characteristic of an individual still makes it irreplaceable and useful for human recognition in practical applications. Gait can be effectively utilized and combined with other biometrics for automatically detecting, recognizing and identifying individuals from a distance.
Face recognition is non-intrusive, and facial attributes are one of the most common features used to recognize an individual. The applications of facial recognition vary from a static, controlled mug-shot authentication to a dynamic, uncontrolled face identification in a cluttered background. Most of the face recognition approaches are based on either the location and shape of facial attributes, such as the eyes, eyebrows, nose, lips, and chin and their spatial relationships, or the overall analysis of the face image that represents a face as a weighted combination of a number of canonical faces. While face recognition is conveniently applied, it is easily affected by several factors including illumination, expression, pose, etc. It is also questionable whether the face itself has a sufficient basis for recognizing a person from a very large number of individuals with an extremely high level of confidence.
The general solution to analyze face and gait video data collected from arbitrary views is to estimate 3-D models. However, the problem of building reliable 3-D models of face and gait with non-rigid face, flexible neck and the articulated human body from low resolution video data is a challenging task. In this book, integrated face and gait recognition approaches are developed that exploit inherent characteristics of human signatures in video that is captured from a distance. Experimental results show the effectiveness of the current systems for human recognition at a distance in video.
This book addresses fundamental problems associated with gait, face and integrated gait and face based human recognition in color and infrared videos acquired at a distance under real-world environments.
For gait-based human recognition the book addresses the problems associated with the representation, the large intra-person variation of gait appearance under different environmental conditions, the lack of discrimination analysis for gait-based human recognition, and the difficulties associated with reliable moving human detection in various situations. Both model-free and model-based approaches are considered for individual recognition under varying contextual, environmental and carrying conditions. This includes the newly developed techniques where the both the model and the data (obtained from multiple cameras) are in 3D. Bayesian-based statistical analysis is performed to evaluate the discriminating power of gait features for human recognition. To improve the performance of moving human detection for both model-free and model-based human recognition, the information from color and infrared videos is combined using automatic image registration methods.
For face recognition in video with people at a distance, the challenges are precise registration of faces in low resolution video data and the robustness of superresolution techniques to variations in pose, lighting, facial expression and the number of video frames. The book presents new video-based techniques for face profilebased recognition and three techniques for super-resolution of frontal and side facial imagery acquired from a video. These techniques are based on (a) closed-loop tracking, (b) free-form deformation, and (c) elastic registration. Objective measures, to evaluate the quality of super-resolved imagery for face recognition, are presented based on different conditions encountered during the video capture.
For integrated gait and face biometrics the challenges are the development of effective techniques at different levels of abstraction. The book presents several systems that integrate information of the side view of face and gait from video data. Several fusion schemes are introduced at the match score and feature levels for the integration of super-resolved face and gait. Both face and gait recognition systems integrate information over multiple frames in a video sequence for improved performance.
Part I Introduction to Gait-Based Individual Recognition at a Distance.Part II Gait-Based Individual Recognition at a Distance.
Gait Representations in Video.
Model-Free Gait-Based Human Recognition in Video.
Discrimination Analysis for Model-Based Gait Recognition.
Model-Based Human Recognition—2D and 3D Gait.
Fusion of Color/Infrared Video for Human Detection.
Part III Face Recognition at a Distance in Video.
Super-Resolution of Facial Images in Video at a Distance.
Evaluating Quality of Super-Resolved Face Images.
Part IV Integrated Face and Gait for Human Recognition at a Distance in Video.
Integrating Face Profile and Gait at a Distance.
Match Score Level Fusion of Face and Gait at a Distance.
Feature Level Fusion of Face and Gait at a Distance.
Part V Conclusions for Integrated Gait and Face for Human Recognition at a Distance in Video.
Conclusions and FutureWork.