2016 Projects

Semantic Face Index
Don Adjeroh, Gianfranco Doretto, Jeremy Dawson (WVU)

Understandable Face Image Quality Assessment
Guodong Guo (WVU)

Cross Audio-to-Visual Speaker Identification in the Wild Using Deep Learning
Jeremy Dawson, Nasser Nasrabadi (WVU)

Nonlinear Mapping using Deep Learning for Thermal-to-visible Night-Time Face Recognition
Thirimachos Bourlai (WVU), Nasser Nasrabadi (WVU), Lawrence Hornak (UGA)

Accelerated Rapid Face in Video Recognition
Chen Liu, Stephanie Schuckers (CU)

Analysis of Vocal and Eye Behavior Countermeasures for Automated Border Control Applications
Aaron Elkins (UA/SDSU), Dmitry Gorodnichy (CBSA), Bradley Walls (UA), Jeffery Proudfoot (Bentley), Nathan Twyman (MST), & Judee Burgoon (UA).

Temporal Analysis of Non-Contact Biometric Modalities to Detect Document Fraud
Jay Nunamaker, Brad Walls (UA

Microfluidic 3D Capillary Network Test Phantom for Subdermal Vascular Imaging
Kwang Oh (UB)

Validation of Biometric Identification of Dairy Cows based on Udder Vein Images
Stephanie Schuckers, Sean Banerjee (CU)

Restoration of Distorted Fingerprints by Deep Learning
Dawson (WVU) and N. M. Nasrabadi (WVU)

Incorporating Biological Models for Iris Authentication in Mobile Environments
Bourlai and A. Clark (WVU)

Cross-spectral Face Recognition Using Thermal Polarization for Night Surveillance
M. Nasrabadi (WVU) and T. Bourlai (WVU)

Cross-Age Face Recognition in Non-Ideal Images
Guodong Guo (WVU)

Wardrobe Models for Long Term Re-Identification and Appearance Prediction
Prof. Napp, Prof. Sethlur, Prof. Govindaraju (UB)

Liveness Detection Competition 2017
Stephanie Schuckers, David Yambay, Mayank Vatsa, Afzel Noore (CU)

Impact of Cultural Factors in Trusting Biometric Technology
Zhaleh Semnani-Azad. Stephanie Schuckers (CU)

 

Summaries:

Semantic Face Index
Don Adjeroh, Gianfranco Doretto, Jeremy Dawson (WVU)

With the increasing use of cheap cameras (e.g., available in cell phones) coupled with the ease of distribution, there is an exponential increase in the size of available face databases. An important key challenge is in the ability to search over these large databases in order to identify a face that may be of interest. This is significantly  compounded in the case of face-in- the-wild, where the query face, the database faces, or both might have been captured in a non-constrained, non-ideal environment, with no control on the image quality, the illumination conditions, pose, partial occlusion, etc. We propose a human-readable indexing scheme to reduce these problems, and thus support a rapid access in very large scale face databases. The index is based on descriptors that are understandable to humans, which makes the approach easy to use, and suitable for law enforcement applications.

 

Understandable Face Image Quality Assessment
Guodong Guo (WVU)

The purpose Face recognition (FR) performance is affected significantly by the face image qualities, especially in real-world applications. Face image qualities vary significantly because of different imaging sensors, compression techniques, video frames, and/or image acquisition conditions/time. It is very challenging to assess face image qualities automatically, quickly and precisely in real world images [1][2]. Recent studies [2][3] have shown that a learning-based paradigm can do better than the traditional heuristic methods, however, all these approaches can only give a single quality score as the “output,” e.g. 90, for an input face image. The “single-value quality score” cannot tell much information to communicate to human assessors. Further, many issues have not been addressed yet, e.g.,  What does a quality score mean? How to interpret a quality score with imaging conditions? Why a face image has a quality score of 50 rather than 60? How well the quality scores characterize the real face image qualities? Can more useful cues (e.g., levels of details) be acquired to develop a complete representation for face image quality assessment? In this project, we propose a new paradigm, called understandable face image quality assessment, to address the issues in quality assessment of face images. We believe that the new paradigm can give a better solution for quality assessment. The objective of this project is to explore a new paradigm and develop a new approach for face image quality assessment with rich information, making quality measures understandable, believable, and more accurate.

 

Cross Audio-to-Visual Speaker Identification in the Wild Using Deep Learning 
Jeremy Dawson, Nasser Nasrabadi (WVU)

Speaker recognition technology has achieved significant performance for some real-world applications.  However, the performance of speaker recognition is still greatly degraded when used in noisy environments. One approach to improve speech recognition/identification is by combining video and audio sources to link the visual features of lip motion with vocal features, two modalities which are correlated and convey complementary information. In this project we are not interested in a baseline improvement in speaker recognition, but instead, we are interested in identifying an individual face from a coupled video/audio clip of several individuals based on data collected in an unrestricted environment (wild). For this effort, we are proposing to use visual lip motion features for a face in a video clip and the co-recorded audio signal features from several speakers to identify the individual who uttered the audio recorded along with the video.  To solve this problem, we are proposing to use an auto-associative deep neural network architecture which is a data-driven model and does not model phonemes or visemes (the visual equivalent of a phoneme). A speech-to-video auto-associative deep network will be used where the network has learned to reconstruct the visual lip features given only speech features as the input. The visual lip feature vector generated by our deep network for an input test speech signal will be compared with a gallery of individual visual lip features for speaker identification. The proposed speech-to-video deep network will be trained with our current WVU voice and video training dataset using the corresponding audio and video features from individuals as inputs to the network. For the audio signal we will use the Mel-frequency cepstral coefficients (MFCC), and for video, we will extract static and temporal visual features of the lip motion.

 

Nonlinear Mapping using Deep Learning for Thermal-to-visible Night-Time Face Recognition 
Thirimachos Bourlai (WVU), Nasser Nasrabadi (WVU), Lawrence Hornak (UGA)

Infrared (IR) thermal cameras are important for night-time surveillance and security applications.  They are especially useful in nighttime scenarios when the subject is far away from the camera. The motivation behind thermal face recognition (FR) is the need for enhanced intelligence gathering capabilities in darkness where active illumination is impractical and when surveillance with visible cameras is not feasible.  However, the acquired thermal face images have to be identified using the images from existing visible face databases. Therefore, cross-spectral face matching between the thermal and visible spectrum is a much desired capability. In cross-modal face recognition, identifying a thermal probe image based on a visible face database is especially difficult because of the wide modality gap between thermal and visible physical phenomenology. In this project we address the cross-spectral (thermal vs. visible) and cross-distance (50 m, 100 m, and 150 m vs. 1 m standoff) face matching problem for night-time FR applications.  Previous research activities [1]-[2] have mainly concentrated on extracting hand-crafted features (i.e., SIFT, SURF, HOG, LBP, wavelets, Gabor jets, kernel functions) by assuming that the two modalities share the same extracted features. However, the relationship between the two modalities is highly non-linear. In this project we investigate non-linear mapping techniques based on deep neural networks (DNN) learning procedures to bridge the modality gap between visible-thermal spectrums while preserving the subject identity information. The nonlinear coupled DNN features will be used by a FR classifier.

 

Accelerated Rapid Face in Video Recognition
Chen Liu, Stephanie Schuckers (CU)

Recently face in video recognition has gained great attention due to the need of such applications arisen from video surveillance and other purposes. Performing real-time face tracking on surveillance videos in live stream and perform face recognition at the same time poses a great computational challenge. In facing this challenge, we propose to attack this problem from both algorithm design and hardware acceleration sides. We propose an innovative key-frame extraction algorithm based on improved quality analysis on facial components from the video stream. Then we will utilize the face(s) extracted from key-frames for face recognition and matching. We will focus on the quality analysis of faces in frame as this is a critical step towards improving the recognition speed, since good quality analysis will reduce the field of what needs to be processed by the face recognition engine. We will employ Graphic Processing Unit (GPU) to accelerate both the key frame extraction and face recognition algorithms through extracting the thread-level and data-level parallelism in order to meet real-time requirement on mobile platform and fast-than-real-time processing speed on server platform. We anticipate this project will pave the way towards designing innovative face-in-video recognition systems that can be referenced by both industry and government agencies on such applications.

 

Analysis of Vocal and Eye Behavior Countermeasures for Automated Border Control Applications
Aaron Elkins (UA/SDSU), Dmitry Gorodnichy (CBSA), Bradley Walls (UA), Jeffery Proudfoot (Bentley), Nathan Twyman (MST), & Judee Burgoon (UA).

 This project will be a follow-on to the current IATS4ABC CITeR project, conducting secondary data analysis on the vocal and eye tracking data that was gathered during the testing of AVATAR at CBSA. It will include statistical analysis  of the distributions of the measures, their degree of homogeneity, and whether anomalies can discriminate between innocent and guilty passengers.

 

Temporal Analysis of Non-Contact Biometric Modalities to Detect Document Fraud
Jay Nunamaker, Brad Walls (UA

Our society is in the midst of a fraudulent document crisis.  According to a January 2016 Politico article, Europe’s trade in fraudulent (i.e. forged and stolen) passports is so out of control that the U.S. has given five European Union (EU) countries the ultimatum to act or risk losing visa-free travel rights.  This is not a new problem, in fact, the same Politico article presents data from Interpol that illustrates lost and stolen travel documents numbered 15 to 16 million in 2010 with the problem growing to greater than 50 million in 2015.  This project will evaluate non-contact biometric modalities coupled with an automated interviewing system that could be used in numerous border crossing scenarios to assist with the detection of fraudulent travel documentation.

 

Microfluidic 3D Capillary Network Test Phantom for Subdermal Vascular Imaging
Kwang Oh (UB)

Leveraging the Sensors and MicroActuators Learning Lab’s (SMALL at University at Buffalo) expertise in both microfluidics and test phantoms we will create a physiologically accurate model of the human finger. It will be acoustically, electrically, and optically equivalent to that of the human finger. This finger test phantom will include dermotographic features, such as ridge valley structures, digital arteries, bone, fat, muscle, and a fully functioning 3 dimensional (3D) capillary network. Subdermal vascular networks can be used another form of a biometric providing a high level of security. Being able to have a controlled test phantom (i.e., blood flow, heart rate, bone structure, fat and muscle thickness, as well as a known capillary design) will allow for advanced sensor/algorithm testing, validation, and calibration.

 

Validation of Biometric Identification of Dairy Cows based on Udder Vein Images
Stephanie Schuckers, Sean Banerjee (CU)

The purpose of this proposal is to develop a biometric recognition system to identify cows based on a NIR image of the vein pattern on the udder.  Safety of food is of critical concern.  In dairy cows, the milk from dairy cows which are infected and/or on antibiotics must be separated from the other milk from the herd, requiring recognition of a specific cow.  Currently, methods such as RF ID tags and ankle bands are used to identify cows.  However, ankle bands can be dislodged.  RF id tags are read when cows enter the milking area, but the cows sometimes get out of order when entering the stalls.  Confirmation of identity once the cow is in the stall would be useful, but at that point the RF ID tag is far away from the back of the cow where it is being milked.  A biometric collected from the back of the cow could be used to confirm identity of the cow at the time the cow is milked.  Iris patterns which have been studied in cows are also too far away. This study will consider the vein pattern of the udder for its biometric recognition properties.  There has been extensive study and commercial products which are based on recognition of the vein pattern of the hand, finger, retina, and sclera of the eye.  The knowledge of this field will be used for development of a system for udder vein recognition in cows.  The focus of this project will be to assess uniqueness and permanence of cow udder veins to validate its potential usefulness for recognizing cows.

 

Restoration of Distorted Fingerprints by Deep Learning
Dawson (WVU) and N. M. Nasrabadi (WVU)

Automatic fingerprint technology has become a highly accurate method for identification of individuals for commercial as well as DoD applications. However, there still exists challenging problems with low-quality or distorted fingerprints. Degradation or distortion of fingerprint can be photometric (non-ideal skin conditions, dirty sensor surface, latent fingerprints) or geometrical (skin distortion due to uncooperative person) [1].  In this project, we are interested in developing an algorithm to correct for geometrical elastic deformations due to flexibility of the fingerprint or the lateral force or torque purposely introduced by an uncooperative person during finger printing. This kind of distortion can be seen in the classical FVC2004 DB1 fingerprint database, and a fingerprint matcher will not be able to identify the individual.  To solve this problem, a number of techniques have been developed that make the fingerprint matcher tolerant to distortion [2, 3] or learn the nonlinear deformation from a training set [1 ,4].  In this project, we are proposing to use a deep learning architecture (an auto-encoder) to learn to correct various types of geometrical distortion by using a large database consisting of pairs of distorted and normal fingerprints of individuals. The input to our deep auto-encoder will be a distorted fingerprint, and its output will be the rectified version of the normal fingerprint.  Our proposed data-driven auto-encoder not only learns implicitly the nonlinear distortion patterns, but also corrects for distortions that are similar to the training dataset. In this project we will use the available FVC2004 DBI and Tsinghua DF databases which have pairs of distorted and normal fingerprints.  Our approach is novel since no one has previously used deep learning auto-encoders for distorted fingerprint corrections.

 

Incorporating Biological Models for Iris Authentication in Mobile Environments
Bourlai and A. Clark (WVU)

For the past decade iris recognition technology has matured from access control to the security of mobile devices. However, recent studies demonstrate the impact of pupil dilation on the matching accuracy of iris recognition algorithms where investigations were held solely from dilation [4 – 6], deformation [1,2], template [3], and state [5] perspectives. Furthermore, working in mobile environments pose additional restrictions such as platform fragmentation and data limitations. Therefore, there is a need to collectively incorporate the physiological aspects of dilation while considering the additional constraints of mobile environments. This proposal aims to consider these perspectives in order to engineer a universal process for improved recognition within mobile environments. The results of this work can advance the iris recognition community via advancing current iris recognition algorithms that are effective across multiple technical environments. Furthermore, these results aim to provide additional insights into the performance metrics within the mobile framework.  The three main tasks are stated in the “Experimental Plan” section below.

 

Cross-spectral Face Recognition Using Thermal Polarization for Night Surveillance
M. Nasrabadi (WVU) and T. Bourlai (WVU)

Face Recognition (FR) in the visible spectrum is sensitive to illumination variations, and is not practical in low-light or nighttime surveillance. In contrast, thermal imaging is ideal for nighttime surveillance and intelligence gathering operations. However, conventional thermal imaging lacks textural details that can be obtained from polarimetric signatures. Such signatures can be used to infer face-based surface features and enhance night-time human identification. In this project, we are proposing to explore novel thermal polarimetric signatures to extract subtle surface features of the face to improve thermal cross-spectral FR performance. The state of polarization at each pixel can be represented by a Stoke vector representation, which is a four-component vector, whose elements are functions of the optical field polarization. The first component of the Stoke vector is the linear polarization (the image itself) and the remaining three components are the differences between the polarizations.  Our proposed algorithm is based on the framework of sparse theory, which uses a set of dictionaries (a multi-polarimetric dictionary), where each dictionary is dedicated for a particular stoke polarization component (stoke image). This multi-polarimetric dictionary will be used to jointly map the information in visible and all the polarimetric Stoke images into a common surrogate feature space (referred to as sparse coefficient vector). Then, a classifier will be designed in this common feature space, from a gallery of visible database, to identify the thermal polarimetric Stoke image probes that will be used for cross-spectral FR. The innovation in our approach is the use of polarimetric Stoke signatures and development of a common surrogate feature space to relate the visible to thermal polarimetric Stoke images. This project will assess the overall performance improvement in the use of polarimetric signatures for cross-spectral FR.

 

Cross-Age Face Recognition in Non-Ideal Images
Guodong Guo (WVU)

Face recognition (FR) performance is affected significantly by the aging in human faces. Aging is inevitable and continuous for all live people. Facial aging can change the facial appearance greatly, making it very challenging to match face images across age progression. While some progresses have been made over the past decade to deal with pose, illumination, and expression changes (PIE), it is still very hard to do cross-age face recognition. Further, it is even harder to perform face recognition with aging in non-ideal conditions. For example, when the facial aging is coupled with pose, illumination, and expression changes, it brings new challenges, which can be called A-PIE (aging, pose, illumination, and expression changes).

In this project, we propose to study a relatively new problem, called cross-age face recognition in non-ideal face images. We will study the influence of aging (A) on face recognition in a quantitative manner, coupled and/or decoupled with the PIE. The objective is to explore the aging effect on face recognition quantitatively, and compare to the traditional pose, illumination, and expression changes (PIE). The outcome of the project will have a significant impact on practical face matching, where a large number of non-ideal face images exist with aging.

 

Wardrobe Models for Long Term Re-Identification and Appearance Prediction
Prof. Napp, Prof. Sethlur, Prof. Govindaraju (UB)

Clothing has been extensively used as a soft biometric for tracking and re-identification (both explicitly. eg [1] and explicitly through color/texture appearance models [2]). We believe that this signal has not been fully exploited. Clothing, specifically a wardrobe (collection of clothes), could be used over much longer time scales than a typical tracking re-identification task. The proposal is to build an explicit wardrobe model and see how it can be used as a soft biometric signal in long-term and long-range identification tasks. Once a wardrobe model is learned it can be used in camera views where subjects are too small for other modalities.

 

Liveness Detection Competition 2017
Stephanie Schuckers, David Yambay, Mayank Vatsa, Afzel Noore (CU)

Fingerprint and Iris recognition systems are vulnerable to artificial presentation attacks, such as molds made of silicone or gelatin for fingerprint and patterned contacts for iris. Liveness Detection, or Presentation Attack Detection (PAD), has been a growing field to combat these types of attacks. LivDet 2015 showcased 13 total competitors for the fingerprint competition and 3 competitors for iris. We propose to host LivDet 2017 with a similar public dataset for training algorithms and systems. In addition, we propose to incorporate updated spoof attacks to showcase advances in the field of spoofing such as 3D printed molds for fingerprint and mobile capture for iris. We will continue testing for submitted fingerprint and iris full hardware/software systems with a particular emphasis on mobile devices. Analysis of performance will enable knowledge of the state-of-the-art in the field, as new technologies and algorithms evolve.

 

Impact of Cultural Factors in Trusting Biometric Technology
Zhaleh Semnani-Azad. Stephanie Schuckers (CU)

Societal acceptance of biometric technology is complex, and highly dependent on trust. People often perceive biometric technology as a ‘machine profiling’ entity where the person being profiled has no access to the knowledge (e.g. database) that is used to categorize that person. This element of biometrics can lower trust which is developed from transparency and disclosure of information. Trust captures the extent to which people are comfortable with being vulnerable to another entity, with the expectation that the entity will not exploit them. Yet, trust is highly contingent on cultural and societal norms. The limited work on trust in biometrics is mostly anecdotal and correlational patterns associated with familiarity and confidence in different types of biometrics. There are two general limitations of current literature. First, to our knowledge there is no research that systematically examines the impact of various cultural factors such as tightness versus looseness of social norms on general trust toward different types of biometrics. Most of the work on cultural influences on trust has been done in the context of interpersonal trust. Second, most of the very limited work studying trust in biometrics has been abstract and suggestive, without empirical validation. To address these issues, the overall objective of our research is to develop  a fundamental understanding of general principles and factors pertaining to trust in biometrics, and how trust mediates acceptance of biometrics across various cultural norms.

.