WASHINGTON--(BUSINESS WIRE)--Face recognition has become a key tool for unobtrusive human identification broadly applied in areas of surveillance, forensic, security, access control, etc. In recent years, although the accuracy of today’s visible-spectrum facial-recognition systems has rapidly increased and are robust to various conditions such as illumination, pose and expressions, there are some challenges that compromise the efficacy of these systems used in broader areas. The application of face paints or cosmetics, for example, is one of these challenges, which can alter the facial signature and severely inhibit biometric facial-recognition systems based on visible imaging.
Now a team of researchers, Drs. Nathaniel Short, Alex Yuffa, Gorden Videen, and Shuowen Hu, from U.S. Army Research Laboratory, Maryland, USA, have developed a new technique exploiting thermal-imaging that potentially could mitigate the impact to facial-recognition performance caused by cosmetics materials. By comparing visible, conventional thermal and polarimetric-thermal images of faces before and after the application of face paint, the researchers have found that surface materials have relatively little impact on polarimetric-thermal imagery, demonstrating the efficacy of polarimetric-thermal imaging for recognizing faces in the presence of make-up. The research team describe the work in the journal Applied Optics, from The Optical Society (OSA).
“Our study has demonstrated polarimetric-thermal imaging can be substantially more robust to face paints, and to a degree cosmetics, for facial recognition than visible imaging,” said Nathaniel Short, the lead researcher, who is on contract with U.S. Army Research Laboratory as part of the Image Processing Branch. “Our experiments show how face paints and cosmetics degrade the performance of traditional facial-recognition methods and we provide a new approach to mitigating this effect using polarimetric-thermal imaging.”
The study is also the first effort to investigate the effect of face paints and cosmetic materials on the polarimetric-thermal facial signature, Short said. The researchers believe a polarimetric-thermal-based system could provide day and night facial-recognition capability for surveillance or security applications, with additional advantages over traditional face-recognition systems, especially in nighttime conditions and in the presence of face paints and cosmetics.
Traditional facial-recognition systems are based on matching clear and well-lit photos taken in the broad light. Recognizing faces using visible-light imaging relies heavily on capturing the reflected light from the key edges of the faces, such as edges of the eyes, nose and mouth, which poses a challenge when faces are covered with cosmetics, as the reflected light from face edges are highly dependent upon the source of illumination and the spectral characteristics of the facial tissue or any material applied to the face. In other words, cosmetics materials distort the perceived shape of the face and undermine the face-recognition accuracy of visual imaging due to the different spectral properties of color pigmentation.
In contrast, an infrared, thermal signature is naturally emitted from the human face and can be acquired passively in low-illumination conditions and even if face paints or cosmetics cover the surface of the skin. In recent years, thermal imaging has been investigated as a new modality for face recognition, especially in poor lighting conditions such as at night. However, since conventional thermal imaging only measures thermal intensity (heat) or temperature of faces, the thermal facial imagery usually lacks detailed shape and texture information. The thermal imagery is also affected by the temperature of the skin and the environment.
Polarimetric-thermal imaging, a maturing thermal mode that records the polarization-state information of thermal infrared emission, allows for the collection of geometric facial information from thermal imagery and may provide advantages over conventional thermal imaging in the presence of face paints or cosmetics, according to Short.
“An electromagnetic wave is composed of an electric field and a magnetic field. In conventional thermal imaging, only the strength of the electric field is recorded and subsequently used in a face-recognition algorithm. In polarimetric-thermal imaging, both direction and strength of an electric field is recorded and subsequently used in the algorithm,” Short explained. “The direction of the electric field is very sensitive to the morphology of the thermally emitting surface (surface slopes and roughness) and invariant to facial temperature differences. In other words, polarimetric imagery contains key edge details of a thermally emitting surface (e.g., face) needed by a facial-recognition algorithm.”
Since the way the human face emits thermal signatures is different than the way it reflects daylight, there is no linear or direct correlation between faces in visible and infrared light. In an early published study, the researchers developed several non-linear mapping techniques and assessed the performance of cross-spectrum polarimetric-thermal imaging to visible facial recognition, achieving a recognition rate of 93.3 percent using a dataset of 60 subjects. Short said, though there may not be a direct correlation between the visible and infrared facial signatures, the key edges of the human face are correlated to a large extent, which can be used to match infrared imagery with regular visible photos.
“We are matching them not in image space, but in a mapped feature space, which yields significantly higher performance,” he said.
As a continuous effort assessing the capability of polarimetric-thermal-imaging facial recognition, in this study, the team examined the effect of various cosmetics materials on the facial signature acquired by polarimetric-thermal imagers and compared how these materials affected the recognition of human faces differently in visible, thermal and polarimetric-thermal imageries.
“We found cosmetics and other face paints can significantly degrade facial recognition in the visible imagery, but have relatively little impact on the polarimetric-thermal facial signature. We believe non-visible spectral-imaging techniques such as polarimetric-thermal imaging may facilitate robust face recognition under a number of challenging conditions,” Short said.
However, the development of the new facial-recognition technique is still at the initial stage. There are several challenges that the researchers plan to work on in the next step.
“One of the major challenges is the limitation of the existing polarimetric-thermal facial database,” Short noted. “Large sample pools are needed to develop and train complex machine-learning techniques such as neural networks, computer programs that attempt to imitate the human brain to make connections and draw conclusions.”
Another fundamental challenge lies in developing algorithms that overcome the large modality gap between visible imaging and polarimetric-thermal imaging for cross-spectrum recognition. Although the researchers have achieved some success, more work needs to be done.
Paper: Nathaniel J. Short, Alex J. Yuffa, Gorden Videen, and Shuowen Hu, "Effects of surface materials on polarimetric-thermal measurements: applications to face recognition," Appl. Opt. 55, 5226-5233 (2016). DOI: 10.1364/AO.55.005226.
About Applied Optics
Applied Optics publishes in-depth peer-reviewed articles emphasizing applications-centered research in optics. Published three times each month, Applied Optics reports significant practical applications of optical science and technology, photonics, lasers, information processing, sensing and environmental optics. It is edited by Ronald Driggers, St. Johns Optical Systems, USA. For more information, visit Applied Optics.
About The Optical Society
Founded in 1916, The Optical Society (OSA) is the leading professional organization for scientists, engineers, students and entrepreneurs who fuel discoveries, shape real-life applications and accelerate achievements in the science of light. Through world-renowned publications, meetings and membership initiatives, OSA provides quality research, inspired interactions and dedicated resources for its extensive global network of optics and photonics experts. For more information, visit osa.org/100.
