Automatic Ear Detection and Segmentation over Partially Occluded Profile Face Images
Automated, non invasive ear detection in images and video is becoming increasingly required in several contexts, including nonivasive biometric identification, biomedical analysis, forensics, and many others. In biometric recognition systems, fast and robust ear detection is a crucial step within the recognition pipeline. Existing approaches to ear detection are susceptible to fail in the presence of typical everyday situations that prevent a crisp imaging of the ears, like partial occlusions, ear accessories, or uncontrolled camera and illumination conditions. Even more, most of the proposed solutions work efficiently only within a previously detected rectangular region of interest, which limits their applicability and lowers the accuracy of the overall detection. In this paper we evaluate the use of Convolutional Neural Networks (CNNs) together with Geometric Morphometrics (GM) for automatic ear detection in the presence of partial occlusions, and a Convex Hull algorithm for the ear area segmentation. A CNN was trained with a set of ear images landmarked by experts using GM to achieve high consistency. After training, the CNN is able to detect ears over profile faces, even in the presence of partial occlusions. We analyze the performance of the proposed ear detection and segmentation method over partially occluded ear images using the CVL Dataset
I. Alberink and A. Ruifrok, “Performance of the FearID earprint identification system.,” Forensic science international, vol. 166, pp. 145–54, mar 2007.
C. Sforza, G. Grandi, M. Binelli, D. G. Tommasi, R. Rosati, and V. F. Ferrario, “Age- and sexrelated changes in the normal human ear.,” Forensic science international, vol. 187, pp. 110.e1–7, may 2009.
M. I. S. Ibrahim, M. S. Nixon, and S. Mahmoodi, “The effect of time on ear biometrics,” in 2011 International Joint Conference on Biometrics (IJCB), pp. 1–6, IEEE, oct 2011.
ˇ Z. Emerˇsiˇc, V. ˇ Struc, and P. Peer, “Ear recognition: More than a survey,” Neurocomputing, vol. 255, pp. 26–39, 2017.
M. L. Zelditch, D. L. Swiderski, H. D. Sheets, and W. L. Fink, “Geometric morphometrics for biologists,” Elsevier, vol. 59, no. 3, p. 457, 2004.
C. Cintas, M. Quinto-S´anchez, V. Acu˜na, C. Paschetta, S. De Azevedo, C. Silva de Cerqueira, V. Ramallo, C. Gallo, G. Poletti, M. C. Bortolini, S. Canizales-Quinteros, R. Francisco, G. Bedoya, A. Ruiz-Linares, R. Gonzal´ez-Jos´e, and C. Delrieux, “Automatic ear detection and feature extraction using Geometric Morphometrics and Convolutional Neural Networks,” IET Biometrics, dec 2016.
F. Solina, P. Peer, B. Batagelj, S. Juvan, and J. Kovaˇc, “Color-based face detection in the” 15 seconds of fame” art installation,” Proceedings of Mirage 2003, Conference on Computer Vision / Computer Graphics, pp. 38–47, 2003.
S. Pflug and C. Busch, “Ear biometrics: a survey of detection, feature extraction and recognition methods,” IET Biometrics, vol. 1, no. 2, p. 114, 2012.
H. Chen and B. Bhanu, “Shape Model-Based 3D Ear Detection from Side Face Range Images,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) -Workshops, vol. 3, pp. 122–122, IEEE.
H. Chen and B. Bhanu, “Contour matching for 3D ear recognition,” in Seventh IEEE Workshop on Applications of Computer Vision, WACV 2005, pp. 123–128, 2007.
S. Attarchi, K. Faez, and A. Rafiei, Advanced Concepts for Intelligent Vision Systems, vol. 5259 of Lecture Notes in Computer Science. Berlin, Heidelberg: Springer Berlin Heidelberg, oct 2008.
S. Ansari and P. Gupta, “Localization of ear using outer helix curve of the ear,” International conferenceon computing:the theoryand applications, pp. 688–692, 2007.
S. Prakash and P. Gupta, “An efficient ear localization technique,” Image and Vision Computing, vol. 30, no. 1, pp. 38–50, 2012.
P. Yan and K.W. Bowyer, “Biometric recognition using 3D ear shape.,” IEEE transactions on pattern analysis and machine intelligence, vol. 29, no. 8, pp. 1297–308, 2007.
A. Pflug, A. Winterstein, and C. Busch, “Robust localization of ears by feature level fusion and context information,” in Biometrics (ICB), 2013 International Conference on, pp. 1–8, IEEE, 2013.
A. Abaza, C. Hebert, and M. A. F. Harrison, “Fast learning ear detection for real-time surveillance,” IEEE 4th International Conference on Biometrics: Theory, Applications and Systems, BTAS 2010, 2010.
S. M. S. Islam, M. Bennamoun, and R. Davies, “Fast and fully automatic ear detection using cascaded adaboost,” 2008 IEEE Workshop on Applications of Computer Vision, WACV, 2008.
L. Yuan, W. Liu, and Y. Li, “Non-negative dictionary based sparse representation classification for ear recognition with occlusion,” Neurocomputing, vol. 171, pp. 540–550, 2016.
A. Kumar and T.-S. T. Chan, “Robust ear identification using sparse representation of local texture descriptors,” Pattern recognition, vol. 46, no. 1, pp. 73–85, 2013.
T.-S. Chan and A. Kumar, “Reliable ear identification using 2-d quadrature filters,” Pattern Recognition Letters, vol. 33, no. 14, pp. 1870–1881, 2012.
R. Purkait and P. Singh, “A test of individuality of human external ear pattern: its application in the field of personal identification.,” Forensic science international, vol. 178, no. 2-3, pp. 112–8, 2008.
I. Ercan, S. T. Ozdemir, A. Etoz, D. Sigirli, R. S. Tubbs, M. Loukas, and I. Guney, “Facial asymmetry in young healthy subjects evaluated by statistical shape analysis,” Journal of Anatomy, vol. 213, no. 6, pp. 663–669, 2008.
P. Mitteroecker and P. Gunz, “Advances in Geometric Morphometrics,” 2009.
D. Slice, “Modern morphometrics,” in Modern morphometrics in physical anthropology, pp. 1–46, 2005.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.
S. Dieleman, K. W. Willett, and J. Dambre, “Rotation-invariant convolutional neural networks for galaxy morphology prediction,” Monthly Notices of the Royal Astronomical Society, vol. 450, no. 2, pp. 1441–1459, 2015.
K. Fukushima, “Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position,” Biological Cybernetics, vol. 36, no. 4, pp. 193–202, 1980.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2323, 1998.
A. Toshev and C. Szegedy, “DeepPose: Human Pose Estimation via Deep Neural Networks,” in Computer Vision and Pattern Recognition (CVPR), pp. 1653–1660, 2014.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” Advances In Neural Information Processing Systems, pp. 1–9, 2012.
Y.-L. Boureau, J. Ponce, and Y. Lecun, “A Theoretical Analysis of Feature Pooling in Visual Recognition,” Proceedings of the 27th International Conference on Machine Learning (2010), pp. 111–118, 2010.
A. Ruiz-Linares, K. Adhikari, V. Acu˜na-Alonzo, M. Quinto-Sanchez, C. Jaramillo, W. Arias, M. Fuentes, M. Pizarro, P. Everardo, F. de Avila, J. G´omez-Vald´es, P. Le´on-Mimila, T. Hunemeier, V. Ramallo, C. C. Silva de Cerqueira, M.-W. Burley, E. Konca, M. Z. de Oliveira, M. R. Veronez, M. Rubio-Codina, O. Attanasio, S. Gibbon, N. Ray, C. Gallo, G. Poletti, J. Rosique, L. Schuler-Faccini, F. M. Salzano, M.-C. Bortolini, S. Canizales-Quinteros, F. Rothhammer, G. Bedoya, D. Balding, and R. Gonzalez-Jos´e, “Admixture in Latin America: geographic structure, phenotypic diversity and self-perception of ancestry based on 7,342 individuals.,” PLoS genetics, vol. 10, p. e1004572, sep 2014.
S. Dieleman, J. Schl¨uter, C. Raffel, E. Olson, S. K. Sønderby, D. Nouri, D. Maturana, M. Thoma, E. Battenberg, J. Kelly, J. D. Fauw, and M. H. et al., “Lasagne: First release.,” Aug. 2015.
L. yuan and Z. C. Mu, “Ear detection based on skin-color and contour information,” in 2007 InternationalConference on Machine Learning and Cybernetics, vol. 4, pp. 2213–2217, Aug 2007.
P. Viola and M. Jones, “Robust real-time object detection,” International Journal of Computer Vision, vol. 57, pp. 137–154, 2001.
C. B. Barber, D. P. Dobkin, and H. Huhdanpaa, “The quickhull algorithm for convex hulls,” ACM Transactions on Mathematical Software, vol. 22, no. 4, pp. 469–483, 1996.
Copyright (c) 2019 Celia Cintas, Claudio Delrieux, Pablo Navarro, Mirsha Quinto-Sánchez, Bruno Pazos, Rolando Gonzalez-José
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.