Fast Facial Landmark Detection and Applications: A Survey

Authors

DOI:

https://doi.org/10.24215/16666038.22.e02

Keywords:

Computer Vision, Edge Computing, Facial Landmarks, Neural Networks, Mobile Applications, Literature Overview

Abstract

Dense facial landmark detection is one of the key elements of face processing pipeline. It is used in virtual face reenactment, emotion recognition, driver status tracking, etc. Early approaches were suitable for facial landmark detection in controlled environments only, which is clearly insufficient. Neural networks have shown an astonishing qualitative improvement for in-the-wild face landmark detection problem, and are now being studied by many researchers in the field. Numerous bright ideas are proposed, often complementary to each other. However, exploration of the whole volume of novel approaches is quite challenging. Therefore, we present this survey, where we summarize state-of-the-art algorithms into categories, provide a comparison of recently introduced in-the-wild
datasets (e.g., 300W, AFLW, COFW, WFLW) that contain images with large pose, face occlusion, taken in unconstrained conditions. In addition to quality, applications require fast inference, and preferably on mobile
devices. Hence, we include information about algorithm inference speed both on desktop and mobile hardware, which is rarely studied. Importantly, we highlight problems of algorithms, their applications, vulnerabilities, and
briefly touch on established methods. We hope that the reader will find many novel ideas, will see how the algorithms are used in applications, which will enable further research. 

Downloads

Download data is not yet available.

References

X. Zhang, X. Zhou, M. Lin, and J. Sun, “Shufflenet: An extremely efficient convolutional neural network for mobile devices,” in 2018

IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 6848–6856. DOI: 10.1109/CVPR.2018.00716.

M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 4510–4520. DOI: 10 . 1109/CVPR.2018.00474.

N. Wang, X. Gao, D. Tao, H. Yang, and X. Li, “Facial feature point detection: A comprehensive survey,” Neurocomputing, vol. 275, pp. 50–65, 2018, ISSN: 0925-2312. DOI: 10 . 1016 / j . neucom . 2017 . 05 . 013. [Online]. Available: https : / / www.sciencedirect .com/science/article /pii/S0925231217308202.

Y. Wu and Q. Ji, “Facial landmark detection: A literature survey,” Int. J. Comput. Vision, vol. 127, no. 2, pp. 115–142, Feb. 2019, ISSN: 0920-5691. DOI: 10 . 1007 / s11263 - 018 -1097-z.

W. Wu, C. Qian, S. Yang, Q. Wang, Y. Cai, and Q. Zhou, “Look at boundary: A boundary-aware face alignment algorithm,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 2129–2138. DOI: 10 .1109/CVPR.2018.00227.

A. Kumar, T. K. Marks, W. Mou, et al., “Luvli face alignment: Estimating landmarks’ location, uncertainty, and visibility likelihood,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 8233–8243. DOI: 10.1109/CVPR42600. 2020.00826.

X. Wang, L. Bo, and L. Fuxin, “Adaptive wing loss for robust face alignment via heatmap regression,” in 2019 IEEE/CVF International

Conference on Computer Vision (ICCV), 2019, pp. 6970–6980. DOI: 10.1109/ICCV.2019.00707.

C. Sagonas, G. Tzimiropoulos, S. Zafeiriou, and M. Pantic, “300 faces in-the-wild challenge: The first facial landmark localization challenge,” in 2013 IEEE International Conference on Computer Vision Workshops, 2013, pp. 397–403. DOI: 10.1109/ICCVW.2013.59.

P. N. Belhumeur, D. W. Jacobs, D. J. Kriegman, and N. Kumar, “Localizing parts of faces using a consensus of exemplars,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 12, pp. 2930– 2940, 2013. DOI: 10.1109/TPAMI.2013.23.

X. Zhu and D. Ramanan, “Face detection, pose estimation, and landmark localization in the wild,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, June 16-21, 2012, IEEE Computer Society, 2012, pp. 2879–2886. DOI: 10.1109/CVPR.2012.6248014.

V. Le, J. Brandt, Z. Lin, L. D. Bourdev, and T. S. Huang, “Interactive facial feature localization,” in Computer Vision - ECCV 2012 - 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part III, ser. Lecture Notes in Computer Science, vol. 7574, Springer, 2012, pp. 679–692. DOI: 10.1007/978-3-642-33712-3_49.

K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maˆıtre, “Xm2vtsdb: The extended m2vts database,” vol. 964, 1999, pp. 965–966.

X. Zhu, Z. Lei, X. Liu, H. Shi, and S. Z. Li, “Face alignment across large poses: A 3d solution,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, IEEE Computer Society, 2016, pp. 146–155. DOI: 10. 1109/CVPR.2016.23.

C. Zhu, X. Li, J. Li, and S. Dai, “Improving robustness of facial landmark detection by defending against adversarial attacks,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2021, pp. 11 751–11 760.

M. K¨ostinger, P. Wohlhart, P. M. Roth, and H. Bischof, “Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization,” in 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), 2011, pp. 2144–2151. DOI: 10.1109/ICCVW.2011.6130513.

S. Qian, K. Sun, W. Wu, C. Qian, and J. Jia, “Aggregation via separation: Boosting facial landmark detector with semi-supervised style translation,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 10 152–10 162. DOI: 10.1109/ICCV.2019. 01025.

X. P. Burgos-Artizzu, P. Perona, and P. Doll´ar, “Robust face landmark estimation under occlusion,” in 2013 IEEE International Conference on Computer Vision, 2013, pp. 1513–1520. DOI: 10.1109/ICCV.2013.191.

G. Ghiasi and C. C. Fowlkes, “Occlusion coherence: Detecting and localizing occluded faces,” CoRR, vol. abs/1506.08347, 2015. arXiv: 1506.08347. [Online]. Available: http://arxiv.org/abs/1506.08347.

S. Zafeiriou, G. Trigeorgis, G. Chrysos, J. Deng, and J. Shen, “The menpo facial landmark localisation challenge: A step towards the solution,” in Computer Vision and Pattern Recognition (CVPR) Workshops, 2017.

J. Deng, A. Roussos, G. Chrysos, et al., “The menpo benchmark for multi-pose 2d and 3d facial landmark localisation and tracking,” International Journal of Computer Vision, pp. 1–26, 2018.

A. Bulat and G. Tzimiropoulos, “How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks),” in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 1021–1030. DOI: 10.1109/ICCV.2017.116.

S. Zafeiriou, G. Chrysos, A. Roussos, E. Ververas, J. Deng, and G. Trigeorgis, “The 3d menpo facial landmark tracking challenge,” in International Conference on Computer Vision (ICCV) Workshops, 2017.

G. G. Chrysos, E. Antonakos, S. Zafeiriou, and P. Snape, “Offline deformable face tracking in arbitrary videos,” in 2015 IEEE International Conference on Computer Vision Workshop, ICCV Workshops 2015, Santiago, Chile, December 7-13, 2015, IEEE Computer Society, 2015, pp. 954–962. DOI: 10.1109/ICCVW. 2015.126.

J. Shen, S. Zafeiriou, G. G. Chrysos, J. Kossaifi, G. Tzimiropoulos, and M. Pantic, “The first facial landmark tracking in-the-wild challenge: Benchmark and results,” in 2015 IEEE International Conference on Computer Vision Workshop, ICCV Workshops 2015, Santiago, Chile, December 7-13, 2015, IEEE Computer Society, 2015, pp. 1003–1011. DOI: 10.1109/ICCVW. 2015.132.

G. Tzimiropoulos, “Project-out cascaded regression with an application to face alignment,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, IEEE Computer Society, 2015, pp. 3659–3667. DOI: 10.1109/CVPR.2015.7298989.

V. Kazemi and J. Sullivan, “One millisecond face alignment with an ensemble of regression trees,” in Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, ser. CVPR ’14, USA: IEEE Computer Society, 2014, pp. 1867–1874, ISBN: 9781479951185. DOI: 10.1109/CVPR.2014.241.

D. E. King, “Dlib-ml: A machine learning toolkit,” J. Mach. Learn. Res., vol. 10, pp. 1755–1758, Dec. 2009, ISSN: 1532-4435.

P. A. Viola and M. J. Jones, “Rapid object detection using a boosted cascade of simple features,” in 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), 8-14 December 2001, Kauai, HI, USA, IEEE Computer Society, 2001, pp. 511–518. DOI: 10.1109/CVPR.2001.990517.

Z. Feng, J. Kittler, M. Awais, P. Huber, and X. Wu, “Wing loss for robust facial landmark localisation with convolutional neural networks,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Los Alamitos, CA, USA: IEEE Computer Society, Jun. 2018, pp. 2235–2245. DOI: 10 . 1109 / CVPR.2018.00238.

Y. Yan, X. Naturel, T. Chateau, S. Duffner, C. Garcia, and C. Blanc, “A survey of deep facial landmark detection,” in RFIAP, Paris, France, Jun. 2018. [Online]. Available: https://hal. archives-ouvertes.fr/hal-02892002.

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition, Ieee, 2009, pp. 248–255.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, IEEE Computer Society, 2016, pp. 770–778. DOI: 10.1109/ CVPR.2016.90.

A. Howard, R. Pang, H. Adam, et al., “Searching for mobilenetv3,” in 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, IEEE, 2019, pp. 1314–1324. DOI: 10.1109/ICCV.2019.00140.

N. Ma, X. Zhang, H. Zheng, and J. Sun, “Shufflenet V2: practical guidelines for efficient CNN architecture design,” in Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XIV, ser. Lecture Notes in Computer Science, vol. 11218, Springer, 2018, pp. 122–138. DOI: 10 . 1007 / 978 - 3 - 030 - 01264 -9_8.

A. Newell, K. Yang, and J. Deng, “Stacked hourglass networks for human pose estimation,” in Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII, ser. Lecture Notes in Computer Science, vol. 9912, Springer, 2016, pp. 483–499. DOI: 10.1007/978-3-319-46484-8_29.

K. Sun, B. Xiao, D. Liu, and J. Wang, “Deep high-resolution representation learning for human pose estimation,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, Computer Vision Foundation / IEEE, 2019, pp. 5693–5703. DOI: 10.1109/CVPR. 2019.00584.

Z. Tang, X. Peng, K. Li, and D. N. Metaxas, “Towards efficient u-nets: A coupled and quantized approach,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 8, pp. 2038–2050, 2020. DOI: 10.1109/TPAMI.2019.2907634.

J. Wang, K. Sun, T. Cheng, et al., “Deep highresolution representation learning for visual recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 10, pp. 3349–3364, 2021. DOI: 10.1109/TPAMI.2020.2983686. [Online]. Available: https : / / doi . org / 10 .1109/TPAMI.2020.2983686.

Y. Liu, A. Jourabloo, W. Ren, and X. Liu, “Dense face alignment,” in 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), 2017, pp. 1619–1628. DOI: 10.1109/ICCVW.2017.190.

X. Dong, Y. Yan, W. Ouyang, and Y. Yang, “Style aggregated network for facial landmark detection,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 379–388. DOI: 10 . 1109 / CVPR . 2018 .00047.

I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, et al., “Generative adversarial networks,” CoRR, vol. abs/1406.2661, 2014. arXiv: 1406.2661. [Online]. Available: http : / / arxiv . org /abs/1406.2661.

J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycleconsistent adversarial networks,” in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2242–2251. DOI: 10.1109/ ICCV.2017.244.

X. Chu,W. Ouyang, H. Li, and X.Wang, “Structured feature learning for pose estimation,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, IEEE Computer Society, 2016, pp. 4715–4723. DOI: 10.1109/CVPR.2016.510.

W. Yang,W. Ouyang, H. Li, and X.Wang, “Endto-end learning of deformable mixture of parts and deep convolutional neural networks for human pose estimation,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, IEEE Computer Society, 2016, pp. 3073–3082. DOI: 10.1109/CVPR.2016.335.

R. B. Girshick, “Fast R-CNN,” in 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015, IEEE Computer Society, 2015, pp. 1440– 1448. DOI: 10.1109/ICCV.2015.169.

X. Guo, S. Li, J. Zhang, et al., “PFLD: A practical facial landmark detector,” CoRR, vol. abs/1902.10859, 2019. arXiv: 1902 . 10859. [Online]. Available: http://arxiv.org/abs/1902.10859.

A. Bulat and G. Tzimiropoulos, “Binarized convolutional landmark localizers for human pose estimation and face alignment with limited resources,” in IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, IEEE Computer Society, 2017, pp. 3726–3734. DOI: 10.1109/ICCV.2017.400.

R. Liu, J. Lehman, P. Molino, et al., “An intriguing failing of convolutional neural networks and the coordconv solution,” in Proceedings of the 32nd International Conference on Neural Information Processing Systems, ser. NIPS’18, Montreal, Canada: Curran Associates Inc., 2018, pp. 9628–9639.

Y. Zhao, Y. Liu, C. Shen, Y. Gao, and S. Xiong, “Mobilefan: Transferring deep hidden representation for face alignment,” Pattern Recognition, vol. 100, p. 107 114, 2020, ISSN: 0031- 3203. DOI: 10 . 1016 / j . patcog . 2019 . 107114. [Online]. Available: https://www.sciencedirect . com / science / article /pii/S0031320319304157.

S. M. Iranmanesh, A. Dabouei, S. Soleymani, H. Kazemi, and N. M. Nasrabadi, “Robust facial landmark detection via aggregation on geometrically manipulated faces,” in 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 319–329. DOI: 10.1109/WACV45572.2020.9093508.

A. Dabouei, S. Soleymani, J. Dawson, and N. Nasrabadi, “Fast geometrically-perturbed adversarial faces,” in 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), 2019, pp. 1979–1988. DOI: 10.1109/WACV.2019.00215.

W. Li, Y. Lu, K. Zheng, et al., “Structured landmark detection via topology-adapting deep graph learning,” in Computer Vision – ECCV 2020, Cham: Springer International Publishing, 2020, pp. 266–283, ISBN: 978-3-030-58545-7.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Y. Bengio and Y. LeCun, Eds., 2015. [Online]. Available: http : / / arxiv.org/abs/1409.1556.

X. Huang, W. Deng, H. Shen, X. Zhang, and J. Ye, “Propagationnet: Propagate points to curve to learn structure information,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, Computer Vision Foundation / IEEE, 2020, pp. 7263–7272. DOI: 10.1109/CVPR42600.2020.00729.

R. Zhang, “Making convolutional networks shift-invariant again,” in Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, K. Chaudhuri and R. Salakhutdinov, Eds., ser. Proceedings of Machine Learning Research, vol. 97, PMLR, 2019, pp. 7324–7334. [Online]. Available: http:// proceedings.mlr.press/v97/zhang19a.html.

H. Yang, J. Lyu, P. Cheng, and X. Tang, “Lddmm-face: Large deformation diffeomorphic metric learning for flexible and consistent face alignment,” CoRR, vol. abs/2108.00690, 2021. arXiv: 2108.00690. [Online]. Available: https://arxiv.org/abs/2108.00690.

J. A. Glaun`es, A. Qiu, M. I. Miller, and L. Younes, “Large deformation diffeomorphic metric curve mapping,” Int. J. Comput. Vis., vol. 80, no. 3, pp. 317–336, 2008. DOI: 10 . 1007 /s11263-008-0141-9.

S. C. Joshi and M. I. Miller, “Landmark matching via large deformation diffeomorphisms,” IEEE Trans. Image Process., vol. 9, no. 8, pp. 1357–1370, 2000. DOI: 10 . 1109 / 83 .855431.

Z. Xu, B. Li, Y. Yuan, and M. Geng, “Anchorface: An anchor-based facial landmark detector across large poses,” in Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, AAAI Press, 2021, pp. 3092–3100. [Online]. Available: https :/ / ojs . aaai . org / index . php / AAAI /article/view/16418.

H. Jin, S. Liao, and L. Shao, “Pixel-in-pixel net: Towards efficient facial landmark detection in the wild,” International Journal of Computer Vision, vol. 129, no. 12, pp. 3174–3194, Dec. 2021, ISSN: 1573-1405. DOI: 10.1007/ s11263-021-01521-4

Y. Huang, H. Yang, C. Li, J. Kim, and F. Wei, “Adnet: Leveraging error-bias towards normal direction in face alignment,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2021, pp. 3080–3090.

X. Lan, Q. Hu, and J. Cheng, “Revisting quantization error in face alignment,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, Oct. 2021, pp. 1521–1530.

A. Bulat, E. Sanchez, and G. Tzimiropoulos, “Subpixel heatmap regression for facial landmark localization,” CoRR, vol. abs/2111.02360, 2021. arXiv: 2111.02360. [Online]. Available: https://arxiv.org/abs/2111.02360.

A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, I. Guyon, U. von Luxburg, S. Bengio, et al., Eds., 2017, pp. 5998–6008. [Online]. Available: https://proceedings.neurips . cc / paper / 2017 / hash / 3f5ee243547dee91fbd053c1c4a845aa -Abstract.html.

D. C. Luvizon, D. Picard, and H. Tabia, “2d/3d pose estimation and action recognition using multitask deep learning,” in 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, Computer Vision Foundation / IEEE Computer Society, 2018, pp. 5137–5146. DOI: 10.1109/CVPR.2018.00539.

K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint face detection and alignment using multitask cascaded convolutional networks,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499– 1503, 2016. DOI: 10 . 1109 / LSP . 2016 . 2603342.

V. Bazarevsky, Y. Kartynnik, A. Vakunov, K. Raveendran, and M. Grundmann, “Blazeface: Sub-millisecond neural face detection on mobile gpus,” CoRR, vol. abs/1907.05047, 2019. arXiv: 1907.05047. [Online]. Available: http: //arxiv.org/abs/1907.05047.

Martin Abadi, Ashish Agarwal, Paul Barham, et al., TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. [Online]. Available: https://www.tensorflow.org/.

W. Liu, D. Anguelov, D. Erhan, et al., “SSD: single shot multibox detector,” in Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I, ser. Lecture Notes in Computer Science, vol. 9905, Springer, 2016, pp. 21–37. DOI: 10.1007/978-3-319-46448-0_2.

S. Minaee, P. Luo, Z. Lin, and K. W. Bowyer, “Going deeper into face detection: A survey,”

CoRR, vol. abs/2103.14983, 2021. arXiv: 2103. 14983. [Online]. Available: https://arxiv.org/abs/2103.14983.

P. Garrido, L. Valgaerts, H. Sarmadi, et al., “Vdub: Modifying face video of actors for plausible visual alignment to a dubbed audio track,” Comput. Graph. Forum, vol. 34, no. 2, pp. 193–204, May 2015, ISSN: 0167-7055. DOI: 10 .1111/cgf.12552.

E. Zakharov, A. Shysheya, E. Burkov, and V. Lempitsky, “Few-shot adversarial learning of realistic neural talking head models,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Los Alamitos, CA, USA: IEEE Computer Society, Nov. 2019, pp. 9458–9467. DOI: 10.1109/ICCV.2019.00955.

C. Finn, P. Abbeel, and S. Levine, “Modelagnostic meta-learning for fast adaptation of deep networks,” in Proceedings of the 34th International Conference on Machine Learning - Volume 70, ser. ICML’17, Sydney, NSW, Australia: JMLR.org, 2017, pp. 1126–1135.

J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and superresolution,” in Computer Vision – ECCV 2016, Cham: Springer International Publishing, 2016, pp. 694–711, ISBN: 978-3-319-46475-6.

R. Zheng, Z. Zhu, B. Song, and C. Ji, “A neural lip-sync framework for synthesizing photorealistic virtual news anchors,” in 25th International Conference on Pattern Recognition, ICPR 2020, Virtual Event / Milan, Italy, January 10-15, 2021, IEEE, 2020, pp. 5286–5293. DOI: 10.1109/ICPR48806.2021.9412187. [Online]. Available: https://doi.org/10. 1109/ICPR48806.2021.9412187.

T. Wang, M. Liu, J. Zhu, A. Tao, J. Kautz, and B. Catanzaro, “High-resolution image synthesis and semantic manipulation with conditional gans,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Los Alamitos, CA, USA: IEEE Computer Society, Jun. 2018, pp. 8798–8807. DOI: 10.1109/CVPR.2018.00917.

J. Zhang, X. Zeng, M. Wang, et al., “Freenet: Multi-identity face reenactment,” in 2020 IEEE/CVF Conference on Computer Vision and

Pattern Recognition (CVPR), 2020, pp. 5325–5334. DOI: 10 . 1109 / CVPR42600 . 2020 .00537.

R. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, and J. Ortega-Garcia, “Deepfakes and beyond: A survey of face manipulation and fake detection,” Information Fusion,vol. 64, pp. 131–148, 2020, ISSN: 1566-2535. DOI: 10 . 1016 / j . inffus . 2020 .06.014. [Online]. Available: https://www.sciencedirect . com / science / article /pii/S1566253520303110.

R. Jabbar, M. Shinoy, M. Kharbeche, K. Al-Khalifa, M. Krichen, and K. Barkaoui, “Driver drowsiness detection model using convolutional neural networks techniques for android application,” in 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT), 2020, pp. 237–242. DOI: 10.1109/ ICIoT48696.2020.9089484.

J. S. Wijnands, J. Thompson, K. A. Nice, G. D. Aschwanden, and M. Stevenson, “Real-time monitoring of driver drowsiness on mobile platforms using 3d neural networks,” Neural Computing and Applications, pp. 1–13, 2019.

W. Kim, W.-S. Jung, and H. K. Choi, “Lightweight driver monitoring system based on multi-task mobilenets,” Sensors, vol. 19, no. 14, 2019, ISSN: 1424-8220. DOI: 10.3390/ s19143200. [Online]. Available: https : / /www.mdpi.com/1424-8220/19/14/3200.

T. Hyuga, K. Kinoshita, K. Nishiyuki, and Y. Hasegawa, “Driver status monitoring system in autonomous driving era,” OMRON TECHNICS, vol. 50.005EN, 2019. [Online]. Available: https : / / www . omron . com /global / en / assets / file / technology /omrontechnics/ vol50 / OMT _ Vol50 _ 005 .pdf.

I. Masi, Y. Wu, T. Hassner, and P. Natarajan, “Deep face recognition: A survey,” in 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), 2018, pp. 471–478. DOI: 10.1109/SIBGRAPI.2018.00067.

C. N. Duong, K. G. Quach, I. Jalata, N. Le, and K. Luu, “Mobiface: A lightweight deep learning face recognition on mobile devices,” in 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS), 2019, pp. 1–6. DOI: 10 . 1109 /BTAS46853 . 2019.9185981.

S. Chen, Y. Liu, X. Gao, and Z. Han, “Mobilefacenets: Efficient cnns for accurate real-time face verification on mobile devices,” in Biometric Recognition, Cham: Springer International Publishing, 2018, pp. 428–438, ISBN: 978-3-319-97909-0.

B. C. Ko, “A brief review of facial emotion recognition based on visual information,” Sensors, vol. 18, no. 2, 2018, ISSN: 1424-8220. DOI: 10.3390/s18020401. [Online]. Available: https://www.mdpi.com/1424-8220/18/2/401.

S. Li and W. Deng, “Deep facial expression recognition: A survey,” IEEE Transactions on Affective Computing, pp. 1–1, 2020. DOI: 10. 1109/TAFFC.2020.2981446.

C. Szegedy, W. Zaremba, I. Sutskever, et al., “Intriguing properties of neural networks,” in 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. [Online]. Available: http : / /arxiv.org/abs/1312.6199.

K. S. Khabarlak and L. S. Koriashkina, “Scoping adversarial attack for improving its quality,” Radio Electronics, Computer Science, Control, no. 2, pp. 108–118, May 2019. DOI: 10.15588/1607 - 3274 - 2019 - 2 - 12. [Online]. Available: http://ric.zntu.edu.ua/article/view/178284.

A. Kurakin, I. J. Goodfellow, and S. Bengio, “Adversarial examples in the physical world,” in 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings, 2017. [Online]. Available: https : / / openreview.net/forum?id=HJGU3Rodl.

K. Khabarlak and L. Koriashkina, “Minimizing perceived image quality loss through adversarial attack scoping,” CoRR, vol. abs/1904.10390, 2019. arXiv: 1904.10390. [Online]. Available: http://arxiv.org/abs/1904.10390.

P.-Y. Chen, H. Zhang, Y. Sharma, J. Yi, and C.-J. Hsieh, “Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models,” in Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, ser. AISec ’17, Dallas, Texas, USA: Association for Computing Machinery, 2017, pp. 15–26, ISBN: 9781450352024. DOI: 10 . 1145 / 3128572 . 3140448.

N. Carlini and D. Wagner, “Adversarial examples are not easily detected: Bypassing ten detection methods,” in Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, ser. AISec ’17, Dallas, Texas, USA: Association for Computing Machinery, 2017, pp. 3–14, ISBN: 9781450352024. DOI: 10 .1145/3128572.3140444.

N. Akhtar and A. Mian, “Threat of adversarial attacks on deep learning in computer vision: A survey,” IEEE Access, vol. 6, pp. 14 410– 14 430, 2018. DOI: 10.1109/ACCESS.2018.2807385.

M. Sharif, S. Bhagavatula, L. Bauer, and M. K. Reiter, “Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition,” in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’16, Vienna, Austria: Association for Computing Machinery, 2016, pp. 1528–1540, ISBN: 9781450341394. DOI: 10.1145/2976749.2978392.

E. Kaziakhmedov, K. Kireev, G. Melnikov, M. Pautov, and A. Petiushko, “Real-world attack on mtcnn face detection system,” in 2019 International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON), 2019, pp. 0422–0427. DOI: 10.1109/SIBIRCON48586.2019.8958122.

Downloads

Published

2022-04-21

How to Cite

Khabarlak, K., & Koriashkina, L. (2022). Fast Facial Landmark Detection and Applications: A Survey. Journal of Computer Science and Technology, 22(1), e02. https://doi.org/10.24215/16666038.22.e02

Issue

Section

Original Articles