|本期目录/Table of Contents|

[1]严 严,陈日伟,王菡子*.基于深度学习的人脸分析研究进展[J].厦门大学学报(自然科学版),2017,56(01):13-24.[doi:10.6043/j.issn.0438-0479.201609024]
 YAN Yan,CHEN Riwei,WANG Hanzi*.Recent Advances on Deep-Learning-Based Face Analysis[J].Journal of Xiamen University(Natural Science),2017,56(01):13-24.[doi:10.6043/j.issn.0438-0479.201609024]
点击复制

基于深度学习的人脸分析研究进展(PDF)
分享到:

《厦门大学学报(自然科学版)》[ISSN:0438-0479/CN:35-1070/N]

卷:
56卷
期数:
2017年01期
页码:
13-24
栏目:
综 述
出版日期:
2017-01-23

文章信息/Info

Title:
Recent Advances on Deep-Learning-Based Face Analysis
文章编号:
0438-0479(2017)01-0013-12
作者:
严 严陈日伟王菡子*
厦门大学 信息科学与技术学院,福建省智慧城市感知与计算重点实验室,福建 厦门 361005
Author(s):
YAN YanCHEN RiweiWANG Hanzi*
Fujian Key Laboratory of Sensing and Computing for Smart City,School of Information Science and Engineering,Xiamen University,Xiamen 361005,China
关键词:
深度学习 卷积神经网络 人脸数据库 人脸识别 人脸分析
Keywords:
deep learning convolutional neural network face database face recognition face analysis
分类号:
TP 391
DOI:
10.6043/j.issn.0438-0479.201609024
文献标志码:
A
摘要:
近年来,基于深度学习的人脸分析取得了巨大的进步,成为计算机视觉领域最为活跃的研究方向之一.为了进一步推动深度学习和人脸分析的研究,结合近年已发表的相关文献,对基于深度学习的人脸分析技术进行综述.首先,简要概述深度学习及其发展历史,并分析深度学习有效性原因.然后,按照任务目的的不同,将人脸分析分成了人脸检测、人脸关键点检测、人脸识别、人脸属性识别等任务进行详细的介绍和讨论,重点分析各种任务现阶段存在的主要问题.接着,介绍人脸分析中常用的人脸数据库.最后,讨论深度学习和人脸分析面临的主要挑战,并给出结论.
Abstract:
In recent years,the face analysis based on deep learning has made great progress,and has become one of the most active research areas in the field of computer vision.In order to further promote the study of deep learning and face analysis,this paper overviews recent advances on the deep-learning-based face analysis techniques in the literature.First,a brief overview of deep learning and its history are given and reasons for the effectiveness of deep learning are also analyzed.Then,according to different objectives,four face analysis tasks,i.e.,face detection,facial key-point detection,face recognition,face attribute recognition,are introduced and discussed in detail,and the key problems existing in these tasks at present are analyzed.After that,the commonly used face databases in the face analysis are described.At last,main challenges of face analysis based deep learning are shown and the conclusion is presented.

参考文献/References:

[1] AHONEN T,HADID A,PIETIKAINEN M.Face description with local binary patterns:application to face recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2006,28(12):2037-2041.
[2] LOWE D J.Distinctive image features from scale-invariant keypoints[J].International Journal of Computer Vision,2004,60(2):91-110.
[3] DALAL N,TRIGGS B.Histograms of oriented gradients for human detection[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2005:886-893.
[4] 梁路宏,艾海舟,徐光祐,等.人脸检测研究综述[J].计算机学报,2002,25(5):449-458.
[5] ZHAO W,CHELLAPPA R,PHILLIPS P J,et al.Face recognition:a literature survey[J].ACM Computing Surveys,2003,35(4):399-458.
[6] 严严,章毓晋.基于视频的人脸识别研究进展[J].计算机学报,2009,32(5):878-886.
[7] LECUN Y,BENGIO Y,HINTON G.Deep learning[J].Nature,2015,521(7553):436-444.
[8] BENGIO Y,COURVILLE A,VINCENT P.Representation learning:a review and new perspectives[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,5(8):1798-1828.
[9] IAN G,YOSHUA B,ARON C.Deep learning[M].Massachusetts:MIT Press,2016.
[10] 余凯,贾磊,陈雨强,等.深度学习的昨天、今天和明天[J].计算机研究与发展,2013,50(9):1799-1804.
[11] 山世光,阚美娜,刘昕,等.深度学习:多层神经网络的复兴与变革[J].科技导报,2016,34(13):60-70.
[12] 尹宝才,王文通,王立春.深度学习研究综述[J].北京工业大学学报,2015,41(1):48-59.
[13] ROWLEY H,BALUJA S,KANADE T.Neural network-based face detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(1):23-38.
[14] FERAUND R,BERNIER O J,VIALLET J,et al.A fast and accurate face detector based on neural networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2001,23(1):42-53.
[15] LAWRENCE S,GILES C L,TSOI A C,et al.Face recognition:a convolutional neural-network approach[J].IEEE Transactions on Neural Networks,1997,8(1):98-113.
[16] SUN Y,WANG X,TANG X.Deep convolutional network cascade for facial point detection[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2013:3476-3483.
[17] ZHANG J,SHAN S,KAN M,et al.Coarse-to-fine auto-encoder networks(CFAN)for real-time face alignment[C]∥Proceedings of the European Conference on Computer Vision.Berlin:Springer,2014:1-16.
[18] LEVI G,HASSNER T.Age and gender classification using convolutional neural networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D C:IEEE,2015:34-42.
[19] LIU P,HAN S,MENG Z,et al.Facial expression recognition via a boosted deep belief network[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2014:1805-1812.
[20] ZHU Z,LUO P,WANG X,et al.Deep learning identity-preserving face space[C]∥Proceedings of the IEEE International Conference on Computer Vision.Washington D C:IEEE,2013:113-120.
[21] ROSENBLATT F.The perceptron:a probabilistic model for information storage and organization in the brain[J].Psychological Review,1958,65(6):386-408.
[22] MINSKY M L,PAPERT S A.Perceptrons:expanded edition[M].Massachusetts:MIT Press,1988:1-308.
[23] LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[24] HINTON G,OSINDERO S,TEH Y W.A fast learning algorithm for deep belief nets[J].Neural Computation,2006,18(7):1527-1554.
[25] SMOLENSKY P.Information processing in dynamical systems:foundations of harmony theory[J].Parallel Distributed Processing,1986,1:194-281.
[26] RIFAI S,VINCENT P,MULLER X,et al.Contractive auto-encoders:explicit invariance during feature extraction[C]∥Proceedings of the International Conference on Machine Learning.Madison:Omnipress,2011:833-840.
[27] WILLIAMS D,HINTON G,Learning representations by back-propagating errors[J].Nature,1986,323:533-536.
[28] GRAVES A,JAITLY N.Towards end-to-end speech recognition with recurrent neural networks[C]∥Proceedings of the International Conference on Machine Learning.Madison:Omnipress,2014:1764-1772.
[29] GERS F A,SCHMIDHUBER J,CUMMINS F.Learning to forget:continual prediction with LSTM[J].Neural Computation,2000,12(10):2451-2471.
[30] KRIZHEVSKY A,SUTSKEVER I,HINTON G.ImageNet classification with deep convolutional neural networks[C]∥Advances in Neural Information Processing Systems.Lake Tahoe:NIPS,2012:1097-1110.
[31] DENG J,DONG W,SOCHER R,et al.ImageNet:a large-scale hierarchical image database[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2009:248-255.
[32] SRIVASTAVA N,HINTON G,KRIZHEVSKY A,et al.Dropout:a simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research,2014,15(1):1929-1958.
[33] NAIR V,HINTON G.Rectified linear units improve restricted Boltzmann machines[C]∥Proceedings of the International Conference on Machine Learning.Madison:Omnipress,2010:807-814.
[34] HE K,ZHANG X,REN S,et al.Delving deep into rectifiers:surpassing human level performance on ImageNet classification[C]∥Proceedings of the IEEE International Conference on Computer Vision.Washington D C:IEEE,2015:1026-1034.
[35] GARCIA C,DELAKIS M.Convolutional face finder:a neural architecture for fast and robust face detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2004,26(11):1408-1423.
[36] ZHANG C,ZHANG Z.Improving multiview face detection with multi-task deep convolutional neural networks[C]∥Proceedings of the IEEE Winter Conference on Applications of Computer Vision.Washington D C:IEEE,2014:1036-1041.
[37] LI H,LIN Z,SHEN X,et al.A convolutional neural network cascade for face detection[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Re-cognition.Washington D C:IEEE,2015:5325-5334.
[38] YANG S,LUO P,LOY C C,et al.From facial parts responses to face detection:a deep learning approach[C]∥Proceedings of the IEEE International Conference on Computer Vision.Washington D C:IEEE,2015:3676-3684.
[39] ZHAN S,TAO Q,LI X.Face detection using representation learning[J].Neurocomputing,2015,187:19-26.
[40] ZHOU E,FAN H,CAO Z,et al.Extensive facial landmark localization with coarse-to-fine convolutional network cascade[C]∥Proceedings of the IEEE International Con-ference on Computer Vision Workshops.Washington D C:IEEE,2013:386-391.
[41] ZHANG Z,LUO P,LOY C C,et al.Facial landmark detection by deep multi-task learning[C]∥Proceedings of the European Conference on Computer Vision.Berlin:Springer,2014:94-108.
[42] JOURABLOO A,LIU X.Large-pose face alignment via CNN-based dense 3D model fitting[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2016:4188-4196.
[43] ZHU X,LEI Z,LIU X,et al.Face alignment across large poses:a 3D solution[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2016:146-155.
[44] ZHANG J,KAN M,SHAN S,et al.Occlusion-free face alignment:deep regression networks coupled with de-corrupt autoencoders[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2016:3428-3437.
[45] TAIGMAN Y,YANG M,RANZATO M,et al.DeepFace:closing the gap to human-level performance in face verification[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2014:1701-1708.
[46] SUN Y,WANG X,TANG X.Deep learning face representation from predicting 10,000 classes[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2014:1891-1898.
[47] SUN Y,CHEN Y,WANG X,et al.Deep learning face representation by joint identification-verification[C]∥Advances in Neural Information Processing Systems.Montreal:NIPS,2014:1988-1996.
[48] SUN Y,WANG X,TANG X.Deeply learned face representations are sparse,selective,and robust[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2015:2892-2900.
[49] SCHROFF F,KALENICHENKO D,PHILBIN J.FaceNet:a unified embedding for face recognition and clustering[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2015:815-823.
[50] WANG X,GUO R,KAMBHAMETTU C.Deeply-learned feature for age estimation[C]∥Proceedings of the IEEE Winter Conference on Applications of Computer Vision 2014.Washington D C:IEEE,2015:534-541.
[51] ROTHE R,TIMOFTE R,GOOL L.DEX:deep expectation of apparent age from a single image[C]∥Proceedings of the IEEE International Conference on Computer Vision Workshops.Washington D C:IEEE,2015:10-15.
[52] KAHOU S,BOUTHILLIER X,LAMBLIN P,et al.EmoNets:multimodal deep learning approaches for emotion recognition in video[J].Journal on Multimodal User Interfaces,2016,10(2):99-111.
[53] TANG X,WANG X,LUO P.Hierarchical face parsing via deep learning[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2012:2480-2487.
[54] YANG J,LEI Z,LI S Z.Learn convolutional neural network for face anti-spoofing[J].Computer Science,2014,9218:373-384.
[55] ZHANG K,Y HUANG,SONG C,et al.Kinship verification with deep convolutional neural networks[C]∥British Machine Vision Conference.British:BMVA,2015:148.1-148.12.
[56] QI Y,ZHANG S,QIN L,et al.Hedged deep tracking[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2016:4303-4311.
[57] VIOLA P,JONES M J.Robust real-time face detection[J].International Journal of Computer Vision,2004,57(2):137-154.
[58] LI Y,SUN B,WU T,et al.Face detection with end-to-end integration of a convnet and a 3D model[C]∥Proceedings of the European Conference on Computer Vision.Berlin:Springer,2016:420-436.
[59] QIN H,YAN J,LI X,et al.Joint training of cascaded CNN for face detection[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2016:3456-3465.
[60] SUBBURAMAN V B,MARCEL S.Fast bounding box estimation based face detection[C]∥Proceedings of ECCV Workshop on Face Detection:Where We Are,and What Next? Berlin:Springer,2010:5325-5334.
[61] COOTES T F,COOPER D,TAYLOR C J,et al.Active shape models-their training and application[J].Computer Vision and Image Understanding,1995,61(1):38-59.
[62] COOTES T F,EDWARDS G J.TAYLOR C J.Active appearance models[C]∥Proceedings of the European Conference on Computer Vision.Berlin:Springer,1998:484-498.
[63] SARAGIH J M,LUCEY S,COHN J.Face alignment through subspace constrained mean-shifts[C]∥Procee-dings of the IEEE International Conference on Computer Vision.Washington D C:IEEE,2009:1034-1041.
[64] LIU X.Discriminative face alignment[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,31(11):1941-1954.
[65] CAO X,WEI Y,WEN F,et al.Face alignment by explicit shape regression[J].International Journal of Computer Vision,2014,107(2):177-190.
[66] SHLIZERMAN I,SEITZ S,MILLER D,et al.The MegaFace benchmark:1 million faces for recognition at scale[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2016:4873-4882.
[67] DEVRIES T,BISWARANJAN K,TAYLOR G W.Multi-task learning of facial landmarks and expression[C]∥Proceedings of the Canadian Conference on Computer and Robot Vision.Washington D C:IEEE,2014:98-103.
[68] ZHANG Z,LUO P,CHEN C L,et al.Learning deep representation for face alignment with auxiliary attributes[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(5):918-930.
[69] KUMAR N,BERG A C,BELHUMEUR P N,et al.Describable visual attributes for face verification and image search[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(10):1962-1977.
[70] KAN M,SHAN S,CHANG H,et al.Stacked progressive auto-encoders(SPAE)for face recognition across poses[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2014:1883-1890.
[71] YIM J,JUNG H,YOO B,et al.Rotating your face using multi-task deep neural network[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Re-cognition.Washington D C:IEEE,2015:676-684.
[72] SUNG K,POGGIO T.Example-based learning for view-based human face detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1999,20(1):39-51.
[73] JAIN V,LEARNED-MILLER E.FDDB:a benchmark for face detection in unconstrained settings,Technical Report UM-CS-2010-009[R].Amherst:University of Massachusetts,2010.
[74] ZHU X,RAMANAN D.Face detection,pose estimation,and landmark localization in the wild[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2012:2879-2886.
[75] YANG B,YAN J,LEI Z,et al.Fine-grained evaluation on face detection in the wild[C]∥Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition Conference and Workshops.Washington D C:IEEE,2015:111-117.
[76] KLARE B F,KLEIN B,TABORSKY E,et al.Pushing the frontiers of unconstrained face detection and recognition:IARPA Janus Benchmark A[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2015:1931-1939.
[77] YANG S,LUO P,CHEN C,et al.WIDER FACE:a face detection benchmark[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2016:5525-5533.
[78] BELHUMEUR P,JACOBS D,KRIEGMAN D,et al.Localization parts of faces using a consensus of exemplars[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2011:545-552.
[79] KOSTINGER M,WOHLHART P,ROTH P M,et al.Annotated facial landmarks in the wild:a large-scale,real-world database for facial landmark localization[C]∥Proceedings of IEEE International Conference on Computer Vision Workshop on Benchmarking Facial Image Analysis Technologies.Washington D C:IEEE,2011:2144-2151.
[80] BURGOS-ARTIZZU X P,PERONA P,DOLLAR P.Robust face landmark estimation under occlusion[C]∥Proceedings of the IEEE International Conference on Computer Vision.Washington D C:IEEE,2013:1513-1520.
[81] XING J,NIU Z,HUANG J,et al.Towards multi-view and partially-occluded face alignment[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2014:1829-1836.
[82] SAGONAS C,ANTONAKOS E,TZIMIROPOULOS G,et al.300 faces in-the-wild challenge:database and results[J].Image and Vision Computing,2016,47:3-18.
[83] HUANG G B,RAMESH M,BERG T,et al.Labeled faces in the wild:a database for studying face recognition in unconstrained environments[R].Amherst:University of Massachusetts,2007.
[84] KUMAR N,BERG A,BELHUMEUR P N,et al.Attri-bute and simile classifiers for face verification[C]∥Proceedings of the International Conference on Computer Vision.Washington D C:IEEE,2009:365-372.
[85] YI D,LEI Z,LIAO S,et al.Learning face representation from scratch[DB/OL].[2016-09-01].https:∥arxiv.org/pdf/1411.7923.pdf.
[86] NG H W,WINKLER S.A data-driven approach to cleaning large face datasets[C]∥Proceedings of the IEEE International Conference on Image Processing.Washington D C:IEEE,2014:343-347.
[87] LYONS M J,BUDYNEK J,AKAMATSU S.Automatic classification of single facial images[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1999,21(12):1357-1362.
[88] LUCEY P,COHN J F,KANADE T,et al.The extended Cohn-Kanade dataset(CK+):a complete dataset for action unit and emotion-specified expression[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D C:IEEE,2010:94-101.
[89] VALSTAR M F,PANTIC M.Induced disgust,happiness and surprise:an addition to the MMI facial expression database[C]∥Proceedings of International Conference on Language Resources and Evaluation.Washington D C:IEEE,2010:65-70.
[90] RICANEK K,TESAFAYE T.MORPH:a longitudinal image database of normal adult age-progression[C]∥Proceedings the IEEE International Conference on Automatic Face and Gesture Recognition.Washington D C:IEEE,2006:341-345.
[91] EIDINGER E,ENBAR R,HASSNER T.Age and gender estimation of unfiltered faces[J].IEEE Transactions on Information Forensics and Security,2014,9(12):2170-2179.
[92] CHEN B,CHEN C,HSU W.Face recognition using cross-age reference coding with cross-age celebrity dataset[J].IEEE Transactions on Multimedia,2015,17(6):804-815.
[93] LIU Z,LUO P,WANG X,et al.Deep learning face attributes in the wild[C]∥Proceedings of the International Conference on Computer Vision.Washington D C:IEEE,2015:3730-3738.
[94] KIM M,KUMAR S,PAVLOVIC V,et al.Face tracking and recognition with visual constraints in real-world vi-deos[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2008:1-8.
[95] WOLF L,HASSNER T,MAOZ I.Face recognition in unconstrained videos with matched background similarity[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2011:529-534.
[96] LU J,HU J,ZHOU X,et al.Neighborhood repulsed metric learning for kinship verification[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington D C:IEEE,2012:2594-2601.
[97] ZHANG Z,YAN J,LIU S,et al.A face antispoofing database with diverse attacks[C]∥Proceedings of the 5th IAPR International Conference on Biometrics.Washington D C:IEEE,2012:26-31.
[98] CHINGOVSKA I,ANJOS A,MARCEL S.On the effectiveness of local binary patterns in face anti-spoofing[C]∥Proceedings of the IEEE International Conference of the Biometrics Special Interest Group.Washington D C:IEEE,2012:1-7.

备注/Memo

备注/Memo:
收稿日期:2016-09-11 录用日期:2016-10-18
基金项目:国家自然科学基金(61571379,61472334)
*通信作者:hanzi.wang@xmu.edu.cn
更新日期/Last Update: 1900-01-01