[1]曹金梦,倪蓉蓉,杨 彪.面向面部表情识别的双通道卷积神经网络[J].南京师范大学学报(工程技术版),2018,18(03):001.[doi:10.3969/j.issn.1672-1292.2018.03.001]
　Cao Jinmeng,Ni Rongrong,Yang Biao.Binary-Channel Convolutional Neural Network forFacial Expression Recognition[J].Journal of Nanjing Normal University(Engineering and Technology),2018,18(03):001.[doi:10.3969/j.issn.1672-1292.2018.03.001]
点击复制

面向面部表情识别的双通道卷积神经网络
分享到：

南京师范大学学报（工程技术版）[ISSN:1006-6977/CN:61-1281/TN]

卷:
18卷
期数:
2018年03期
页码:
001
栏目:
人工智能算法与应用专栏
出版日期:
2018-09-30

文章信息/Info

Title:
Binary-Channel Convolutional Neural Network forFacial Expression Recognition
文章编号:
1672-1292(2018)03-0001-09
作者:
曹金梦1; 倪蓉蓉2; 杨 彪1
(1.常州大学信息科学与工程学院,江苏 常州 213164)(2.常州纺织服装职业技术学院机电系,江苏 常州 213164)
Author(s):
Cao Jinmeng1; Ni Rongrong2; Yang Biao1
(1.School of Information Science and Engineering,Changzhou University,Changzhou 213164,China)(2.Department of Mechanical and Electrical,Changzhou Vocational Institute of Textile and Garment,Changzhou 213164,China)
关键词:
人脸检测; 面部表情识别; 双通道; 卷积神经网络; LBP图像
Keywords:
face detection; facial expression recognition; binary-channel; convolutional neural network; LBP images
分类号:
TP391.41
DOI:
10.3969/j.issn.1672-1292.2018.03.001
文献标志码:
A
摘要:
面部表情识别是机器感知人类情绪变化的重要途径,但表情识别受不同个体及情绪强弱差异影响较大,难以手动设计准确的特征. 提出一种基于双通道卷积神经网络的面部表情识别方法,首先对采集得到的人脸图像进行预处理以限制分析范围,同时分析人脸灰度图像与对应的LBP图像以兼顾全局与细节特征; 针对双通道输入数据,利用不同参数的卷积神经网络自动提取面部特征,通过加权融合分类网络进行特征融合,并利用softmax分类不同表情. 实验结果表明,该算法能够以较高的准确率识别6种基本面部表情(高兴、悲伤、愤怒、沮丧、恐惧及惊讶). 该方法性能优于基于手动设计特征的面部表情识别方法及单通道CNN方法,相比于其他双通道CNN方法,能通过更简单的处理得到近似的识别结果.
Abstract:
Facial expression recognition is an important way for machines to understand emotional changes of human beings. However,accurate hand-crafted features are hard to extract due to the fact that emotion recognition may be severely influenced by individual differences and emotional intensity differences. A facial expression recognition approach based on binary-channel convolutional neural network(BC-CNN)is proposed in this paper. Initially,the sampled facial images are pre-processed for region limitation of further analysis. Global and detailed features are both considered by studying facial gray-scale images and corresponding LBP images simultaneously. For binary-channel input data,two CNNs with different parameters are employed to extract facial features automatically and further,a weighted merge classify network(WMCN)is used for feature fusion and the softmax is finally used to recognize different facial expressions. Experimental results indicate that our algorithm can recognize six fundamental facial expressions(happy,sadness,angry,disgust,fear and surprise)with high accuracy. Its performance is better than that of those methods based on hand-crafted features or single channel CNN. Compared with other binary-channel CNN approach,our approach is easier to implement with similar recognition accuracy.

参考文献/References:

[1] 黄建,李文书,高玉娟. 人脸表情识别研究进展[J]. 计算机科学,2016,43(增刊2):123-126.
HUANG J,LI W S,GAO Y J. Research advance of facial expression recognition[J]. Computer science,2016,43(Suppl. 2):123-126.(in Chinese)
[2]CHEN C R,WONG W S,CHIU C T. A 0.64 mm real-time cascade face detection design based on reduced two-field extraction[J]. IEEE transactions on very large scale integration system,2011,19(11):1937-1948.
[3]BARNOUTI N H,ALDABBAGH S S M,MATTI W E,et al. Face detection and recognition using viola-jones with PCA-LDA and square euclidean distance[J]. International journal of advanced computer science and applications,2016,7(5):371-377.
[4]DEMIRKUS M,PRECUP D,CLARK J,et al. Multi-layer temporal graphical model for head pose estimation in real-world videos[C]//2014 IEEE International Conference on Image Processing. Paris,France:IEEE,2014:3392-3396.
[5]JAIN S,HU C,AGGARWAL J. Facial expression recognition with temporal modeling of shapes[C]//2011 International Conference on Computer Vision Workshops. Barcelona,Spain:IEEE,2011:1642-1649.
[6]ALI G,IQBAL M A,CHOI T S. Boosted NNE collections for multicultural facial expression recognition[J]. Pattern recognition,2016,55:14-27.
[7]李宇成,刘昆. 一种基于人脸分割的PCA和SVM人脸识别方法[J]. 微型机与应用,2016,35(15):51-53.
LI Y C,LIU K. PCA and SVM face recognition method based on human face segmentation[J]. Microcomputer & its applications,2016,35(15):51-53.(in Chinese)
[8]PU X,FAN K,CHEN X,et al. Facial expression recognition from image sequences using twofold random forest classifier[J]. Neurocomputing,2015,168:1173-1180.
[9]LI H,LIN Z,SHEN X,et al. A convolutional neural network cascade for face detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston,USA:IEEE,2015:5325-5334.
[10]LECUN Y,BENGIO Y,HINTON G. Deep learning[J]. Nature,2015,521(7553):436-444.
[11]ZHONG L,LIU Q S,YANG P,et al. Learning multi-scale active facial patches for expression analysis[J]. IEEE transactions on cybernetics,2015,45(8):1499-1510.
[12]ZHANG W,ZHANG Y,MA L,et al. Multimodal learning for facial expression recognition[J]. Pattern recognition,2015,48(10):3191-3202.
[13]WALECKI R,RUDOVIC O,PAVLOVIC V,et al. Variable-state latent conditional random fields for facial expression recognition and action unit detection[C]//11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition. Ljubljana,Slovenia:IEEE,2015:1-8.
[14]郑胤,陈权崎,章毓晋. 深度学习及其在目标和行为识别中的新进展[J]. 中国图象图形学报,2014,19(2):175-184.
ZHENG Y,CHEN Q Q,ZHANG Y J. Deep learning and its new progress in object and behavior recognition[J]. Journal of image and graphics,2014,19(2):175-184.(in Chinese)
[15]何俊,蔡建峰,房灵芝,等. 基于LBP/VAR与DBN模型的人脸表情识别[J]. 计算机应用研究,2016,33(8):2509-2513.
HE J,CAI J F,FANG L Z,et al. Facial expression recognition based on LBP/VAR and DBN model[J]. Application research of computer,2016,33(8):2509-2513.(in Chinese)
[16]LOPES A T,DE AGUIAR E,DE SOUZA A F,et al. Facial expression recognition with convolutional neural networks:coping with few data and the training sample order[J]. Pattern recognition,2017,61:610-628.
[17]YU Z,ZHANG C. Image based static facial expression recognition with multiple deep network learning[C]//Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. London,UK:ACM,2015:435-442.v[18]SIMARD P,STEINKRAUS D,PLATT J C. Best practices for convolutional neural networks applied to visual document analysis[C]//Proceedings of the 7th International Conference on Document Analysis and Recognition. Edinburgh,Scotland:IEEE,2003:958-963.
[19]GUO Z,ZHANG L,ZHANG D. A completed modeling of local binary pattern operator for texture classification[J]. IEEE transactions on image processing,2010,19(6):1657-1663.
[20]SUN Y,LIANG D,WANG X,et al. Deepid3:Face recognition with very deep neural networks[DB/OL]. [2018-06-10]. https://arxiv.org/abs/1502.00873.
[21]SRIVASTAVA N,HINTON G E,KRIZHEVSKY A,et al. Dropout:a simple way to prevent neural networks from overfitting[J]. Journal of machine learning research,2014,15(1):1929-1958.
[22]THEODORIDIS S. Improving the way neural networks learn[M]//MICHAEL A N. Neural networks and deep learning. Pittsburgh:Academic Press,2015.
[23]LUCEY P,COHN J F,KANADE T,et al. The extended cohn-kanade dataset(ck+):a complete dataset for action unit and emotion-specified expression[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops(CVPRW). San Francisco,USA:IEEE,2010:94-101.
[24]EKMAN P. Strong evidence for universals in facial expressions:a reply to Russell’s mistaken critique[J]. Psychological bulletin,1994,115(2):268-287.
[25]LYONS M,AKAMATSU S,KAMACHI M,et al. Coding facial expressions with gabor wavelets[C]//Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition. Nara,Japan:IEEE,1998:200-205.
[26]ALY S,ABBOTT A L,TORKI M. A multi-modal feature fusion framework for kinect-based facial expression recognition using Dual Kernel Discriminant Analysis(DKDA)[C]//IEEE Winter Conference on Applications of Computer Vision. NewYork,USA:IEEE,2016:1-10.
[27]RIVERA A R,CASTILLO J R,CHAE O O. Local directional number pattern for face analysis:face and expression recognition[J]. IEEE transactions on image processing,2013,22(5):1740-1752.

相似文献/References:

[1]谢 非,龚 俊,王元祥,等.基于肤色增强和分块PCA的人脸表情识别方法[J].南京师范大学学报(工程技术版),2017,17(02):049.[doi:10.3969/j.issn.1672-1292.2017.02.008]
　Xie Fei,Gong Jun,Wang Yuanxiang,et al.A Facial Expression Recognition Method Based onSkin Color Enhancement and Block PCA[J].Journal of Nanjing Normal University(Engineering and Technology),2017,17(03):049.[doi:10.3969/j.issn.1672-1292.2017.02.008]
[2]吕 晶,薛亚非,谷静平.像素邻域差向量协方差特征的多视角人脸检测[J].南京师范大学学报(工程技术版),2022,22(02):073.[doi:10.3969/j.issn.1672-1292.2022.02.011]
　Lü Jing,Xue Yafei,Gu Jingping.Multi-view Face Detection Based on the Pixel Differential Covariance Feature[J].Journal of Nanjing Normal University(Engineering and Technology),2022,22(03):073.[doi:10.3969/j.issn.1672-1292.2022.02.011]

备注/Memo

备注/Memo:
收稿日期:2018-04-18.
基金项目:国家自然基金(61501060)、江苏省科技厅青年基金(BK20150271)、江苏省道路载运工具新技术应用重点实验室开放课题(ZMF15020068).
通讯联系人:杨彪,博士,讲师,研究方向:机器视觉、模式识别. E-mail:yb6864171@cczu.edu.cn
更新日期/Last Update: 2018-09-30