|Table of Contents|

Binary-Channel Convolutional Neural Network forFacial Expression Recognition(PDF)

南京师范大学学报（工程技术版）[ISSN:1006-6977/CN:61-1281/TN]

Issue:
2018年03期
Page:
1-
Research Field:
人工智能算法与应用专栏
Publishing date:

Info

Title:
Binary-Channel Convolutional Neural Network forFacial Expression Recognition
Author(s):
Cao Jinmeng1; Ni Rongrong2; Yang Biao1
(1.School of Information Science and Engineering,Changzhou University,Changzhou 213164,China)(2.Department of Mechanical and Electrical,Changzhou Vocational Institute of Textile and Garment,Changzhou 213164,China)
Keywords:
face detection; facial expression recognition; binary-channel; convolutional neural network; LBP images
PACS:
TP391.41
DOI:
10.3969/j.issn.1672-1292.2018.03.001
Abstract:
Facial expression recognition is an important way for machines to understand emotional changes of human beings. However,accurate hand-crafted features are hard to extract due to the fact that emotion recognition may be severely influenced by individual differences and emotional intensity differences. A facial expression recognition approach based on binary-channel convolutional neural network(BC-CNN)is proposed in this paper. Initially,the sampled facial images are pre-processed for region limitation of further analysis. Global and detailed features are both considered by studying facial gray-scale images and corresponding LBP images simultaneously. For binary-channel input data,two CNNs with different parameters are employed to extract facial features automatically and further,a weighted merge classify network(WMCN)is used for feature fusion and the softmax is finally used to recognize different facial expressions. Experimental results indicate that our algorithm can recognize six fundamental facial expressions(happy,sadness,angry,disgust,fear and surprise)with high accuracy. Its performance is better than that of those methods based on hand-crafted features or single channel CNN. Compared with other binary-channel CNN approach,our approach is easier to implement with similar recognition accuracy.

References:

[1] 黄建,李文书,高玉娟. 人脸表情识别研究进展[J]. 计算机科学,2016,43(增刊2):123-126.
HUANG J,LI W S,GAO Y J. Research advance of facial expression recognition[J]. Computer science,2016,43(Suppl. 2):123-126.(in Chinese)
[2]CHEN C R,WONG W S,CHIU C T. A 0.64 mm real-time cascade face detection design based on reduced two-field extraction[J]. IEEE transactions on very large scale integration system,2011,19(11):1937-1948.
[3]BARNOUTI N H,ALDABBAGH S S M,MATTI W E,et al. Face detection and recognition using viola-jones with PCA-LDA and square euclidean distance[J]. International journal of advanced computer science and applications,2016,7(5):371-377.
[4]DEMIRKUS M,PRECUP D,CLARK J,et al. Multi-layer temporal graphical model for head pose estimation in real-world videos[C]//2014 IEEE International Conference on Image Processing. Paris,France:IEEE,2014:3392-3396.
[5]JAIN S,HU C,AGGARWAL J. Facial expression recognition with temporal modeling of shapes[C]//2011 International Conference on Computer Vision Workshops. Barcelona,Spain:IEEE,2011:1642-1649.
[6]ALI G,IQBAL M A,CHOI T S. Boosted NNE collections for multicultural facial expression recognition[J]. Pattern recognition,2016,55:14-27.
[7]李宇成,刘昆. 一种基于人脸分割的PCA和SVM人脸识别方法[J]. 微型机与应用,2016,35(15):51-53.
LI Y C,LIU K. PCA and SVM face recognition method based on human face segmentation[J]. Microcomputer & its applications,2016,35(15):51-53.(in Chinese)
[8]PU X,FAN K,CHEN X,et al. Facial expression recognition from image sequences using twofold random forest classifier[J]. Neurocomputing,2015,168:1173-1180.
[9]LI H,LIN Z,SHEN X,et al. A convolutional neural network cascade for face detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston,USA:IEEE,2015:5325-5334.
[10]LECUN Y,BENGIO Y,HINTON G. Deep learning[J]. Nature,2015,521(7553):436-444.
[11]ZHONG L,LIU Q S,YANG P,et al. Learning multi-scale active facial patches for expression analysis[J]. IEEE transactions on cybernetics,2015,45(8):1499-1510.
[12]ZHANG W,ZHANG Y,MA L,et al. Multimodal learning for facial expression recognition[J]. Pattern recognition,2015,48(10):3191-3202.
[13]WALECKI R,RUDOVIC O,PAVLOVIC V,et al. Variable-state latent conditional random fields for facial expression recognition and action unit detection[C]//11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition. Ljubljana,Slovenia:IEEE,2015:1-8.
[14]郑胤,陈权崎,章毓晋. 深度学习及其在目标和行为识别中的新进展[J]. 中国图象图形学报,2014,19(2):175-184.
ZHENG Y,CHEN Q Q,ZHANG Y J. Deep learning and its new progress in object and behavior recognition[J]. Journal of image and graphics,2014,19(2):175-184.(in Chinese)
[15]何俊,蔡建峰,房灵芝,等. 基于LBP/VAR与DBN模型的人脸表情识别[J]. 计算机应用研究,2016,33(8):2509-2513.
HE J,CAI J F,FANG L Z,et al. Facial expression recognition based on LBP/VAR and DBN model[J]. Application research of computer,2016,33(8):2509-2513.(in Chinese)
[16]LOPES A T,DE AGUIAR E,DE SOUZA A F,et al. Facial expression recognition with convolutional neural networks:coping with few data and the training sample order[J]. Pattern recognition,2017,61:610-628.
[17]YU Z,ZHANG C. Image based static facial expression recognition with multiple deep network learning[C]//Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. London,UK:ACM,2015:435-442.v[18]SIMARD P,STEINKRAUS D,PLATT J C. Best practices for convolutional neural networks applied to visual document analysis[C]//Proceedings of the 7th International Conference on Document Analysis and Recognition. Edinburgh,Scotland:IEEE,2003:958-963.
[19]GUO Z,ZHANG L,ZHANG D. A completed modeling of local binary pattern operator for texture classification[J]. IEEE transactions on image processing,2010,19(6):1657-1663.
[20]SUN Y,LIANG D,WANG X,et al. Deepid3:Face recognition with very deep neural networks[DB/OL]. [2018-06-10]. https://arxiv.org/abs/1502.00873.
[21]SRIVASTAVA N,HINTON G E,KRIZHEVSKY A,et al. Dropout:a simple way to prevent neural networks from overfitting[J]. Journal of machine learning research,2014,15(1):1929-1958.
[22]THEODORIDIS S. Improving the way neural networks learn[M]//MICHAEL A N. Neural networks and deep learning. Pittsburgh:Academic Press,2015.
[23]LUCEY P,COHN J F,KANADE T,et al. The extended cohn-kanade dataset(ck+):a complete dataset for action unit and emotion-specified expression[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops(CVPRW). San Francisco,USA:IEEE,2010:94-101.
[24]EKMAN P. Strong evidence for universals in facial expressions:a reply to Russell’s mistaken critique[J]. Psychological bulletin,1994,115(2):268-287.
[25]LYONS M,AKAMATSU S,KAMACHI M,et al. Coding facial expressions with gabor wavelets[C]//Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition. Nara,Japan:IEEE,1998:200-205.
[26]ALY S,ABBOTT A L,TORKI M. A multi-modal feature fusion framework for kinect-based facial expression recognition using Dual Kernel Discriminant Analysis(DKDA)[C]//IEEE Winter Conference on Applications of Computer Vision. NewYork,USA:IEEE,2016:1-10.
[27]RIVERA A R,CASTILLO J R,CHAE O O. Local directional number pattern for face analysis:face and expression recognition[J]. IEEE transactions on image processing,2013,22(5):1740-1752.

Memo

Memo:
-
Last Update: 2018-09-30