|Table of Contents|

Face Forgery Detection Based on Facial Micro-Movements(PDF)

南京师范大学学报（工程技术版）[ISSN:1006-6977/CN:61-1281/TN]

Issue:
2024年04期
Page:
28-36
Research Field:
计算机科学与技术
Publishing date:

Info

Title:
Face Forgery Detection Based on Facial Micro-Movements
Author(s):
Wang Xiaopeng1; Zhu Feng1; Li Lei2; Liu Sinan3; Tan Xiaoyang1
(1.College of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 210016,China)
(2.Nanjing Research Institute of Electronics Engineering,Nanjing 210022,China)
(3.School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023,China)
Keywords:
face forgery detection; Transformer network; multivariate time series; attention mechanism; deepfakes
PACS:
TP391
DOI:
10.3969/j.issn.1672-1292.2024.04.003
Abstract:
With the rapid development of deep learning,facial video forgery techniques have become increasingly sophisticated,posing a significant threat to social security. Although image-based facial video authenticity detection methods have achieved remarkable progress and demonstrated certain robustness and generalization capabilities,existing video stream-based methods often suffer from high input dimensionality and substantial computational overhead,which remains inadequately addressed. To tackle these challenges,this paper proposes a facial video authenticity detection method based on multivariate time series analysis. Specifically,a novel modeling approach based on facial micro-movements is designed to convert video streams into multivariate time series,effectively reducing input dimensionality. Furthermore,an enhanced Transformer network is developed to improve its ability to model time series features. Experimental results show that the proposed method achieves performance comparable to state-of-the-art approaches in terms of accuracy and generalization,demonstrating promising application potential.

References:

[1]GOODFELLOW I J,POUGET A J,MIRZA M,et al. Generative adversarial nets[C]//Proceedings of the 27th Interrational Conferene on Neural Information Processing Systems. Montreal,Canada:NIPS,2014.
[2]RÖSSLER A,COZZOLINO D,VERDOLIVA L,et al. FaceForensics++:Learning to detect manipulated facial images[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision(ICCV). Seoul,ROK:IEEE,2019.
[3]QUAN W Z,WANG K,YAN D M,et al. Distinguishing between natural and computer-generated images using convolutional neural networks[J]. IEEE Transactions on Information Forensics and Security,2018,13(11):2772-2787.
[4]AFCHAR D,NOZICK V,YAMAGISHI J,et al. MesoNet:A compact facial video forgery detection network[C]//Proceedings of the 2018 IEEE International Workshop on Information Forensics and Security(WIFS). Hong Kong,China:IEEE,2018.
[5]DURALL R,KEUPER M,PFREUNDT F,et al. Unmasking deepfakes with simple features[EB/OL].(2019-11-02)[2024-05-12]. https://doi.org/10.48550/arXiv.1911.00686.
[6]QIAN Y Y,YIN G J,SHENG L,et al. Thinking in frequency:Face forgery detection by mining frequency-aware clues[C]//Proceedings of the European Conference on Computer Vision-ECCV2000. Glasgov,UK:Springer,2020.
[7]WANG J K,WU Z X,CHEN J J,et al. M2TR:Multi-modal multi-scale transformers for deepfake detection[C]//Proceedings of the 2022 International Conference on Multimedia Retrieval. Newark,USA:ICMR,2022.
[8]PIPIN S J,PURBA R,PASHA M F. Deepfake video detection using spatiotemporal convolutional network and photo response non uniformity[C]//Proceedings of the 2022 IEEE International Conference of Computer Science and Information Technology(ICOSNIKOM). Laguboti,Indonesia:IEEE,2022.
[9]GANIYUSUFOGLU I,NGÔ L M,SAVOV N,et al. Spatio-temporal features for generalized detection of deepfake videos[EB/OL].(2020-10-22)[2024-05-12]. https://doi.org/10.48550/arXiv.2010.11844.
[10]GÜERA D,DELP E. Deepfake video detection using recurrent neural networks[C]//Proceedings of the 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance(AVSS). Auckland,New Zealand:IEEE,2018.
[11]HALIASSOS A,VOUGIOUKAS K,PETRIDIS S,et al. Lips don't lie:A generalisable and robust approach to face forgery detection[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Nashville,USA:IEEE,2021.
[12]JUNG T,KIM S,KIM K. DeepVision:Deepfakes detection using human eye blinking pattern[J]. IEEE Access,2020,8:83144-83154.
[13]WANG R,MA L,XU J F,et al. FakeSpotter:A simple yet robust baseline for spotting AI-synthesized fake faces[C]//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. Yokohama,Japan:IJCAI,2020.
[14]SABIR E,CHENG J X,JAISWAL A,et al. Recurrent convolutional strategies for face manipulation detection in videos[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR2019). Long Beach,USA:IEEE,2019.
[15]CHU B L,YOU W K,YANG Z,et al. Protecting world leader using facial speaking pattern against deepfakes[J]. IEEE Signal Processing Letters,2022,29:2078-2082.
[16]SUN Z K,HAN Y J,HUA Z Y,et al. Improving the efficiency and robustness of deepfakes detection through precise geometric features[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Nashville,USA:IEEE,2021.
[17]QI H,GUO Q,XU J F,et al. DeepRhythm:Exposing deepfakes with attentional visual heartbeat rhythms[C]//Proceedings of the 28th ACM International Conference on Multimedia(MM'20). Seattle,USA:ACM,2020.
[18]WANG H,KLÄSER A,SCHMID C,et al. Action recognition by dense trajectories[C]//Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Colorado Springs,USA:IEEE,2011.
[19]EKMAN P,FRIESEN W. Facial Action Coding System:A Technique for the Measurement of Facial Movement[M]. Palo Alto,USA:Consulting Psychologists Press,1978.
[20]ROSTEN E,DRUMMOND T. Machine learning for high-speed corner detection[C]//Proceedings of the 9th European Conference on Computer Vision. Graz,Austria:Springer,2006.
[21]LUCAS B,KANADE T. An iterative image registration technique with an application to stereo vision[C]//Proceedings of the 7th International Joint Conference on Artificial Intelligence. Vancouver,Canada:IJCAI,1981.
[22]FISCHLER M A,BOLLES R C. Random sample consensus:A paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM,1981,24(6):381-395.
[23]LI Y Z,YANG X,SUN P,et al. Celeb-DF:A large-scale challenging dataset for deepfake forensics[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Seattle,USA:IEEE,2020.
[24]CIFTCI U A,DEMIR I,YIN L J. FakeCatcher:Detection of synthetic portrait videos using biological signals[J/OL]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020. https://doi.org/10.1109/TPAMI.2020.3009287.
[25]WANG G X,ZHOU J H,WU Y. Exposing deep-faked videos by anomalous co-motion pattern detection[EB/OL].(2020-08-11)[2024-05-12]. https://doi.org/10.48550/arXiv.2008.04848.
[26]LI X D,LANG Y N,CHEN Y F,et al. Sharp multiple instance learning for deepfake video detection[C]//Proceedings of the 28th ACM International Conference on Multimedia. Seattle,USA:ACM,2020.

Memo

Memo:
-
Last Update: 2024-12-15