[1]汪小鹏,朱 峰,李 磊,等.基于人脸微动作的伪造视频检测[J].南京师范大学学报(工程技术版),2024,24(04):028-36.[doi:10.3969/j.issn.1672-1292.2024.04.003]
　Wang Xiaopeng,Zhu Feng,Li Lei,et al.Face Forgery Detection Based on Facial Micro-Movements[J].Journal of Nanjing Normal University(Engineering and Technology),2024,24(04):028-36.[doi:10.3969/j.issn.1672-1292.2024.04.003]
点击复制

基于人脸微动作的伪造视频检测
分享到：

南京师范大学学报（工程技术版）[ISSN:1006-6977/CN:61-1281/TN]

卷:
24卷
期数:
2024年04期
页码:
028-36
栏目:
计算机科学与技术
出版日期:
2024-12-15

文章信息/Info

Title:
Face Forgery Detection Based on Facial Micro-Movements
文章编号:
1672-1292(2024)04-0028-09
作者:
汪小鹏1; 朱 峰1; 李 磊2; 刘司南3; 谭晓阳1
(1.南京航空航天大学计算机科学与技术学院,江苏 南京 210016)
(2.中国电子科技集团公司第二十八研究所,江苏 南京 210022)
(3.南京邮电大学计算机学院,江苏 南京 210023)
Author(s):
Wang Xiaopeng1; Zhu Feng1; Li Lei2; Liu Sinan3; Tan Xiaoyang1
(1.College of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 210016,China)
(2.Nanjing Research Institute of Electronics Engineering,Nanjing 210022,China)
(3.School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023,China)
关键词:
人脸伪造检测; Transformer网络; 多元变量时间序列; 注意力机制; 深度伪造
Keywords:
face forgery detection; Transformer network; multivariate time series; attention mechanism; deepfakes
分类号:
TP391
DOI:
10.3969/j.issn.1672-1292.2024.04.003
文献标志码:
A
摘要:
随着深度学习技术的快速发展,人脸视频伪造技术日益精进,其逼真效果对社会安全构成了严重威胁. 尽管基于静态图像的人脸视频真伪检测方法已取得显著进展,并展现出一定的鲁棒性和泛化能力,但现有基于视频流的检测方法通常面临输入维度过高和计算开销过大的问题,这一领域仍缺乏深入研究. 为了解决上述问题,提出了一种基于多变量时间序列分析的人脸视频真伪鉴别方法. 首先,设计了一种基于人脸微动作的建模方法,将视频流转化为多变量时间序列,从而显著降低输入维度. 随后,改进了Transformer网络结构,以增强其对时间序列特征的建模能力. 实验结果表明,所提出的方法在准确性与泛化性方面均与当前主流方法相当,展现了良好的应用潜力.
Abstract:
With the rapid development of deep learning,facial video forgery techniques have become increasingly sophisticated,posing a significant threat to social security. Although image-based facial video authenticity detection methods have achieved remarkable progress and demonstrated certain robustness and generalization capabilities,existing video stream-based methods often suffer from high input dimensionality and substantial computational overhead,which remains inadequately addressed. To tackle these challenges,this paper proposes a facial video authenticity detection method based on multivariate time series analysis. Specifically,a novel modeling approach based on facial micro-movements is designed to convert video streams into multivariate time series,effectively reducing input dimensionality. Furthermore,an enhanced Transformer network is developed to improve its ability to model time series features. Experimental results show that the proposed method achieves performance comparable to state-of-the-art approaches in terms of accuracy and generalization,demonstrating promising application potential.

参考文献/References:

[1]GOODFELLOW I J,POUGET A J,MIRZA M,et al. Generative adversarial nets[C]//Proceedings of the 27th Interrational Conferene on Neural Information Processing Systems. Montreal,Canada:NIPS,2014.
[2]RÖSSLER A,COZZOLINO D,VERDOLIVA L,et al. FaceForensics++:Learning to detect manipulated facial images[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision(ICCV). Seoul,ROK:IEEE,2019.
[3]QUAN W Z,WANG K,YAN D M,et al. Distinguishing between natural and computer-generated images using convolutional neural networks[J]. IEEE Transactions on Information Forensics and Security,2018,13(11):2772-2787.
[4]AFCHAR D,NOZICK V,YAMAGISHI J,et al. MesoNet:A compact facial video forgery detection network[C]//Proceedings of the 2018 IEEE International Workshop on Information Forensics and Security(WIFS). Hong Kong,China:IEEE,2018.
[5]DURALL R,KEUPER M,PFREUNDT F,et al. Unmasking deepfakes with simple features[EB/OL].(2019-11-02)[2024-05-12]. https://doi.org/10.48550/arXiv.1911.00686.
[6]QIAN Y Y,YIN G J,SHENG L,et al. Thinking in frequency:Face forgery detection by mining frequency-aware clues[C]//Proceedings of the European Conference on Computer Vision-ECCV2000. Glasgov,UK:Springer,2020.
[7]WANG J K,WU Z X,CHEN J J,et al. M2TR:Multi-modal multi-scale transformers for deepfake detection[C]//Proceedings of the 2022 International Conference on Multimedia Retrieval. Newark,USA:ICMR,2022.
[8]PIPIN S J,PURBA R,PASHA M F. Deepfake video detection using spatiotemporal convolutional network and photo response non uniformity[C]//Proceedings of the 2022 IEEE International Conference of Computer Science and Information Technology(ICOSNIKOM). Laguboti,Indonesia:IEEE,2022.
[9]GANIYUSUFOGLU I,NGÔ L M,SAVOV N,et al. Spatio-temporal features for generalized detection of deepfake videos[EB/OL].(2020-10-22)[2024-05-12]. https://doi.org/10.48550/arXiv.2010.11844.
[10]GÜERA D,DELP E. Deepfake video detection using recurrent neural networks[C]//Proceedings of the 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance(AVSS). Auckland,New Zealand:IEEE,2018.
[11]HALIASSOS A,VOUGIOUKAS K,PETRIDIS S,et al. Lips don't lie:A generalisable and robust approach to face forgery detection[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Nashville,USA:IEEE,2021.
[12]JUNG T,KIM S,KIM K. DeepVision:Deepfakes detection using human eye blinking pattern[J]. IEEE Access,2020,8:83144-83154.
[13]WANG R,MA L,XU J F,et al. FakeSpotter:A simple yet robust baseline for spotting AI-synthesized fake faces[C]//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. Yokohama,Japan:IJCAI,2020.
[14]SABIR E,CHENG J X,JAISWAL A,et al. Recurrent convolutional strategies for face manipulation detection in videos[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR2019). Long Beach,USA:IEEE,2019.
[15]CHU B L,YOU W K,YANG Z,et al. Protecting world leader using facial speaking pattern against deepfakes[J]. IEEE Signal Processing Letters,2022,29:2078-2082.
[16]SUN Z K,HAN Y J,HUA Z Y,et al. Improving the efficiency and robustness of deepfakes detection through precise geometric features[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Nashville,USA:IEEE,2021.
[17]QI H,GUO Q,XU J F,et al. DeepRhythm:Exposing deepfakes with attentional visual heartbeat rhythms[C]//Proceedings of the 28th ACM International Conference on Multimedia(MM'20). Seattle,USA:ACM,2020.
[18]WANG H,KLÄSER A,SCHMID C,et al. Action recognition by dense trajectories[C]//Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Colorado Springs,USA:IEEE,2011.
[19]EKMAN P,FRIESEN W. Facial Action Coding System:A Technique for the Measurement of Facial Movement[M]. Palo Alto,USA:Consulting Psychologists Press,1978.
[20]ROSTEN E,DRUMMOND T. Machine learning for high-speed corner detection[C]//Proceedings of the 9th European Conference on Computer Vision. Graz,Austria:Springer,2006.
[21]LUCAS B,KANADE T. An iterative image registration technique with an application to stereo vision[C]//Proceedings of the 7th International Joint Conference on Artificial Intelligence. Vancouver,Canada:IJCAI,1981.
[22]FISCHLER M A,BOLLES R C. Random sample consensus:A paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM,1981,24(6):381-395.
[23]LI Y Z,YANG X,SUN P,et al. Celeb-DF:A large-scale challenging dataset for deepfake forensics[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Seattle,USA:IEEE,2020.
[24]CIFTCI U A,DEMIR I,YIN L J. FakeCatcher:Detection of synthetic portrait videos using biological signals[J/OL]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020. https://doi.org/10.1109/TPAMI.2020.3009287.
[25]WANG G X,ZHOU J H,WU Y. Exposing deep-faked videos by anomalous co-motion pattern detection[EB/OL].(2020-08-11)[2024-05-12]. https://doi.org/10.48550/arXiv.2008.04848.
[26]LI X D,LANG Y N,CHEN Y F,et al. Sharp multiple instance learning for deepfake video detection[C]//Proceedings of the 28th ACM International Conference on Multimedia. Seattle,USA:ACM,2020.

备注/Memo

备注/Memo:
收稿日期:2024-05-12.
基金项目:国家自然科学基金项目(61373060).
通讯作者:谭晓阳,博士,教授,研究方向:机器学习. E-mail:x.tan@nuaa.edu.cn
更新日期/Last Update: 2024-12-15