[1]陈 斌,樊飞燕,陆天易.骨骼双流注意力增强图卷积人体姿态识别[J].南京师范大学学报(工程技术版),2024,24(04):057-67.[doi:10.3969/j.issn.1672-1292.2024.04.006]
　Chen Bin,Fan Feiyan,Lu Tianyi.Bone Dual-Stream Attention Enhancement Graph Convolving Human Posture Recognition[J].Journal of Nanjing Normal University(Engineering and Technology),2024,24(04):057-67.[doi:10.3969/j.issn.1672-1292.2024.04.006]
点击复制

骨骼双流注意力增强图卷积人体姿态识别
分享到：

南京师范大学学报（工程技术版）[ISSN:1006-6977/CN:61-1281/TN]

卷:
24卷
期数:
2024年04期
页码:
057-67
栏目:
计算机科学与技术
出版日期:
2024-12-15

文章信息/Info

Title:
Bone Dual-Stream Attention Enhancement Graph Convolving Human Posture Recognition
文章编号:
1672-1292(2024)04-0057-11
作者:
陈 斌; 樊飞燕; 陆天易
(南京师范大学信息化建设管理处,江苏 南京 210023)
Author(s):
Chen Bin; Fan Feiyan; Lu Tianyi
(Information Construction Management Division,Nanjing Normal University,Nanjing 210023,China)
关键词:
姿态识别; 时空双域; 注意力机制; 图卷积; 骨骼特征; 运动信息表示
Keywords:
posture recognition; time and space double domain; attention mechanism; figure convolution; skeletal features; movement information representation
分类号:
TP39; TH691.9
DOI:
10.3969/j.issn.1672-1292.2024.04.006
文献标志码:
A
摘要:
为解决骨骼关键点分类算法中运动时间线中运动关联信息的价值分析缺乏,以及骨骼节点关联性和依赖关系信息含义丢失问题,提出了一种骨骼双流注意力增强图卷积人体姿态识别模型. 以提取骨骼特征节点为基础,构建骨骼关节点之间空域连接矩阵和运动时间线时域信息矩阵,在此基础上进行双流骨骼节点信息处理. 利用通道注意力机制对上下文处理的优势,构解关键节点间依赖关系以及全局骨骼运动含义,构建邻域节点加权的双域骨骼拓扑. 在Kinetics和NTU RGB+D两个数据集上的对比验证显示,该模型在不同数据集上均有较好的执行效果. 与领域内较具代表性的主流方法的横向比对显示,该模型在选定的9种行为姿态的识别精度上均优于其他模型. 该方法在人体姿态识别上体现了较优的识别率及稳定性,并佐证了时空双域骨骼特征信息的挖掘价值.
Abstract:
In order to solve the lack of value analysis of motion correlation information in the loss of meaning of skeletal nodes and dependency information,the paper proposes a model of bone dual-stream attention enhancement graph convolving human posture recognition. The airspace connection matrix and time domain information matrix between bone joints are constructed on the basis of extracting bone feature nodes. With this basis,dual-flow bone node information processing is performed. Taking advantage of the channel attention mechanism for context processing,decturing key node dependencies and global bone motion implications,a two-domain bone topology weighted by neighborhood nodes is constructed. The comparative validation on two datasets Kinetics and NTU RGB+D shows that the model performs better on different datasets. Horizontal comparison with the more representative mainstream methods in the field is shown,the model outperforms the other models in the recognition accuracy of the nine selected behavioral poses. This method reflects the better recognition rate and stability in human posture recognition,and proves the mining value of spatial-temporal dual-domain bone feature information.

参考文献/References:

[1]YANG X D,TIAN Y L. Effctive 3D action recognition using eigenjoints[J]. Journal of Visual Communication and Image Representation,2014,25(1):2-11.
[2]石跃祥,朱茂清. 基于骨架动作识别的协作卷积Transformer网络[J]. 电子与信息学报,2023,45(4):1485-1493.
[3]CHAUDHRY R,RAVICHANDRAN A,HAGER G,et al. Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami,USA:IEEE,2009.
[4]姜权晏,吴小俊,徐天阳. 用于骨架行为识别的多维特征嵌合注意力机制[J]. 中国图象图形学报,2022,27(8):2391-2403.
[5]周风余,尹建芹,杨阳,等. 基于时序深度置信网络的在线人体动作识别[J]. 自动化学报,2016,42(7):1030-1039.
[6]PENG W,HONG X P,CHEN H Y,et al. Learning graph convolutional network for skeleton-based human action recognition by neural searching[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York,USA:AAAI,2020.
[7]马钰锡,谭励,董旭,等. 面向智能监控的行为识别[J]. 中国图象图形学报,2019,24(2):282-290.
[8]DU W B,WANG Y L,QIAO Y. Recurrent spatial-temporal attention network for action recognition in videos[J]. IEEE Transactions on Image Processing,2018,27(3):1347-1360.
[9]XIA R J,LI Y S,LUO W H. LAGA-Net:Local-and-global attention network for skeleton based action recognition[J]. IEEE Transactions on Multimedia,2022,24:2648-2661.
[10]JIANG X H,XU K,SUN T F. Action recognition scheme based on skeleton representation with DS-LSTM network[J]. IEEE Transactions on Circuits and Systems for Video Technology,2020,30(7):2129-2140.
[11]SI C Y,CHEN W T,WANG W,et al. An attention enhanced graph convolutional LSTM network for skeleton-based action recognition[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach,USA:IEEE,2019.
[12]李扬志,袁家政,刘宏哲. 基于时空注意力图卷积网络模型的人体骨架动作识别算法[J]. 计算机应用,2021,41(7):1915-1921.
[13]田志强,邓春华,张俊雯. 基于骨骼时序散度特征的人体行为识别算法[J]. 计算机应用,2021,41(5):1450-1457.
[14]SHI L,ZHANG Y F,CHENG J,et al. Skeleton-based action recognition with multi-stream adaptive graph convolutional networks[J]. IEEE Transactions on Image Processing,2020,29:9532-9545.
[15]CAO B,XIA H,LIU Z. A video abnormal behavior recognition algorithm based on deep learning[C]//Proceedings of the 2021 IEEE 4th Advanced Information Management,Communicates,Electronic and Automation Control Conference(IMCEC). Chongqing,China:IEEE,2021.
[16]薛盼盼,刘云,李辉,等. 基于时域扩张残差网络和双分支结构的人体行为识别[J]. 控制与决策,2022,37(11):2993-3002.
[17]NAVEENKUMAR M,DOMNIC S. Spatio temporal joint distance maps for skeleton-based action recognition using convolutional neural networks[J]. International Journal of Image and Graphics,2021,21(5):s0219467821400015.
[18]钱银中,沈一帆. 姿态特征与深度特征在图像动作识别中的混合应用[J]. 自动化学报,2019,45(3):626-636.
[19]YANG H Y,GU Y Z,ZHU J C,et al. PGCN-TCA:Pseudo graph convolutional network with temporal and channel-wise attention for skeleton-based action recognition[J]. IEEE Access,2020,8:10040-10047.
[20]SULTANI W,CHEN C,SHAH M,et al. Real-world anomaly detection in surveillance videos[C]//Proceedings of the 2018 IEEE/AVF Conference on Computer Vision and Pattern Recognition. Salt Lake City,USA:IEEE,2018.
[21]PENG W,SHI J G,ZHAO G Y. Spatial temporal graph deconvolutional network for skeleton-based human action recognition[J]. IEEE Signal Processing Letters,2021,28:244-248.
[22]YU W,YANG K Y,YAO H X,et al. Exploiting the complementary strengths of multi-layer CNN features for image retrieval[J]. Neurocomputing,2017,237:235-241.
[23]LIU J,SHAHROUDY A,WANG G,et al. Skeleton-based online action prediction using scale selection network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(6):1453-1467.
[24]PENG W,SHI J,VARANKA T,et al. Rethinking the ST-GCNs for 3D skeleton-based human action recognition[J]. Neurocomputing,2021,454:45-53.
[25]MIRZA A,SIDDIQI I. Recognition of cursive video text using a deep learning framework[J]. IET Image Processing,2020,14(14):3444-3455.
[26]ZHANG S Y,YANG Y,XIAO J,et al. Fusing geometric features for skeleton-based action recognition using multilayer LSTM networks[J]. IEEE Transactions on Multimedia,2018,20(9):2230-2343.
[27]CHEN C,LIU B,WAN S H,et al. An edge traffic flow detection scheme based on deep learning in an intelligent transportation system[J]. IEEE Transactions on Intelligent Transportation Systems,2021,22(3):1840-1852.
[28]张全贵,蔡丰,李志强. 基于耦合多隐马尔可夫模型和深度图像数据的人体动作识别[J]. 计算机应用,2018,38(2):454-457.
[29]LI C,XIE C Y,ZHANG B C,et al. Memory attention networks for skeleton-based action recognition[J]. IEEE Transactions on Neural Networks and Learning Systems,2022,33(9):4800-4814.
[30]LIU J,SHAHROUDY A,PEREZ M,et al. NTU RGB+D 120:A large-scale benchmark for 3D human activity understanding[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(10):2684-2701.
[31]CAO Z,HIDALGO G,SIMON T,et al. Openpose:realtime multi-person 2D pose estimation using part affinity fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,43(1):172-186.
[32]HUANG L J,HUANG Y,OUYANG W L,et al. Part-level graph convolutional network for skeleton-based action recognition[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York,USA:AAAI,2020.
[33]HATIRNZA E,SAH M,DIREKOGLU C A. Novel framework and concept-based semantic search Interface for abnormal crowd behaviour analysis in surveillance videos[J]. Multimedia Tools & Applications,2020,79(25/26):17579-17617.
[34]王佳铖,鲍劲松,刘天元,等. 基于工件注意力的车间作业行为在线识别方法[J]. 计算机集成制造系统,2021,27(4):1099-1107.
[35]苏江毅,宋晓宁,吴小俊,等. 多模态轻量级图卷积人体骨架行为识别方法[J]. 计算机科学与探索,2021,15(4):733-742.
[36]JI S W,XU W,YANG M,et al. 3D convolutional neural networks for human action recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(1):221-231.
[37]黄海新,王瑞鹏,刘孝阳. 基于3D卷积的人体行为识别技术综述[J]. 计算机科学,2020,47(增刊2):139-144.
[38]ZHANG B W,WANG Y D,HOU W X,et al. Flexmatch:Boosting semi-supervised learning with curriculum pseudo labeling[C]//Proceedings of the 35th Conference on Neural Information Processing System(NeurIPS 2021). Online,Canada:NeurIPS,2021.
[39]CHEN P,GAO Y,MA A J. Multi-level attentive adversarial learning with temporal dilation for unsupervised video domain adaptation[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Waikoloa,USA:IEEE,2022.

备注/Memo

备注/Memo:
收稿日期:2024-06-20.
基金项目:江苏省现代教育技术研究2023年度智慧校园专项课题(2023-R-107311).
通讯作者:陈斌,博士,高级工程师,研究方向:模式识别、机器学习、大数据分析方面的研究. E-mail:60167@njnu.edu.cn
更新日期/Last Update: 2024-12-15