|Table of Contents|

Bone Dual-Stream Attention Enhancement Graph Convolving Human Posture Recognition(PDF)

南京师范大学学报（工程技术版）[ISSN:1006-6977/CN:61-1281/TN]

Issue:
2024年04期
Page:
57-67
Research Field:
计算机科学与技术
Publishing date:

Info

Title:
Bone Dual-Stream Attention Enhancement Graph Convolving Human Posture Recognition
Author(s):
Chen Bin; Fan Feiyan; Lu Tianyi
(Information Construction Management Division,Nanjing Normal University,Nanjing 210023,China)
Keywords:
posture recognition; time and space double domain; attention mechanism; figure convolution; skeletal features; movement information representation
PACS:
TP39; TH691.9
DOI:
10.3969/j.issn.1672-1292.2024.04.006
Abstract:
In order to solve the lack of value analysis of motion correlation information in the loss of meaning of skeletal nodes and dependency information,the paper proposes a model of bone dual-stream attention enhancement graph convolving human posture recognition. The airspace connection matrix and time domain information matrix between bone joints are constructed on the basis of extracting bone feature nodes. With this basis,dual-flow bone node information processing is performed. Taking advantage of the channel attention mechanism for context processing,decturing key node dependencies and global bone motion implications,a two-domain bone topology weighted by neighborhood nodes is constructed. The comparative validation on two datasets Kinetics and NTU RGB+D shows that the model performs better on different datasets. Horizontal comparison with the more representative mainstream methods in the field is shown,the model outperforms the other models in the recognition accuracy of the nine selected behavioral poses. This method reflects the better recognition rate and stability in human posture recognition,and proves the mining value of spatial-temporal dual-domain bone feature information.

References:

[1]YANG X D,TIAN Y L. Effctive 3D action recognition using eigenjoints[J]. Journal of Visual Communication and Image Representation,2014,25(1):2-11.
[2]石跃祥,朱茂清. 基于骨架动作识别的协作卷积Transformer网络[J]. 电子与信息学报,2023,45(4):1485-1493.
[3]CHAUDHRY R,RAVICHANDRAN A,HAGER G,et al. Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami,USA:IEEE,2009.
[4]姜权晏,吴小俊,徐天阳. 用于骨架行为识别的多维特征嵌合注意力机制[J]. 中国图象图形学报,2022,27(8):2391-2403.
[5]周风余,尹建芹,杨阳,等. 基于时序深度置信网络的在线人体动作识别[J]. 自动化学报,2016,42(7):1030-1039.
[6]PENG W,HONG X P,CHEN H Y,et al. Learning graph convolutional network for skeleton-based human action recognition by neural searching[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York,USA:AAAI,2020.
[7]马钰锡,谭励,董旭,等. 面向智能监控的行为识别[J]. 中国图象图形学报,2019,24(2):282-290.
[8]DU W B,WANG Y L,QIAO Y. Recurrent spatial-temporal attention network for action recognition in videos[J]. IEEE Transactions on Image Processing,2018,27(3):1347-1360.
[9]XIA R J,LI Y S,LUO W H. LAGA-Net:Local-and-global attention network for skeleton based action recognition[J]. IEEE Transactions on Multimedia,2022,24:2648-2661.
[10]JIANG X H,XU K,SUN T F. Action recognition scheme based on skeleton representation with DS-LSTM network[J]. IEEE Transactions on Circuits and Systems for Video Technology,2020,30(7):2129-2140.
[11]SI C Y,CHEN W T,WANG W,et al. An attention enhanced graph convolutional LSTM network for skeleton-based action recognition[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach,USA:IEEE,2019.
[12]李扬志,袁家政,刘宏哲. 基于时空注意力图卷积网络模型的人体骨架动作识别算法[J]. 计算机应用,2021,41(7):1915-1921.
[13]田志强,邓春华,张俊雯. 基于骨骼时序散度特征的人体行为识别算法[J]. 计算机应用,2021,41(5):1450-1457.
[14]SHI L,ZHANG Y F,CHENG J,et al. Skeleton-based action recognition with multi-stream adaptive graph convolutional networks[J]. IEEE Transactions on Image Processing,2020,29:9532-9545.
[15]CAO B,XIA H,LIU Z. A video abnormal behavior recognition algorithm based on deep learning[C]//Proceedings of the 2021 IEEE 4th Advanced Information Management,Communicates,Electronic and Automation Control Conference(IMCEC). Chongqing,China:IEEE,2021.
[16]薛盼盼,刘云,李辉,等. 基于时域扩张残差网络和双分支结构的人体行为识别[J]. 控制与决策,2022,37(11):2993-3002.
[17]NAVEENKUMAR M,DOMNIC S. Spatio temporal joint distance maps for skeleton-based action recognition using convolutional neural networks[J]. International Journal of Image and Graphics,2021,21(5):s0219467821400015.
[18]钱银中,沈一帆. 姿态特征与深度特征在图像动作识别中的混合应用[J]. 自动化学报,2019,45(3):626-636.
[19]YANG H Y,GU Y Z,ZHU J C,et al. PGCN-TCA:Pseudo graph convolutional network with temporal and channel-wise attention for skeleton-based action recognition[J]. IEEE Access,2020,8:10040-10047.
[20]SULTANI W,CHEN C,SHAH M,et al. Real-world anomaly detection in surveillance videos[C]//Proceedings of the 2018 IEEE/AVF Conference on Computer Vision and Pattern Recognition. Salt Lake City,USA:IEEE,2018.
[21]PENG W,SHI J G,ZHAO G Y. Spatial temporal graph deconvolutional network for skeleton-based human action recognition[J]. IEEE Signal Processing Letters,2021,28:244-248.
[22]YU W,YANG K Y,YAO H X,et al. Exploiting the complementary strengths of multi-layer CNN features for image retrieval[J]. Neurocomputing,2017,237:235-241.
[23]LIU J,SHAHROUDY A,WANG G,et al. Skeleton-based online action prediction using scale selection network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(6):1453-1467.
[24]PENG W,SHI J,VARANKA T,et al. Rethinking the ST-GCNs for 3D skeleton-based human action recognition[J]. Neurocomputing,2021,454:45-53.
[25]MIRZA A,SIDDIQI I. Recognition of cursive video text using a deep learning framework[J]. IET Image Processing,2020,14(14):3444-3455.
[26]ZHANG S Y,YANG Y,XIAO J,et al. Fusing geometric features for skeleton-based action recognition using multilayer LSTM networks[J]. IEEE Transactions on Multimedia,2018,20(9):2230-2343.
[27]CHEN C,LIU B,WAN S H,et al. An edge traffic flow detection scheme based on deep learning in an intelligent transportation system[J]. IEEE Transactions on Intelligent Transportation Systems,2021,22(3):1840-1852.
[28]张全贵,蔡丰,李志强. 基于耦合多隐马尔可夫模型和深度图像数据的人体动作识别[J]. 计算机应用,2018,38(2):454-457.
[29]LI C,XIE C Y,ZHANG B C,et al. Memory attention networks for skeleton-based action recognition[J]. IEEE Transactions on Neural Networks and Learning Systems,2022,33(9):4800-4814.
[30]LIU J,SHAHROUDY A,PEREZ M,et al. NTU RGB+D 120:A large-scale benchmark for 3D human activity understanding[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(10):2684-2701.
[31]CAO Z,HIDALGO G,SIMON T,et al. Openpose:realtime multi-person 2D pose estimation using part affinity fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,43(1):172-186.
[32]HUANG L J,HUANG Y,OUYANG W L,et al. Part-level graph convolutional network for skeleton-based action recognition[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York,USA:AAAI,2020.
[33]HATIRNZA E,SAH M,DIREKOGLU C A. Novel framework and concept-based semantic search Interface for abnormal crowd behaviour analysis in surveillance videos[J]. Multimedia Tools & Applications,2020,79(25/26):17579-17617.
[34]王佳铖,鲍劲松,刘天元,等. 基于工件注意力的车间作业行为在线识别方法[J]. 计算机集成制造系统,2021,27(4):1099-1107.
[35]苏江毅,宋晓宁,吴小俊,等. 多模态轻量级图卷积人体骨架行为识别方法[J]. 计算机科学与探索,2021,15(4):733-742.
[36]JI S W,XU W,YANG M,et al. 3D convolutional neural networks for human action recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(1):221-231.
[37]黄海新,王瑞鹏,刘孝阳. 基于3D卷积的人体行为识别技术综述[J]. 计算机科学,2020,47(增刊2):139-144.
[38]ZHANG B W,WANG Y D,HOU W X,et al. Flexmatch:Boosting semi-supervised learning with curriculum pseudo labeling[C]//Proceedings of the 35th Conference on Neural Information Processing System(NeurIPS 2021). Online,Canada:NeurIPS,2021.
[39]CHEN P,GAO Y,MA A J. Multi-level attentive adversarial learning with temporal dilation for unsupervised video domain adaptation[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Waikoloa,USA:IEEE,2022.

Memo

Memo:
-
Last Update: 2024-12-15