[1]李少凡,高尚兵,张莹莹,等.基于轻量化网络与嵌入式的分心行为协同检测系统[J].南京师范大学学报(工程技术版),2023,23(01):025-32.[doi:10.3969/j.issn.1672-1292.2023.01.004]
　Li Shaofan,Gao Shangbing,Zhang Yingying,et al.Collaborative Detection System for Distraction Behavior Based on Lightweight Network and Embedded Platform[J].Journal of Nanjing Normal University(Engineering and Technology),2023,23(01):025-32.[doi:10.3969/j.issn.1672-1292.2023.01.004]
点击复制

基于轻量化网络与嵌入式的分心行为协同检测系统
分享到：

南京师范大学学报（工程技术版）[ISSN:1006-6977/CN:61-1281/TN]

卷:
23卷
期数:
2023年01期
页码:
025-32
栏目:
计算机科学与技术
出版日期:
2023-03-15

文章信息/Info

Title:
Collaborative Detection System for Distraction Behavior Based on Lightweight Network and Embedded Platform
文章编号:
1672-1292(2023)01-0025-08
作者:
李少凡1; 2; 高尚兵1; 2; 张莹莹1; 2; 黄 想1; 杨苏强1; 郭筱宇1
(1.淮阴工学院计算机与软件工程学院,江苏 淮安 223001) (2.淮阴工学院江苏省物联网移动互联技术工程实验室,江苏 淮安 223001)
Author(s):
Li Shaofan1; 2; Gao Shangbing1; 2; Zhang Yingying1; 2; Huang Xiang1; Yang Suqiang1; Guo Xiaoyu1
(1.Faculty of Computer and Software Engineering,Huaiyin Institute of Technology,Huai'an 223001,China) (2.Laboratory for Internet of Things and Mobile Internet Technology of Jiangsu Province,Huaiyin Institute of Technology,Huai'an 223001,China)
关键词:
协同检测; 人物交互; 轻量级网络; 智能交通; 深度学习
Keywords:
collaborative detection; human object interaction; lightweight network; intelligent transportation; deep learning
分类号:
TP391
DOI:
10.3969/j.issn.1672-1292.2023.01.004
文献标志码:
A
摘要:
分心驾驶是交通事故发生的主要原因之一. 针对目前分心驾驶检测手段单一、检测种类少、检测效率低的问题,提出一种基于轻量化网络与嵌入式的分心行为协同检测系统. 首先,结合Ghost模块和通道注意力机制提出一种轻量化目标检测网络YOLO-Ghost,采用CSPGBottleck构建GhostDarknet作为主干网络,同时构建一种具有多尺度注意力机制的多特征融合模块SE-FPN来进行特征融合,根据固有检测场景进行检测头优化,以CIOU(complete-IOU)作为损失函数. 采用YOLO-Ghost识别和定位局部特征,提出APJ(anchor position judge)对手动分心行为进行判定; 协同检测方面,利用MobileNetv3与YOLO-Ghost协同进行人脸关键点回归和视线估计; 最后利用检测出的多模态信息对驾驶员当前行驶状态进行联合判定. 实验结果表明,YOLO-Ghost的准确率和检测速度优于其他主流方法. 将算法部署到嵌入式设备中,在NVIDIA Jetson TX1上实现了20FPS的实时检测性能,准确性和实时性均达到检测要求.
Abstract:
Distracted driving is the main cause of traffic accident. In order to solve the problems of fewer kinds of distracted driving detection and poor detection efficiency, a collaborative detection system for distraction behavior based on the lightweight network and embedded platform is proposed. First of all, a lightweight object detection network YOLO-Ghost is proposed by combining Ghost module and channel attention mechanism, the CSPGBottleck is proposed to build GhostDarknet as the backbone network, and a multi-feature fusion module SE-FPN with a multi-scale attention mechanism is proposed for feature fusion. A more comprehensive CIOU(complete-IOU)function is considered as the loss function. YOLO-Ghost is used to identify and locate local features, and APJ(anchor position judge)is proposed to judge manual distraction behavior. Secondly, MobileNetv3 and YOLO-Ghost are used to perform face key point regression and gaze estimation. Finally, the detected multimodal information is used to jointly determine the current driving state of the driver. The experimental results show that the YOLO-Ghost achieves the higher accuracy and speed than other main stream methods. At the same time, when the algorithm is deployed to the embedded device, it obtains 20FPS real-time detection performance on the NVIDIA Jetson TX1 and the accuracy and real-time performance reach the detection requirements.

参考文献/References:

[1]王飞,胡荣林,金鹰. 基于3D-CBAM注意力机制的人体动作识别[J]. 南京师范大学学报(工程技术版),2021,21(1):49-56.
[2]DONAHUE J,HENDRICKS L A,GUADARRAMA S,et al. Long-term recurrent convolutional networks for visual recognition and description[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Boston,USA:IEEE,2015.
[3]TRAN D,BOURDEV L,FERGUS R,et al. Learning spatiotemporal features with 3d convolutional networks[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision(CVPR). Santiago,Chile:IEEE,2015.
[4]GUO G D,LAI A. A survey on still image based human action recognition[J]. Pattern Recognition,2014,47(10):3343-3361.
[5]YAN C,COENEN F,ZHANG B L. Driving posture recognition by joint application of motion history image and pyramid histogram of oriented gradients[J]. International Journal of Vehicular Technology,2014,846:719413.
[6]SHARMA G,JURIE F,SCHMID C. Discriminative spatial saliency for image classification[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Providence,USA:IEEE,2012.
[7]KOESDWIADY A,BEDAWI S M,OU C,et al,End-to-end deep learning for driver distraction recognition[C]//Proceedings of the 2017 International Conference on Image Analysis and Recognition. Montreal,Canada:Springer,2017.
[8]HU Y C,LU M Q,LU X B. Driving behaviour recognition from still images by usingmulti-stream fusion CNN[J]. Machine Vision and Applications,2019,30(5):851-865.
[9]OU C J,OUALI C,KARRAY F. Transfer learning based strategy for improving driver distraction recognition[C]//Proceedings of the 2018 International Conference on Image Analysis and Recognition. Pvoa de Varzim,Portugal:Springer,2018.
[10]BAHETI B,GAJRE S,TALBAR S. Detection of distracted driver using convolutional neural network[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW). Salt Lake City,USA:IEEE,2018.
[11]LE T H N,ZHENG Y,ZHU C C,et al. Multiple scale faster-RCNN approach to driver's cell-phone usage and hands on steering wheel detection[C]//Proceedings of the IEEE Conferenceon Computer Vision and Pattern Recognition Workshops(CVPRW). Las Vegas,USA:IEEE,2016.
[12]梁秦嘉,刘怀,陆飞. 基于改进YOLOv3的运动目标分类检测算法研究[J]. 南京师范大学学报(工程技术版),2021,21(4):27-32.
[13]LIN T Y,DOLLÁR P,GIRSHICK R,et al. Feature pyramid networks for object detection[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu,USA:IEEE,2017.
[14]LIU S,QI L,QIN H F,et al. Path aggregation network for instance segmentation[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City,USA:IEEE,2018.
[15]YUN S D,HAN D Y,CHUN S H,et al. CutMix:regularization strategy to train strong classifiers with localizable features[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision(ICCV). Seoul,Korea:IEEE,2019.
[16]REZATOFIGHI H,TSOI N,GWAK J Y,et al. Generalized intersection over union:a metric and a loss for bounding box regression[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Long Beach,USA:IEEE,2019.
[17]HAN K,WANG Y H,TIAN Q,et al. GhostNet:more features from cheap operations[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Seattle,USA:IEEE,2020.
[18]HU J,SHEN L,ALBANIE S,et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,42(8):2011-2023.
[19]殷业瑜,高家全,李莹. 面向印花图案检索的特征融合方法研究[J]. 南京师大学报(自然科学版),2022,45(2):118-125.
[20]REN S Q,HE K M,GIRSHICK R,et al. Faster R-CNN:towards real time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137-1149.
[21]LIU W,ANGUELOV D,ERHAN D,et al. SSD:single shot multibox detector[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam,The Netherlands:Springer,2016.
[22]ZHOU X Y,WANG D Q,KRÄHENBÜHL P. Objects as points[EB/OL]. arXiv Preprint arXiv:1904.07850v2,2019.
[23]LIN T Y,GOYAL P,GIRSHICK R,et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(2):318-327.
[24]JOCHER G. Yolov5[EB/OL]. [2020-08-09]. https://github. com/ultralyc-s/yolov5.

相似文献/References:

[1]樊隽,金烨.网络入侵检测中协同信息的安全传输(英文)[J].南京师范大学学报(工程技术版),2002,02(03):042.
　Fan Jun.The Secure Transfer of the Cooperative Data in Network-Based Intrusion Detection System[J].Journal of Nanjing Normal University(Engineering and Technology),2002,02(01):042.

备注/Memo

备注/Memo:
收稿日期:2022-09-15.
基金项目:国家重点研发计划项目(2018YFB1004904)、国家自然科学基金面上项目(62076107)、江苏省高校自然科学研究重大项目(18KJA520001)、江苏省产学研合作项目(BY2022334)、淮阴工学院研究生科技创新计划项目(HGYK202216).
通讯作者:高尚兵,博士,教授,研究方向:机器学习、计算机视觉、模式识别和数据挖掘. E-mail:luxiaofen_2002@126.com
更新日期/Last Update: 2023-03-15