[1]凌 旭,谢 非,杨继全,等.基于通道加权FPN的熔池检测与特征提取方法[J].南京师范大学学报(工程技术版),2023,23(02):016-24.[doi:10.3969/j.issn.1672-1292.2023.02.003]
　Ling Xu,Xie Fei,Yang Jiquan,et al.High Speed Molten Pool Image Detection Based on Weighted Fusion and Parameter Extraction[J].Journal of Nanjing Normal University(Engineering and Technology),2023,23(02):016-24.[doi:10.3969/j.issn.1672-1292.2023.02.003]
点击复制

基于通道加权FPN的熔池检测与特征提取方法
分享到：

南京师范大学学报（工程技术版）[ISSN:1006-6977/CN:61-1281/TN]

卷:
23卷
期数:
2023年02期
页码:
016-24
栏目:
计算机科学与技术
出版日期:
2023-06-15

文章信息/Info

Title:
High Speed Molten Pool Image Detection Based on Weighted Fusion and Parameter Extraction
文章编号:
1672-1292(2023)02-0016-09
作者:
凌 旭1; 2; 谢 非1; 2; 3; 杨继全1; 2; 杜 军4; 苗立国3; 5; 锁红波3
(1.南京师范大学电气与自动化工程学院,江苏 南京 210023)
(2.南京师范大学江苏省三维打印装备与制造重点实验室,江苏 南京 210023)
(3.南京中科煜宸激光技术有限公司,江苏 南京 210023)
(4.西安交通大学机械制造系统工程国家重点实验室,陕西 西安 710049)
(5.沈阳工业大学机械工程学院,辽宁 沈阳 110870)
Author(s):
Ling Xu1; 2; Xie Fei1; 2; 3; Yang Jiquan1; 2; Du Jun4; Miao Liguo3; 5; Suo Hongbo3
(1.School of Electrical and Automation Engineering,Nanjing Normal University,Nanjing 210023,China)
(2.Jiangsu Key Laboratory of 3D Printing Equipment and Manufacturing,Nanjing Normal University,Nanjing 210023,China)
(3.Nanjing Zhongke Raycham Laser Technology Co.,Ltd.,Nanjing 210023,China)
(4.The State Key Laboratory for Manufacturing Systems Engineering,Xi'an Jiaotong University,Xi'an 710049,China)
(5.School of Machanical Engineering,Shenyang University of Technology,Shenyang 110870,China)
关键词:
熔池检测; 增材制造; 语义分割; 特征融合; 神经网络
Keywords:
molten pool detection; additive manufacturing; semantic segmentation; feature fusion; neural network
分类号:
TG441.7
DOI:
10.3969/j.issn.1672-1292.2023.02.003
文献标志码:
A
摘要:
为了获取使用熔池特征参数对增材制造进行反馈控制的输入参数,提出了一种基于通道加权FPN的激光增材制造熔池语义分割算法和基于图像像素阈值的熔池方向、面积和宽度特征参数提取算法. 语义分割算法主要包含轻量级的主干神经网络、通道加权特征FPN网络. 实验结果表明,熔池图像的分割速度可达79.76张/s,mIoU和mAP分别可达90.53%和95.79%,且模型大小仅为90MB. 与其他相同类型的深度学习模型相比,该算法在保证精度的同时,提高了检测速度,减少了模型参数量和大小. 熔池图像特征参数提取算法则结合了相机拍摄的原始图像与分割完成的图像的像素阈值分布情况,能够准确分析并计算出熔池的方向、宽度与面积特征参数.
Abstract:
In order to obtain the input parameters for feedback control of additive manufacturing using molten pool feature parameters,we propose a laser additive manufacturing molten pool semantic segmentation algorithm based on channel-weighted FPN,and a molten pool orientation,area and width feature parameter extraction algorithm based on image pixel threshold. The semantic segmentation algorithm mainly consists of a lightweight backbone neural network and a channel-weighted feature FPN network. The experimental results show that the segmentation speed of the molten pool image can reach 79.76 frames/s,and the mIoU and mAP can reach 90.53% and 95.79%,respectively,and the model size is only 90MB. Compared with other similar types of deep learning models,this algorithm improves the detection speed and reduces the model parameter size and size while ensuring accuracy. The molten pool image feature parameter extraction algorithm combines the pixel threshold distribution of the original image captured by the camera and the segmented image,and can accurately analyze and calculate the orientation,width and area feature parameters of the molten pool.

参考文献/References:

[1]吴陈铭,戴澄恺,王昌凌,等. 多自由度3D打印技术研究进展综述[J]. 计算机学报,2019,42(9):1918-1938.
[2]叶福兴,王永辉,娄智. 激光增材制造过程中激光与粉末的相互作用研究现状[J]. 中国表面工程,2021,34(2):1-12.
[3]WU D,CHEN H B,HUANG Y M,et al. Monitoring of weld joint penetration during variable polarity plasma arc welding based on the keyhole characteristics and PSO-ANFIS[J]. Journal of Materials Processing Technology,2017,239:113-124.
[4]CHEN Y H,CHEN B,YAO Y Z,et al. A spectroscopic method based on support vector machine and artificial neural network for fiber laser welding defects detection and classification[J]. NDT & E International,2019,108:102176.
[5]VANDONE A,BARALDO S,VALENTE A,et al. Vision-based melt pool monitoring system setup for additive manufacturing[J]. Procedia CIRP,2019,81:747-752.
[6]ZHANG Z F,WEN G R,CHEN S B. Audible sound-based intelligent evaluation for aluminum alloy in robotic pulsed GTAW:Mechanism,feature selection,and defect detection[J]. IEEE Transactions on Industrial Informatics,2017,14(7):2973-2983.
[7]李子晗,忻建文,肖笑,等. 热导型等离子弧焊电弧物理特性和熔池动态行为[J]. 金属学报,2021,57(5):693-702.
[8]MAZZOLENI L,DEMIR A G,CAPRIO L,et al. Real-time observation of melt pool in selective laser melting:Spatial,temporal,and wavelength resolution criteria[J]. IEEE Transactions on Instrumentation and Measurement,2020,69(4):1179-1190.
[9]刘晓刚,闫红方,张荣. 基于形态学多尺度多结构的熔池图像边缘检测[J]. 热加工工艺,2019,48(5):216-219.
[10]杨启,田虎成,闫昭华,等. 激光近净成形中熔池宽度实时监控系统的研究[J]. 激光与红外,2019,49(9):1060-1067.
[11]LI X B,LI T,SHI B W,et al. The influence of substrate tilt angle on the morphology of laser cladding layer[J]. Surface and Coatings Technology,2020,391:125706.
[12]顾振杰,雷剑波,张传鹏,等. 镍硅硼合金粉末激光熔覆中熔池光谱检测分析[J]. 中国激光,2014,41(11):95-102.
[13]雷凯云,秦训鹏,刘华明,等. 基于神经网络的宽带激光熔覆熔池特征参数预测[J]. 光电子·激光,2018,29(11):1212-1220.
[14]YANG Z,LU Y,YEUNG H,et al. Investigation of deep learning for real-time melt pool classification in additive manufacturing[C]//Proceedings of the 2019 IEEE 15th International Conference on Automation Science and Engineering(CASE). Vancouver,Canada:IEEE,2019.
[15]LU J,HE H Y,SHI Y M,et al. Quantitative prediction for weld reinforcement in arc welding additive manufacturing based on molten pool image and deep residual network[J]. Additive Manufacturing,2021,41:101980.
[16]王仁杰,史圣泰. 基于机器视觉的增材制造激光熔覆熔池边缘检测和行为参数分析[J]. 中国金属通报,2021(13):209-210.
[17]CAI W,JIANG P,SHU L S,et al. Real-time identification of molten pool and keyhole using a deep learning-based semantic segmentation approach in penetration status monitoring[J]. Journal of Manufacturing Processes,2022,76:695-707.
[18]JIAO W H,WANG Q Y,CHENG Y C,et al. End-to-end prediction of weld penetration:A deep learning and transfer learning based method[J]. Journal of Manufacturing Processes,2021,63(2):191-197.
[19]ZHANG Z H,LI B,ZHANG W F,et al. Real-time penetration state monitoring using convolutional neural network for laser welding of tailor rolled blanks[J]. Journal of Manufacturing Systems,2020,54:348-360.
[20]ZHANG Y X,YOU D Y,GAO X D. Welding defects detection based on deep learning with multiple optical sensors during disk laser welding of thick plates[J]. Journal of Manufacturing Systems,2019,51:87-94.
[21]XU Y L,FANG G,LV N. Computer vision technology for seam tracking in robotic GTAW and GMAW[J]. Robotics and Computer-Integrated Manufacturing,2015,32:25-36.
[22]LIN T Y,DOLLAR P,GIRSHICK R,et al. Feature pyramid networks for object detection[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu,USA:IEEE,2017.
[23]HOWARD A,SANDLER M,CHEN B,et al. Searching for MobileNetV3[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision(ICCV). Seoul,Korea:IEEE,2020.
[24]ZHANG M X,WANG Z J,SUN T Z,et al. Salient object detection by pyramid networks with gating[C]//Proceedings of the 2019 IEEE International Conference on Robotics and Biomimetics(ROBIO). Dali,China:IEEE,2019.
[25]HE K M,ZHANG X Y,REN S Q,et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Las Vegas,USA:IEEE,2016.
[26]SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL]. arXiv Preprint arXiv:1409.1556,2014.
[27]BOCHKOVSKIY A,WANG C Y,LIAO H Y M. Yolov4:Optimal speed and accuracy of object detection[J/OL]. arXiv Preprint arXiv:2004.10934,2020.
[28]CHEN L C,ZHU Y K,PAPANDREOU G,et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the 15th European Conference on Computer Vision(ECCV). Munich,Germany:Springer,2018.
[29]ZHAO H S,SHI J P,QI X J,et al. Pyramid Scene Parsing Network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu,USA:IEEE,2017.
[30]RONNEBERGER O,FISCHER P,BROX T. U-Net:Convolutional Networks for Biomedical Image Segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention(MICCAI). Munich,Germany:MICCAI,2015.
[31]张亚红,覃科. 一种形态学与Canny算子融合的焊接熔池边缘检测算法[J]. 桂林航天工业学院学报,2017,22(1):9-13.

备注/Memo

备注/Memo:
收稿日期:2022-07-05.
基金项目:国家重点研发计划项目(2017YFB1103200)、 江苏省科技成果转化项目(BA2020004)、 2020年江苏省省级工业和信息产业转型升级专项资金项目(JITC-2000AX0676-71)、 江苏省研究生科研与实践创新计划项目(SJCX21_0582).
通讯作者:谢非,博士,副教授,研究方向:金属增材制造、机器视觉与深度学习、数据融合与处理、嵌入式系统开发等. E-mail:xiefei@njnu.edu.cn
更新日期/Last Update: 2023-06-15