[1]於佳乐,黄 坤,张 潇,等.基于图像生成的多模态视网膜图像配准方法[J].南京师范大学学报(工程技术版),2023,23(01):010-17.[doi:10.3969/j.issn.1672-1292.2023.01.002]
 Yu Jiale,Huang Kun,Zhang Xiao,et al.Multi-Modal Retinal Image Registration Method Based on Image Generation[J].Journal of Nanjing Normal University(Engineering and Technology),2023,23(01):010-17.[doi:10.3969/j.issn.1672-1292.2023.01.002]
点击复制

基于图像生成的多模态视网膜图像配准方法
分享到:

南京师范大学学报(工程技术版)[ISSN:1006-6977/CN:61-1281/TN]

卷:
23卷
期数:
2023年01期
页码:
010-17
栏目:
计算机科学与技术
出版日期:
2023-03-15

文章信息/Info

Title:
Multi-Modal Retinal Image Registration Method Based on Image Generation
文章编号:
1672-1292(2023)01-0010-08
作者:
於佳乐黄 坤张 潇陈 强
(南京理工大学计算机科学与工程学院,江苏 南京 210094)
Author(s):
Yu JialeHuang KunZhang XiaoChen Qiang
(School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China)
关键词:
图像配准多模态视网膜图像图像生成
Keywords:
image registrationmulti-modalretinal imagesimage generation
分类号:
TP391
DOI:
10.3969/j.issn.1672-1292.2023.01.002
文献标志码:
A
摘要:
针对多模态视网膜的全局粗配准,提出了一种基于图像生成的多模态视网膜配准方法. 不同于当前主流的提取多模态视网膜图像血管再进行配准的方法,使用GAN模型对不同模态的视网膜图像进行像素级映射,再通过特征点匹配的方式计算图像仿射矩阵,实现图像粗配准. 基于彩色眼底图像与荧光素血管造影图像的实验结果表明,该方法与当前主流的视网膜粗配准方法相比,具有快速且鲁棒的优势.
Abstract:
A multi-modal retinal registration method based on image generation is proposed for global coarse registration of multi-modal retinal images. Unlike the current mainstream methods that extract retinal vascular structures for registration, this method uses GAN model to perform pixel-level mapping of different modal retinal images. Then, the affine matrix is calculated through feature point matching to achieve image rough registration. Experimental results based on color fundus images and fluorescein angiography images demonstrate that this method has the advantages of faster speed and robust performance compared with current mainstream retinal registration methods.

参考文献/References:

[1]MACGILLIVRAY T J,TRUCCO E,CAMERON J R,et al. Retinal imaging as a source of biomarkers for diagnosis,characterization and prognosis of chronic illness or long-term conditions[J]. The British Journal of Radiology,2014,87(1040):20130832.
[2]ZHANG L,FAN H,ZHANG J,et al. Multi-modal and multi-vendor retina image registration[J]. Biomedical Optics Express,2018,9(2):410-422.
[3]WANG Y Q,ZHANG J K,AN C H,et al. A segmentation based robust deep learning framework for multimodal retinal image registration[C]//Proceedings of the 2020 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP). Barcelona,Spain:IEEE,2020.
[4]MA J Y,JIANG X Y,FAN A X,et al. Image matching from handcrafted to deep features:a survey[J]. International Journal of Computer Vision,2021,129(1):23-79.
[5]汪前进,朱斌,李存华. 基于特征点的图像拼接方法的研究与应用[J]. 南京师范大学学报(工程技术版),2016,16(3):48-53.
[6]李鹏程,曾毓敏,张梦. 基于改进Harris的图像拼接算法[J]. 南京师范大学学报(工程技术版),2014,14(1):70-75.
[7]CHEN J,TIAN J,LEE N,et al. A partial intensity invariant feature descriptor for multimodal retinal image registration[J]. IEEE Transactions on Biomedical Engineering,2010,57(7):1707-1718.
[8]HARRIS C,STEPHENS M. A combined corner and edge detector[C]//TAYLOR C J. Proceedings of the Alvey Vision Conference. Manchester,UK:Alvey Vision Club,1988.
[9]BAY H,TUYTELAARS T,GOOL L V. Surf:speeded up robust features[C]//Proceedings of the 9th European Conference on Computer Vision. Graz,Austria:Springer,2006.
[10]WANG G,WANG Z C,CHEN Y F,et al. Robust point matching method for multimodal retinal image registration[J]. Biomedical Signal Processing and Control,2015,19:68-76.
[11]ZHANG H T,LIU X H,WANG G,et al. An automated point set registration framework for multimodal retinal image[C]//Proceedings of the 24th International Conference on Pattern Recognition(ICPR). Beijing,China:IEEE,2018.
[12]WANG Y Q,ZHANG J K,CAVICHINI M,et al. Robust content-adaptive global registration for multimodal retinal images using weakly supervised deep-learning framework[J]. IEEE Transactions on Image Processing,2021,30:3167-3178.
[13]SHE Y Y,ZHOU M,LI Q L,et al. Retinal image registration based on features of vessel-segmented image[C]//Proceedings of the 14th International Congress on Image and Signal Processing,BioMedical Engineering and Informatics(CISP-BMEI). Shanghai,China:IEEE,2021.
[14]AN C H,WANG Y Q,ZHANG J K,et al. Self-supervised rigid registration for multimodal retinal images[J]. IEEE Transactions on Image Processing,2022,31:5733-5747.
[15]SARLIN P E,DETONE D,MALISIEWICZ T,et al. Superglue:Learning feature matching with graph neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle,USA:IEEE,2020.
[16]SCHIFFERS F,YU Z K,ARGUIN S,et al. Synthetic fundus fluorescein angiography using deep neural networks[C]//Proceedings of the Bildverarbeitung für die Medizin 2018. Erlangen,Germany:Springer,2018.
[17]LI K,YU L Q,WANG S J,et al. Unsupervised retina image synthesis via disentangled representation learning[C]//Proceedings of the 4th International Workshop on Simulation and Synthesis in Medical Imaging. Shenzhen,China:Springer,2019.
[18]ISOLA P,ZHU J Y,ZHOU T,et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu,USA:IEEE,2017.
[19]WANG T C,LIU M Y,ZHU J Y,et al. High-resolution image synthesis and semantic manipulation with conditional gans[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City,USA:IEEE,2018.
[20]LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra,Greece:IEEE,1999.
[21]BAY H,TUYTELAARS T,GOOL L V. Surf:speeded up robust features[C]//Proceedings of the 9th European Conference on Computer Vision. Graz,Austria:Springer,2006.
[22]CHOY C B,GWAK J Y,SAVARESE S,et al. Universal correspondence network[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona,Spain:NIPS,2016.
[23]YI K M,TRULLS E,LEPETIT V,et al. Lift:Learned invariant feature transform[J]. arXiv Preprint arXiv:1603.09114,2016.
[24]吴玲玉,蓝洋,夏海英. 基于卷积神经网络的眼底图像配准研究[J]. 广西师范大学学报(自然科学版),2021,39(5):122-133.
[25]ZHANG J K,WANG Y Q,DAI J,et al. Two-step registration on multi-modal retinal images via deep neural networks[J]. IEEE Transactions on Image Processing,2021,31:823-838.
[26]FISCHLER M A,BOLLES R C. Random sample consensus:a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM,1981,24(6):381-395.
[27]DETONE D,MALISIEWICZ T,RABINOVICH A. Superpoint:self-supervised interest point detection and description[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City,USA:IEEE,2018.
[28]RONNEBERGER O,FISCHER P,BROX T. U-net:convolutional networks for biomedical image segmentation[J]. arXiv Preprint arXiv:1505.04597,2015.
[29]SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv Preprint arXiv:1409.1556,2014.
[30]ALIPOUR S H M,RABBANI H,AKHLAGHI M R. Diabetic retinopathy grading by digital curvelet transform[J]. Computational and Mathematical Methods in Medicine,2012(4):761901.
[31]DAN T T,HU Y,HAN C,et al. Fusion of multi-source retinal fundus images via automatic registration for clinical diagnosis[J]. Neurocomputing,2021,459:370-382.

相似文献/References:

[1]叶云龙,杨明.一种基于多模态模型的随机子空间分类集成算法[J].南京师范大学学报(工程技术版),2009,09(04):057.
 Ye Yunlong,Yang Ming.A Multi-modality-based Random Subspace Classifier Ensemble Algorithm[J].Journal of Nanjing Normal University(Engineering and Technology),2009,09(01):057.
[2]韩天翊,林荣恒.一种基于决策层融合的多模态情感识别方法[J].南京师范大学学报(工程技术版),2022,22(02):035.[doi:10.3969/j.issn.1672-1292.2022.02.006]
 Han Tianyi,Lin Rongheng.A Multimodal Emotion Recognition Method Based on Decision Level Fusion[J].Journal of Nanjing Normal University(Engineering and Technology),2022,22(01):035.[doi:10.3969/j.issn.1672-1292.2022.02.006]

备注/Memo

备注/Memo:
收稿日期:2022-09-15.
基金项目:国家自然科学基金项目(62172223和61671242)、中央高校基本科研业务费专项资金项目(30921013105).
通讯作者:陈强,教授,博士生导师,研究方向:图像处理和分析. E-mail:chen2qiang@njust.edu.cn
更新日期/Last Update: 2023-03-15