|Table of Contents|

Multi-Modal Retinal Image Registration Method Based on Image Generation(PDF)

南京师范大学学报(工程技术版)[ISSN:1006-6977/CN:61-1281/TN]

Issue:
2023年01期
Page:
10-17
Research Field:
计算机科学与技术
Publishing date:

Info

Title:
Multi-Modal Retinal Image Registration Method Based on Image Generation
Author(s):
Yu JialeHuang KunZhang XiaoChen Qiang
(School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China)
Keywords:
image registrationmulti-modalretinal imagesimage generation
PACS:
TP391
DOI:
10.3969/j.issn.1672-1292.2023.01.002
Abstract:
A multi-modal retinal registration method based on image generation is proposed for global coarse registration of multi-modal retinal images. Unlike the current mainstream methods that extract retinal vascular structures for registration, this method uses GAN model to perform pixel-level mapping of different modal retinal images. Then, the affine matrix is calculated through feature point matching to achieve image rough registration. Experimental results based on color fundus images and fluorescein angiography images demonstrate that this method has the advantages of faster speed and robust performance compared with current mainstream retinal registration methods.

References:

[1]MACGILLIVRAY T J,TRUCCO E,CAMERON J R,et al. Retinal imaging as a source of biomarkers for diagnosis,characterization and prognosis of chronic illness or long-term conditions[J]. The British Journal of Radiology,2014,87(1040):20130832.
[2]ZHANG L,FAN H,ZHANG J,et al. Multi-modal and multi-vendor retina image registration[J]. Biomedical Optics Express,2018,9(2):410-422.
[3]WANG Y Q,ZHANG J K,AN C H,et al. A segmentation based robust deep learning framework for multimodal retinal image registration[C]//Proceedings of the 2020 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP). Barcelona,Spain:IEEE,2020.
[4]MA J Y,JIANG X Y,FAN A X,et al. Image matching from handcrafted to deep features:a survey[J]. International Journal of Computer Vision,2021,129(1):23-79.
[5]汪前进,朱斌,李存华. 基于特征点的图像拼接方法的研究与应用[J]. 南京师范大学学报(工程技术版),2016,16(3):48-53.
[6]李鹏程,曾毓敏,张梦. 基于改进Harris的图像拼接算法[J]. 南京师范大学学报(工程技术版),2014,14(1):70-75.
[7]CHEN J,TIAN J,LEE N,et al. A partial intensity invariant feature descriptor for multimodal retinal image registration[J]. IEEE Transactions on Biomedical Engineering,2010,57(7):1707-1718.
[8]HARRIS C,STEPHENS M. A combined corner and edge detector[C]//TAYLOR C J. Proceedings of the Alvey Vision Conference. Manchester,UK:Alvey Vision Club,1988.
[9]BAY H,TUYTELAARS T,GOOL L V. Surf:speeded up robust features[C]//Proceedings of the 9th European Conference on Computer Vision. Graz,Austria:Springer,2006.
[10]WANG G,WANG Z C,CHEN Y F,et al. Robust point matching method for multimodal retinal image registration[J]. Biomedical Signal Processing and Control,2015,19:68-76.
[11]ZHANG H T,LIU X H,WANG G,et al. An automated point set registration framework for multimodal retinal image[C]//Proceedings of the 24th International Conference on Pattern Recognition(ICPR). Beijing,China:IEEE,2018.
[12]WANG Y Q,ZHANG J K,CAVICHINI M,et al. Robust content-adaptive global registration for multimodal retinal images using weakly supervised deep-learning framework[J]. IEEE Transactions on Image Processing,2021,30:3167-3178.
[13]SHE Y Y,ZHOU M,LI Q L,et al. Retinal image registration based on features of vessel-segmented image[C]//Proceedings of the 14th International Congress on Image and Signal Processing,BioMedical Engineering and Informatics(CISP-BMEI). Shanghai,China:IEEE,2021.
[14]AN C H,WANG Y Q,ZHANG J K,et al. Self-supervised rigid registration for multimodal retinal images[J]. IEEE Transactions on Image Processing,2022,31:5733-5747.
[15]SARLIN P E,DETONE D,MALISIEWICZ T,et al. Superglue:Learning feature matching with graph neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle,USA:IEEE,2020.
[16]SCHIFFERS F,YU Z K,ARGUIN S,et al. Synthetic fundus fluorescein angiography using deep neural networks[C]//Proceedings of the Bildverarbeitung für die Medizin 2018. Erlangen,Germany:Springer,2018.
[17]LI K,YU L Q,WANG S J,et al. Unsupervised retina image synthesis via disentangled representation learning[C]//Proceedings of the 4th International Workshop on Simulation and Synthesis in Medical Imaging. Shenzhen,China:Springer,2019.
[18]ISOLA P,ZHU J Y,ZHOU T,et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu,USA:IEEE,2017.
[19]WANG T C,LIU M Y,ZHU J Y,et al. High-resolution image synthesis and semantic manipulation with conditional gans[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City,USA:IEEE,2018.
[20]LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra,Greece:IEEE,1999.
[21]BAY H,TUYTELAARS T,GOOL L V. Surf:speeded up robust features[C]//Proceedings of the 9th European Conference on Computer Vision. Graz,Austria:Springer,2006.
[22]CHOY C B,GWAK J Y,SAVARESE S,et al. Universal correspondence network[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona,Spain:NIPS,2016.
[23]YI K M,TRULLS E,LEPETIT V,et al. Lift:Learned invariant feature transform[J]. arXiv Preprint arXiv:1603.09114,2016.
[24]吴玲玉,蓝洋,夏海英. 基于卷积神经网络的眼底图像配准研究[J]. 广西师范大学学报(自然科学版),2021,39(5):122-133.
[25]ZHANG J K,WANG Y Q,DAI J,et al. Two-step registration on multi-modal retinal images via deep neural networks[J]. IEEE Transactions on Image Processing,2021,31:823-838.
[26]FISCHLER M A,BOLLES R C. Random sample consensus:a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM,1981,24(6):381-395.
[27]DETONE D,MALISIEWICZ T,RABINOVICH A. Superpoint:self-supervised interest point detection and description[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City,USA:IEEE,2018.
[28]RONNEBERGER O,FISCHER P,BROX T. U-net:convolutional networks for biomedical image segmentation[J]. arXiv Preprint arXiv:1505.04597,2015.
[29]SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv Preprint arXiv:1409.1556,2014.
[30]ALIPOUR S H M,RABBANI H,AKHLAGHI M R. Diabetic retinopathy grading by digital curvelet transform[J]. Computational and Mathematical Methods in Medicine,2012(4):761901.
[31]DAN T T,HU Y,HAN C,et al. Fusion of multi-source retinal fundus images via automatic registration for clinical diagnosis[J]. Neurocomputing,2021,459:370-382.

Memo

Memo:
-
Last Update: 2023-03-15