[1]许博鸣,刘晓峰,业巧林,等.基于卷积神经网络面向自然场景建筑物识别技术的移动端应用[J].南京师范大学学报(工程技术版),2019,19(03):037.[doi:10.3969/j.issn.1672-1292.2019.03.006]
　Xu Boming,Liu Xiaofeng,Ye Qiaolin,et al.A Convolutional Neural Network Based on Mobile Application forIdentification of Buildings in Natural Scene[J].Journal of Nanjing Normal University(Engineering and Technology),2019,19(03):037.[doi:10.3969/j.issn.1672-1292.2019.03.006]
点击复制

基于卷积神经网络面向自然场景建筑物识别技术的移动端应用
分享到：

南京师范大学学报（工程技术版）[ISSN:1006-6977/CN:61-1281/TN]

卷:
19卷
期数:
2019年03期
页码:
037
栏目:
计算机工程
出版日期:
2019-09-30

文章信息/Info

Title:
A Convolutional Neural Network Based on Mobile Application forIdentification of Buildings in Natural Scene
文章编号:
1672-1292(2019)03-0037-08
作者:
许博鸣1; 刘晓峰1; 业巧林1; 张福全1; 周京正2
(1.南京林业大学信息科学技术学院,江苏 南京 210042)(2.中华人民共和国公安部科技信息化局,北京 100741)
Author(s):
Xu Boming1; Liu Xiaofeng1; Ye Qiaolin1; Zhang Fuquan1; Zhou Jingzheng2
1.College of Information Science and Technology,Nanjing Forestry University,Nanjing 210042,China)(2.Bureau of Science and Information Technology,The Ministry of Public Security of the People’s Republic of China,Beijing 100741,China
关键词:
迁移学习; 深度学习; 卷积神经网络; 移动平台移植
Keywords:
transfer learning; deep learning; convolutional neural network; mobile system transplantation
分类号:
TP311
DOI:
10.3969/j.issn.1672-1292.2019.03.006
文献标志码:
A
摘要:
由于自然场景中背景噪声的存在,以及光照、旋转、拍摄角度等复杂因素的干扰,使得自然场景中对建筑物的图像识别难度较大. 针对传统建筑物提取方法对人为设计的依赖,以及对建筑物边缘特征提取算法的改进,基于卷积神经网络(convolutional neural networks,CNN)对自然场景中地标建筑物进行分类的图像识别技术,以及将CNN模型移植到移动端实现复杂场景的快速识别的现实需求,通过Keras框架获取MobileNet瓶颈层后加入新的分类器进行迁移学习,对输入图片进行大量的图像增强技术和测试集增强技术,经过3个阶段的迁移学习,480次迭代后在3个测试集上分别达到98.2%、95.6%、97.2%的准确率. 相比其他的特征提取算法,CNN具有平移不变形以及自动提取等优点,在较短的时间内获得较高准确率的同时,MobileNet的权重仅有15.3 MB,兼顾计算量和精度,可以广泛移植到移动端设备. 基于模型移植的移动端系统兼具拍照识别、相册识别、菜单展示等功能,为移动平台用户提供一个方便简捷的工具来快速准确地判断自然场景中建筑物的信息.
Abstract:
It is very difficult to identify the image of buildings in natural scenes due to the presence of background noise in natural scenes and the interference of complex factors such as illumination,rotation,and shooting angle. Aiming at the dependence of traditional building extraction methods on human design and the improvement of building edge feature extraction algorithm,the paper uses a convolutional neural network based on the image recognition technology to classify landmark buildings in natural scenes and the realistic demand to transplant CNN models to mobile terminals for fast identification of complex scenes,obtains bottleneck layer of MobileNet through Keras,and adds a new classifier for transfer learning. A large number of data augmentation and test set augmentation are applied to the input image. After three versions of transfer learning,the accuracy of 98.2%,95.6% and 97.2% are achieved within 480 iterations in three test set. Compared with other feature extraction algorithms,CNN has the advantages of non-transformation and automatic extraction of features,and achieves a higher accuracy in a shorter period of time. At the same time,MobileNet weight only occupies 15.3 MB with high precision and less calculation,which can be widely transplanted to mobile devices. The system based on model transplantation has the functions of photo recognition,photo album recognition,menu display,etc. Which provides a mobile platform users with a convenient and simple tool to quickly and accurately obtain the information of buildings in natural scenes.

参考文献/References:

[1] 范荣双,陈洋,徐启恒,等. 基于深度学习的高分辨率遥感影像建筑物提取方法[J]. 测绘学报,2019,48(1):34-41.
FAN R S,CHEN Y,XU Q H,et al. A high-resolution remote sensing image building extraction method based on deep learning[J]. Acta geodaetica et cartographica sinica,2019,48(1):34-41.(in Chinese)
[2]李红,刘芳,杨淑媛,等. 基于深度支撑值学习网络的遥感图像融合[J]. 计算机学报,2016,39(8):1583-1596.
LI H,LIU F,YANG S Y,et al. Remote sensing image fusion based on deep support value learning networks[J]. Chinese journal of computers,2016,39(8):1583-1596.(in Chinese)
[3]QIN Q M,CHEN S J,WANG W J,et al. The building recognition of high resolution satellite remote sensing image based on wavelet analysis[C]//2005 International Conference on Machine Learning and Cybernetics. Guangzhou,China:IEEE,2005,7:4533-4538.
[4]ALI G,ABEDELKARIM J. Autonomous building detection using edge properties and image color invariants[J]. Buildings,2018,8(5):65-75.
[5]WANG L,XU Y,LI Y. A Voxel-based 3D building detection algorithm for airborne LIDAR point clouds[J]. Journal of the Indian society of remote sensing,2019,47(2):349-358.
[6]KIM Y,LEE K,CHOI K,et al. Building recognition for augmented reality based navigation system[C]//The Sixth IEEE International Conference on Computer and Information Technology(CIT’06). Bhubaneswar,India:IEEE,2006:131-131.
[7]KRIZHEVSKY A,SUTSKEVER I,HINTON G. ImageNet classification with deep convolutional neural networks[J]. Advances in neural information processing systems,2012,25(2):1-9.
[8]焦李成,杨淑媛,刘芳,等. 神经网络70年:回顾与展望[J]. 计算机学报,2016,39(8):1697-1716.
JIAO L C,YANG S Y,LIU F,et al. Seventy years beyond neural networks:retrospect and prospect[J]. Chinese journal of computers,2016,39(8):1697-1716.(in Chinese)
[9]周飞燕,金林鹏,董军. 卷积神经网络研究综述[J]. 计算机学报,2017,40(6):1229-1251.
ZHOU F Y,JIN L P,DONG J. Review of convolutional neural network[J]. Chinese journal of computers,2017,40(6):1229-1251.(in Chinese)
[10]周健航,杨绪兵,张福全,等. 马氏度量下局部化广义特征值最接近支持向量机[J]. 南京师大学报(自然科学版),2018,41(4):65-71.
ZHOU J H,YANG X B,ZHANG F Q,et.al. Localized GEPSVM based on mahalanobis metric[J]. Journal of Nanjing normal university(natural science edition),2018,41(4):65-71.(in Chinese)
[11]寇振宇,杨绪兵,张福全,等. L1范数最大间隔分类器设计[J]. 南京师大学报(自然科学版),2018,41(4):59-64.
KOU Z Y,YANG X B,ZHANG F Q,et.al. Design of L1 norm maximum margin classifier[J]. Journal of Nanjing normal university(natural science edition),2018,41(4):59-64.(in Chinese)
[12]曲延云,郑南宁,李翠华,等. 基于支持向量机的显著性建筑物检测[J]. 计算机研究与发展,2007,44(1):141-147.
QU Y Y,ZHENG N N,LI C H,et al. Salient building detection based on SVM[J]. Journal of computer research and development,2007,44(1):141-147.(in Chinese)
[13]SHAHZAD M,MAURER M,FRAUNDORFER F,et al. Buildings detection in VHR SAR images using fully convolution neural networks[J]. IEEE transactions on geoscience and remote sensing,2018,57(2):1-17.
[14]ZHAO K,KANG J,JUNG J,et al. Building extraction from satellite images using mask R-CNN with building boundary regularization[C]//CVPR Workshops. Salt Lake City,United States,2018:247-251.
[15]TERMRITTHIKUN C,KANPRACHAR S,MUNEESAWANG P. NU-LiteNet:mobile landmark recognition using convolutional neural networks[EB/OL]. [2019-10-02] http://arxiv.org/abs/1810.01074.
[16]CHOI S,LEE J,LEE K,et al. A 9.02 mW CNN stereo based real time 3D hand gesture recognition processor for smart mobile devices[C]//2018 IEEE International Solid State Circuits Conference-(ISSCC). San Francisco,United States:IEEE,2018:220-222.
[17]HA K D,HYUN L S,CHEOL S B. MUNet:Macro unit based convolutional neural network for mobile devices[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City,United States:2018:1668-1676.
[18]HOWARD A G,ZHU M,CHEN B,et al. Mobilenets:efficient convolutional neural networks for mobile vision applications[EB/OL]. arXiv preprint arXiv:1704.04861,2017.
[19]IOFFE S,SZEGEDY C. Batch normalization:accelerating deep network training by reducing internal covariate shift[EB/OL]. arXiv preprint arXiv:1502.03167,2015.
[20]RUMELHART D E,HINTON G E,WILLIAMS R J. Learning representations by back-propagating errors[J]. Cognitive modeling,1988,5(3):1.
[21]HINTON G E,SRIVASTAVA N,KRIZHEVSKY A,et al. Improving neural networks by preventing co-adaptation of feature detectors[EB/OL]. arXiv preprint arXiv:1207.0580,2012.
[22]ZEILER M D,FERGUS R. Visualizing and understanding convolutional networks[C]//European Conference On Computer Vision. Zurich,Switzerland:Springer,Cham,2014:818-833.
[23]YOSINSKI J,CLUNE J,BENGIO Y,et al. How transferable are features in deep neural networks?[C]//Advances in Neural Information Processing Systems. Vancouver,British Columbia,Canada,2014:3320-3328.
[24]SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. arXiv preprint arXiv:1409.1556,2014.
[25]SZEGEDY C,LIU W,JIA Y,et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston,United States,2015:1-9.

相似文献/References:

[1]程显毅,胡海涛,季国华,等.基于深度学习监控场景下的多尺度目标检测算法研究[J].南京师范大学学报(工程技术版),2018,18(03):033.[doi:10.3969/j.issn.1672-1292.2018.03.005]
　Cheng Xianyi,Hu Haitao,Ji Guohua,et al.Research on Algorithm of Multi-Scale Target DetectionBased on Deep Learning in Monitoring Scenario[J].Journal of Nanjing Normal University(Engineering and Technology),2018,18(03):033.[doi:10.3969/j.issn.1672-1292.2018.03.005]
[2]陈 扬,曾 诚,程 成,等.一种基于CNN的足迹图像检索与匹配方法[J].南京师范大学学报(工程技术版),2018,18(03):039.[doi:10.3969/j.issn.1672-1292.2018.03.006]
　Chen Yang,Zeng Cheng,Cheng Cheng,et al.A CNN-based Approach to Footprint Image Retrieval and Matching[J].Journal of Nanjing Normal University(Engineering and Technology),2018,18(03):039.[doi:10.3969/j.issn.1672-1292.2018.03.006]
[3]王俊淑,张国明,胡 斌.基于深度学习的推荐算法研究综述[J].南京师范大学学报(工程技术版),2018,18(04):033.[doi:10.3969/j.issn.1672-1292.2018.04.006]
　Wang Junshu,Zhang Guoming,Hu Bin.A Survey of Deep Learning Based Recommendation Algorithms[J].Journal of Nanjing Normal University(Engineering and Technology),2018,18(03):033.[doi:10.3969/j.issn.1672-1292.2018.04.006]
[4]郝 坤,张天坤,史振威.基于时空特征的热带气旋强度预测方法[J].南京师范大学学报(工程技术版),2019,19(03):001.[doi:10.3969/j.issn.1672-1292.2019.03.001]
　Hao Kun,Zhang Tiankun,Shi Zhenwei.An Tropical Cyclone Intensity Prediction MethodBased on Spatial-Temporal Features[J].Journal of Nanjing Normal University(Engineering and Technology),2019,19(03):001.[doi:10.3969/j.issn.1672-1292.2019.03.001]
[5]任媛媛,张显峰,马永建,等.基于卷积神经网络的无人机遥感影像农村建筑物目标检测[J].南京师范大学学报(工程技术版),2019,19(03):029.[doi:10.3969/j.issn.1672-1292.2019.03.005]
　Ren Yuanyuan,Zhang Xianfeng,Ma Yongjian,et al.Target Detection of Rural Buildings in UAV Remote Sensing ImagesBased on Convolutional Neural Network[J].Journal of Nanjing Normal University(Engineering and Technology),2019,19(03):029.[doi:10.3969/j.issn.1672-1292.2019.03.005]
[6]吴燕如,珠 杰,管美静.基于深度学习的藏文现代印刷物版面检测技术研究[J].南京师范大学学报(工程技术版),2021,21(01):044.[doi:10.3969/j.issn.1672-1292.2021.01.007]
　Wu Yanru,Zhu Jie,Guan Meijing.Research on Layout Inspection Technology of ModernTibetan Prints Based on Deep Learning[J].Journal of Nanjing Normal University(Engineering and Technology),2021,21(03):044.[doi:10.3969/j.issn.1672-1292.2021.01.007]
[7]梁秦嘉,刘 怀,陆 飞.基于改进YOLOv3模型的交通视频目标检测算法研究[J].南京师范大学学报(工程技术版),2021,21(02):047.[doi:10.3969/j.issn.1672-1292.2021.02.008]
　Liang Qinjia,Liu Huai,Lu Fei.Traffic Video Target Detection Algorithm Based on Improved YOLOv3[J].Journal of Nanjing Normal University(Engineering and Technology),2021,21(03):047.[doi:10.3969/j.issn.1672-1292.2021.02.008]
[8]苏 叶,李 婧,徐寅林.手骨X光片骨龄预测中图像预处理的研究[J].南京师范大学学报(工程技术版),2021,21(02):054.[doi:10.3969/j.issn.1672-1292.2021.02.009]
　Su Ye,Li Jing,Xu Yinlin.Research on Image Preprocessing in Predicting the Bone Age ofHand Bone X-ray Films[J].Journal of Nanjing Normal University(Engineering and Technology),2021,21(03):054.[doi:10.3969/j.issn.1672-1292.2021.02.009]
[9]王立凯,曲维光,魏庭新,等.基于深度学习的中文零代词识别[J].南京师范大学学报(工程技术版),2021,21(04):019.[doi:10.3969/j.issn.1672-1292.2021.04.004]
　Wang Likai,Qu Weiguang,Wei Tingxin,et al.Identification of Chinese Zero Pronouns Based on Deep Learning[J].Journal of Nanjing Normal University(Engineering and Technology),2021,21(03):019.[doi:10.3969/j.issn.1672-1292.2021.04.004]
[10]李庆涛,林培光,王基厚,等.基于板块效应的深度学习股价走势预测方法[J].南京师范大学学报(工程技术版),2022,22(01):030.[doi:10.3969/j.issn.1672-1292.2022.01.005]
　Li Qingtao,Lin Peiguang,Wang Jihou,et al.Deep Learning Stock Price Forecasting Method Based on Plate Effect[J].Journal of Nanjing Normal University(Engineering and Technology),2022,22(03):030.[doi:10.3969/j.issn.1672-1292.2022.01.005]

备注/Memo

备注/Memo:
收稿日期:2019-07-05.
基金项目:国家自然科学基金(61871444、31670554)、南京林业大学大学生创新训练计划项目(2017NFUSPITP231).
通讯联系人:刘晓峰,博士,讲师,研究方向:网络信息抽取与信息检索、社交网络数据挖掘. E-mail:liuxiaofeng@njfu.edu.cn
更新日期/Last Update: 2019-09-30