多变的光照条件及天气状况将会严重影响水下光学图像的成像质量，为提升水下目标检测的稳定性及检测精度，基于深度神经网络模型，对结合光学图像和声呐图形的多模态方法进行研究。首先，针对实时神经网络检测器架构YOLOv7，通过改进该检测器，使其适用于多模态输入。其次，为了有效地结合来自不同模态的影响特征，提出全新的融合模型YOLOv7-Fusion，并通过引入CE-Fusion模块，实现融合效率和准确度的提升。最后，为了解决数据集缺少的问题，利用快速风格和图像处理算法转化的方法，生成人工数据集。所设计的算法及模型目标识别准确率为0.995，具有较高检测精度；Fps为43.4，具有较高处理效率。该模型可支持真实应用，适用于不同类型的水下场景。
Lighting and weather conditions seriously affect the quality of underwater optical images. To improve the stability and detection accuracy of underwater target detection, a multi-modal method combining optical images and sonar graphics is studied based on the deep neural network model. Firstly, the architecture of real-time neural network detector YOLOv7 is studied, and the detector is improved to be suitable for multi-mode input. Secondly, in order to effectively combine the influence characteristics from different modes, YOLOv7-Fusion was proposed and CE-Fusion module was introduced to improve fusion efficiency and accuracy. Finally, in order to solve the problem of the lack of data set, fast style and image processing algorithm transformation is used to generate artificial data set. The target recognition accuracy of the designed algorithm and model is 0.995 and Fps is 43.4, with high detection accuracy and processing efficiency. Therefore, the model can support real applications and is suitable for different underwater scenes.
2023,45(12): 122-127 收稿日期：2023-01-16
DOI：10.3404/j.issn.1672-7619.2023.12.023
分类号：TB566
基金项目：国家自然科学基金资助项目（62006102）；镇江市重点研发计划（社会发展）项目（SH2022013）
作者简介：葛慧林(1989-),男,硕士,副研究员,研究方向为深度学习、水下信息感知
参考文献：
[1] LIRMAN D, GRACIAS N R, GINTERT B E, et al. Development and application of a video-mosaic survey technology to document the status of coral reef communities[J]. Environmental Monitoring and Assessment, 2007, 125(1): 59–73
[2] JOHANNSSON H, KAESS M, ENGLOT B J, et al. Imaging sonar-aided navigation for autonomous underwater harbor surveillance[C] //IEEE/RSJ International Conference on Intelligent Robots & Systems. IEEE, 2010: 4396-4403.
[3] FERREIRA F, VERUGGIO G, CACCIA M, et al. A survey on real-time motion estimation techniques for underwater robots[J]. Journal of Real-Time Image Processing, 2016, 11(4): 693–711
[4] FOLEY B P, DELLAPORTA K, SAKELLARIOU D, et al. The 2005 Chios ancient shipwreck survey: new methods for underwater archaeology[J]. Hesperia, 2009: 269-305.
[5] 巩文静, 田杰, 李宝奇, 等. 基于改进MobilenetV2网络的声光图像融合水下目标分类方法[J]. 应用声学, 2022, 41(3): 462–470
GONG W J, TIAN J, LI B Q, et al. Acoustic-optical image fusion underwater target classification method based on improved MobilenetV2[J]. Journal of Applied Acoustics, 2022, 41(3): 462–470
[6] MORONI D, PASCALI M A, REGGIANNINI M, et al. Underwater scene understanding by optical and acoustic data integration[J]. Acoustical Society of America, 2013, 17: 070085
[7] MAJUMDER S, SCHEDING S, DURRANT-WHYTE H F. Multisensor data fusion for underwater navigation[J]. Robotics and Autonomous Systems, 2001, 35(2): 97–108
[8] NEGAHDARIPOUR S, TAATIAN A. 3-D motion and structure estimation for arbitrary scenes from 2-D optical and sonar video[C]// IEEE OCEANS. IEEE, 2008: 1–8.
[9] BABAEE M, NEGAHDARIPOUR S. 3-D object modeling from 2-D occluding contour correspondences by opti-acoustic stereo imaging[J]. Computer Vision & Image Understanding, 2015, 132: 56–74
[10] GE Q, RUAN F, QIAO B, et al. Side-scan sonar image classification based on style transfer and pre-trained convolutional neural networks[J]. Electronics, 2021, 10(15): 1823
[11] XI J, YE X, LI C. Sonar image target detection based on style transfer learning and random shape of noise under zero shot target[J]. Remote Sensing, 2022, 14(24): 6260
[12] LEE S, PARK B, KIM A. Deep learning based object detection via style-transferred underwater sonar images[J]. IFAC-PapersOnLine, 2019, 52(21): 152–155
[13] LI C, YE X, CAO D, et al. Zero shot objects classification method of side scan sonar image based on synthesis of pseudo samples[J]. Applied Acoustics, 2021, 173: 107691
[14] YU Y, ZHAO J, GONG Q, et al. Real-time underwater maritime object detection in side-scan sonar images based on transformer-YOLOv5[J]. Remote Sensing, 2021, 13(18): 3555
[15] HUANG C, ZHAO J, YU Y, et al. Comprehensive sample augmentation by fully considering SSS imaging mechanism and environment for shipwreck detection under zero real samples[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1–14
[16] SONG Y, LIU P. Segmentation of sonar images with intensity inhomogeneity based on improved MRF[J]. Applied Acoustics, 2020, 158: 107051
