Write your message
                  Back to the articles list | Back to browse issues page

XML Persian Abstract Print

Implementation of a combined acoustic-optical obstacle avoidance system in autonomous underwater vehicle using wall following and stall following methods
Qolamreza Nadaliniachary *1 , Alireza Ahmadyfard1
1- Shahrood university of technology
Abstract:   (37 Views)
Obstacle detection and avoidance in marine environments remains a major challenge for autonomous underwater vehicles (AUVs) due to adverse lighting conditions, high noise of acoustic data, and high cost of long-range sonars. In this paper, we present an integrated acousto-optic framework for obstacle detection and safe path planning, which is developed based on the fusion of camera data and two consecutive sonar scans, using deep learning networks and the artificial potential field (APF) algorithm. In the surface motion phase, real camera images are combined with sonar data, while in the subsurface motion phase, estimated virtual images of obstacles are combined with sonar data to maintain detection stability. For obstacle detection, a U-NET network with a MobileNetV2 encoder and an LSTM network are used to predict the positions and motion of moving obstacles. In addition, two methods, the wall-following method (WFM) and the stagnation-following method (SFM), are introduced to convert the optimal path obtained from the APF into target images suitable for network training. Experimental results based on real and simulated data show that the proposed method provides high detection accuracy in both surface and subsurface phases, such that the safe path detection accuracy with two consecutive sonar scans using WFM based on the IOU criterion reaches 0.965. In addition, the integration of camera and sonar data increases the detection range of underwater obstacles by approximately 15 m compared to using sonar alone and significantly reduces the steering correction angle.
Keywords: auv, Obstacle avoidance, Sonar, Integrated acoustic-optic detection, Deep learning
Full-Text [PDF 1595 kb]   (12 Downloads)    
Type of Study: Research Paper | Subject: Submarine Hydrodynamic & Design
Received: 2026/03/17 | Accepted: 2026/06/27
فایل صوتی چکیده مقاله [MP3 2846 KB]  (3 Download)
References
1. B. Wang, J. Liu, Z. Deng, and M. Fu, "A Model Free Calibration Method of Inertial Navigation System and Doppler Sensors," IEEE Sens. J., vol. 21, no. 2, pp. 2219-2229, 2021. https://doi.org/10.1109/JSEN.2020.3015845 [DOI:10.1109/JSEN.2020.3015845.]
2. M. Sabet and H. Nourmohammadi, "Water velocity sensor with the ability to estimate the sideslip angle based on Bernoulli's law for use in autonomous underwater vehicles," Ocean Eng., vol. 263, p. 112252, 2022. https://doi.org/10.1016/j.oceaneng.2022.112252 [DOI:10.1016/j.oceaneng.2022.112252.]
3. H. Ling, T. Zhu, W. He, Z. Zhang, and H. Luo, "Cooperative search method for multiple AUVs based on target clustering and path optimization," Nat. Comput., vol. 20, no. 1, pp. 3-10, 2019. https://doi.org/10.1007/s11047-019-09749-3 [DOI:10.1007/s11047-019-09749-3.]
4. B. Bourdin, M. J. Hasan, S. Kannan, and R. Prabhu, "Adaptive path-planning for AUVs in dynamic underwater environments using sonar data," in Artificial Intelligence for Security and Defence Applications II, H. Bouma, Y. Yitzhaky, R. Prabhu, and H. J. Kuijf, Eds., SPIE, 2024, p. 42. https://doi.org/10.1117/12.3031644 [DOI:10.1117/12.3031644.] [PMID]
5. C. Zhang, P. Cheng, B. Du, B. Dong, and W. Zhang, "AUV path tracking with real-time obstacle avoidance via reinforcement learning under adaptive constraints," Ocean Eng., vol. 256, p. 111453, 2022. https://doi.org/10.1016/j.oceaneng.2022.111453 [DOI:10.1016/j.oceaneng.2022.111453.]
6. C. Cheng, Q. Sha, B. He, and G. Li, "Path planning and obstacle avoidance for AUV: A review," Ocean Eng., vol. 235, p. 109355, 2021. https://doi.org/10.1016/j.oceaneng.2021.109355 [DOI:10.1016/j.oceaneng.2021.109355.]
7. S. Zhang, R. Wu, K. Xu, J. Wang, and W. Sun, "R-CNN-Based Ship Detection from High Resolution Remote Sensing Imagery," Remote Sens., vol. 11, no. 6, p. 631, 2019. https://doi.org/10.3390/rs11060631 [DOI:10.3390/rs11060631.]
8. J. Xu, X. Sun, D. Zhang, and K. Fu, "Automatic Detection of Inshore Ships in High-Resolution Remote Sensing Images Using Robust Invariant Generalized Hough Transform," IEEE Geosci. Remote Sens. Lett., vol. 11, no. 12, pp. 2070-2074, 2014. https://doi.org/10.1109/LGRS.2014.2319082 [DOI:10.1109/lgrs.2014.2319082.]
9. G. Liu, Y. Zhang, X. Zheng, X. Sun, K. Fu, and H. Wang, "A New Method on Inshore Ship Detection in High-Resolution Satellite Images Using Shape and Context Information," IEEE Geosci. Remote Sens. Lett., vol. 11, no. 3, pp. 617-621, 2014. https://doi.org/10.1109/LGRS.2013.2272492 [DOI:10.1109/lgrs.2013.2272492.]
10. M. Tello, C. Lopez-Martinez, and J. J. Mallorqui, "A Novel Algorithm for Ship Detection in SAR Imagery Based on the Wavelet Transform," IEEE Geosci. Remote Sens. Lett., vol. 2, no. 2, pp. 201-205, 2005, doi: https://doi.org/10.1109/LGRS.2005.845033 [DOI:10.1109/lgrs.2005.845033.]
11. "Review of Ship Surveillance Technologies Based on High-Resolution Wide-Swath Synthetic Aperture Radar Imaging." doi: [DOI:10.12000/JR14144.]
12. W. Tao, "Unified mean shift segmentation and graph region merging algorithm for infrared ship target segmentation," Opt. Eng., vol. 46, no. 12, p. 127002, 2007, doi: https://doi.org/10.1117/1.2823159 [DOI:10.1117/1.2823159.]
13. S. Greenberg, "Region-of-interest-based algorithm for automatic target detection in infrared images," Opt. Eng., vol. 44, no. 7, p. 77002, 2005, doi: https://doi.org/10.1117/1.1951547 [DOI:10.1117/1.1951547.]
14. M. Ren, Y. Cao, Z. Tang, J. Yang, B. Wang, and X. Geng, "One effective method for ship recognition in ship locks," in Signal Processing, Sensor Fusion, and Target Recognition VIII, I. Kadar, Ed., SPIE, 1999, p. 467. doi: https://doi.org/10.1117/12.357188 [DOI:10.1117/12.357188.]
15. W. Ouyang, X. Wang, C. Zhang, and X. Yang, "Factors in finetuning deep model for object detection with long-tail distribution," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 864-873. doi: https://doi.org/10.1109/CVPR.2016.100 [DOI:10.1109/CVPR.2016.100.]
16. J. Cao, Y. Pang, and X. Li, "Pedestrian detection inspired by appearance constancy and shape symmetry," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1316-1324. doi: https://doi.org/10.1109/CVPR.2016.147 [DOI:10.1109/CVPR.2016.147.] [PMID]
17. S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137-1149, 2017, doi: https://doi.org/10.1109/TPAMI.2016.2577031 [DOI:10.1109/tpami.2016.2577031.] [PMID]
18. J. Redmon and A. Farhadi, "YOLO9000: better, faster, stronger," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 7263-7271. doi: https://doi.org/10.1109/CVPR.2017.690 [DOI:10.48550/arXiv.1612.08242.]
19. W. Liu et al., "SSD: Single Shot MultiBox Detector," in Computer Vision - ECCV 2016, Springer International Publishing, 2016, pp. 21-37. doi: https://doi.org/10.1007/978-3-319-46448-0_2 [DOI:10.1007/978-3-319-46448-0_2.]
20. R. Girshick, "Fast r-cnn," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1440-1448. doi: https://doi.org/10.1109/ICCV.2015.169 [DOI:10.1109/ICCV.2015.169.]
21. J. Yan, J. Meng, and J. Zhao, "Bottom detection from backscatter data of conventional side scan sonars through 1D-UNet," Remote Sens., vol. 13, no. 5, p. 1024, 2021. https://doi.org/10.3390/rs13051024 [DOI:10.3390/rs13051024.]
22. J. Li, P. Jiang, and H. Zhu, "A local region-based level set method with Markov random field for side-scan sonar image multi-level segmentation," IEEE Sens. J., vol. 21, no. 1, pp. 510-519, 2020. https://doi.org/10.1109/JSEN.2020.3013649 [DOI:10.1109/JSEN.2020.3013649.]
23. Z. Yan, J. Zhang, and J. Tang, "Modified water wave optimization algorithm for underwater multilevel thresholding image segmentation," Multimed. Tools Appl., vol. 79, no. 43, pp. 32415-32448, 2020. https://doi.org/10.1007/s11042-020-09664-1 [DOI:10.1007/s11042-020-09664-1.]
24. J. Lu, S. Zhang, H. Zhang, and L. Lin, "An Improved Target Detection Method Based on Sonar Image Processing," in 2019 Photonics & Electromagnetics Research Symposium - Fall (PIERS - Fall), IEEE, 2019, pp. 1124-1128. https://doi.org/10.1109/PIERS-Fall48861.2019.9021810 [DOI:10.1109/piersfall48861.2019.9021810.] [PMCID]
25. T. Guerneve, K. Subr, and Y. Petillot, "Three-dimensional reconstruction of underwater objects using wide-aperture imaging SONAR," J. F. Robot., vol. 35, no. 6, pp. 890-905, 2018. https://doi.org/10.1002/rob.21783 [DOI:10.1002/rob.21783.]
26. W. Li, M. Liu, S. Zhang, R. Zheng, and J. Lan, "Underwater Sonar Visual Tracking with Obstacle Solving Method," in 2020 39th Chinese Control Conference (CCC), IEEE, 2020, pp. 3044-3049. https://doi.org/10.23919/CCC50068.2020.9189400 [DOI:10.23919/ccc50068.2020.9189400.]
27. X. Wang, J. Jiao, J. Yin, W. Zhao, X. Han, and B. Sun, "Underwater sonar image classification using adaptive weights convolutional neural network," Appl. Acoust., vol. 146, pp. 145-154, 2019. https://doi.org/10.1016/j.apacoust.2018.11.003 [DOI:10.1016/j.apacoust.2018.11.003.]
28. S. Lee, B. Park, and A. Kim, "Deep learning based object detection via style-transferred underwater sonar images," IFAC-PapersOnLine, vol. 52, no. 21, pp. 152-155, 2019. https://doi.org/10.1016/j.ifacol.2019.12.299 [DOI:10.1016/j.ifacol.2019.12.299.]
29. M. Valdenegro-Toro, "Object recognition in forward-looking sonar images with Convolutional Neural Networks," in OCEANS 2016 MTS/IEEE Monterey, IEEE, 2016, pp. 1-6. https://doi.org/10.1109/OCEANS.2016.7761140 [DOI:10.1109/oceans.2016.7761140.]
30. J. Kim, H. Cho, J. Pyo, B. Kim, and S.-C. Yu, "The convolution neural network based agent vehicle detection using forward-looking sonar image," in OCEANS 2016 MTS/IEEE Monterey, 2016, pp. 1-5. https://doi.org/10.1109/OCEANS.2016.7761209 [DOI:10.1109/OCEANS.2016.7761209.]
31. X. Zhou, K. Yang, and R. Duan, "Deep Learning Based on Striation Images for Underwater and Surface Target Classification," IEEE Signal Process. Lett., vol. 26, no. 9, pp. 1378-1382, 2019. https://doi.org/10.1109/LSP.2019.2919102 [DOI:10.1109/lsp.2019.2919102.]
32. C. Lin, G. Han, J. Du, Y. Bi, L. Shu, and K. Fan, "A path planning scheme for AUV flock-based Internet-of-Underwater-Things systems to enable transparent and smart ocean," IEEE Internet Things J., vol. 7, no. 10, pp. 9760-9772, 2020. https://doi.org/10.1109/JIOT.2020.2988285 [DOI:10.1109/JIOT.2020.2988285.]
33. R. Kot, "Fuzzy inference-based adaptive sonar control for collision avoidance in autonomous underwater vehicles," Polish Marit. Res., 2024. https://doi.org/10.2478/pomr-2024-0058 [DOI:10.2478/pomr-2024-0058.]
34. R. Cui, Y. Li, and W. Yan, "Mutual Information-Based Multi-AUV Path Planning for Scalar Field Sampling Using Multidimensional RRT*," IEEE Trans. Syst. Man, Cybern. Syst., vol. 46, no. 7, pp. 993-1004, 2016. https://doi.org/10.1109/TSMC.2015.2500027 [DOI:10.1109/tsmc.2015.2500027.]
35. X. Cao, D. Zhu, and S. X. Yang, "Multi-AUV Target Search Based on Bioinspired Neurodynamics Model in 3-D Underwater Environments," IEEE Trans. Neural Networks Learn. Syst., vol. 27, no. 11, pp. 2364-2374, 2016. https://doi.org/10.1109/TNNLS.2015.2482501 [DOI:10.1109/tnnls.2015.2482501.] [PMID]
36. Y. Liu and W. Xu, "Application of improved Hopfield Neural Network in path planning," in Journal of Physics: Conference Series, 2020, p. 12154. https://doi.org/10.1088/1742-6596/1544/1/012154 [DOI:10.1088/1742-6596/1544/1/012154.]
37. D. Zhu, H. Huang, and S. X. Yang, "Dynamic Task Assignment and Path Planning of Multi-AUV System Based on an Improved Self-Organizing Map and Velocity Synthesis Method in Three-Dimensional Underwater Workspace," IEEE Trans. Cybern., vol. 43, no. 2, pp. 504-514, 2013. https://doi.org/10.1109/TSMCB.2012.2210212 [DOI:10.1109/tsmcb.2012.2210212.] [PMID]
38. M. Chen and D. Zhu, "A Workload Balanced Algorithm for Task Assignment and Path Planning of Inhomogeneous Autonomous Underwater Vehicle System," IEEE Trans. Cogn. Dev. Syst., vol. 11, no. 4, pp. 483-493, 2019, https://doi.org/10.1109/TCDS.2018.2866984 [DOI:10.1109/tcds.2018.2866984.]
39. R. Gladstone, Y. Moshe, A. Barel, and E. Shenhav, "Distance estimation for marine vehicles using a monocular video camera," in 2016 24th European Signal Processing Conference (EUSIPCO), 2016, pp. 2405-2409. https://doi.org/10.1109/EUSIPCO.2016.7760680 [DOI:10.1109/EUSIPCO.2016.7760680.]
40. V. Antoska-Knights, Z. Gacovski, and S. Deskovski, "Obstacles avoidance algorithm for mobile robots, using the potential fields method," Univ. J. Electr. Electron. Eng, vol. 5, pp. 75-84, 2017, https://doi.org/10.13189/ujeee.2017.050402 [DOI:10.13189/UJEEE.2017.050402.]
41. W. Ding et al., "Research on obstacle avoidance of multi-AUV cluster formation based on virtual structure and artificial potential field method★," Comput. Electr. Eng., vol. 117, p. 109250, 2024, https://doi.org/10.1016/j.compeleceng.2024.109250 [DOI:10.1016/j.compeleceng.2024.109250.]
42. Zia S V, Poyan Rad M. An obstacle avoidance algorithm for an autonomous underwater vehicle based on fuzzy-virtual potential function and forward looking sonar. marineeng 2021; 17 (33) :11-22 http://marine-eng.ir/article-1-790-fa.html
Send email to the article author

Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Creative Commons License
International Journal of Maritime Technology is licensed under a

Creative Commons Attribution-NonCommercial 4.0 International License.