Write your message

XML Persian Abstract Print


1- Postdoctoral
2- professor, sharif university of technology
Abstract:   (35 Views)

Nowadays, the use of advanced methods for controlling the movements of underwater robots has led to improved efficiency and enhanced operational quality. In this research, a method based on reinforcement learning has been developed for the depth control of AUV robots. This method learns the robot's movement pattern based on a reward criterion and makes the optimal decision for motion and control surface adjustments accordingly. Depth control using reinforcement learning improves the robot's performance and selects the most optimal control signal based on the robot's conditions and rewards. In this study, a linear dynamic model of pitch motion was used to develop the depth control model. For each desired state, the scenario is repeated 500 times to update the Q-matrix during simulation. Subsequently, by assigning rewards to each signal, the optimal value is determined. After completing the scenario, the optimal value from the Q-matrix is selected to determine the control signal for the fin. The results showed that the use of reinforcement learning significantly enhances the quality of the AUV robot's control system, resulting in minimal overshoot and oscillation in performance.

Full-Text [PDF 1088 kb]   (10 Downloads)    
Type of Study: Research Paper | Subject: Submarine Hydrodynamic & Design
Received: 2025/05/13 | Accepted: 2025/07/17

References
1. Zhang, Y., et al. "Deep Reinforcement Learning for Autonomous Underwater Vehicle Path Planning and Control." IEEE Transactions on Robotics, 2022.
2. Li, H., & Wang, J. "PPO-based Control of AUVs in Dynamic Underwater Environments." Ocean Engineering, 2021.
3. Chen, X., et al. "Transfer Learning in Reinforcement Learning for AUV Motion Control." Journal of Marine Science and Technology, 2023.
4. Liu, R., et al. "Multi-Agent Reinforcement Learning for Cooperative AUV Navigation." Autonomous Robots, 2023.
5. Wang, L., & Zhang, K. "Energy-Efficient Multi-AUV Control Using MADDPG." IEEE Journal of Oceanic Engineering, 2022.
6. Patel, S., et al. "Spiking Neural Networks for Real-Time AUV Control." Neural Networks, 2023.
7. Kim, H., & Park, S. "LSTM-Based Reinforcement Learning for Dynamic AUV Motion Planning." Ocean Engineering, 2023.
8. Zhao, Y., et al. "Hybrid RL-SNN Architectures for Autonomous Underwater Vehicles." Journal of Intelligent & Robotic Systems, 2024.
9. Hasanvand, A. and Seif, M.S., 2024. Investigation the effect of length-to-diameter ratio on six-DOF helical and zigzag maneuvers of the SUT glider with internal actuators. Ocean Engineering, 295, p.116819. [DOI:10.1016/j.oceaneng.2024.116819]
10. Hasanvand, A. and Seif, M.S., 2024. Development of state space models for underwater gliders equipped with embedded actuators in vertical and horizontal planes. Ocean Engineering, 309, p.118344. [DOI:10.1016/j.oceaneng.2024.118344]
11. Hasanvand, A. and Seif, M.S., 2024. Adaptive path-following control for high-underactuated underwater glider under hydrodynamic coefficient uncertainties. Journal of Marine Science and Technology, 29(4), pp.1000-1017. [DOI:10.1007/s00773-024-01032-0]
12. Wang, D., Wan, J., Shen, Y., Qin, P. and He, B., 2022. Hyperparameter Optimization for the LSTM Method of AUV Model Identification Based on Q-Learning. Journal of Marine Science and Engineering, 10(8), p.1002. [DOI:10.3390/jmse10081002]
13. Jun, B.H., Park, J.Y., Lee, F.Y., Lee, P.M., Lee, C.M., Kim, K., Lim, Y.K. and Oh, J.H., 2009. Development of the AUV 'ISiMI'and a free running test in an Ocean Engineering Basin. Ocean engineering, 36(1), pp.2-14. [DOI:10.1016/j.oceaneng.2008.07.009]

Send email to the article author


Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Creative Commons License
International Journal of Maritime Technology is licensed under a

Creative Commons Attribution-NonCommercial 4.0 International License.