PSN-PPO: A Policy-Splitting Network-Based Deep Reinforcement Learning Approach for Autonomous Navigation of Unmanned Surface Vehicles (opens in a new tab)
The paper introduces PSN-PPO, a deep reinforcement learning method for autonomous navigation of unmanned surface vehicles (USVs) that separates navigation and obstacle avoidance into two jointly trained policy branches. The approach uses a navigation branch for course guidance and an avoidance branch for bounded corrections, modulated by a learnable risk gate. Simulations across five scenarios—including obstacle-free navigation, static obstacles, and dynamic encounters—show PSN-PPO improves efficiency and collision avoidance compared to standard PPO. The method was tested using a three-degree-of-freedom vessel motion model in a closed-loop framework.