Towards Robust Deep Reinforcement Learning Agent for the Path Following of Autonomous Ships Amid Perception Sensor Noise

In the era of maritime autonomous surface ships (MASSs), intelligent agents are projected to make safety-critical decisions without human intervention. Considering the various disturbances associated with the maritime environment, enhancing their robustness during safety-critical operations is pivotal, including those related to path following. The aim of this study is to propose a methodology that enhances the robustness of path following for a MASS amid perception sensor noise by controlling the state space parameter of a deep reinforcement learning agent. The agent is trained to follow a predefined path at various noise levels between a minimum and maximum value, and a robustness metric based on the cross-track error is defined. The case study considers a container vessel that uses light detection and ranging for the situation awareness of its surrounding environment. Simulation results suggest that when the state space parameter related to the value of the noise level is controlled, the robustness is enhanced up to 5,668% from its maximum trained value by not violating the cross-track error threshold. When the state space parameter is not controlled, an enhancement of up to 112% is noted, highlighting the effectiveness of the proposed methodology. This study contributes towards the development of agents capable of making robust decisions during safety-critical operations under uncertainty.

Keywords: Maritime autonomous surface ship, Deep reinforcement learning, Deep deterministic policy gradient, Path following, Robustness, Perception senor noise, LIDAR, State space manipulation.