Reinforcement Learning for Black-box Safety Validation of Autonomous Marine Vessels

Background & motivation

  • The Zeabuz autonomous mobility system is a complex, software intensive system, enabled by artificial intelligence and subject to an unpredictable operating environment. 
  • This makes formal safety proofs (practically) impossible. Instead, one needs to resort to statistical considerations in the safety argumentation. In other words, there is a need to argue – based on the accumulated experience, testing, verification and validation activities – that the system is sufficiently safe.
  • To solve these issues, we need a systematic and effective way of designing, running, and evaluating simulation scenarios that together give sufficient confidence in the safety.

Problem description

  • The thesis will employ, adapt and extend a method called adaptive stress testing (AST) to find likely failure events of the autonomy system, using reinforcement learning (RL) and other AI and ML techniques.
  • The AST method has previously been used to validate aircraft collision avoidance systems, which is a similar problem.

Work proposal

  • Perform a literature review of AST and similar methods.
  • Discuss how the reviewed methods can be deployed for an autonomous passenger ferry, specifically the Zeabuz ferry. 
  • Design and implement an AST system and connect this to an existing digital twin setup provided by Zeabuz. 
  • Perform initial simulations with the AST system and identify potential weaknesses in the autonomy system.
  • Create additional Zeabuz scenarios for system validation and stress testing.  Use them in comprehensive simulations with the digital twin, using the AST system. 
  • Compare and contrast to existing approaches, and discuss strengths and weaknesses based on simulation results.  
  • To test the method, the candidate may – in cooperation with the Zeabuz team – also introduce known errors in the autonomy system under test and investigate if the AST system is able to find these errors.

Figure 1: An illustration of the adaptiv stress testing method.