Reinforcement Learning for Black-box Safety Validation of Autonomous Marine Vessels
Background & motivation
- The Zeabuz autonomous mobility system is a complex, software intensive system, enabled by artificial intelligence and subject to an unpredictable operating environment.
- This makes formal safety proofs (practically) impossible. Instead, one needs to resort to statistical considerations in the safety argumentation. In other words, there is a need to argue – based on the accumulated experience, testing, verification and validation activities – that the system is sufficiently safe.
- To solve these issues, we need a systematic and effective way of designing, running, and evaluating simulation scenarios that together give sufficient confidence in the safety.
- The thesis will employ, adapt and extend a method called adaptive stress testing (AST) to find likely failure events of the autonomy system, using reinforcement learning (RL) and other AI and ML techniques.
- The AST method has previously been used to validate aircraft collision avoidance systems, which is a similar problem.
- Perform a literature review of AST and similar methods.
- Discuss how the reviewed methods can be deployed for an autonomous passenger ferry, specifically the Zeabuz ferry.
- Design and implement an AST system and connect this to an existing digital twin setup provided by Zeabuz.
- Perform initial simulations with the AST system and identify potential weaknesses in the autonomy system.
- Create additional Zeabuz scenarios for system validation and stress testing. Use them in comprehensive simulations with the digital twin, using the AST system.
- Compare and contrast to existing approaches, and discuss strengths and weaknesses based on simulation results.
- To test the method, the candidate may – in cooperation with the Zeabuz team – also introduce known errors in the autonomy system under test and investigate if the AST system is able to find these errors.
Figure 1: An illustration of the adaptiv stress testing method.