Ensuring Safety in Highly Automated Vehicles: A Review of Testing and Validation Methods for Robustness and Reliability

The future of mobility is set to be reformed as the rapidly increasing use of driver assistance systems and highly automated vehicles (HAVs) show their great potential. The use of deep neural networks in autonomous driving systems has led to significant progress in this area. However, the increase in accidents involving HAVs highlights the need for effective testing and validation methods to increase the overall safety of these vehicles. With many technology companies and manufacturers aiming to put Level 4 and 5 vehicles into operation soon, the safety of HAVs remains a major concern. Rigorous testing and validation against potential failures and misbehaviour are required to ensure the reliability and robustness of these systems. This paper provides an overview of state of the art in testing and evaluation methods for machine learning-based HAVs. A literature review on these topics is provided to give valuable insights to researchers, practitioners and policymakers. As such, the review describes different types of validation, verification and testing methods, including real-world testing, simulation testing, hardware-in-the-loop testing, adversarial robustness, and methods used for explainability and interpretability in AI. The advantages and limitations are discussed and current challenges are highlighted. Finally, open research questions and future directions in the field are identified.

Keywords: Highly automated vehicles, Autonomous vehicles, Robustness, Reliability, Testing, Validation, Machine learning, Safety.

Download PDF