A Systematic Review of Risk Metrics for AI and Autonomous Systems

Supervisory risk control (SRC) is a concept for risk-aware operational decision-making, enhancing the safety and intelligence of autonomous systems. Autonomous systems may support and exceed human performance, but new types of risks are introduced, for example, related to mission complexity and challenges with situation awareness. There are many types of autonomous systems, both crewed and uncrewed, operating in low to high degrees of autonomy, and systems may also switch in between these. The foundation of SRC is constituted by risk assessment and control system design, as well as artificial intelligence (AI). One or more risk models are integrated with the mission planning and/or guidance layer of the autonomous control system. A challenge is, however, how to measure the risk in a way that represents both safe systems and operations, and that can be utilized by the control system, e.g., for path planning. Furthermore, the human supervisor also needs information about the risks to support situation awareness. Hence, risk metrics that sufficiently integrate spatial and temporal information, evaluate “instantaneous” and "long-term" risk, as well as the consider the effect of uncertainty are needed. Therefore, the objective of this paper is to provide an overview of existing metrics for measuring risk and evaluate their usefulness for autonomous systems operating in unstructured environments. The paper also suggests potential directions for further research and development in the area.

Keywords: Risk metrics, Autonomous systems, Robotics, Artificial intelligence, Supervisory risk control, Decision-support.