Meaningful Human Control in Digitalization, Automation/AI, and Remote Oversight

Meaningful human control (MHC) of safety critical systems is a important goal as digitalization, automation/artificial intelligence (AI), and remote oversight are implemented. In the EU/AI regulation, the concept of human oversight is introduced, especially for safety critical operations. MHC and human oversight are challenging because they depend on human strengths and weaknesses, system design, knowledge and training, and organizational factors like responsibilities, staffing, and work processes. MHC is more useful than human oversight because it ensures that systems, technology, and organizational structures are designed to keep humans in control of safety-critical operations, thereby preventing disasters. However, to be useful, MHC needs to be defined and specified. This paper aims to define MHC by addressing three key areas: design, operations, and learning. Key design issues for MHC include adopting a system approach, using human-centred design best practices, conducting task analysis to manage cognitive workload, creating consistent interfaces for quick situational understanding, designing alarms to support situational awareness (SA), and establishing work processes that promote shared SA across teams. Key operational issues include ensuring safety, managing change (MoC), addressing error traps and training, and maintaining physical and mental conditions to enable MHC in all situations. In a critical situation, we observe that it can take 10 minutes to observe, understand and act correctly in crises. Main issues in learning from accidents must be to identify root causes including poor concepts/design and trying to understand reasons for human SA and actions. We have used "Human Error" as a starting point for analysis. Learning and understanding should drive change and improvement in governing values, prioritizing learning over blame.

Keywords: Meaningful human control, Safety, human factors.