Proceedings of the

The 33rd European Safety and Reliability Conference (ESREL 2023)
3 – 8 September 2023, Southampton, UK

Expert Evaluation of ChatGPT Performance for Risk Management Process based on ISO 31000 Standard

M.K.S. Al-Mhdawi1, Abroon Qazi2, Ammar Alzarrad3, Nicholas Dacre4,a, Farzad Pour Rahimian5, Mohanad K. Buniya6 and Hanqin Zhang4,b

1School of Computing, Engineering & Digital Technologies, Teesside University, UK; Department of Civil, Structural and Environmental Engineering, Trinity College Dublin, the University of Dublin, Ireland.

2School of Business Administration, American University of Sharjah, UAE.

3Department of Civil Engineering, Marshall University, USA.

4Southampton Business School, University of Southampton, UK.

5School of Computing, Engineering & Digital Technologies, Teesside University, Middlesbrough, UK.

6School Civil & Environmental Engineering Department, University Technology Petronas, Malaysia.

ABSTRACT

ChatGPT is widely known for its ability to facilitate knowledge exchange, support research endeavours, and enhance problem-solving across various scientific disciplines. However, to date, no empirical research has been undertaken to evaluate ChatGPT's performance against established standards or professional guidelines. Consequently, the present study aims to evaluate the performance of ChatGPT for the risk management (RM) process based on ISO 31000 standard using expert evaluation. The authors (1) identified the key indicators for measuring the performance of ChatGPT in managing construction risks based on ISO 31000 and determined the key assessment criteria for evaluating the identified indicators using a focus group session with Iraqi experts; and (2) quantitatively analysed the level of performance of ChatGPT under a fuzzy environment. The findings indicated that ChatGPT's overall performance was high. Specifically, its ability to provide relevant risk mitigation strategies was identified as its strongest aspect. However, the research also revealed that ChatGPT's consistency in risk assessment and prioritisation was the least effective aspect. This research serves as a foundation for future studies and developments in the field of AI-driven risk management, advancing our theoretical understanding of the application of AI models like ChatGPT in real-world risk scenarios.

Keywords: ChatGPT, ChatGPT performance, AI, Risk, Risk management, ISO 31000.


Download PDF