A Multi-Scale LSTM with Multi-Head Self-Attention Embedding Mechanism for the Remaining Useful Life Prediction of Hot Strip Mill Rollers

Remaining useful life(RUL) prediction of intelligent equipment plays a crucial role in avoiding major safety accidents and substantial economic losses from degradation failures. Recently, many studies focused on deep learning-based data-driven methods, such as long short-term memory (LSTM) neural networks, which used multi-dimensions monitoring signals or features to predict the RULs. However, most existing methods are inability to acquire valid temporal information from long-term time series. Moreover, the input data containing much redundant information leads to imprecise RUL prediction results. To overcome the aforementioned weakness, a multi-scale LSTM neural network with multi-head self-attention embedding mechanism(MLSTM-MHA) is proposed in this article for RUL prediction. Firstly, the memory cells of LSTM are divided into several parts according to different temporal trend types, such as local trends, medium trends, and long trends. Fusing all types of memory cells can capture additional trend information and improve the performance of LSTM in learning time series. Secondly, the multi-head self-attention mechanism is embedded in the forgetting gate and input gate structure of LSTM, which can participate in training the MLSTM-MHA network and adaptively recalculates the network weights. The redundant information is assigned lower weights due to lower values by the attention module. Finally, a hot strip mill roller dataset is used to validate the superiority of the proposed method. Compared with the existing data-driven RUL prediction methods, the proposed method has a more accurate predictive ability.

Keywords: Remaining useful life prediction, LSTM, Self-attention mechanism, Redundant information.

Download PDF