Proceedings of the
The Nineteenth International Conference on Computational Intelligence and Security (CIS 2023)
December 1 – 4, 2023, Haikou, China
Evolutionary Reinforcement Learning by Rank-one Evolution Strategy with Population Size Control
School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China.
ABSTRACT
Deep reinforcement learning has made significant breakthroughs in numerous fields and is considered a meaningful way to achieve general artificial intelligence. However, the existing deep reinforcement learning algorithms still face some difficulties in dealing with real problems, such as long time range and sparse reward credit allocation, lack of effective diversified exploration and sensitivity to the selection of hyperparameters. To this end, we propose an evolutionary reinforcement learning algorithm, RPSA-RL, which combines an evolutionary algorithm based on population search with a deep reinforcement learning algorithm that utilizes problem gradient information. We evaluated the performance of RPSA-RL with state-of-the-art reinforcement learning algorithms on six common DRL continuous control tasks in the OpenAI Gym test bed. The results show that the proposed algorithm is superior to or better than the most advanced algorithm, which proves the effectiveness of the proposed algorithm.
Keywords: Deep reinforcement learning, Evolutionary algorithm, Noise processing method, Population size adaptation.
Download PDF
School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China.
ABSTRACT
Deep reinforcement learning has made significant breakthroughs in numerous fields and is considered a meaningful way to achieve general artificial intelligence. However, the existing deep reinforcement learning algorithms still face some difficulties in dealing with real problems, such as long time range and sparse reward credit allocation, lack of effective diversified exploration and sensitivity to the selection of hyperparameters. To this end, we propose an evolutionary reinforcement learning algorithm, RPSA-RL, which combines an evolutionary algorithm based on population search with a deep reinforcement learning algorithm that utilizes problem gradient information. We evaluated the performance of RPSA-RL with state-of-the-art reinforcement learning algorithms on six common DRL continuous control tasks in the OpenAI Gym test bed. The results show that the proposed algorithm is superior to or better than the most advanced algorithm, which proves the effectiveness of the proposed algorithm.
Keywords: Deep reinforcement learning, Evolutionary algorithm, Noise processing method, Population size adaptation.
Download PDF