Article Details

Abstract

Automated Guided Vehicle (AGV) serves as a crucial transportation means in intelligent logistics systems, and the AGV path planning and obstacle avoidance algorithm is a significant research hotspot in the domain of mobile robots. To tackle the issues of slow convergence speed and imbalanced exploration and exploitation in AGV path planning using the Q-learning algorithm under the existing warehouse environment, this paper proposes an improved Q-learning algorithm combined with gravitational potential field, along with a dynamic adjustment of the greedy coefficient. Firstly, Aiming at the problem of low learning efficiency in traditional Q-learning algorithm planning, it builds a gravitational potential field from the AGV to the target point, directs the AGV to constantly move towards the direction of target point, reduces the initial algorithm blindness, and enhances the targeting performance of initial stage. Next, the algorithm balance problem of exploration and exploitation is solved, and the greedy coefficient is dynamically improved. Simulation experiments demonstrate that while the exploration rate rises, there is a certain enhancement in algorithm stability as well.

Keywords

Q-learning algorithm; Reinforcement Learning; Artificial Potential Field algorithm; AGV; path planning

References

1. [1] CHEN X，LIU S，ZHAO J，et al. Autonomous Port Management based AGV Path Planning and Optimization Via an Ensemble Reinforcement Learning Framework [J/OL].Ocean and Coastal Management，2024，251：107087（2024-03-10）.https://doi.org/10.1016/j.ocecoaman.2024.107087.
2. [2] Zhu Lei, Fan Jizhuang, Zhao Jie, etc Global Path Planning and Local Obstacle Avoidance of Mining Accident Search Robot Based on Grid Method [J]. Journal of Central South University: Natural Science Edition, 2011, 42 (11): 3421-3428
3. [3] Yu Xiang, Jiang Chen, Duan Sirui, etc Improved A * algorithm and artificial potential field method for path planning [J]. Journal of System Simulation, 2024, 36 (3): 782-794
4. [4] Lin Wenxuan, Xie Wenjun, Zhang Peng, etc Three dimensional path planning of unmanned aerial vehicles based on group optimization and improved particle swarm algorithm [J]. Fire and Command Control, 2023, 48 (1): 20-25+32
5. [5] XIN J，LI Z，ZHANG Y，et al. Efficient Real-Time Path Planning with Self-Evolving Particle Swarm Optimization in Dynamic Scenarios [J].Unmanned Systems，2024，12（2）：215-226.
6. [6] Yang Hailan, Qi Yongqiang, Rong Dan AGV path planning based on memristive reinforcement learning in storage environment [J]. Computer Engineering and Applications, 2023, 59 (17): 318-327
7. [7] CUI Y，REN J，ZHANG Y. Path Planning Algorithm for Unmanned Surface Vehicle based on Optimized Ant Colony Algorithm [J].IEEJ Transactions on Electrical and Electronic Engineering，2022，17（7）：1027-1037.
8. [8] LI D D，WANG L，CAI J C，et al. Research on Path Planning of Mobile Robot based on Improved Genetic Algorithm [J/OL].International Journal of Modeling, Simulation, and Scientific Computing，2023，14（6）：2341030[2024-03-16].https://doi.org/10.1142/S1793962323410301.
9. [9] Xu Xiaosu, Yuan Jie A Path Planning Method for Mobile Robots Based on Improved Reinforcement Learning [J]. Chinese Journal of Inertial Technology, 2019, 27 (3): 314-320
10. [10] SURESH K S，VENKATESAN R，VENUGOPAL S. Mobile Robot path Planning Using Multi-Objective Genetic Algorithm in Industrial Automation [J].Soft Computing，2022，26（15）：7387-7400.
11. [11] Ji Hong, Zhao Zhongyi, Wang Yingli, etc Research on Multi AGV Path Planning and Scheduling System in Complex Environment [J]. Mechanical Design, 2023, 40 (6): 110-115
12. [12] Wang Zhiwei, Zou Yanli, Liu Tanghuimei, etc Path planning based on improved Q-learning algorithm and DWA [J]. Sensors and Microsystems, 2023, 42 (9): 148-152
13. [13] Ren Xuegan, Ge Yingfei AGV path planning based on improved potential field ant colony algorithm [J]. Journal of Nanjing Institute of Engineering: Natural Science Edition, 2021, 19 (1): 36-41
14. [14] Duan Jianmin, Chen Qianglong Research on Q-Learning Path Planning Algorithm Using Prior Knowledge [J]. Electro Optics and Control, 2019, 26 (9): 29-33

About the authors

Geng Hua (1985-), male, Han ethnicity, from Handan, Hebei Province, lecturer, PhD, research direction: modeling and control of complex systems; Feng Tao (1987-), female, Han ethnicity, from Handan, Hebei Province, is currently pursuing a master's degree in mobile robot path planning.