Article Details

Abstract

Gesture recognition is of great significance to realize human-computer interaction. In order to realize high-precision target detection and recognition under dynamic conditions, this paper is based on YOLOv5 target detection firstly, and determines the coordinate information of the target gesture by using the feature pyramid structure and multi-scale fusion structural features inside the algorithm. Then it uses the MediaPipe model to detect the key points of the hand, determines the vector angle of the hand joints, and analyzes the finger bending situation, so as to judge the specific gesture. Using the methods of position determination and implementation by using separate models for action classification effectively improves the problem that the reduced recognition rate of gestures caused by factors such as rotation and occlusion in dynamic conditions. The training samples are selected from six categories in the open-source gesture dataset HaGRID. The experimental test results demonstrate that the mean value of one-hand recognition detection accuracy of the combined algorithm is up to mAP@0.5 and the detection speed is up to 40 FPS, and the model size is 88.5 MB.

Keywords

gesture recognition; YOLOv5; MediaPipe; hand joint point detection; gesture dataset

References

1. [1] Chen Ruizhi, Xie Xiaoyu, Luo Ying The Development, Characteristics, and Implications of Human Computer Interaction Questions in Science Education Assessment [J]. Chinese Examinations, 2024 (8): 79-88
2. [2] Gao Jiarui, Du Hongbo, Cai Jin, etc Research and Application of Gesture Recognition Algorithm Based on MediaPipe [J]. Intelligent Computers and Applications, 2024, 14 (4): 157-161
3. [3] Yuan Yin, Yang Supeng, Chen Mengyuan SLAM Algorithm Based on YOLOv5 in Dynamic Scenarios [J/OL]. Journal of Chongqing Technology and Business University: Natural Science Edition, 2024: 1-9 [2400-08-23] http://kns.cnki.net/kcms/detail/50.1155.N.20240816.0914.002.html.
4. [4] Wen Shuoyi Research on Gesture Sequence Recognition Based on Deep Learning [D]. Tianjin: Tianjin Vocational and Technical Normal University, 2024
5. [5] Zhou Nan Research on Gesture Recognition Algorithm Based on Convolutional Neural Network [D]. Guilin: Guilin University of Technology, 2022
6. [6] BAI L，DING X W，LIU Y，et al. Multi-scale Feature Fusion Optical Remote Sensing Target Detection Method [J].Optoelectronics Letters，2025，21（4）：226-233.
7. [7] ABONDIO P，BRUNO F. Single-cell Pan-omics, Environmental Neurology, and Artificial Intelligence: The Time for Holistic Brain Health Research [J].Neural Rregeneration Rresearch，2025，20（6）：1703-1704.
8. [8] Luo Yi Research on Hand Pose Estimation Based on Deep Learning [D]. Guiyang: Guizhou University, 2023
9. [9] Liu Xiaoke Research on Human Joint Point Detection Algorithm Based on Deep Learning [D]. Qinhuangdao: Yanshan University, 2019
10. [10] Cai Yu Research on Gesture Recognition Algorithm Based on Deep Learning [D]. Hohhot: Inner Mongolia University, 2023
11. [11] Qiu Feng Real time dynamic gesture recognition based on deep learning [D]. Hangzhou: Zhejiang University, 2020
12. [12] Chen Ruimin Research on key technology of gesture recognition and its application in intelligent laboratories [D]. University of Chinese Academy of Sciences (Chinese Academy of Sciences Shanghai Institute of Technical Physics), 2017
13. [13] Yang Wenji Research on Gesture Interaction Technology for Home Service Robots [D]. Qinhuangdao: Yanshan University, 2015
14. [14] Wang Fengyan Research on Gesture Recognition Algorithm Based on Multimodal Input [D]. Hefei: University of Science and Technology of China, 2017
15. [15] Yin Kang Research on Gesture Trajectory Recognition Based on Multi Feature Fusion Network [D]. Chengdu: University of Electronic Science and Technology of China, 2019
16. [16] Bai Yunchao Recognition and Semantic Representation Technology of Gesture Behavior [D]. Nanjing: Nanjing University, 2020
17. [17] Wang Qiangyu Research on Dynamic Gesture Recognition Technology Based on Deep Neural Networks [D]. Beijing: China University of Mining and Technology (Beijing), 2019

About the authors

Chen Wudong (2003-), male, Han ethnicity, from Jingzhou, Hubei Province, is currently pursuing a bachelor's degree with a research focus on embedded systems.