Article Details

Abstract

To address the communication barriers and safety needs of people with hearing and speech impairments, this study designs and implements an embedded bidirectional translation system for this group. Its core contribution lies in a wearable system-level solution with low cost, low power consumption, and high integration. With STM32F103C8T6 as the core, the system integrates flex sensors and an optimized threshold recognition algorithm to realize sign-language-to-speech conversion, and uses the ASRPRO speech module and a serial screen to realize speech-to-sign-language conversion. The system integrates GPS/GSM modules to support one-click sending of emergency help messages containing real-time location, and provides multi channel status feedback through LEDs and vibration. Test results show that the gesture recognition accuracy of the system exceeds 92%, the bidirectional translation response is timely, and the communication function is reliable. Compared with vision based schemes, the system has significant advantages in cost, power consumption, and user privacy protection, thus providing an efficient and safe accessible communication tool for people with hearing and speech impairments.

Keywords

multi-sensor; speech recognition; gesture recognition; bidirectional translation; STM32

References

1. [1] Cheng Jie, Chen Jianhua, Xu Shuhan Intelligent Gloves and Translation System for Sign Language Collection and Input [J]. Electronic Products World, 2023, 30 (8): 61-65
2. [2] Xu Wangqi Research on Low Cost Indoor Navigation System Based on Bluetooth Positioning and Deep Learning [D]. Hangzhou: Zhejiang Gongshang University, 2025
3. [3] Hu Jiawei, Feng Zhenghui, Chen Guojun, etc Design of Community Environment Intelligent Monitoring System Integrating Multi Sensor Data [J]. Wireless Internet Technology, 2023, 20 (12): 16-18
4. [4] Jiang Weijia, Han Yali, Shi Jinfei, etc Research on Sign Language Translation System Based on Data Gloves [J]. Electronic Devices, 2024, 47 (4): 1128-1132
5. [5] Fu Jiajun Design of an accessible two-way interactive system for sign language real-time translation for deaf and mute people [D]. Tianjin: Tianjin University of Technology, 2022
6. [6] Li Bingyan, Sun Xuanyu, Dan Dongyang, etc Development of Sign Language Translation Gloves Based on STM32 [J]. Internet of Things Technology, 2025, 15 (5): 127-132
7. [7] Xia Mingming, Chai Xinyu, Yang Jinguang, etc Design of a Multi Sensor based Gesture Recognition Device [J]. Integrated Circuit Applications, 2021, 38 (12): 292-293
8. [8] Liu Yu, Li Renxiang, Yang Mengfan, etc Design and Implementation of Voice Controlled Garbage Can [J]. Wireless Internet Technology, 2024, 21 (17): 24-26, 30
9. [9] Zhang Yuxi, Zhao Wanli Intelligent biomimetic bidirectional sign language automatic translation system based on machine learning [J]. Automation and Instrumentation, 2022 (1): 176-180
10. [10] Cui Zongxing, Qi Zhongzheng, Sheng Haonan Innovative Design and Application of Intelligent Sign Language Translation Gloves [J]. Internet of Things Technology, 2025, 15 (6): 2-3

Funding

Key Research and Development Plan of Jiangsu Province Science and Technology Program in 2023 (BE2023345); Guiding Project for Science and Technology Plan of Suqian City in 2023 (Z2023135); Innovation and Entrepreneurship Training Program for College Students of Suqian University in 2025 (X2025141600431); 2024 Suqian College "Practical Teaching" Special Teaching Reform Research Project (2024SJJX04); Research Project on Innovation and Entrepreneurship Education at Suqian College in 2024 (2024 cxcy03)

About the authors

Lu Jugu (2004-), male, Han ethnicity, from Suzhou, Jiangsu, undergraduate student, research interests include software engineering; Corresponding author: Lyu Kun (1989-), male, Han ethnicity, from Suqian, Jiangsu, laboratory technician, master's degree, research interests include computer software and theory; Liu Zhiwen (2005-), male, Han ethnicity, from Lianyungang, Jiangsu, undergraduate student, research interests include software engineering; Wang Jiale (2006-), male, Han ethnicity, from Xuzhou, Jiangsu, undergraduate student, research interests include software engineering.