Article Details

Abstract

Aiming at the problems of difficult multi-source data alignment, high fusion complexity, and insufficient real-time decision-making performance of service robots in unstructured environments, a method combining a cross-modal attention fusion network and an Actor-Critic decision architecture is proposed. Multi-dimensional semantic representation is achieved through spatial registration of RGB-D vision and laser point cloud, together with the fusion of visual, laser, voice, and tactile features. Spatio-temporal convolutional networks are used to model dynamic sequence features, and an end-to-end mapping from perception to decision is constructed. Experiments based on the TIAGo robot platform show that the target recognition accuracy of this method reaches 91.4%, the scene segmentation mIoU is 79.6%, and the decision response time is 76 ms. The task success rate exceeds 91% in each of the three typical scenarios: household object grasping, medical personnel following, and commercial navigation obstacle avoidance, which verifies the effectiveness of the proposed method in terms of perception accuracy, real-time performance and robustness.

Keywords

service robot; multimodal perception; feature fusion; real-time decision-making; Attention Mechanism

References

1. [1] Nkwocha C L，Adewumi A，Folorunsho S O，et al. A Comprehensive Review of Sensing, Control, and Networking in Agricultural Robots: From Perception to Coordination [J/OL].Robotics，2025，14（11）：159.https://doi.org/10.3390/robotics14110159.
2. [2] Yu Xinyi, He Wei, Ou Linlin Research on Robot Grasping Strategy Integrating Multimodal Perception [J]. Small Microcomputer System, 2026, 47 (4): 894-901
3. [3] Olufade M，Bankole E A，Victor-Igun O O，et al. Multimodal Sensor Fusion for Autonomous Systems: Integrating Data from Various Sensors to Improve Decision-making in Autonomous Vehicles and Robotics [J].Journal of Basic and Applied Research International，2025：37-54.
4. [4] Xie Yuanlong, Wang Shuting, Cheng Xiang, etc Overview of Large Model Driven Tailored Intelligent Robot Navigation Technology [J]. Journal of Central China Normal University (Natural Science Edition), 2025, 59 (5): 677-693
5. [5] Yin Chenkun, Ji Hongxuan, Zhang Yanxin Autonomous decision-making of search and rescue robots based on heterogeneous strategy hierarchical reinforcement learning in complex interactive scenarios [J]. Journal of Beijing Institute of Technology, 2023, 49 (4): 403-414
6. [6] Mao Xiaohan, Zhu Chenming, Wang Tai, etc M3Fusion: A Unified Multi view Multi modal Fusion Framework for Embodied 3D Perception [J]. Chinese Science: Information Science, 2026, 56 (2): 312-326
7. [7] Guo Jie, Shi Jinzan, Song Bin, etc Attention aware contrastive fusion network for multimodal recommendation [J]. Journal of Xi'an University of Electronic Science and Technology, 2026, 53 (3): 164-180
8. [8] Wei Debin, Li Jing, Pan Chengsheng Research on end-to-end QoS class mapping of satellite networks based on SDN [J]. Computer Simulation, 2024, 41 (7): 55-60
9. [9] Zhang Hanwen, Feng Guancheng, Zhang Hongjun, etc Actor Critic Multi Robot Navigation Method with Distributed RVO Constraints [J]. Journal of Xi'an University of Arts and Sciences (Natural Science Edition), 2026, 29 (1): 41-49
10. [10] Zhao Jihong, Dong Sha, Hu Xiaoyan, etc End to end dynamic collaborative slicing method based on inter domain delay game in multi domain networks [J]. Computer Application Research, 2024, 41 (6): 1820-1824

About the authors

Chen Zhengbin (1989-), male, Han ethnicity, from Bazhong, Sichuan, is an engineer with a bachelor's degree. His research interests include robotics and artificial intelligence.