Article Details

Title · Authors · Abstract · Keywords · DOI · References

Pedestrian Detection Method Based on Visible and Infrared Image Fusion

ZHU Jufeng

(School of Artificial Intelligence, Guangxi Minzu University, Nanning 530006, China)

Section：Artificial Intelligence · Vol. 2026, Issue 9 · Pages：75-79 · 2096-4706（2026）09-0075-05 · CLC number：TP391.4；TP183 · Document code：A · DOI：10.19850/j.cnki.2096-4706.2026.09.014

Abstract

To address the issues of low pedestrian detection accuracy in low-light environments and susceptibility to false detections in complex environments, a multispectral object detection algorithm based on the Cross-Modal Feature Fusion Network (CFFNet) is proposed, utilizing the YOLO framework and fusing images from different spectral modalities (e.g., RGB and thermal infrared images). Improving upon Transformer-based fusion methods, a Cross-Modal Feature Interaction Module (CFI) is designed, incorporating a cross-attention mechanism. This module mainly consists of two parts: a Spatial Feature Compression Module (SFS) and a Cross-Modal Information Complementary Module (CICM). SFS reduces computational cost while preserving important image features as much as possible. CICM focuses on efficiently mining and utilizing Complementary feature information between different modalities. Experimental results show that, compared to the Iterative Cross-Attention Guided Feature Fusion Algorithm (ICAFusion), CFFNet improves mAP@0.5 and mAP@0.5:0.95 by 3.8% and 1.3% respectively on the FLIR multispectral dataset, accurately detecting targets and reducing false detections.

Keywords

multispectral object detection; Transformer; cross-attention mechanism; YOLO; object detection

References

  1. [1] Luo Huilan, Peng Shan, Chen Hongkun A review of the latest research progress on difficult problems in object detection [J]. Computer Engineering and Applications, 2021, 57 (5): 36-46
  2. [2] Hu Jun, Yu Wenpeng Research progress on YOLO series algorithms for target detection in drone aerial photography [J]. Modern Information Technology, 2025, 9 (20): 27-34
  3. [3] WAGNER J，FISCHER V，HERMAN M， et al. Multispectral Pedestrian Detection using Deep Fusion Convolutional Neural Networks [C]//Proceedings of 24th European Symposium on Artificial Neural Networks.Bruges：[s.n.]，2016：509-514.
  4. [4] LIU J J，ZHANG S T，WANG S，et al. Multispectral Deep Neural Networks for Pedestrian Detection [J/OL].arXiv:1611.02644 [cs.CV].[2025-10-27].https://doi.org/10.48550/arXiv.1611.02644.
  5. [5] SHEN J F，CHEN Y F，LIU Y，et al. ICAFusion: Iterative Cross-Attention Guided Feature Fusion for Multispectral Object Detection[J/OL].arXiv:2308.07504 [cs.CV].[2025-10-03].https://doi.org/10.48550/arXiv.2308.07504.
  6. [6] FANG Q Y，HAN D P，WANG Z K. Cross-Modality Fusion Transformer for Multispectral Object Detection [J/OL].arXiv:2111.00273 [eess.IV].[2025-11-02].https://doi.org/10.48550/arXiv.2111.00273.
  7. [7] LI C Y，SONG D，TONG R F，et al. Illumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detection [J].Pattern Recognition，2019，85：161-171.
  8. [8] ZHANG L，ZHU X Y，CHEN X Y，et al. Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection [C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV).Seoul：IEEE，2019：5127-5137.
  9. [9] HWANG S，PARK J，KIM N，et al. Multispectral Pedestrian Detection: Benchmark Dataset and Baseline [C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Boston：IEEE，2015：1037-1045.
  10. [10] ZHANG L，LIU Z Y，ZHANG S F，et al. Cross-modality Interactive Attention Network for Multispectral Pedestrian Detection [J].Information Fusion，2019，50：20-29.
  11. [11] ZHANG H，FROMONT E，LEFÈVRE S，et al.Guided Attentive Feature Fusion for Multispectral Pedestrian Detection [C]//2021 IEEE Winter Conference on Applications of Computer Vision (WACV).Waikoloa：IEEE，2021：72-80.
  12. [12] VASWANI A，SHAZEER N，PARMAR N，et al. Attention is all You Need [C]//NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems.Long Beach：Curran Associates Inc.，2017：56000-6010.
  13. [13] DOSOVITSKIY A，BEYER L，KOLESNIKOV A，et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale [J/OL].arXiv:2010.11929 [cs.CV].[2025-09-26].https://doi.org/10.48550/arXiv.2010.11929.
  14. [14] SUN C，MYERS A，VONDRICK C，et al. VideoBERT: A Joint Model for Video and Language Representation Learning [C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV).Seoul：IEEE，2019：7464-7473.
  15. [15] Teledyne. Teledyne FLIR dataset for algorithm training [EB/OL]. [2021-11-03] https://www.flir.cn/oem/adas/adas-dataset-form/.
  16. [16] ZHANG H，FROMONT E，LEFEVRE S，et al. Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blocks [C]//2020 IEEE International Conference on Image Processing (ICIP).Abu Dhabi：IEEE，2020：276-280.
  17. [17] GitHub. ultralytics/yolov5. [EB/OL].[2023-08-21].https://github.com/ultralytics/yolov5.
  18. [18] ZHENG Z H，WANG P，REN D W，et al. Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation [J].IEEE Transactions on Cybernetics，2021，52（8）：8574-8586.
  19. [19] LIN M，CHEN Q，YAN S C. Network In Network [J/OL].arXiv:1312.4400 [cs.NE].[2025-10-19].https://doi.org/10.48550/arXiv.1312.4400.

About the authors

Zhu Jufeng (August 2000-), male, Hui ethnic, from Huainan, Anhui, is currently pursuing a master's degree in multimodal and object detection.