Article Details

Abstract

To address the challenges of cross-modal semantic inconsistency and imprecise object boundaries in single-step prediction for RGB-Thermal salient object detection, a collaborative inference framework that advances from semantic consistency alignment to spatial structure refinement is proposed. First, a collaborative semantic mask estimation module is designed. By employing a pretrained Vision Transformer (ViT) to capture global semantic context, this module explicitly explores stable salient object categories across both modalities through cross-modal attention mechanisms and statistical co-occurrence metrics, generating semantically reliable initial prior masks to mitigate the bias caused by the dominance of a single modality. Subsequently, a diffusion-inspired iterative optimization module is proposed. Building upon the denoising principle of diffusion models, this module enables sufficient information propagation through multi-step global interactions in the latent space, thereby achieving progressive refinement of mask boundaries and enhancing spatial coherence. Experimental evaluations on multiple benchmark datasets and co-saliency detection tasks demonstrate that the proposed method significantly improves detection accuracy and generalization capability.

Keywords

RGB-T salient object detection; collaborative semantic mask; diffusion-inspired iterative optimization; cross-modal fusion

References

1. [1] Wu Jintao, Wang Anzhi, Ren Chunhong Overview of RGB-T Significant Object Detection [J]. Infrared Technology, 2025, 47 (1): 1-9
2. [2] Liu Z Y，Tan Y C，He Q，et al. SwinNet: Swin Transformer Drives Edge-Aware RGB-D and RGB-T Salient Object Detection [J].IEEE Transactions on Circuits and Systems for Video Technology，2022，32（7）：4486–4497.
3. [3] Chen G，Shao F，Chai X L，et al. Modality-Induced Transfer-Fusion Network for RGB-D and RGB-T Salient Object Detection [J].IEEE Transactions on Circuits and Systems for Video Technology，2023，33（4）：1787-1801.
4. [4] Liu Z Y，Huang X S，Zhang G H，et al. Scribble-Supervised RGB-T Salient Object Detection [C]//2023 IEEE International Conference on Multimedia and Expo (ICME)，2023：2369-2374.
5. [5] Lyu C T，Wan B，Zhou X F，et al. Lightweight Cross-Modal Information Mutual Reinforcement Network for RGB-T Salient Object Detection [J/OL].Entropy，2024，26（2）：130.https://doi.org/10.3390/e26020130.
6. [6] Yang Chao, Guan Zheng, Wang Xue, etc RGB-T saliency object detection with saliency enhancement and global perception [J]. Journal of Computer Aided Design and Graphics, 2025, 37 (8): 1359-1370
7. [7] Caron M，Touvron H，Misra I，et al. Emerging Properties in Self-Supervised Vision Transformers [C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV)，2021：9630-9640.
8. [8] Fan Q，Fan D P，Fu H Z，et al. Group Collaborative Learning for Co-Salient Object Detection [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)，2021：12283-12293.
9. [9] Zhang K H，Wu Y，Dong M L，et al. Deep Object Co-segmentation and Co-saliency Detection via High-order Spatial-semantic Network Modulation [J].IEEE Transactions on Multimedia，2022，25：5733-5746.
10. [10] Zhu Z Y，Zhang Z，Lin Z，et al. Co-salient Object Detection with Co-representation Purification [J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2023，45（7）：8193-8205.
11. [11] Su Y K，Deng J L，Sun R Z，et al. A Unified Transformer Framework for Group-based Segmentation: Co-segmentation, Co-saliency Detection and Video Salient Object Detection [J].IEEE Transactions on Multimedia，2023，26：313-325.
12. [12] Chakraborty S，Samaras D. Self-supervised Co-salient Object Detection via Feature Correspondences at Multiple Scales [C]/Computer Vision - ECCV 2024，2024：231-250.
13. [13] Wang G Z，Li C L，Ma Y P，et al. RGB-T Saliency Detection Benchmark: Dataset, Baselines, Analysis and a Novel Approach [C]//Image and Graphics Technologies and Applications(IGTA 2018)，2018：359-369.
14. [14] Tu Z Z，Xia T，Li C L，et al. RGB-T image Saliency Detection via Collaborative Graph Learning [J].IEEE Transactions on Multimedia，2020，22（1）：160-173.
15. [15] TU Z Z，MA Y，LI Z，et al. RGBT Salient Object Detection: A Large-Scale Dataset and Benchmark [J].IEEE Transactions on Multimedia，2022，25：4163-4176.
16. [16] Zhang Z，Jin W D，Xu J，et al. Gradient-induced Co-saliency Detection [C]//Computer Vision - ECCV 2020，2020：455-472.
17. [17] Fan D P，Lin Z，Ji G P，et al.Taking a Deeper Look at Co-salient Object Detection [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)，2020：2916-2926.
18. [18] Zhang D，Han J，Li C，et al. Detection of Co-salient Objects by Looking Deep and Wide [J].International Journal of Computer Vision，2016，120（2）：215-232.
19. [19] Lin T-Y，Maire M，Belongie S，et al. Microsoft COCO: Common Objects in Context [C]//Computer Vision - ECCV 2014，2014：740-755.
20. [20] Fan D P，Chendg M M，Liu Y，et al. Structure-Measure: A New Way to Evaluate Foreground Maps [C]//Proceedings of the IEEE International Conference on Computer Vision，2017：4558-4567.
21. [21] Fan D P，Gong C，Cao Y，et al. Enhanced-alignment Measure for Binary Foreground Map Evaluation [C]//Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence，2018：698-704.
22. [22] Achanta R，Hemami S，Estrada F，et al. Frequency-tuned Salient Region Detection [C]//2009 IEEE Conference on Computer Vision and Pattern Recognition，2009：1597-1604.
23. [23] Cheng M M，Warrell J，Lin W Y，et al. Efficient Salient Region Detection with Soft Image Abstraction [C]//2013 IEEE International Conference on Computer Vision，2013：1529-1536.

Funding

Scientific Research Plan Project of Hubei Provincial Department of Education (B2024187); Key Project of Hanjiang Normal University Science Research Program "Research on Collaborative Segmentation Method Based on Deep Learning" (2024A05)

About the authors

Chen Yasong (1993-), male, Han nationality, from Shiyan, Hubei, teaching assistant, master's student, research direction: computer vision, multimodal learning; Tan Wei (1995-), male, Tujia ethnic group, from Enshi, Hubei, teaching assistant, master's student, research direction: medical image segmentation, multimodal learning; Guo Huimin (1992-), female, Han ethnicity, from Xiangyang, Hubei, teaching assistant, master's student, research direction: computer applications, big language modeling; Wang Mengge (1995-), female, Han nationality, from Pingdingshan, Henan Province, lecturer, master's student, research direction: graphic image processing, machine learning; Yang Jing (1980-), female, Han ethnicity, from Shiyan, Hubei, associate professor, undergraduate, research direction: network and information security.