Article Details

Title · Authors · Abstract · Keywords · DOI · References

Clothing Image Conditional Retrieval Based on Cross-Attention

TIAN Liren, ZHANG Xiaoyun

(School of Computer, Hunan University of Technology, Zhuzhou 412007, China)

Section：Artificial Intelligence · Vol. 2025, Issue 6 · Pages：83-88 · 2096-4706（2025）06-0083-06 · CLC number：TP391.4 · Document code：A · DOI：10.19850/j.cnki.2096-4706.2025.06.016

Abstract

Clothing image conditional retrieval refers to the use of text as feedback to guide the adjustment of clothing image content, and using it as a retrieval condition to achieve the purpose of retrieval. The key to the task of clothing image conditional retrieval is how to effectively combine the semantic space of clothing images and text. To this end, this paper proposes a hybrid feature cross-attention clothing image retrieval method using the multimodal pre-training model. Firstly, the multimodal pre-training model is fine-tuned to make it suitable for downstream tasks. Secondly, the cross-attention mechanism is used to obtain the fusion features. Finally, experiments are carried out on the FashionIQ dataset. The results show that the retrieval recall rate of this method is higher and the retrieval results are more accurate.

Keywords

clothing image; conditional retrieval; cross-attention; multimodal fusion

References

  1. [1] CHEN N，YANG D，LI MENGLU. Survey on Clothing Image Retrieval with Cross-domain [J]. Complex & Intelligent Systems，2022，8（6）：5531-5544.
  2. [2] WU H，GAO Y P，GUO X X，et al. Fashion IQ:A New Dataset Towards Retrieving Images by Natural Language Feedback [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Nashville：IEEE，2021：11302-11312.
  3. [3] Zhang Shunyao, Li Huawang, Zhang Yonghe, etc Image retrieval algorithm based on independent attention mechanism [J]. Computer Science, 2023, 50 (S1): 328-333
  4. [4] DALAL N，TRIGGS B. Histograms of Oriented Gradients for Human Detection [C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).San Diego：IEEE，2005，886-893.
  5. [5] LIU Z W，LUO P，QIU S，et al. DeepFashion:Powering Robust Clothes Recognition and Retrieval with Rich Annotations [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas：IEEE，2016，1096-1104.
  6. [6] Miao Lanxin, Lei Yu, Zeng Pengpeng, etc Image text retrieval network based on granularity perception and semantic aggregation [J]. Computer Science, 2022 (11): 134-140
  7. [7] VO N，JIANG L，SUN C，et al. Composing Text and Image for Image Retrieval-an Empirical Odyssey [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Beach：IEEE，2019：6432-6441.
  8. [8] CHEN Y B，BAZZANI L. Learning Joint Visual Semantic Matching Embeddings for Language-guided Retrieval [C]//Computer Vision–ECCV 2020:16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16.Cham：Springer，2020：136-152.
  9. [9] CHEN Y B，GONG S G，BAZZANI L. Image Search with Text Feedback by Visiolinguistic Attention Learning [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Seattle：IEEE，2020：2998-3008.
  10. [10] LEE S，KIM D，HAN B. Cosmo:Content-style Modulation for Image Retrieval with Text Feedback [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Nashville：IEEE，2021：802-812.
  11. [11] DODDS E，CULPEPPER J，HERDADE S，et al. Modality-agnostic Attention Fusion for Visual Search with Text Feedback [J/OL].arXiv:2007.00145 [cs.CV].（2020-6-30）.https://arxiv.org/abs/2007.00145.
  12. [12] YANG Y C，WANG M，ZHOU W G，et al. Cross-modal Joint Prediction and Alignment for Composed Query Image Retrieval [C]//MM'21:Proceedings of the 29th ACM International Conference on Multimedia. Chengdu：ACM，2021：3303-3311.
  13. [13] GU C B，BU J J，ZHANG Z，et al. Image Search with Text Feedback by Deep Hierarchical Attention Mutual Information Maximization [C]//MM'21:Proceedings of the 29th ACM International Conference on Multimedia. Chengdu：ACM，2021：4600-4609.
  14. [14] BALDRATI A，BERTINI M，URICCHIO T，et al. Conditioned and Composed Image Retrieval Combining and Partially Fine-tuning CLIP-based Features [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). New Orleans：IEEE，2022：4955-4964.
  15. [15] ZHAO Y D，SONG Y Q，JIN Q，Progressive Learning for Image Retrieval with Hybrid-Modality Queries [C]//SIGIR'22:Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. Madrid：ACM，2022：1012-1021.
  16. [16] RADFORD A，KIM J W，HALLACY C，et al. Learning Transferable Visual Models From Natural Language Supervision [J/OL].arXiv:2103.00020 [cs.CV].（2021-2-26）.https://arxiv.org/abs/2103.00020.
  17. [17] OORD A V D，LI Y Z，VINYALS O. Representation Learning with Contrastive Predictive Coding [J/OL].arXiv:1807.03748 [cs.LG].（2018-7-10）.https://arxiv.org/abs/1807.03748.
  18. [18] GOODFELLOW I，POUGET-ABADIE J，MIRZA M，et al. Generative Adversarial Networks [J].Communications of the ACM，2020，63（11）：139-144.

About the authors

Tian Liren (1998-), male, Han ethnicity, born in Dianjiang, Chongqing, is a graduate student studying computer vision.