Article Details

Abstract

Existing supervised methods based on category labels often suffer from coarse-grained supervision signals, which fail to capture subtle differences among samples within the same category, thus limiting their effectiveness in fine-grained retrieval tasks. To address this issue, this paper proposes a Fine-grained Semantic-guided Multi-modal Hashing (FSMH). By automatically mining the content semantic structures of multi-modal data, FSMH constructs fine-grained semantic supervision signals to guide the hash code learning process. Consequently, the generated hash codes can better preserve fine-grained semantic differences and correlations across modalities, achieving deep cross-modal fusion and accurate retrieval at the semantic level. Experiments on two widely used multi-modal datasets demonstrate that FSMH significantly outperforms current state-of-the-art unsupervised and supervised hashing methods across different hash code lengths, validating its effectiveness.

Keywords

fine-grained semantics; multi-modal hashing; semantic supervision; cross-modal retrieval

References

1. [1] Alexandr A，Piotr I. Near-optimal Hashing Algorithms for Approximate Nearest Neighbor in High Dimensions [J].Communications of the ACM，2008，51（1）：117-122.
2. [2] Liu X L，He J F，Lang B. Multiple Feature Kernel Hashing for Large-scale Visual Search [J].Pattern Recognition，2014，47（2）：748-757.
3. [3] Yang R，Shi Y L，Xu X S. Discrete Multi-view Hashing for Effective Image Retrieval [C]//Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval (ICMR)， Association for Computing Machinery，2017：175-183.
4. [4] Lu X，Zhu L，Li J J，et al. Efficient Supervised Discrete Multi-view Hashing for Large-scale Multimedia Search [J].IEEE Transactions on Multimedia，2019：2048-2060.
5. [5] Lu X，Zhu L，Cheng Z Y，et al. Flexible Online Multi-modal Hashing for Large-scale Multimedia Retrieval [C]//MM'19: Proceedings of the 27th ACM International Conference on Multimedia，2019：1129-1137.
6. [6] Liu L Y，Zhang Z，Huang Z. Flexible Discrete Multi-view Hashing with Collective Latent Feature Learning [J].Neural Processing Letters，2020，52：1765-1791.
7. [7] Lu X，Zhu L，Cheng Z Y，et al. Online Multi-modal Hashing with Dynamic Query-adaption [C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval，2019：715-724.
8. [8] Zhu L，Lu X，Cheng Z Y，et al. Deep Collaborative Multi-view Hashing for Large-scale Image Search [J].IEEE Transactions on Image Processing: a publication of the IEEE Signal Processing Society，2020，29：4643-4655.
9. [9] Zheng C Q，Zhu L，Lu X，et al. Fast Discrete Collaborative Multi-modal Hashing for Large-scale Multimedia Retrieval [J].IEEE Transactions on Knowledge and Data Engineering，2019：2171-2184.
10. [10] Yin Qiyue, Huangyan, Zhang Junge, etc A review of cross modal retrieval based on deep learning [J]. Chinese Journal of Image and Graphics, 2021, 26 (6): 1368-1388
11. [11] Chen Feng, Meng Zuqiang Research on Heterogeneous Multimodal Data Retrieval Based on Hash Algorithm [J]. Computer Science, 2019, 46 (10): 49-54

About the authors

Zheng Xiaozhi (1989-), male, Han nationality, from Weihai, Shandong, engineer, master's student, research direction: medical information retrieval, big language modeling; Corresponding author: Sun Zhuo (1981-), male, Han ethnicity, from Jinan, Shandong, assistant engineer, research direction: medical information retrieval, big language modeling.