基于特征增强的对抗哈希跨模态检索

Feature Boosting Adversarial Hashing for Cross-Modal Retrieval

关键词:  特征增强  跨模态检索  稀疏矩阵  哈希子空间学习  对抗学习

Key words:  feature boosting  cross-modal retrieval  sparse matrix  hashing subspace learning  adversarial learning

摘要:

在跨模态检索任务中，哈希方法由于其检索效率高效、储存成本低廉而被广泛应用。但是，这些方法很少关注如何去弥补主体网络将高维特征转换为哈希码的过程中所丢失的特征信息。为解决这些问题，本文提出了一种特征增强对抗跨模态哈希（Feature Boosting Adversarial Hashing for Cross-Modal，FBAH）方法。FBAH方法将子空间学习与对抗学习相结合，来减少不同模态数据的差异性。另外，构造一种类残差模块，它可以将筛选出具有区别性的特征绕过主体网络直接输入到哈希空间进行特征增强。这样，生成的哈希码能够具有更多的原始特征信息。最后，通过带有分支网络的线性分类器在标签空间进行两种方式的预测，并最小化与真实标签的差距来保证语义的不变性。本文选择两个跨模态检索任务中常用的大型数据集进行大量实验，结果表明FBAH方法的性能优于目前7种较为先进的跨模态哈希方法。

Abstract:

In cross-modal retrieval tasks, hashing method is widely used because of its high retrieval efficiency and low storage cost.However, these methods pay little attention on how to compensate for the loss of feature information in the process of transforming high-dimensional features into hash codes.To solve these problems, this article proposes a Feature Boosting Adversarial Hashing for Cross-Modal (FBAH).FBAH combines subspace learning with adversarial learning to reduce the difference of different modal data.In addition, a module similar to residual structure is constructed, which can bypass the main network and directly input the selected distinguishing features into the hash space for feature boosting.In this way, the generated hash code can have more original feature information.Finally, the linear classifier with branch network is used to make two kinds of prediction in label space, and the gap with the real label is minimized to ensure the invariance of semantics.In this article, two large datasets commonly used in cross-modal retrieval tasks are selected for a large number of experiments.The results show that the performance of FBAH is superior to seven existing advanced cross-modal hashing methods.