Deep code operation network for multilabel image retrieval Abstract   Deep hashing methods have been extensively studied for largescale image search and achieved promising results in recent years. However, there are two major limitations of previous deep hashing methods for multilabel image retrieval: the first one concerns the flexibility for users to express their query intention (socalled the intention gap), and the second one concerns the exploitation of rich similarity structures of the semantic space (socalled the semantic gap). To address these issues, we propose a novel Deep Code Operation Network (CoNet), in which a user is allowed to simultaneously present multiple images instead of a single one as hisher query, and then the system triggers a series of code operators to extract the hidden relations among them. In this way, a set of new queries are automatically constructed to cover usersxe2x80x99 real complex query intention, without the need of explicitly stating them. The CoNet is trained with a newly proposed marginadaptive triplet loss function, which effectively encourages the system to incorporate the hierarchical similarity structures of the semantic space into the learning procedure of the code operations. The whole system has an endtoend differentiable architecture, equipped with an adversarial mechanism to further improve the quality of the final intention representation. Experimental results on four multilabel image datasets demonstrate that our method significantly improves the stateoftheart in performing complex multilabel retrieval tasks with multiple query images.

Deep Multilabel Hashing for Image Retrieval Due to its low storage cost and fast query speed, hashing has been widely applied to approximate nearest neighbor search for largescale image retrieval, while deep hashing further improves the retrieval quality by learning a good image representation. However, existing deep hash methods simplify multilabel images into singlelabel processing, so the rich semantic information from multilabel is ignored. Meanwhile, the imbalance of similarity information leads to the wrong sample weight in the loss function, which makes unsatisfactory training performance and lower recall rate. In this paper, we propose Deep MultiLabel Hashing (DMLH) model that generates binary hash codes which retain the semantic relationship of multilabel of the image. The contributions of this new model mainly include the following two aspects: (1) A novel sample weight calculation model adaptively adjusts the weight of the sample pair by calculating the semantic similarity of the multilabel image pairs. (2) The sample weight crossentropy loss function, which is designed according to the similarity of the image, adjusts the balance of similar image pairs and dissimilar image pairs. Extensive experiments demonstrate that the proposed method can generate hash codes which achieve better retrieval performance on two benchmark datasets, NUSWIDE and MSCOCO.