Paper:
Greedy Hash: Towards Fast Optimization for Accurate Hash Coding in CNN
To convert the input into binary code, hashing algorithm has been widely used for approximate nearest neighbor search on large-scale image sets due to its computation and storage efficiency. Deep hashing further improves the retrieval quality by combining the hash coding with deep neural network. However, a major difficulty in deep hashing lies in the discrete constraints imposed on the network output, which generally makes the optimization NP hard. In this work, we adopt the greedy principle to tackle this NP hard problem by iteratively updating the network toward the probable optimal discrete solution in each iteration. A hash coding layer is designed to implement our approach which strictly uses the sign function in forward propagation to maintain the discrete constraints, while in back propagation the gradients are transmitted intactly to the front layer to avoid the vanishing gradients. In addition to the theoretical derivation, we provide a new perspective to visualize and understand the effectiveness and efficiency of our algorithm. Experiments on benchmark datasets show that our scheme outperforms state-of-the-art hashing methods in both supervised and unsupervised tasks.

Review:
The paper presents a greedy approach to train a deep neural network to directly produce binary codes that build on the straight through estimator.  During forward propagation the model uses the sgn output whereas at the back-propagation stage it passes derivatives as it the output were a simple linear function.  There are relevant papers that already proposed such an approach and that are not referred to as earlier work: [1] Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation, Bengio Y. et al. [2] Techniques for learning binary stochastic feedforward neural networks, Tapani R. et al.  The experimental setting is not very clear and I would suggest the authors to better explain  the supervised setting. Do they produce a binary code of length k and then classify it  with a single final output layer? If so then I would like to see an ablation study on this. What is the relative contribution of using this loss vs the contrastive or triplet loss that is  used in many of the other approaches?   What is the influence of the pre-trained network? What would be the performance if the method were to be trained from scratch end-to-end? This would allow for a better assessment of the greedy relaxation proposed here proposed for binarizing the output layer.  Regarding other works that have tackled the problem of hashing with neural networks I see missing: [3] Multimodal similarity-preserving hashing, Masci J. et al. [4] Semantic Hashing, Salakhutdinov R. et al  The comparison in 3.5 claims a performance margin that is larger at short code lengths wrt other methods,  however I see only one of the algorithms in the figure. Please report more.  Overall an interesting application paper that is clearly written, but that should include further references and  readjust the contribution section considering that the straight through is not novel.   ============================================================== After having read the rebuttal I decided to increase my score as the authors addressed all my concerns.