Paper:
LEARNING WHAT AND WHERE TO ATTEND
Most recent gains in visual recognition have originated from the inclusion of attention mechanisms in deep convolutional networks (DCNs). Because these networks are optimized for object recognition, they learn where to attend using only a weak form of supervision derived from image class labels. Here, we demonstrate the benefit of using stronger supervisory signals by teaching DCNs to attend to image regions that humans deem important for object recognition. We first describe a large-scale online experiment (ClickMe) used to supplement ImageNet with nearly half a million human-derived "top-down" attention maps. Using human psychophysics, we confirm that the identified top-down features from ClickMe are more diagnostic than "bottom-up" saliency features for rapid image categorization. As a proof of concept, we extend a state-of-the-art attention network and demonstrate that adding ClickMe supervision significantly improves its accuracy and yields visual features that are more interpretable and more similar to those used by human observers.

Review:
This paper proposes a new approach to use more informative signals (than only class labels), specifically, regions humans deem important on images, to improve deep convolutional neural networks. They collected a large dataset by implementing a game on clickme.ai and showed that using this information results in both i) improved classification accuracy and ii) more interpretable features. 
I think this is good work and should be accepted. The main contribution is three fold: i) a publicly available dataset that many researchers can use, ii) a network module to incorporate this human information that might be inserted into many networks to improve performance, and iii) some insights on the effect of such human supervision and the relation between features that humans deem important to those that neural nets deem important. 
Some suggestions on how to improve the paper:
1. I find Sections 3 & 4 hard to track - some missing details and notation issues. Several variables are introduced without detailing the proper dimensions, e.g., the global feature attention vector g (which is shown in the figure actually). The relation between U and u_k isn't clear. Also, it will help to put a one-sentence summary of what this module does at the beginning of Section 3, like the last half-sentence in the caption of Figure 3. I was quite lost until I see that. Some more intuition is needed, on W_expand and W_shrink; maybe moving some of the "neuroscience motivation" paragraph up into the main text will help. Bold letters are used to denote many different things - in  Section 4 as a set of layers, in other places a matrix/tensor, and even an operation (F). 
2. Is there any explanation on why you add the regularization term to every layer in a network? This setup seems to make it easy to explain what happens in Figure 4. One interesting observation is that after your regularization, the GALA features with ClickMe maps exhibit minimal variation across layers (those shown). But without this supervision the features are highly different. What does this mean? Is this caused entirely by the regularization? Or there's something else going on, e.g., this is evidence suggesting that with proper supervision like human attention regions, one might be able to use a much shallower network to achieve the same performance as a very deep one?
3. Using a set of 10 images to compute the correlation between ClickMe and Clicktionary maps isn't ideal - this is even less than the number of categories among the images. I'm also not entirely convinced that "game outcomes from the first and second half are roughly equal" says much about humans not using a neural net-specific strategy, since you can't rule out the case that they learned to play the game very quickly (in the first 10 of the total 380 rounds). 
4. Title - this paper sound more "human feedback" to me than "humans-in-the-loop", because the loop has only 1 iteration.  Because you are collecting feedback from humans but not yet giving anything back to them. Maybe change the title?