Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: Main Contribution:

This paper is trying to bridge the gap between CNN and the human  visual system by proposing a metric  (angular visual distance) and validate that this metric is correlated to the human visual hardness and this metric has a stronger relation  compared to the softmax score which has been viewed as a metric measuring the hardness of images in CNNs. Furthermore, this paper proposed a reasonable explanation for this observation, i.e., the norm is possibly not correlated to the human visual hardness and validate through the experiment. Finally, this paper shows that this metric is also useful  in other applications. 

Innovative Part:

The metric proposed in this paper is based on an interesting and also innovative observation that  samples in each class will concentrate in a convex cone in the embedding space (e.g., shown in Figure 1) and the norm has no information on the visual hardness. I like this observation since several existing theoretical results have similar implications although in far simpler settings. For example, [1] shows that for LINEAR model with logistic loss, gradient descent converges to the maximum margin classifier while the norm (corresponding to ||x||_2 in this paper) diverges to infinity with log(T) rate.  If we are looking into the Figure 1, we will see that ten convex cones almost form an equal partition of  the two-dimensional space and this indicates that the classifier is very similar to the classifier with the maximum margin in the angular space (NOT in the Euclidean space). The observation is quite intuitive and has strong theoretical foundation, which is the main reason that I vote for the acceptance of this paper. 

Drawbacks:

This paper also have several drawbacks but I do believe they can be addressed very easily. 

1. The introduction is not well-written, especially the second paragraph. I strongly recommend modifying the introduction. 

For the first three sentences of the second paragraph, do you mean that CNNs are constructed based on some properties of the human visual systems and thus they should have had some connections but they actually fundamentally differ in practice? Otherwise, if these two are fundamentally different with each other, what is the point of showing some connections between them? 

For the sentence "we use this dataset to verify our hypothesis", what is the hypothesis? Do you mean the hypothesis that human visual hardness should have had connections to the classifying hardness for CNNs?

 For the sentence "Given a CNN, we propose a novel score function that has strong correlation with human visual hardness", I am not sure whether the word "strong" can be used here. 

2. In table 1, I am not sure whether the author should assume that all audiences have some background on z-score, although I can understand it. I would also encourage the authors to use other correlation metrics with more intuitive explanations (e.g., correlation coefficients). 

3. For the experiment, I would like to recommend authors adding the following experiments.

3.1) Show that on other datasets (e.g., CIFAR 10, 100), AVH converges fast to a plateau while the norm constantly diverges to infinity. 
3.2) Introducing several other measurements to show the correlation. 
3.3) I also would like to see similar results in Table 1 for different models. 






[1] Soudry, Daniel, et al. "The implicit bias of gradient descent on separable data." The Journal of Machine Learning Research 19.1 (2018): 2822-2878.
review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: This paper defined Angular Visual Hardness (AVH) based on the angle between image feature embedding and the weights of the target class. The authors compared the correlation between AVH and human selection frequency with model confidence and feature norm. The results show that both AVH and model confidence have correlation, but AVH has a stronger correlation than model confidence. Differently from the conjecture of [41], feature norm was not correlated with human selection frequency. 

Next, the training dynamics of AVH are analyzed. The results show that feature norm keeps increasing during training, whereas AVH hits a plateau very early even when the accuracy or loss is still improving. Also, AVH correlates human selection frequency across different deep models, and it also correlates the model’s accuracies. 

As an application of AVH, the authors applied it to sample selection of self-training for domain adaption. The proposed selection method based on AVC could improve a state-of-the-art self-training method, of which sample selection is based on model confidence of CNN. 

Overall, the experimental contribution of this paper is good, and the experimental conditions seem to be correct. 

Minor problems. 
In the analysis of the dynamics of training, the authors compared the AVH with feature norm. How about the dynamics of model confidence? Is it similar to the feature norm? 

The curves of different levels of hardness are missing in Fig.14. 


review_title: Official Blind Review #2

final_score: rating: 1: Reject
review_content: The paper proposes, when given a CNN, an image and its label, a measure called angular visual hardness (AVH). The paper shows that AVH correlates with human selection frequency (HSF) [RRSS19].

Pros:
1. The authors experimented with a representative set of trained models in my opinion. (More on this in Con2)
2. In Section 6, the authors acknowledge a substantive counter-example/argument. (More on this in Con2) 

Cons (I put the ones that weighed the most on my decision first):
1. The presentation is confusing, and at times self-contradictory. For example, in Section 1, the paper asserts that “the two systems differ in what they view as hard examples that appear ambiguous or uncertain“ but then proceeds to claim that AVH being a CNN-derived quantity (more on this in Con2) correlates well with HSF. In fact, [RRSS19] (heavily cited here) seems to suggest exactly the opposite that harder examples for humans are also hard for CNNs. This is not very surprising as high accuracies of these CNNs imply their agreement with human judgment: we are learning a dataset labeled by humans. (More on this in Que2)
2. AVH is a function of a particular CNN (architecture and parameter values) and the target class label _in addition_ to the input image. These dependencies make AVH a measure of the ambiguity of an image very problematic. Granted that the paper presents evidence that AVH correlates with HSF for a number of _trained_ models but they will be of different values. 
3. The work is not well self-contained. HSF, the core quantity studied is not introduced with sufficient details. (See Que1 and Sug1)

Some possible mistakes/typos:
1. Feature vector x and class weight w in general do not lie on S^n. Indeed your definition of A(u, v) only relies on u, v being nonzero.
2. There is a missing { above Definition 1.
3. In Definition 1, “for any x” -> for any (x, y). 
4. In References, [10] is a duplicate of [11].
5. The captions in Figures 5, 6, and 7 in Appendix A might be wrong. They say ||x|| whereas the y-axis in the plots is labeled AVH.

Questions (I listed the important ones first):
1. What is human visual hardness (HVH)? How is HSF related to HVH? Why is being selected from a pool of images (in the procedure described in [RRSS19]) a good measure of HVH?
2. Since the class logit is exactly <x, w_c>, with arccos being a decreasing function, I expect AVH to behave very much like the opposite of model confidence (Definition 2). And this seems to be confirmed in Table 1 (performing a confidence calibration on validation set might increase this further). I wonder how AVH is different from model confidence _qualitatively_ and consequentially what insights do we gain (or should we expect to gain) by studying AVH instead of model confidence?
3. Degradation levels (DL) are mentioned early on but the experiments and figures were not shown in the main text (deferred till Appendix). What is the rationale? 
4. The middle row in Figure 3 has a small range of ~1e-4. Is that expected? Can you provide some simple arguments? The closeness of the initial and final values of AVH in the AlexNet plot also concerns me.
5. How is the visualization in Figure 1 generated? It is not immediately clear to me how the high dimensional space is projected into 2D. My concern is that though suggestive in the figure, the category weights w_c in general do not spread out evenly. Do they? I would suggest reporting the angular separation of the category weights (maybe by showing them in a CxC matrix).
6. In Figure 2, what happens to the dark stripes? Are there no data points with the specified range of HSF values?

Minor issues (factored little to none in my decision):
1. There are 60+ citations but their relevance to the current seems questionable in many cases. Many of them are accompanied by little or no technical comparison when they are mentioned. In particular, in Section 2 on the related work from psychology/neuroscience, little specifics are discussed to contextualize the current work. 
2. Many arguments come across as (highly) speculative and imprecise. As a result, I find the reasoning and logical story diluted and hard to follow.
3. The comparison with feature norm seems poorly motivated. The other quantities, namely AVH and model confidence, both depend on the class label. 
4. The term hardness has a rich history and connotation in the algorithmic analysis literature. I would suggest using a different term, as the hardness of a problem usually reflects some intrinsic aspects of its structure and not dependent on some algorithm.

Suggestions:
1. If DL is not important to the core results, it will help simplify and focus the presentation by leaving them out entirely.
2. Try to be more concise and more precise in the presentation. It might also benefit from more formalism wherever possible, and more procedural details, when human studies or notions is involved. The latter seems to be a lesson from [RRSS19] (in regard to reproducibility).

In summary, I do not recommend accepting the current article. 

(To authors and other reviewers) Please do not hesitate to directly point out my misunderstandings. I am open to acknowledging mistakes and revising my assessment accordingly.
review_title: Official Blind Review #1

Meta-review: This paper proposes a new measure for CNN and show its correlation to human visual hardness. The topic of this paper is interesting, and it sparked many interesting discussions among reviews. After reviewing each others’ comments, reviewers decided to recommend reject due to a few severe concerns that are yet to be address. In particular, reviewer 1 and 2 both raised concerns about potentially misleading and perhaps confusing statements around the correlation between HSF and accuracy. A concrete step was suggested by a reviewer - reporting correlation between accuracy and HSF. A few other points were raised around its conflict/agreement with prior work [RRSS19], or self-contradictory statements as pointed out by Reviewer 1 and 2 (see reviewer 2’s comment). We hope authors would use this helpful feedback to improve the paper for the future submission. 

Decision: Reject