Venue: NeurIPS 2020 Workshop SVRHM

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Here the authors show that using a reward-based optimization function in a categorization task can lead to better performance of a DNN with adversarial examples than more traditional approaches. The paper is relatively clear, and the result is rather modest but interesting. I only have a couple of comments about how this paper/poster could be improved:

- The writing is very unclear, full of typos and grammatical errors. The paper needs to be thoroughly proofread before acceptance. Examples in lines 32, 33, 35, 51, 68, 98, 167, 169, 171, 172, 173.

- Figure 2 is not useful to understand the model architecture. It consist of mostly text and full of terms (acronyms? function names?) that are not explained in the figure caption. A better graphical representation of the model architecture, together with explanation of terms in the caption would improve the paper greatly.

- The final result is rather modest, and there is no discussion of the prior relevant literature combining RL with deep learning. For example, some researchers have used deep learning for state representation learning in RL (e.g., Bohmer et al., 2015; de Bruin et al., 2018). Because the authors do not offer links to that literature, it is not clear how their approach relates to those previous efforts and to what extent it is novel.
review_title: Review of "Reinforcement Based Learning on Classification Task Could Yield Better Generalisation and Adversarial Accuracy"

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper examines the generalization properties and adversarial robustness in CNNs trained via a reinforcement learning method (vanilla policy gradient) with those trained using the cross-entropy loss. The paper reports higher adversarial accuracy and smaller generalization gap in the neural network model trained with reinforcement learning. 

Complementing supervised training of neural networks with RL-based methods is a promising approach. However, I have some concerns with the way the experiments are conducted in this paper and I would like to see them resolve before this paper is presented in this venue. 

My main concern is regarding the control CE model trained on FGSM attack. The FGSM attack has been shown to be unsuitable for adversarial training (AT) due to "label leaking" [1]. Current AT methods unanimously advise to train on iterative attacks like PGD-Linf. My concern is also amplified by the results reported in Table-1 where the CE model's adversarial performance seems to be getting substantially worse with more training. 

Other comments/concerns: 

It is claimed that: "Humans do not learn to classify on the basis error between "target label" and "predicted label" but instead they predicts a label and then some other people give them feedback which results into a positive or negative reward for the learner and on the basis of this reinforcements they learn to classify different stimuli." The true objective and learning algorithm in the human brain is being debated and NOT known. RL is only one of these mechanisms and potentially not the primary one in the sensory cortices.

line79: How was the checkpoint picked? My impression is that the checkpoints might have been selected based on the test accuracy which will be incorrect. In any case, this should be clarified here. 

Several statements in the abstract are unclear: 
- "An empirical evaluation on cifar10 dataset showed that our method outperforms the same model architecture trained using cross-entropy loss function" --> outperformed at what? 
- "At the same time, our method generalizes better to the training data" --> do you mean generalizes better to the TEST data?

The paper needs better proof-reading, there are more than few typos or grammatical mistakes in the text. e.g. in abstract: "on the basis error" --> "on the basis of error", line68: fasten --> improve? 

[1] Kurakin, Alexey, Ian Goodfellow, and Samy Bengio. "Adversarial machine learning at scale." arXiv preprint arXiv:1611.01236 (2016).
review_title: Interesting idea but experiments may need to be improved

final_score: rating: 7: Good paper, accept
review_content: This work investigates using a policy loss from Reinforcement Learning (RL) to train an image-classifying CNN on CIFAR10. The authors suggest that an RL policy loss may be closer to the way humans learn compared to crossentropy. This network shows some improvements in robustness compared to a network trained using the classic crossentropy loss -  it has higher adversarial robustness and less difference between training and testing error. However, it does not quite reach performance level of the crossentropy trained network on non-adversarial images. 

I think this approach is interesting and fits the SVRHM workshop well. The experiments are generally well done. Therefore, I suggest accepting this contribution. I have some suggestions to potentially improve the paper. 

-	None of the networks, including the crossentropy one, surpass 60% test accuracy on CIFAR10. Why is this? The state of the art is 1-2% error rate, and to my understanding it is easy nowadays to reach at least 80%. This low performance may suggest that the crossentropy network trains suboptimally, which would question the results in general. Can the authors explain this low performance and show that it is not a problem? If it comes from the network architecture, why not choose a better performing architecture?
-	There is “individual variability” in deep learning, i.e., the initial weights of a network impact performance. This is especially true in deep RL. There is a concern that the initialization, and not the training procedure, might drive the results here. Comparing networks with the same random seed would fix this problem. Alternatively, several networks could be trained and the average & standard deviation of the performance could be plotted.
-	The authors suggest that their approach might be closer to how humans learn. Is there any scientific evidence for this?
-	The clarity of presentation could also be improved.

review_title: Review for Reinforcement Based Learning on Classification Task Could Yield Better Generalisation and Adversarial Accuracy 

Decision: Accept (Poster)