Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:
The paper propose a method for generating adversarial examples in image recognition problems. The Adversarial scheme is inspired in the one proposed by Goodgellow  et al 2015 (AT) that introduces small perturbations to the data in the direction that increases the error. Such a perturbations are random (they have not structure) and lack of interpretation for a human user. The proposal is to limit the perturbations to just three kind of global motion fields: shift, centered rotation and scale (zoom in/out). Since the motions are small in scale, the authors use a first-order Taylor series approximation  (as in classical optical flow). This approximation allows to obtain close formulas for the perturbed examples; i.e. the correction factor of the Back-propagation computed derivatives w.r.t. original example. As result, the method is computational efficient respect to the AT and the perturbations are interpretable. 
Experiments demonstrate that with the MNIST database is not obtained an improvement in the error reduction but a reduction of the computational time. However, with ta more general recognition problem conducted with the CIFAR-10 database, the use of the proposed method improves both the error and the computational time, when compared with AT and Virtual Adversarial Train. 

Comments:

1. The paper presents a series os typos: FILED (title), obouve, freedm, nerual,; please check carfully.

2. The Derivation of eq. (13) should be explained, It could be said that (12) can be casted as a eigenvalue problem [for example: $ max_{\tilde v} \| \nabla_p L^T \tilde v \|^2 \;\; s.t. \| v\|=1  $] and (13) is the largest eigenvalue of $ \nabla_p L \nabla_p L^T $]

3. The improvement in the error results in the db CIFAR-10 is good enough to see merit in the proposal approach. Maybe other perturbations with closed formula could be considered and linear combinations of them
review_title: Train improvement by adversarial trail examples generated with differential motion fields 

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper tackles the overfitting problem when training neural networks based on regularization technique. More precisely, the authors propose new regularization terms that are related to the underlying virtual geometrical transformations (shift, rotation and scale) of the input data (signal, image and video). By formalizing the geometrical transformation process of a given image, the authors deduce constraints on the objective function which depend on the magnitude of the applied transformation. The proposed method is compared to three methods: one baseline and two methods of the literature (AT and VAT). The comparison is done on three datasets (synthetic data, MNIST and CIFAR10) in terms of test errors (for classification problems) and running time.

The paper is well formalized and the idea is interesting. The regularization approach is novel compared to the methods of the literature. 

Main concerns: 
1)	The experimental validation of the proposed approach is not consistent:
The description of the baseline method is not detailed in the paper. 
A priori, the baseline should naturally be the method without your regularization terms.
But, this seems to be contrary with what you displayed in Figure 3. 
Indeed, in Figure 3, there is three different graphs for the baseline method (i.e., one for each regularization term). It seems that the baseline method depends on the different kinds of regularization term, why? Same question for AT and VAT methods. 
In practice, what is the magnitude of the perturbations? 
Please, explain the axis of all the figures. 
Please, explain how do you mix your different regularization terms in your method that you call VMT-all? 
All the following points are related to the experiment for which you presented the results in Table 2: 
Please, provide the results of all your methods on the synthetic dataset (only VMT-shift is provided). What is VMF? Do you mean VMT? 
For the evaluations, it would be more rigorous to re-implement also the state-of-the-art methods for which you only give the results that they report in their paper. Especially, because you re-implemented AT with L-2 constraint, so, it seems straightforward to re-implement also AT with L-infinite constraint. Same remark for the dropout regularization technique, which is easy to re-implement on the dense layers of your neural networks, within the Tensorflow framework. 
As you mentioned, your main contribution is related to running time, thus, you should give the running time in all experiments. 

2)	The method seems to be a tradeoff between accuracy and running time:
The VAT method performs better than all your methods in all the datasets. 
The baseline method is faster than all the methods (Table 3). 
This being said, the proposed method should be clearly presented in the paper as a tradeoff between accuracy and running time. 
3)	The positioning of the proposed approach is not so clear:  
As mentioned above, your method is a tradeoff between accuracy and running time. But you also mentioned (top of page 2) that the contribution of your paper is also related to the interpretability in terms of ‘’Human perception’’. Indeed, you clearly mentioned that the methods of the literature lacks interpretability. You also mentioned that your method is more ‘’geometrically’’ interpretable than methods of the literature. The link between interpretability in terms of “human perception” and “geometry” is not obvious. Anyway, the interpretability point is not sufficiently demonstrated, or at least, discussed in the paper. 

4)	Many typos in the paper : 
Section 1: “farward-backward”
Section 2.1: “we define the movement field V of as a n+1…”
Section 2.2: “lable” - “the another” - “of how it are generated” – Sentence “Since V is normalized.” seems incomplete… - \mathcal{L} not defined - Please, precise the simplifications like \mathcal{L}_{\theta} to \mathcal{L} 
Section 3: “DISCUSSTION”
Section 4.1: “negtive”
Figure 2: “negetive”
Table 2: “VMF”
Section 4.2: “Tab 2.3” does not exist 
Section 4.3: “consists 9 convolutional” – “nerual networks”…
Please, always use the \eqref latex command to refer to equations.
There is many others typos in the paper, so, please proofread the paper…

review_title: The paper is well formalized and the idea is interesting. The regularization approach is novel compared to the methods of the literature. However, the experimental validation of the proposed approach is not consistent and the positioning of the proposed approach is not so clear. So, I  suggest to reject the paper in his actual form.

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes to regularize neural networks by the invariance to certain types of transforms. This is framed into a minimax problem, which yields a closed form regularization when constrained to simple types of transforms. 

The basic idea of using derivative to measure sensitivity has been widely known, and is related to tangent propagation and influence function. Please comment on the connection and difference. What is the substantial novelty of this current approach? 

The empirical results are not particularly impressive. The performance is not as good as  (and seems significantly worse than) AT and VAT on MNIST. Could you provide an explanation? On CIFAR10, VMT-all is only comparable with VAT. Although VMT is faster than VAT, it seems not a significant advantage since is not faster in a magnitude. 

The writing need to be significantly improved. Currently there are lot of typos and grammar errors, e.g., \citep vs. \citet; randon, abouve, batchszie;  \mathcal{Z}^n is undefined when it first appears.

In VMT-all, how do you decide the relative importance of the three different regularizations?  

Is Figure 3 the regularization on the training or testing set? Could you explain why it reflects generalization ability? 
review_title: Review

Meta-review: R1 thought the proposed method was novel and the idea interesting. However, he/she raised concerns with consistency in the experimental validation, the trade-off between accuracy and running time, and the positioning/motivation, specifically the claim about interpretability. The authors responded to these concerns, and R1 upgraded their score. R2 didn’t raise major concerns or strengths. R3 questioned the novelty of the work and the experimental validations. All reviewers raised concerns with the writing. Though I think the work is interesting, issues raised about experiments and writing make me hesitant to go against the overall recommendation of the reviewers, which is just below the bar. I think this is a paper that could make a good workshop contribution.

Decision: Invite to Workshop Track