Kati8/ICLR_2023_2025 huggingface dataset:
authors: Wei Tao
authors: Lei Bao
authors: Sheng Long
authors: Jun Chen
authors: Qing Tao
reviews: 
Comments: The paper proposes a new adversarial attack algorithm based on gradient clipping, momentum, and normalized gradient. The algorithm is proven to converge and experiments show it has superior performance than existing attack algorithms.
Strengths: 1. The paper extends gradient clipping into the setting of adversarial attack which is a novel contribution.
2. Theoretical analysis is provided for the proposed algorithm to show it is convergent.
Weaknesses: 1. The paper mentioned a few times that signsgd is non-convergent, but did not provide any example of its non-convergent. The claim should be consolidated and more importantly, for the adversarial attack problem. My intuition tells me signsgd might be convergent for adversarial attack problems with box (l1) constraint, it may diverge for l2 constraints.
2. Gradient clipping is shown to accelerate training theoretically in prior works, I missed any solid acceleration discussions in the paper.
Recommendation: 5: marginally below the acceptance threshold
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 3 good
reviews: 
Comments: The authors propose a clipped momentum method (clipped NGM), in which the normalized-gradient momentum (NGM) is clipped as its update direction with a radius-varying clipping rule.  They show that the clipped NGM attains its averaging convergence for general
constrained convex problems. Numerical experiments are given to demonstrate the efficiency of the clipped NGM.
Strengths: Numerical experiments and comparisons with the sign-momentum MI-FGSM are given in some adversarial attack problems.
Weaknesses: The basic idea of the proposed algorithm, clipped NGM, looks a bit unnatural to me. The authors state that they propose the algorithm based on replacing the sign operator in the sign-momentum algorithm-MIFGSM by a clipping operator. As we all know, in the sign-momentum-based algorithm, using the sign operator, the normalization of the gradient in the momentum update is natural as one would lose the magnitude information once taking the sign operator. For the clipped NGM, it looks like the normalization of the gradient could be unnecessary, as one already has another energy-clipping operator in the iterative updates.  Without considering the normalization gradient step in the momentum updates, the algorithm looks similar to the one studied in (Mai & Johansson, 2021). From this point of view, numerical comparisons with the clipped momentum algorithm by  (Mai & Johansson, 2021) should be better given.

The presentation could be further improved. The introduction of the motivation and the algorithms looks like a composition of pieces of different algorithms in different areas.  

In the introduction of the background of some related algorithms, the authors mention Adam algorithm and say that the drawback of Adam is its non-convergence issue. However, to my best knowledge, Adam indeed can converge in some parameter settings. Besides, Adam is one of the popular algorithms in ML area. However, the authors do not provide any numerical comparisions with Adam algorithm. 

The derived theoretical results are for the convex constraints problems with bounded gradient and domain assumptions, which is a bit restricted in a world filled with non-convex optimization problems.
Recommendation: 5: marginally below the acceptance threshold
Reviewer confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 2 fair
reviews: 
Comments: This paper proposes a clipped normalized-gradient heavy-ball momentum method for generating adversarial examples. Moreover, the authors also provide the convergence analysis of the proposed algorithm. Some experimental results show the performance of the proposed method.
Strengths: 1. This paper proposes a clipped normalized-gradient heavy-ball momentum method for generating adversarial examples. 
2. Moreover, the authors also provide the convergence analysis of the proposed algorithm. 
3. Some experimental results show the performance of the proposed method.
Weaknesses: Although the paper is theoretically and experimental sound, there are still some questions need to be discussed in this paper:
1.	The novelty of this paper is limited. In other words, the main techniques used in this paper include one clipped technique and a heavy-ball momentum acceleration method. However, all the techniques are widely used in previous works. 
2.	The convergence analysis of the proposed algorithm is provided for the constrained convex problem. But in fact, the problem of generating adversarial examples is non-convex. Therefore, the authors should provide the convergence analysis of the proposed algorithm for solving both convex and non-convex optimization problems.
3.	Eq. (15) may be questionable. Note that the clipping operator in Eq. (8) should be element-wise, and thus Eq. (15) includes the element-wise product of two vectors.
4.	The experimental results are not convincing. The authors should compare the proposed algorithm with more recently proposed algorithms.
5.	Both the English language and equations in this paper need to be improved. For instance, what’s the definition of the (L_0, L-1)-smooth function?
Recommendation: 5: marginally below the acceptance threshold
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 2 fair
Clarity: 3 good