wanyu/IteraTeR_full_sent huggingface dataset:
Original:  Informally , we say that a point (x^\star, y^\star) is an \varepsilon-greedy min-max equilibrium point of a function f: \mathbb{R^d \times \mathbb{R}^d \rightarrow \mathbb{R} }  if y^\star is a second-order local maximum for f(x^\star,\cdot) and, roughly, x^\star is a local minimum for a greedy optimization version of the function \max _y f(x, y ) which can be efficiently estimated using greedy algorithms.
Revised:  Informally , we say that a point (x^\star, y^\star) is an \varepsilon-greedy min-max equilibrium point of a function f: \mathbb{R^d \times \mathbb{R}^d \rightarrow \mathbb{R} } if y^\star is a second-order local maximum for f(x^\star,\cdot) and, roughly, x^\star is an \varepsilon-approximate local minimum for a greedy optimization version of the function \max _y f(x, y ) which can be efficiently estimated using greedy algorithms.
Edit intent: clarity
Original:  Informally , we say that a point (x^\star, y^\star) is an \varepsilon-greedy min-max equilibrium point of a function f: \mathbb{R^d \times \mathbb{R}^d \rightarrow \mathbb{R} }  if y^\star is a second-order local maximum for f(x^\star,\cdot) and, roughly, x^\star is a local minimum for a greedy optimization version of the function \max _y f(x, y ) which can be efficiently estimated using greedy algorithms.
Revised:  Informally , we say that a point (x^\star, y^\star) is an \varepsilon-greedy min-max equilibrium point of a function f: \mathbb{R^d \times \mathbb{R}^d \rightarrow \mathbb{R} } if y^\star is a second-order local maximum for f(x^\star,\cdot) and, roughly, x^\star is a local minimum for a "greedy approximation" to the function \max _y f(x, y ) which can be efficiently estimated using greedy algorithms.
Edit intent: clarity
Original:  Informally , we say that a point (x^\star, y^\star) is an \varepsilon-greedy min-max equilibrium point of a function f: \mathbb{R^d \times \mathbb{R}^d \rightarrow \mathbb{R} }  if y^\star is a second-order local maximum for f(x^\star,\cdot) and, roughly, x^\star is a local minimum for a greedy optimization version of the function \max _y f(x, y ) which can be efficiently estimated using greedy algorithms.
Revised:  Informally , we say that a point (x^\star, y^\star) is an \varepsilon-greedy min-max equilibrium point of a function f: \mathbb{R^d \times \mathbb{R}^d \rightarrow \mathbb{R} } if y^\star is a second-order local maximum for f(x^\star,\cdot) and, roughly, x^\star is a local minimum for a greedy optimization version of the function \max _z f(x, y ) which can be efficiently estimated using greedy algorithms.
Edit intent: meaning-changed
Original:  Informally , we say that a point (x^\star, y^\star) is an \varepsilon-greedy min-max equilibrium point of a function f: \mathbb{R^d \times \mathbb{R}^d \rightarrow \mathbb{R} }  if y^\star is a second-order local maximum for f(x^\star,\cdot) and, roughly, x^\star is a local minimum for a greedy optimization version of the function \max _y f(x, y ) which can be efficiently estimated using greedy algorithms.
Revised:  Informally , we say that a point (x^\star, y^\star) is an \varepsilon-greedy min-max equilibrium point of a function f: \mathbb{R^d \times \mathbb{R}^d \rightarrow \mathbb{R} } if y^\star is a second-order local maximum for f(x^\star,\cdot) and, roughly, x^\star is a local minimum for a greedy optimization version of the function \max _y f(x, z ) which can be efficiently estimated using greedy algorithms.
Edit intent: meaning-changed
Original:  Informally , we say that a point (x^\star, y^\star) is an \varepsilon-greedy min-max equilibrium point of a function f: \mathbb{R^d \times \mathbb{R}^d \rightarrow \mathbb{R} }  if y^\star is a second-order local maximum for f(x^\star,\cdot) and, roughly, x^\star is a local minimum for a greedy optimization version of the function \max _y f(x, y ) which can be efficiently estimated using greedy algorithms.
Revised:  Informally , we say that a point (x^\star, y^\star) is an \varepsilon-greedy min-max equilibrium point of a function f: \mathbb{R^d \times \mathbb{R}^d \rightarrow \mathbb{R} } if y^\star is a second-order local maximum for f(x^\star,\cdot) and, roughly, x^\star is a local minimum for a greedy optimization version of the function \max _y f(x, y ) which can be efficiently estimated using second-order optimization algorithms.
Edit intent: fluency
Original:  The existence follows from an algorithm that converges from any starting point to such a point in a number of gradient and function evaluations f(x,y), and \nabla^2_y f(x,y), } that is polynomial in \frac{1{\varepsilon}}  , the dimension d, and the bounds on f and its first three derivatives.
Revised:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and \nabla^2_y f(x,y), } that is polynomial in \frac{1{\varepsilon}} , the dimension d, and the bounds on f and its first three derivatives.
Edit intent: meaning-changed
Original:  The existence follows from an algorithm that converges from any starting point to such a point in a number of gradient and function evaluations f(x,y), and \nabla^2_y f(x,y), } that is polynomial in \frac{1{\varepsilon}}  , the dimension d, and the bounds on f and its first three derivatives.
Revised:  The existence follows from an algorithm that converges from any starting point to such a point in a number of gradient and function evaluations f(x,y), and \nabla^2_y f(x,y), } that is polynomial in {\varepsilon}} , the dimension d, and the bounds on f and its first three derivatives.
Edit intent: meaning-changed
Original:  The existence follows from an algorithm that converges from any starting point to such a point in a number of gradient and function evaluations f(x,y), and \nabla^2_y f(x,y), } that is polynomial in \frac{1{\varepsilon}}  , the dimension d, and the bounds on f and its first three derivatives.
Revised:  The existence follows from an algorithm that converges from any starting point to such a point in a number of gradient and function evaluations f(x,y), and \nabla^2_y f(x,y), } that is polynomial in \frac{1{\varepsilon}} 1/\varepsilon , the dimension d, and the bounds on f and its first three derivatives.
Edit intent: meaning-changed
Original:  The existence follows from an algorithm that converges from any starting point to such a point in a number of gradient and function evaluations f(x,y), and \nabla^2_y f(x,y), } that is polynomial in \frac{1{\varepsilon}}  , the dimension d, and the bounds on f and its first three derivatives. Our results do not require convexity, monotonicity, or special starting points .
Revised:  The existence follows from an algorithm that converges from any starting point to such a point in a number of gradient and function evaluations f(x,y), and \nabla^2_y f(x,y), } that is polynomial in \frac{1{\varepsilon}} , the dimension d, and the bounds on f and its Lipschitz constant. In addition to existence, our model retains many desirable properties of the min-max model. For instance, it empowers the min-player to make updates that take into account the max-player's response, and in the case of strong convexity/concavity it corresponds to a global min-max solution with duality gap O(\epsilon^2) .
Edit intent: meaning-changed
Original:  In this paper, we propose an optimization model, the \varepsilon-greedy adversarial equilibrium, which can serve as a computationally tractable alternative to the min-max optimization model.
Revised:  In this paper, we propose an optimization model, the \varepsilon-greedy adversarial equilibrium, and show that it can serve as a computationally tractable alternative to the min-max optimization model.
Edit intent: clarity
Original:  Roughly, we say  a point (x^\star, y^\star) is an \varepsilon-greedy adversarial equilibrium if y^\star is an \varepsilon-approximate local maximum for f(x^\star,\cdot), and x^\star is an \varepsilon-approximate local minimum for a "greedy approximation" to the function \max_z f(x, z) which can be efficiently estimated using second-order optimization algorithms.
Revised:  Roughly, we say that a point (x^\star, y^\star) is an \varepsilon-greedy adversarial equilibrium if y^\star is an \varepsilon-approximate local maximum for f(x^\star,\cdot), and x^\star is an \varepsilon-approximate local minimum for a "greedy approximation" to the function \max_z f(x, z) which can be efficiently estimated using second-order optimization algorithms.
Edit intent: fluency
Original:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and  \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d,  and the bounds on f and its Lipschitz constant .
Revised:  We prove the existence of such a point for any smooth function which is bounded and has Lipschitz Hessian. To prove existence, we introduce an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d, and the bounds on f and its Lipschitz constant .
Edit intent: meaning-changed
Original:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and  \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d,  and the bounds on f and its Lipschitz constant .
Revised:  The existence follows from an algorithm that converges from any starting point to an \varepsilon-greedy adversarial equilibrium in a number of evaluations to f, \nabla_{y f(x,y), and \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d, and the bounds on f and its Lipschitz constant .
Edit intent: meaning-changed
Original:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and  \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d,  and the bounds on f and its Lipschitz constant .
Revised:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations of the function f, the max-player's gradient \nabla_y f(x,y), and \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d, and the bounds on f and its Lipschitz constant .
Edit intent: meaning-changed
Original:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and  \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d,  and the bounds on f and its Lipschitz constant .
Revised:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and its Hessian \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d, and the bounds on f and its Lipschitz constant .
Edit intent: meaning-changed
Original:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and  \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d,  and the bounds on f and its Lipschitz constant .
Revised:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and \nabla^2_y f(x,y), that is polynomial in the dimension d, and the bounds on f and its Lipschitz constant .
Edit intent: meaning-changed
Original:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and  \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d,  and the bounds on f and its Lipschitz constant .
Revised:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d, 1/\varepsilon, and the bounds on f and its Lipschitz constant .
Edit intent: meaning-changed
Original:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and  \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d,  and the bounds on f and its Lipschitz constant . In addition to existence, our model retains many desirable properties of the min-max model. For instance, it empowers the min-player to make updates that take into account the max-player's response, and in the case of strong convexity/concavity it corresponds to a global min-max solution with duality gap O(\epsilon^2) .
Revised:  The existence follows from an algorithm that converges from any starting point to such a point in a number of evaluations to f, \nabla_{y f(x,y), and \nabla^2_y f(x,y), that is polynomial in 1/\varepsilon, the dimension d, and the bounds on f and its Lipschitz constant .
Edit intent: coherence
Original:  The point our algorithm converges to is stable when the maximizing player can respond using any sequence of steps which increase the loss at each step, and the minimizing player is empowered to simulate the maximizing player 's response for arbitrarily many steps but is restricted to move according to updates sampled from a stochastic gradient oracle.
Revised:  To achieve convergence, we 1) give a novel approximation to the minimizing player is empowered to simulate the maximizing player 's response for arbitrarily many steps but is restricted to move according to updates sampled from a stochastic gradient oracle.
Edit intent: clarity
Original:  The point our algorithm converges to is stable when the maximizing player can respond using any sequence of steps which increase the loss at each step, and the minimizing player is empowered to simulate the maximizing player 's response for arbitrarily many steps but is restricted to move according to updates sampled from a stochastic gradient oracle.
Revised:  The point our algorithm converges to is stable when the maximizing player can respond using any sequence of steps which increase the loss at each step, and the global strategy of the maximizing player 's response for arbitrarily many steps but is restricted to move according to updates sampled from a stochastic gradient oracle.
Edit intent: clarity
Original:  The point our algorithm converges to is stable when the maximizing player can respond using any sequence of steps which increase the loss at each step, and the minimizing player is empowered to simulate the maximizing player 's response for arbitrarily many steps but is restricted to move according to updates sampled from a stochastic gradient oracle.
Revised:  The point our algorithm converges to is stable when the maximizing player can respond using any sequence of steps which increase the loss at each step, and the minimizing player is empowered to simulate the max-player based on first-order algorithms such as gradient ascent, and 2) empower the min-player to look ahead and simulate the max-player 's response for arbitrarily many steps but is restricted to move according to updates sampled from a stochastic gradient oracle.
Edit intent: meaning-changed
Original:  The point our algorithm converges to is stable when the maximizing player can respond using any sequence of steps which increase the loss at each step, and the minimizing player is empowered to simulate the maximizing player 's response for arbitrarily many steps but is restricted to move according to updates sampled from a stochastic gradient oracle.
Revised:  The point our algorithm converges to is stable when the maximizing player can respond using any sequence of steps which increase the loss at each step, and the minimizing player is empowered to simulate the maximizing player 's response for arbitrarily many steps , but restrict the min-player to move according to updates sampled from a stochastic gradient oracle.
Edit intent: fluency
Original:  We apply our algorithm to train GANs on Gaussian mixtures, MNIST and CIFAR-10. We observe that our algorithm trains stably and avoids mode collapse, while achieving a training time per iteration and memory requirement similar to gradient descent-ascent.
Revised:  We apply our algorithm to train GANs on synthetic and real-world datasets, does not cycle, results in GANs that seem to avoid mode collapse, while achieving a training time per iteration and memory requirement similar to gradient descent-ascent.
Edit intent: meaning-changed
Original: We present a first-order algorithm for nonconvex-nonconcave min-max optimization problems such as those that arise in training GANs . Our algorithm provably converges in time polynomial in the dimension and smoothness parameters of the loss function .
Revised: We propose a variant of the standard min-max optimization problems such as those that arise in training GANs . Our algorithm provably converges in time polynomial in the dimension and smoothness parameters of the loss function .
Edit intent: clarity
Original: We present a first-order algorithm for nonconvex-nonconcave min-max optimization problems such as those that arise in training GANs . Our algorithm provably converges in time polynomial in the dimension and smoothness parameters of the loss function .
Revised: We present a first-order algorithm for nonconvex-nonconcave min-max framework for GANs to learn a distribution, where the discriminator can update its strategy in a greedy manner until it reaches a first-order stationary point. We give an algorithm to train such a GAN and show that it provably converges from any initial point to an approximate local equilibrium for this framework . Our algorithm provably converges in time polynomial in the dimension and smoothness parameters of the loss function .
Edit intent: meaning-changed
Original: We present a first-order algorithm for nonconvex-nonconcave min-max optimization problems such as those that arise in training GANs . Our algorithm provably converges in time polynomial in the dimension and smoothness parameters of the loss function .
Revised: We present a first-order algorithm for nonconvex-nonconcave min-max optimization problems such as those that arise in training GANs . Our algorithm runs in time polynomial in the dimension and smoothness parameters of the loss function .
Edit intent: clarity
Original: We present a first-order algorithm for nonconvex-nonconcave min-max optimization problems such as those that arise in training GANs . Our algorithm provably converges in time polynomial in the dimension and smoothness parameters of the loss function .
Revised: We present a first-order algorithm for nonconvex-nonconcave min-max optimization problems such as those that arise in training GANs . Our algorithm provably converges in time polynomial in the smoothness parameters of the loss function .
Edit intent: clarity
Original: We present a first-order algorithm for nonconvex-nonconcave min-max optimization problems such as those that arise in training GANs . Our algorithm provably converges in time polynomial in the dimension and smoothness parameters of the loss function . To achieve convergence, we 1) give a novel approximation to the global strategy of the max-player based on first-order algorithms such as gradient ascent, and 2) empower the min-player to look ahead and simulate the max-player's response for arbitrarily many steps, but restrict the min-player to move according to updates sampled from a stochastic gradient oracle. Our algorithm, when used to train GANs on synthetic and real-world datasets, does not cycle, results in GANs that seem to avoid mode collapse, and achieves a training time per iteration and memory requirement similar to gradient descent-ascent  .
Revised: We present a first-order algorithm for nonconvex-nonconcave min-max optimization problems such as those that arise in training GANs . Our algorithm provably converges in time polynomial in the dimension and smoothness parameters of the loss function and independent of the dimension, and allows for loss functions that can be nonconvex and nonconcave in the parameters of the generator and discriminator. Empirically, GANs trained using our algorithm consistently learn a greater number of modes than gradient descent-ascent .
Edit intent: meaning-changed
Original:  Our algorithm, when used to train GANs on synthetic and real-world datasets, does not cycle, results in GANs that seem to avoid mode collapse, and achieves a training time per iteration and memory requirement similar to gradient descent-ascent  .
Revised:  Our algorithm, when used to train GANs on synthetic and real-world datasets, does not cycle, results in GANs that seem to avoid mode collapse, and achieves a training time per iteration and memory requirement similar to gradient descent-ascent (GDA), optimistic mirror descent (OMD), and unrolled GANs when applied to a synthetic Gaussian mixture dataset. Moreover, they perform significantly better on CIFAR-10 than OMD and GDA when comparing the mean and standard deviation of the Inception Score respectively .
Edit intent: meaning-changed
Original: We propose a variant of the standard min-max framework for GANs to learn a distribution, where the discriminator can update its strategy in a greedy manner until it reaches a first-order stationary point.
Revised: Motivated by the recent work of Mangoubi and Vishnoi (STOC 2021), we propose a variant of the standard min-max framework for GANs to learn a distribution, where the discriminator can update its strategy in a greedy manner until it reaches a first-order stationary point.
Edit intent: meaning-changed
Original: We propose a variant of the standard min-max framework for GANs to learn a distribution, where the discriminator can update its strategy in a greedy manner until it reaches a first-order stationary point.
Revised: We propose a variant of the standard min-max framework for GANs to learn a distribution, where the discriminator can update its strategy in a greedy manner until it reaches a *first-order* stationary point.
Edit intent: coherence
Original:  We give an algorithm to train such a GAN and show that it provably converges from any initial point to an approximate local equilibrium for this framework . Our algorithm runs in time polynomial in the smoothness parameters of the loss function and independent of the dimension, and allows for loss functions that can be nonconvex and nonconcave in the parameters of the generator and discriminator.
Revised:  We give an algorithm to train such a GAN and show that it provably converges from any initial point to an approximate local equilibrium for our framework from any initialization and for nonconvex-nonconcave loss functions. Compared to the second-order algorithm of Mangoubi and Vishnoi, whose iteration bound is polynomial in the smoothness parameters of the loss function and independent of the dimension, and allows for loss functions that can be nonconvex and nonconcave in the parameters of the generator and discriminator.
Edit intent: meaning-changed
Original:  Our algorithm runs in time polynomial in the smoothness parameters of the loss function and independent of the dimension, and allows for loss functions that can be nonconvex and nonconcave in the parameters of the generator and discriminator.
Revised:  Our algorithm runs in time polynomial in the dimension, our algorithm is first-order and its iteration bound is independent of dimension. We empirically evaluate our algorithm on challenging nonconvex-nonconcave test-functions and loss functions that can be nonconvex and nonconcave in the parameters of the generator and discriminator.
Edit intent: meaning-changed