  We revisit the concept of "adversary" in online learning, motivated by
solving robust optimization and adversarial training using online learning
methods. While one of the classical setups in online learning deals with the
"adversarial" setup, it appears that this concept is used less rigorously,
causing confusion in applying results and insights from online learning.
Specifically, there are two fundamentally different types of adversaries,
depending on whether the "adversary" is able to anticipate the exogenous
randomness of the online learning algorithms. This is particularly relevant to
robust optimization and adversarial training because the adversarial sequences
are often anticipative, and many online learning algorithms do not achieve
diminishing regret in such a case.
  We then apply this to solving robust optimization problems or (equivalently)
adversarial training problems via online learning and establish a general
approach for a large variety of problem classes using imaginary play. Here two
players play against each other, the primal player playing the decisions and
the dual player playing realizations of uncertain data. When the game
terminates, the primal player has obtained an approximately robust solution.
This meta-game allows for solving a large variety of robust optimization and
multi-objective optimization problems and generalizes the approach of
arXiv:1402.6361.
