  Online learning has been successfully applied to many problems in which data
are revealed over time. In this paper, we provide a general framework for
studying multi-agent online learning problems in the presence of delays and
asynchronicities. Specifically, we propose and analyze a class of adaptive dual
averaging schemes in which agents only need to accumulate gradient feedback
received from the whole system, without requiring any between-agent
coordination. In the single-agent case, the adaptivity of the proposed method
allows us to extend a range of existing results to problems with potentially
unbounded delays between playing an action and receiving the corresponding
feedback. In the multi-agent case, the situation is significantly more
complicated because agents may not have access to a global clock to use as a
reference point; to overcome this, we focus on the information that is
available for producing each prediction rather than the actual delay associated
with each feedback. This allows us to derive adaptive learning strategies with
optimal regret bounds, at both the agent and network levels. Finally, we also
analyze an "optimistic" variant of the proposed algorithm which is capable of
exploiting the predictability of problems with a slower variation and leads to
improved regret bounds.
