ICLR 2025

Published as a conference paper at ICLR 2025
CONGO:
COMPRESSIVE ONLINE GRADIENT OPTIMIZATION
Jeremy Carleton1∗
Prathik Vijaykumar1
Divyanshu Saxena2
Dheeraj Narasimha3
Srinivas Shakkottai1
Aditya Akella2
1 Texas A&M University, 2 The University of Texas at Austin, 3 Inria
ABSTRACT
We address the challenge of zeroth-order online convex optimization where the
objective function’s gradient exhibits sparsity, indicating that only a small number
of dimensions possess non-zero gradients. Our aim is to leverage this sparsity to
obtain useful estimates of the objective function’s gradient even when the only
information available is a limited number of function samples. Our motivation
stems from the optimization of large-scale queueing networks that process time-
sensitive jobs. Here, a job must be processed by potentially many queues in
sequence to produce an output, and the service time at any queue is a function of
the resources allocated to that queue. Since resources are costly, the end-to-end
latency for jobs must be balanced with the overall cost of the resources used. While
the number of queues is substantial, the latency function primarily reacts to resource
changes in only a few, rendering the gradient sparse. We tackle this problem by
introducing the Compressive Online Gradient Optimization framework which
allows compressive sensing methods previously applied to stochastic optimization
to achieve regret bounds with an optimal dependence on the time horizon without
the full problem dimension appearing in the bound. For specific algorithms, we
reduce the samples required per gradient estimate to scale with the gradient’s
sparsity factor rather than its full dimensionality. Numerical simulations and real-
world microservices benchmarks demonstrate CONGO’s superiority over gradient
descent approaches that do not account for sparsity.
1
INTRODUCTION
Online convex optimization (OCO) plays an important role in managing distributed applications like
web caching, advertise...