Title: A Split-Client Approach to Second-Order Optimization
Primary area: optimization
Keywords: Second-order optimization; Delayed Hessians; Split client

Abstract: Second-order methods promise faster convergence but are rarely used in practice because Hessian computations and decompositions are far more expensive than gradients. We propose a \emph{split-client} framework where gradients and curvature are computed asynchronously by separate clients. This abstraction captures realistic delays and inexact Hessian updates while avoiding the manual tuning required by Lazy Hessian methods. Focusing on cubic regularization, we show that our approach retains strong convergence guarantees and achieves a provable wall-clock speedup of order $\sqrt{\tau}$, where $\tau$ is the relative time needed to compute and decompose the Hessian compared to a gradient step. Since $\tau$ can be orders of magnitude larger than one in high-dimensional problems, this improvement is practically significant. Experiments on synthetic and real datasets confirm the theory: asynchronous curvature consistently outperforms vanilla and Lazy Hessian baselines, while maintaining second-order accuracy.

TL;DR: \textbf{TL;DR:} We propose an asynchronous split-client framework for cubic Newton methods that overlaps gradient and Hessian computations, yielding a $\sqrt{\tau}$ wall-clock speedup and outperforming Lazy Hessian baselines in practice.