  We develop an online adjoint algorithm for the optimization of partial
differential equations (PDEs). Traditional adjoint algorithms must solve a new
adjoint PDE at each optimization iteration, which can be computationally costly
since there may be a large number of iterations. The online adjoint algorithm
updates the design variables in continuous-time, and thus constantly makes
progress towards minimizing the objective function. This is in contrast to the
traditional adjoint algorithm which must solve an entire PDE to take a single
optimization step. The proposed online adjoint algorithm relies upon a
time-relaxed adjoint PDE which provides an estimate of the direction of
steepest descent. The algorithm updates this estimate continuously in time, and
it asymptotically converges to the exact direction of steepest descent as $t
\rightarrow \infty$. We rigorously prove that the online adjoint algorithm
converges to a critical point of the objective function for optimizing the PDE.
Under appropriate technical conditions, we also prove a convergence rate for
the algorithm.
