  Large-scale finite element simulations of complex physical systems governed
by partial differential equations crucially depend on adaptive mesh refinement
(AMR) to allocate computational budget to regions where higher resolution is
required. Existing scalable AMR methods make heuristic refinement decisions
based on instantaneous error estimation and thus do not aim for long-term
optimality over an entire simulation. We propose a novel formulation of AMR as
a Markov decision process and apply deep reinforcement learning (RL) to train
refinement policies directly from simulation. AMR poses a new problem for RL in
that both the state dimension and available action set changes at every step,
which we solve by proposing new policy architectures with differing generality
and inductive bias. The model sizes of these policy architectures are
independent of the mesh size and hence scale to arbitrarily large and complex
simulations. We demonstrate in comprehensive experiments on static function
estimation and the advection of different fields that RL policies can be
competitive with a widely-used error estimator and generalize to larger, more
complex, and unseen test problems.
