  The emerging push of the differentiable programming paradigm in scientific
computing is conducive to training deep learning turbulence models using
indirect observations. This paper demonstrates the viability of this approach
and presents an end-to-end differentiable framework for training deep neural
networks to learn eddy viscosity models from indirect observations derived from
the velocity and pressure fields. The framework consists of a Reynolds-averaged
Navier-Stokes (RANS) solver and a neural-network-represented turbulence model,
each accompanied by its derivative computations. For computing the
sensitivities of the indirect observations to the Reynolds stress field, we use
the continuous adjoint equations for the RANS equations, while the gradient of
the neural network is obtained via its built-in automatic differentiation
capability. We demonstrate the ability of this approach to learn the true
underlying turbulence closure when one exists by training models using
synthetic velocity data from linear and nonlinear closures. We also train a
linear eddy viscosity model using synthetic velocity measurements from direct
numerical simulations of the Navier-Stokes equations for which no true
underlying linear closure exists. The trained deep-neural-network turbulence
model showed predictive capability on similar flows.
