  This paper presents a nonlinear model reduction method for systems of
equations using a structured neural network. The neural network takes the form
of a "three-layer" network with the first layer constrained to lie on the
Grassmann manifold and the first activation function set to identity, while the
remaining network is a standard two-layer ReLU neural network. The Grassmann
layer determines the reduced basis for the input space, while the remaining
layers approximate the nonlinear input-output system. The training alternates
between learning the reduced basis and the nonlinear approximation, and is
shown to be more effective than fixing the reduced basis and training the
network only. An additional benefit of this approach is, for data that lie on
low-dimensional subspaces, that the number of parameters in the network does
not need to be large. We show that our method can be applied to scientific
problems in the data-scarce regime, which is typically not well-suited for
neural network approximations. Examples include reduced order modeling for
nonlinear dynamical systems and several aerospace engineering problems.
