  In this paper, we study the properties of nonparametric least squares
regression using deep neural networks. We derive non-asymptotic upper bounds
for the prediction error of the empirical risk minimizer for feedforward deep
neural regression. Our error bounds achieve the minimax optimal rate and
significantly improve over the existing ones in the sense that they depend
linearly or quadratically on the dimension d of the predictor, instead of
exponentially on d. We show that the neural regression estimator can circumvent
the curse of dimensionality under the assumption that the predictor is
supported on an approximate low-dimensional manifold. This assumption differs
from the structural condition imposed on the target regression function and is
weaker and more realistic than the exact low-dimensional manifold support
assumption in the existing literature. We investigate how the prediction error
of the neural regression estimator depends on the structure of neural networks
and propose a notion of network relative efficiency between two types of neural
networks, which provides a quantitative measure for evaluating the relative
merits of different network structures. Our results are derived under weaker
assumptions on the data distribution, the target regression function and the
neural network structure than those in the existing literature.
