  In this article we study the stochastic gradient descent (SGD) optimization
method in the training of fully-connected feedforward artificial neural
networks with ReLU activation. The main result of this work proves that the
risk of the SGD process converges to zero if the target function under
consideration is constant. In the established convergence result the considered
artificial neural networks consist of one input layer, one hidden layer, and
one output layer (with $d \in \mathbb{N}$ neurons on the input layer, $H \in
\mathbb{N}$ neurons on the hidden layer, and one neuron on the output layer).
The learning rates of the SGD process are assumed to be sufficiently small and
the input data used in the SGD process to train the artificial neural networks
is assumed to be independent and identically distributed.
