  In this work, we address the question whether a sufficiently deep quantum
neural network can approximate a target function as accurate as possible. We
start with simple but typical physical situations that the target functions are
physical observables, and then we extend our discussion to situations that the
learning targets are not directly physical observables, but can be expressed as
physical observables in an enlarged Hilbert space with multiple replicas, such
as the Loshimidt echo and the Renyi entropy. The main finding is that an
accurate approximation is possible only when the input wave functions in the
dataset do not exhaust the entire Hilbert space that the quantum circuit acts
on, and more precisely, the Hilbert space dimension of the former has to be
less than half of the Hilbert space dimension of the latter. In some cases,
this requirement can be satisfied automatically because of the intrinsic
properties of the dataset, for instance, when the input wave function has to be
symmetric between different replicas. And if this requirement cannot be
satisfied by the dataset, we show that the expressivity capabilities can be
restored by adding one ancillary qubit where the wave function is always fixed
at input. Our studies point toward establishing a quantum neural network
analogy of the universal approximation theorem that lays the foundation for
expressivity of classical neural networks.
