  In this work, we investigate the universal representation capacity of the
Matrix Product States (MPS) from the perspective of boolean functions and
continuous functions. We show that MPS can accurately realize arbitrary boolean
functions by providing a construction method of the corresponding MPS structure
for an arbitrarily given boolean gate. Moreover, we prove that the function
space of MPS with the scale-invariant sigmoidal activation is dense in the
space of continuous functions defined on a compact subspace of the
$n$-dimensional real coordinate space $\mathbb{R^{n}}$. We study the relation
between MPS and neural networks and show that the MPS with a scale-invariant
sigmoidal function is equivalent to a one-hidden-layer neural network equipped
with a kernel function. We construct the equivalent neural networks for several
specific MPS models and show that non-linear kernels such as the polynomial
kernel which introduces the couplings between different components of the input
into the model appear naturally in the equivalent neural networks. At last, we
discuss the realization of the Gaussian Process (GP) with infinitely wide MPS
by studying their equivalent neural networks.
