Before revision:  We prove that when a PDE's coefficients are representable by small neural networks, the parameters required to approximate its solution scale polynomially with the input dimension d and are proportional to the parameter counts of the coefficient neural networks. Our proof is based on constructing a neural network which simulates gradient descent  in an appropriate Hilbert space which converges to the solution of the PDE.
After revision:  We prove that when a PDE's coefficients are representable by small neural networks, the parameters required to approximate its solution scale polynomially with the input dimension d and are proportional to the parameter counts of the coefficient networks. To this we end, we develop a proof technique that simulates gradient descent in an appropriate Hilbert space which converges to the solution of the PDE.