## 2 Materials and Methods
### A Related Classical Approach  
Both Problem 1 and Problem 2 are linear, and accordingly, we intended to construct the ANN as a linear map. In this case, it can be identified with a matrix \(A\in\mathbb{R}^{N\times M}\) such that we have the approximation:  
\[[\Psi_{\mathbf{y}_{j}}(\mathbf{x}_{1}),\Psi_{\mathbf{y}_{j}}(\mathbf{x}_{2}), \ldots,\Psi_{\mathbf{y}_{j}}(\mathbf{x}_{M})]^{T}\approx A[\Psi_{\mathbf{y}_{ j}}(\mathbf{z}_{1}),\Psi_{\mathbf{y}_{j}}(\mathbf{z}_{2}),\ldots,\Psi_{ \mathbf{y}_{j}}(\mathbf{z}_{N})]^{T}. \tag{5}\]  
Keeping this in mind, any input vector:  
\[[u(\mathbf{z}_{1}),u(\mathbf{z}_{2}),\ldots,u(\mathbf{z}_{N})]^{T}=[g(\mathbf{ z}_{1}),g(\mathbf{z}_{2}),\ldots,g(\mathbf{z}_{N})]^{T}\]  
in the ANN as the boundary data is first approximated as  
\[[g(\mathbf{z}_{1}),g(\mathbf{z}_{2}),\ldots,g(\mathbf{z}_{N})]^{T}\approx\sum _{j=1}^{K}b_{j}[\mathbf{\Psi}_{\mathbf{y}_{j}}(\mathbf{z}_{1}),\Psi_{\mathbf{ y}_{j}}(\mathbf{z}_{2}),\ldots,\Psi_{\mathbf{y}_{j}}(\mathbf{z}_{N})]^{T}, \tag{6}\]  
where \(\{b_{j}\}_{j=1}^{K}\) are unknown real coefficients. Indeed, taking \(K\geq N\) and appropriate values of \(\{\mathbf{y}_{j}\}_{j=1}^{K}\) such that  
\[\text{rank}\Big{\{}[\Psi_{\mathbf{y}_{j}}(\mathbf{z}_{1}),\Psi_{\mathbf{y}_{ j}}(\mathbf{z}_{2}),\ldots,\Psi_{\mathbf{y}_{j}}(\mathbf{z}_{N})]^{T}\Big{\}}_{j=1}^{ K}=N,\]  
the approximation in (6) becomes an equality.  
This is the case in real-life situations, since we try to produce a large number \(K\) of learning data pairs. Moreover, using the singular property of \(\Psi\), for each \(\mathbf{z}_{k}\), we can choose \(\mathbf{y}_{k}\) in its vicinity such that  
\[|\Psi_{\mathbf{y}_{k}}(\mathbf{z}_{k})|\geq\sum_{j\
eq k}|\Psi_{\mathbf{y}_{ k}}(\mathbf{z}_{j})|.\]  
This means that the matrix composed by the vectors:  
\[\Big{\{}[\Psi_{\mathbf{y}_{j}}(\mathbf{z}_{1}),\Psi_{\mathbf{y}_{j}}(\mathbf{ z}_{2}),\ldots,\Psi_{\mathbf{y}_{j}}(\mathbf{z}_{N})]^{T}\Big{\}}_{j=1}^{N}\]  
will be diagonally dominant, such that it is non-singular. Then, the rank of its columns is indeed \(N\).  
In this case, using (5) and (6) with equality, the neural network with the input \([g(\mathbf{z}_{1}),g(\mathbf{z}_{2}),\ldots,g(\mathbf{z}_{N})]^{T}\) gives  
\[\sum_{j=1}^{K}b_{j}[\Psi_{\mathbf{y}_{j}}(\mathbf{x}_{1}),\Psi_{\mathbf{y}_{j }}(\mathbf{x}_{2}),\ldots,\Psi_{\mathbf{y}_{j}}(\mathbf{x}_{M})]^{T}\approx \sum_{j=1}^{K}b_{j}A[\Psi_{\mathbf{y}_{j}}(\mathbf{z}_{1}),\Psi_{\mathbf{y}_{ j}}(\mathbf{z}_{2}),\ldots,\Psi_{\mathbf{y}_{j}}(\mathbf{z}_{N})]^{T},\]  
where the left-hand side is the prediction of \(u\) at \(\mathbf{x}_{1},\mathbf{x}_{2},\ldots,\mathbf{x}_{M}\). In summary, the linear ANN-based approximation can be recognized as an approximation:  
\[u\approx\sum_{j=1}^{K}b_{j}\Psi_{\mathbf{y}_{j}}.\]  
This is exactly the idea of the _method of fundamental solutions_ (MFS), which was initiated in [14]. Due to its simplicity and meshless property, this numerical method became extremely popular in the engineering community. At the same time, a rigorous and general convergence theory for the MFS is still missing. In any case, such a theory is based on the approximation property of the functions \(\{\Psi_{y_{j}}\}_{j=1}^{K}\). A nice summary of the related results can be found in [15].  
Furthermore, the approximation property of our neural network depends on the density properties of the above function space.