# Meshfree Variational-Physics-Informed Neural Networks (MF-VPINN): An Adaptive Training Strategy
## 2 Problem Formulation
### Problem Discretization  
In order to numerically solve Problem (4), one needs to choose suitable finite-dimensional approximations of the trial space \(\bar{u}+V\) and of the test space \(V\). A Galerkin formulation is considered when we consider a finite-dimensional space \(V_{h}^{\text{trial}}\) for the trial space \(\bar{u}+V_{h}^{\text{trial}}\) and a finite-dimensional test space \(V_{h}^{\text{test}}\), with \(V_{h}^{\text{trial}}=V_{h}^{\text{test}}\), whereas a Petrov-Galerkin formulation is considered otherwise. In this work, we consider a Petrov-Galerkin formulation in which the trial space is approximated by a set of functions \(V^{\mathcal{NN}}\) of the form \(V^{\mathcal{NN}}=\bar{u}+V_{h}^{\text{trial}}\), with \(V_{h}^{\text{trial}}\) represented by a neural network suitably modified to enforce the Dirichlet boundary conditions, and the test space is a space \(V_{h}\) of piecewise linear functions. The neural network considered in the following is a standard fully connected feed-forward neural network. Given the number \(L\) of layers and a set of matrices \(A_{\ell}\in\mathbb{R}^{N_{\ell}\times N_{\ell-1}}\) and vectors \(b_{\ell}\in\mathbb{R}^{N_{\ell}}\), \(\ell=1,\dots,L\) containing the neural network's trainable weights, the function \(w:\mathbb{R}^{n}\to\mathbb{R}\) associated with the considered neural network architecture is:  
\[ \begin{split}& x_{0}=\mathbf{x},\\ & x_{\ell}=\rho(A_{\ell}x_{\ell-1}+b_{\ell}),\qquad\ell=1,\dots,L -1,\\ & w(\mathbf{x})=A_{L}x_{L-1}+b_{L}.\end{split}  \tag{5}\]  
where \(\rho:\mathbb{R}\to\mathbb{R}\) is a nonlinear function applied element-wise to the vector \(A_{\ell}x_{\ell-1}+b_{\ell}\). In this section, we use \(\rho(x)=\tanh(x)\); other common choices include, but are not limited to, \(\rho(x)=\text{ReLU}(x)=\max\{0,x\}\), \(\rho(x)=\text{ReLU}(x)=\max\{0,x^{p}\}\) for \(1<p\in\mathbb{N}\), \(\rho(x)=1/(1+e^{-x})\) and \(\rho(x)=\log(1+e^{x})\). Note that, in order to represent a function \(w:\mathbb{R}^{n}\to\mathbb{R}\), the layer widths \(N_{\ell}\) of the first and last layers are chosen as \(N_{0}=n\) and \(N_{L}=1\). We denote by \(W^{\mathcal{NN}}\) the set of functions that can be represented as in (5) for any combination of the neural network weights and by \(\mathbf{w}^{\mathcal{NN}}\) the vector containing all the trainable weights of the neural network. The function \(w\) defined in (5) is independent of the differential problem that has to be solved and is, in most papers on PINNs or related models, trained to minimize both the residual of the equation and a term penalizing the discrepancy between \(w_{|\Gamma}\) and \(g\). Instead, we add a non-trainable layer \(B\) to the neural network architecture in order to automatically enforce the required boundary conditions without the need to learn them during the training. As described in [49], the operator \(B\) acts on the neural network output as  
\[Bw=\phi w+\bar{g}, \tag{6}\]  
where \(\phi:\Omega\to\mathbb{R}\) is a function vanishing on \(\Gamma\) and strictly positive inside \(\Omega\), and \(\bar{g}:\Omega\to\mathbb{R}\) is a suitable extension of \(g:\Gamma\to\mathbb{R}\). The advantages of such an approach are also described in [50]. Then, the discrete trial space approximating \(\bar{u}+V\) can be defined as  
\[V^{\mathcal{NN}}=\left\{v^{\mathcal{NN}}\in\bar{u}+V:v^{\mathcal{NN}}=Bw\text { for some }w\in W^{\mathcal{NN}}\right\}.\]  
On the other hand, the discrete test space \(V_{h}\) is not associated with the neural network and only contains known test functions. In standard VPINNs, one generates a triangulation \(\mathcal{T}\) of the domain \(\Omega\) and then defines \(V_{h}\) as the space of functions that coincide with a polynomial of order \(p\in\mathbb{N}\) inside each element of \(\mathcal{T}\). Instead, we want to construct a discrete space \(V_{h}\) of functions independent from a global triangulation \(\mathcal{T}\).