Original: M ESH -F REE U NSUPERVISED L EARNING -B ASED PDE S OLVER OF F ORWARD AND I NVERSE PROBLEMS
Revised: M ESH -F REE U NSUPERVISED L EARNING -B ASED PDE S OLVER OF F ORWARD AND I NVERSE PROBLEMS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We introduce a novel neural network-based partial differential equations solver for forward and inverse problems.
Revised: We introduce a novel neural network-based partial differential equations solver for forward and inverse problems.

Original: The solver is grid free, mesh free and shape free, and the solution is approximated by a neural network.
Revised: The solver is grid free, mesh free and shape free, and the solution is approximated by a neural network.

Original: We employ an unsupervised approach such that the input to the network is a points set in an arbitrary domain, and the output is the set of the corresponding function values.
Revised: We employ an unsupervised approach such that the input to the network is a points set in an arbitrary domain, and the output is the set of the corresponding function values.

Original: The network is trained to minimize deviations of the learned function from the PDE solution and satisfy the boundary conditions.
Revised: The network is trained to minimize deviations of the learned function from the PDE solution and satisfy the boundary conditions.

Original: The resulting solution in turn is an explicit smooth differentiable function with a known analytical form.
Revised: The resulting solution in turn is an explicit smooth differentiable function with a known analytical form.

Original: Unlike other numerical methods such as ﬁnite differences and ﬁnite elements, the derivatives of the desired function can be analytically calculated to any order.
Revised: Unlike other numerical methods such as ﬁnite differences and ﬁnite elements, the derivatives of the desired function can be analytically calculated to any order.

Original: This framework therefore, enables the solution of high order non-linear PDEs.
Revised: This framework therefore, enables the solution of high order non-linear PDEs.

Original: The proposed algorithm is a uniﬁed formulation of both forward and inverse problems where the optimized loss function consists of few elements: ﬁdelity terms of Land L ∞ norms, boundary conditions constraints and additional regularizers.
Revised: The proposed algorithm is a uniﬁed formulation of both forward and inverse problems where the optimized loss function consists of few elements: ﬁdelity terms of Land L ∞ norms, boundary conditions constraints and additional regularizers.

Original: This setting is ﬂexible in the sense that regularizers can be tailored to speciﬁc problems.
Revised: This setting is ﬂexible in the sense that regularizers can be tailored to speciﬁc problems.

Original: We demonstrate our method on a free shape 2D second order elliptical system with application to Electrical Impedance Tomography (EIT).
Revised: We demonstrate our method on several free shape 2D second order systems with application to Electrical Impedance Tomography (EIT), diffusion and wave equations.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Partial differential equations are fundamental in science and mathematics with wide applications in medical imaging, signal processing, computer vision, remote sensing, electromagnetism and more.
Revised: Partial differential equations are fundamental in science and mathematics with wide applications in medical imaging, signal processing, computer vision, remote sensing, electromagnetism, economics and more.

Original: Classical methods such as ﬁnite differences, ﬁnite volume and ﬁnite elements are numerical discretization-based methods where the domain is divided into a uniform grid or polygon mesh.
Revised: Classical methods such as ﬁnite differences, ﬁnite volume and ﬁnite elements are numerical discretization-based methods where the domain is divided into a uniform grid or polygon mesh.

Original: The differential equation is then reduced to a system of algebraic equations.
Revised: The differential equation is then reduced to a system of algebraic equations.

Original: These methods may have some limitations: the solution is numeric and may suffer from high condition number, highly dependent on the discretization and even the second derivative is sensitive to noise.
Revised: These methods may have some limitations: the solution is numeric and may suffer from high condition number, highly dependent on the discretization and even the second derivative is sensitive to noise, see for example

Revised: Garcia (2017); LeVeque (2007); Thomas (1995); Reddy (2005) and references therein.

Original: In the last few years, deep learning and neural network-based algorithms are extensively used in pattern recognition, image processing, computer vision and more.
Revised: In the last few years, deep learning and neural network-based algorithms are extensively used in pattern recognition, image processing, computer vision and more.

Original: Recently, the deep learning approach had been adopted to the ﬁeld of PDEs as well by converting the problem into a machine learning one.
Revised: Recently, the deep learning approach had been adopted to the ﬁeld of PDEs as well by converting the problem into a machine learning one.

Original: In Supervised learning , the network maps an input to an output based on example input-output pairs.
Revised: In Supervised learning , the network maps an input to an output based on example input-output pairs.

Original: This strategy is used in inverse problems, where the input to the network is a set of observations/measurements (e.g. x-ray tomography, ultrasound) and the output is the set of parameters of interest (tissue density etc.)
Revised: This strategy is used in inverse problems, where the input to the network is a set of observations/measurements (e.g. x-ray tomography, ultrasound) and the output is the set of parameters of interest (tissue density etc.)

Original: Feigin et al.
Revised: Feigin et al.

Original: ; Lucas et al.
Revised: ; Lucas et al.

Original: ; McCann et al.
Revised: ; McCann et al.

Original: Seo et al.
Revised: Seo et al.

Original: Unsupervised learning on the other hand is a self-learning mechanism where the natural structure presents within a set of data points is inferred.
Revised: Unsupervised learning on the other hand is a self-learning mechanism where the natural structure present within a set of data points is inferred.

Original: Algorithms for forward and inverse problems in partial differential equations via unsupervised learning were recently introduced.
Revised: Algorithms for forward and inverse problems in partial differential equations via unsupervised learning were recently introduced.

Original: The indirect approach utilizes a neural network as a component in the solution.
Revised: The indirect approach utilizes a neural network as a component in the solution.

Original: Li et al.
Revised: Li et al.

Original: (2018) for example, proposed the NETT (Network Tikhonov) approach to inverse problems.
Revised: (2018) for example, proposed the NETT (Network Tikhonov) approach to inverse problems.

Original: NETT considers regularized solutions having a small value of a regularizer deﬁned by a trained neural network.
Revised: NETT considers regularized solutions having a small value of a regularizer deﬁned by a trained neural network.

Original: Khoo & Ying (2018) introduced a novel neural network architecture, SwitchNet, for solving the wave equation based inverse scattering problems via providing maps between the scatterers and the scattered ﬁeld.
Revised: Khoo & Ying (2018) introduced a novel neural network architecture, SwitchNet, for solving the wave equation based inverse scattering problems via providing maps between the scatterers and the scattered ﬁeld.

Original: Han et al.
Revised: Han et al.

Original: (2018) developed a deep learning-based approach that can handle general high-dimensional parabolic PDEs.
Revised: (2018) developed a deep learning-based approach that can handle general high-dimensional parabolic PDEs.

Original: To this end, the PDEs are reformulated using backward stochastic differential equations.
Revised: To this end, the PDEs are reformulated using backward stochastic differential equations.

Original: The latter is solved by a temporal discretization and the gradient of the unknown solution at each time step is approximated by  neural network.
Revised: The latter is solved by a temporal discretization and the gradient of the unknown solution at each time step is approximated by a neural network.

Original: Li et al.
Revised: Li et al.

Original: (2019) approximate the  solution map of linear and nonlinear problems  by a deep network.
Revised: (2019) approximate the inverse solution map of linear and nonlinear problems directly by a deep network.

Original: Direct algorithms solve the forward  problem PDEs by directly approximating the solution with a deep neural network.
Revised: Direct algorithms solve the forward and inverse problem PDEs by directly approximating the solution with a deep neural network.

Original: The network parameters are determined by the optimization of a cost function such that the optimal solution satisﬁes the PDE, boundary conditions and initial conditions.
Revised: The network parameters are determined by the optimization of a cost function such that the optimal solution satisﬁes the PDE, boundary conditions and initial conditions.

Original: Chiaramonte & Kiener (2017) addressed the forward problem by constructing a one layer network which satisﬁes the PDE within the domain.
Revised: Chiaramonte & Kiener (2017) addressed the forward problem by constructing a one layer network which satisﬁes the PDE within the domain.

Original: The boundary conditions were analytically integrated in the cost function.
Revised: The boundary conditions were analytically integrated in the cost function.

Original: They demonstrated their algorithm on the Laplace and hyperbolic conservation law PDEs.
Revised: They demonstrated their algorithm on the Laplace and hyperbolic conservation law PDEs.

Original: Sirignano & Spiliopoulos (2017) proposed a deep learning forward problem solver for high dimensional PDEs.
Revised: Sirignano & Spiliopoulos (2017) proposed a deep learning forward problem solver for high dimensional PDEs.

Original: Their algorithm was demonstrated on the American option free-boundary equation.
Revised: Their algorithm was demonstrated on the American option freeboundary equation.

Original: Raissi et al.
Revised: Raissi et al.

Original: (2017) focused on continuous time models and solved the Burgers and Shr¨odinger equations.
Revised: (2017) focused on continuous time models and solved the Burgers and Schr¨odinger equations, and Xu & Darve (2019) introduced a novel direct method for the inverse problem and demonstrated their algorithm on various PDEs.

Original: In this work we address the forward and inverse PDE problems via a direct unsupervised method.
Revised: In this work we address the forward and inverse PDE problems via a direct unsupervised method.

Original: Our key contributions are four fold: (1) inverse problems can be solved in the same framework as the forward problems.
Revised: Our key contributions are four fold: (1) inverse problems can be solved in the same framework as the forward problems.

Original: In the  forward part we extend the standard L 2 -based ﬁdelity term in the cost function by adding L ∞ -like norm.
Revised: In the both forward and inverse parts we extend the standard L 2 -based ﬁdelity term in the cost function by adding L ∞ -like norm.

Original: Moreover, (3) some regularization terms which impose a-priori knowledge on the solution can be easily incorporated.
Revised: Moreover, (3) some regularization terms which impose a-priori knowledge on the solution can be easily incorporated.

Original: (4) An important feature of our construction is the ability to handle free-form domain in a mesh free manner.
Revised: (4) Our construction exempliﬁes the ability to handle free-form domain in a mesh free manner.

Original: We demonstrate our algorithm by  a second order elliptic equation, in particular the Electrical Impedance Tomography (EIT) application on circular and three other arbitrary domains.
Revised: Point (1) is essential for full tomography-like solutions as will be explained in the sequel. The extension of the loss function by the L ∞ -like norm is fundamental. This term promotes a strong solution of the PDE. The L 2 term, used in recent studies, aims only for weak solutions of the PDE.

Original: M ATHEMATICAL F ORMULATION
Revised: Weak solutions may differ from the strong solutions by a set of isolated points where the function is not continuous. This is not a merely theoretical issue but strongly affects the quality of the result as we empirically demonstrate. In unsupervised learning of ill-posed problems regularization is crucial.

Original: Let Ω be a bounded open and connected subset of R d , and
Revised: Choosing the right regularizer and the ability to incorporate it in the formulation is of prime importance.

Original: A =
Revised: Our formalism integrates such regularizations in a natural way.

Original: A ( x ) = ( a ij ( x )) be any given d × d symmetric positive deﬁnite matrix of functions for 1 ≤ i, j ≤ d .
Revised: We demonstrate our algorithm by a second order elliptic equation, in particular the Electrical Impedance Tomography (EIT) application on a circular and three other arbitrary domains.

Original: Let b = b ( x ) = ( b j ( x )) be any given n-tuple of functions and let c
Revised: We additionally solve the inverse problem of diffusion and wave equations.

Original: = c ( x ) be any given function. A second order operator Lis said to be in divergence form, if L acting on some u has the form
Revised: Let Ω be a bounded open and connected subset of R d where d is the dimension. A differential operator L acting on a function u ( x ) : R d → R is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where we use the Einstein summation convention.
Revised: where D n is the nth order d-dimensional derivative and a 0 ( x ) , . . . , a n ( x ) are the coefﬁcients.

Original: Consider the partial differential problem with Dirichlet boundary conditions
Revised: Consider the partial differential problem with Dirichlet boundary conditions

Original: [Equation]
Revised: [Equation]

Original:  The forward problem solves u  given the coefﬁcients θ := { a ij ( x ) , b j ( x ) , c ( x ) } while the inverse problem determines the coefﬁcients set θ given u  .
Revised: where f ( x ) : R d → R is a given function. The forward problem solves u ( x ) given the coefﬁcients θ := { a 0 ( x ) , . . . , a n ( x ) } while the inverse problem determines the coefﬁcients set θ given u ( x ) .

Original: The proposed algorithm approximates the solutions in both problems by neural networks u ( x ; w u ) ,  aij ( x ; w ij ) , b j ( x ; w b ) , c ( x ; w c )  such that the networks are parameterized by w u ,  w ij , w b , w c , and the input to the network is x ∈ R d .
Revised: The proposed algorithm approximates the solutions in both problems by neural networks u ( x ; w u ) , { a j ( x ; w a j ) } such that the networks are parameterized by w u , { w a j } , and the input to the network is x ∈ R d .

Original: Figure 1 depicts a network architecture of u in R 2 .
Revised: Figure 1 depicts a network architecture of u in R 2 .

Original: The network consists of few fully connected layers with tanh activation and linear sum in the last layer.
Revised: The network consists of few fully connected layers with tanh activation and linear sum in the last layer.

Original: The network is trained to satisfy the PDE with boundary conditions by minimizing a cost function.
Revised: The network is trained to satisfy the PDE with boundary conditions by minimizing a cost function.

Original: In the forward problem
Revised: In the forward problem

Original: [Equation]
Revised: [Equation]

Original: and in the inverse problem
Revised: and in the inverse problem

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: The ﬁrst two terms enforce the solution to satisfy the equation.
Revised: The ﬁrst two terms enforce the solution to satisfy the equation.

Original: The ﬁrst term minimizes the error in  L 2 sense while the second term minimizes the maximal error.
Revised: The ﬁrst term minimizes the error in the L 2 sense while the second term minimizes the maximal error.

Original: Ideally, the solution to the PDE has to be satisﬁed at every point x .
Revised: Ideally, the solution to the PDE has to be satisﬁed at every point x .

Original: This is considered as a strong solution.
Revised: This is considered as a strong solution.

Original: In the deep learning framework the optimization is carried out on mini batches of points.
Revised: In the deep learning framework the optimization is carried out on mini batches of points.

Original: The ﬁrst term minimizes the operator in the integral or average sense and is therefore insensitive to a point jump in the value of the integrand which yields a weak solution.
Revised: The ﬁrst term minimizes the equation in the integral or average sense and is therefore insensitive to a point jump in the value of the integrand. It therefore yields a weak solution.

Original: This manifest itself by point discontinuities  u ( x ) that areseen empirically  as well.
Revised: This manifests itself by point discontinuities of u ( x ) that are empirically seen as well.

Original: The L ∞ term promotes a strong solution since it takes care of possible error points.
Revised: The L ∞ term promotes a strong solution since it handles possible discontinuous points where the value of the week solution is different than the strong one. We consider this discrepancy as an error. Practically, we use a relaxed version of the L ∞ norm where we take the top- K errors. Then, at every mini batch the training procedure minimizes the equation at

Revised: K different points having the largest errors.

Original: The third term imposes boundary conditions where u 0 and θ 0 are the boundary values of u ( x ) and θ ( x ) .
Revised: The third term imposes boundary conditions where uand θ 0 are the boundary values of u ( x ) and θ ( x ) .

Revised: We used the L 1 norm on the boundary so that the solution is robust to measurements noise.

Original: The last term is a regularizer which can be tailored to the application.
Revised: The last term is a regularizer which can be tailored to the application.

Original: There are few advantages of this setting.
Revised: There are few advantages of this setting.

Original: First, the solutions are smooth analytic functions and are therefore analytically differentiable .
Revised: First, the solution is a network and therefore an explicit function. It is a smooth analytic function and is therefore analytically differentiable .

Original: It was recently proven that for such functions, e.g. Lipchitz with constant ν , the approximation of the function by a fully connected network Θ is given by
Revised: It was recently proven that for such functions, e.g. Lipchitz with constant ν , the approximation of the function by a fully connected network Γ is given by

Original: [Equation]
Revised: [Equation]

Original: where p ∈ [0 , ∞ ) , N number of neurons in a layer and L number of layers Shen et al.
Revised: where p ∈ [0 , ∞ ) , N number of neurons in a layer, and L number of layers Shen et al.

Original: In addition, this framework enables setting a prior  knowledge on the solution by designing the regularizers R F and R I .
Revised: In addition, the proposed framework enables setting a prior physical knowledge on the solution by designing the regularizers R F and R I .

Original: Moreover, the training procedure is mesh free.
Revised: Moreover, the training procedure is mesh free.

Original: Unlike ﬁnite differences or ﬁnite elements methods, we use random points in the domain and its boundary in the course of the optimization of equation 3 and equation 4, see Figure 2.
Revised: Unlike ﬁnite differences or ﬁnite elements methods, we use random points in the domain and its boundary in the course of the optimization of equation 3 and equation 4, see Figure 2.

Original: This means that the solution does not depend upon a coordinate mesh and we can also deﬁne an arbitrary regular domain Ω .
Revised: This means that the solution does not depend upon a coordinate mesh and we can also deﬁne an arbitrary regular domain Ω .

Original:  A PPLICATION TO E LECTRICAL I MPEDANCE T OMOGRAPHY
Revised:  A PPLICATION TO E LECTRICAL I MPEDANCE T OMOGRAPHY

Original: Let us address a special case of equation 1,
Revised: Consider the following elliptic equation which is a special case of equation 1,

Original: [Equation]
Revised: [Equation]

Original:  We assume that 0 < σ ( x ) ∈ C 1 (Ω) , which guarantees existence and uniqueness of a solution u ∈ C 2 (Ω) Evans (2010).
Revised: where ∇ := ( ∂/∂ x , ∂/∂ y ) . We assume that 0 < σ ( x ) ∈ C 1 (Ω) , which guarantees existence and uniqueness of a solution u ∈ C 2 (Ω) Evans (2010).

Original: The elliptical system equation 5 was addressed by Siltanen et al.
Revised: The elliptical system equation 5 was addressed by Siltanen et al.

Original: (2000) in the context of Electrical
Revised: (2000) in the context of Electrical

Original: Impedance Tomography (EIT) which is a reconstruction method for the inverse conductivity problem.
Revised: Impedance Tomography (EIT) which is a reconstruction method for the inverse conductivity problem.

Original: The function σ indicates the electrical conductivity density, and u is the electrical potential.
Revised: The function σ indicates the electrical conductivity density, and u is the electrical potential.

Original: An electrical current
Revised: An electrical current

Original: [Equation]
Revised: [Equation]

Original: is applied on electrodes on the surface ∂ Ω , where κ is the angle in polar coordinate system along the domain boundary,  ϕ is the phase and ν is the normal unit.
Revised: is applied on electrodes on the surface ∂ Ω , where κ is the angle in polar coordinate system along the domain boundary, n is the current frequency, ϕ is the phase and ν is the normal unit.

Original: The resulting voltage u | ∂ Ω = u  is measured through the electrodes.
Revised: The resulting voltage u | ∂ Ω = u 0 is measured through the electrodes.

Original: The conductivity σ is determined from the knowledge of the Dirichlet-to-Neumann map or voltage-to-current map
Revised: The conductivity σ is determined from the knowledge of the Dirichlet-to-Neumann map or voltage-to-current map

Original: [Equation]
Revised: [Equation]

Original: Mueller & Siltanen (2012); Alsaker & Mueller (2018) [Equation]
Revised: Mueller & Siltanen (2012); Alsaker & Mueller (2018); Fan & Ying (2019).

Original: Bera et al.

Original: [Equation]

Original: Fan & Ying (2019).

Original: We demonstrate our framework by solving the forward and inverse problem of equation 5 which is a ﬁrst step towards a full tomography.
Revised: We demonstrate our framework by solving the forward and inverse problem of equation 5 which is a ﬁrst step towards a full tomography.

Original: Following Mueller & Siltanen (2012), we simulate the voltage measurement u | ∂ Ω by the Finite Elements Method (FEM) given three variants of a conductivity phantom σ ( x ) depicted in Figure 3.
Revised: Following Mueller & Siltanen (2012), we simulate the voltage measurement u | ∂ Ω by the Finite Elements Method (FEM) given three variants of a conductivity phantom σ ( x ) depicted in Figure 3.

Original: We calculate the FEM solution with different triangle mesh densities and select as ground truth the one such that ﬁner meshes do not improve the numerical solution.
Revised: We calculate the FEM solution with different triangle mesh densities and select as ground truth the one such that ﬁner meshes do not improve the numerical solution.

Original: With our suggested method, the forward problem determines the electrical potential u in the whole domain Ω given σ , while the inverse problem uses the approximated u and calculates the conductivity σ given that σ | ∂ Ω = σ 0 .
Revised: With our suggested method, the forward problem determines the electrical potential u in the whole domain Ω given σ , while the inverse problem uses the approximated u and calculates the conductivity σ given that σ | ∂ Ω = σ 0 .

Original:  F ORWARD P ROBLEM
Revised:  F ORWARD P ROBLEM

Original: In the forward problem the conductivity σ ( x i ) and boundary conditions u 0 ( x b ) are given for random points set { x i } ∈ Ω ⊂ R 2 , { x b } ∈ ∂ Ω ⊂ R 2 with sets size of N s and N b respectively.
Revised: In the forward problem the conductivity σ ( x i ) and boundary conditions u 0 ( x b ) are given for random points set { x i } ∈ Ω ⊂ R 2 , { x b } ∈ ∂ Ω ⊂ R 2 with sets size of N s and N b respectively.

Original: A neural network having the architecture shown in Figure 1 approximates u ( x ) .
Revised: A neural network having the architecture shown in Figure 1 approximates u ( x ) .

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: The cost function equation 3 is then rewritten as
Revised: The cost function equation 3 is then rewritten as

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term is the L 2 norm of the differential operator, the second term is a relaxed version of the inﬁnity norm where we take the mean value of the top-K values of |L i | .
Revised: The ﬁrst term is the L 2 norm of the differential operator, the second term is a relaxed version of the inﬁnity norm where we take the mean value of the top-K values of |L i | .

Revised: There is some balance between small K which accounts for few error points and can have negligible effect in the training procedure, and a large K which resembles the effect of the L 2 norm.

Revised: In all our experiments we empirically selected K = 40 as an intermediate value.

Original: The third term imposes the boundary conditions and the last term serves as a regularizer of the network parameters. 
Revised: The third term imposes the boundary conditions and the last term serves as a regularizer of the network parameters. [Table] [Table] [Table]

Original:  The ﬁrst phantom is shown in Figure 3 left.
Revised: [Table] The ﬁrst phantom is shown in Figure 3 left.

Original: The impedance values associated with the background ellipses and circle were 1 , 5 and 2 respectively.
Revised: The impedance values associated with the background ellipses and circle were 1 , 5 and 2 respectively.

Original: The original piecewise constant function σ was slightly smoothed by a Gaussian kernel.
Revised: The original piecewise constant function σ was slightly smoothed by a Gaussian kernel.

Original: Figure 4 shows the forward problem results for current ψ with n = 1 and ϕ = π/ 8 .
Revised: Figure 10 in the Appendix shows the forward problem results for current ψ with n = 1 and ϕ = π/ 8 .

Original: The left column is the FEM solution which is referred to as ground truth, where the top row indicates the solution u  and the bottom row the derivative of u  with respect to  x calculated as the ﬁnite difference approximation of the FEM result.
Revised: The left column is the FEM solution which is referred to as ground truth, where the top row indicates the solution u ( x ) and the bottom row the derivative of u ( x ) with respect to the ﬁrst coordinate x calculated as the ﬁnite difference approximation of the FEM result.

Original: The middle column depicts the outcome of the proposed method where ∂u/∂x is an analytical derivative of our result.
Revised: The middle column depicts the outcome of the proposed method where ∂u/∂x is an analytical derivative of our result.

Original: The right column shows the outcome of the DGM method Sirignano & Spiliopoulos (2017) which is a special case of equation 3 with λ = 1 and µ = 0 .
Revised: The right column shows the outcome of the DGM method Sirignano & Spiliopoulos (2017) which is a special case of equation 3 with λ = 1 , µ = 0 , L 2 norm of the boundary conditions constraint with no regularizers.

Original: Quantitative results of the Mean square error (MSE) and PSNR are summarized in Table 1.
Revised: Quantitative results of the mean square error (MSE) and peak signal-to-noise ratio (PSNR) are summarized in Table 1.

Original: Clearly, the proposed method outperforms the DGM method since the weighting parameters and the L ∞ norm  play a signiﬁcant role in the loss function.
Revised: Clearly, the proposed method outperforms the DGM method since the weighting parameters, the L ∞ norm and the network weights regularization play a signiﬁcant role in the loss function.

Original: [Figure]
Revised: [Table]

Original: MSE and PSNR are noted in line 2 of Table [Figure] [Table]
Revised: 9.17e37.

Original: [Table]
Revised: 3 , Ω π/1.74e31.

Original: [Table] [Table] [Table] [Table]
Revised: 2.24e41.

Original: [Table]
Revised: [Table]

Original: The forward problem was repeated using phantom 2 where the background and circle conductivities were 1 and 0 .
Revised: The forward problem was repeated using phantom 2 where the background and circle conductivities were 1 and 0 .

Original: 2 respectively (Figure 3, middle).
Revised: 2 respectively (Figure 3, middle).

Original: Four different current combinations were applied.
Revised: Four different current combinations were applied.

Original: Quantitative results are summarized in lines 1 - 4 of Table 2.
Revised: Quantitative results are summarized in rows 1 - 4 of Table 2.

Original: Figure 5 shows the results for both u and ∂u/∂x for n = 2 and ϕ = π/ 2 .
Revised: Figure 11 in the Appendix shows the results for both u and ∂u/∂x for n = 2 and ϕ = π/ 2 .

Original: The right column presents the relative error deﬁned as e ( x, y )
Revised: The right column presents the relative error deﬁned as e ( x, y )

Original: = ( u fem ( x, y ) − u ( x, y )) / max( u fem ) .
Revised: = ( u fem ( x, y ) − u ( x, y )) / max( u fem ) .

Original:  I NVERSE P ROBLEM FOR EIT
Revised:  I NVERSE P ROBLEM FOR EIT

Original: In the inverse problem, the electrical potential u ( x ) is known while σ ( x ) is unknown.
Revised: In the inverse problem, the electrical potential u ( x ) is known while σ ( x ) is unknown.

Original: Since we have a network which approximates u ( x ) , we can evaluate it at any point x .
Revised: Since we have a network which approximates u ( x ) , we can evaluate it at any point x .

Original: The objective function equation 4 then takes the form
Revised: The objective function equation 4 then takes the form

Original: [Equation]
Revised: [Equation]

Original: As in the forward problem, the ﬁrst two terms enforce σ to satisfy the PDE, where L i is deﬁned in equation 6.
Revised: As in the forward problem, the ﬁrst two terms enforce σ to satisfy the PDE, where L i is deﬁned in equation 7.

Original: The third term imposes the boundary conditions, and the fourth regularizes the network parameters.
Revised: The third term imposes the boundary conditions, and the fourth regularizes the network parameters.

Original:  The last term is  the total variation regularization ( p = 1 ) which promotes the solution towards a piecewise constant function.
Revised: A physical a priori knowledge regarding σ is exploited in this application. The conductivity is assumed to have well deﬁned sub-regions. We therefore design the ﬁfth term to have sparse edges via the total variation regularization ( p = 1 ).

Original: Additional inverse problem generalization may exploit multiple u approximations for several currents ψ j .
Revised: Additional inverse problem generalization may exploit multiple u approximations for several currents ψ j .

Original: The σ calculation thus, simultaneously relies on all { u j } , resulting a more stable solution,
Revised: The σ calculation thus, simultaneously relies on all { u j } , resulting a more stable solution. Let

Original: [Equation]
Revised: [Equation]

Original: MSE and PSNR are summed up in lines 1 - 2 of Table 3.

Original: [Table]

Original: Then equation 8 is generalized to
Revised: Then equation 9 is generalized to

Original: [Equation]
Revised: [Equation]

Original: and

Original: I (cid:16) σ

Original: [Equation]

Original: x

Original: [Equation]
Revised: [Equation]

Original: Reconstruction results by the generalized inverse problem are shown in Figure 6 and lines 1 - 2 in Table 3.
Revised: Adequate reconstruction results by the generalized inverse problem are shown in Figure 4 and rows 1 - 2 in Table 3.

Original: In both phantoms we used four current combinations ψ n,ϕ : (1 , 0) , (1 , π/ 4) , (2 , 0) , (2 , π/ 4) , and σ 0 = 1 .
Revised: In both phantoms we used four current combinations ψ n,ϕ : (1 , 0) , (1 , π/ 4) , (2 , 0) , (2 , π/ 4) , and σ 0 = 1 .

Original:  F REE S HAPE G EOMETRY
Revised:  F REE S HAPE G EOMETRY

Original: We applied the proposed method to arbitrary domains Ω 1 , Ω 2 and Ω 3 , see Figure 7.
Revised: We applied the proposed method to arbitrary domains Ω 1 , Ω 2 and Ω 3 , see Figure 5.

Original: The random sample points within the domain and along its boundary can be easily obtained as can be seen in Figure 7 right.
Revised: The random sample points within the domain and along its boundary can be easily obtained as can be seen in Figure 5 right.

Original: The forward problem results applied on phantom 3 are presented in lines 5 - 10 of
Revised: Forward problem results applied on phantom 3 are presented in rows 5 - 10 of

Original: Table 2.
Revised: Table 2.

Original: Figure 8 shows the results for n = 2 and ϕ =
Revised: Figure 12 in the Appendix shows the results for n = 2 and ϕ =

Original: π/ 4 for the three domains.
Revised: π/ 4 for the three domains.

Original: The outcome of the generalized inverse problem equation 11 is shown in Figure 12 and lines 3 - 5 ofTable 3.
Revised: The outcome of the generalized inverse problem equation 12 is shown in Figure 6 and rows 3 - 5 of

Original: [Figure]
Revised: Table 3.

Original:  A DDITIONAL I NVERSE P ROBLEM E XAMPLES
Revised:  A DDITIONAL I NVERSE P ROBLEM E XAMPLES

Original: In this section we exemplify the proposed algorithm with additional inverse problem setting introduced by Xu & Darve (2019).
Revised: In this section we exemplify the proposed algorithm with additional inverse problem setting introduced by Xu & Darve (2019).

Original: Unknown parameters which are parameterized by neural networks are inferred out of few measurements.
Revised: Unknown parameters which are approximated by neural networks are inferred out of few measurements.

Original: The differential operators are approximated in ﬁnite differences schemes discretized by ∆ t and ∆ h , the temporal and spatial grid spacings.
Revised: The differential operators are approximated in ﬁnite differences schemes discretized by ∆ t and ∆ h , the temporal and spatial grid spacings.

Revised: In the two following examples we applied the cost function equation 4 with total variation and network parameters regularizations.

Original: 7. D IFFUSION E QUATION
Revised: 7. D IFFUSION E QUATION

Original: Diffusion PDE is given by
Revised: Diffusion PDE is given by

Original: [Equation]
Revised: [Equation]

Original: where  the conductivity C ( x ) is the unknown parameter.
Revised: where t denotes the time and the conductivity C ( x ) is the unknown parameter.

Original: In the following we solve the equation in 2D where x :=
Revised: In the following we solve the equation in 2D where x :=

Original: ( x, y ) .
Revised: ( x, y ) .

Original: Given two measurements at adjacent times u ( x, y, t 1 ) and u ( x, y, t 1 + ∆ t ) , the loss function then takes the form
Revised: Given two measurements at adjacent times u ( x, y, t 1 ) and u ( x, y, t 1 +∆ t ) , the conductivity C ( x, y ) is approximated as a neural network and is determined as the minimizer of equation 4 with

Original: [Equation]
Revised: [Equation]

Original: where D 2 is the central scheme approximation for the Laplacian on 3x3 stencil.
Revised: where D 2 is the central scheme approximation for the Laplacian on 3x3 stencil.

Original: The analytical 2D conductivity function is given by
Revised: The analytical 2D conductivity function is given by

Original: [Equation]
Revised: [Equation]

Original: −

Original: [Equation]

Original: xy )

Original: [Equation]

Original: , and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: The measurements were taken at t 1 = 0 .
Revised: The measurements were taken at t 1 = 0 .

Original: 1 with ∆ t = 0 .
Revised: 1 with ∆ t = 0 .

Original: 001 and ∆ h = 0 .
Revised: 001 and ∆ h = 0 .

Original: The result is shown in
Revised: The result is shown in

Original: Figure 9.
Revised: Figure 7.

Original: As can be seen the smoothness regularizer and the L ∞ terms increase the reconstruction accuracy.
Revised: As can be seen the smoothness regularizer and the L ∞ terms increase the reconstruction accuracy.

Original: In addition, we added few Gaussian noise levels to the measurements.
Revised: In addition, we added few Gaussian noise levels to the measurements.

Original: Quantitative results are summarized in lines 1 − 4 of Table 4.
Revised: Quantitative results are summarized in rows 1 − 4 of Table 4.

Original: W AVE E QUATION
Revised: [Figure]

Original: The second example is the wave equation given by
Revised: The second example is the wave equation given by

Original: [Equation]
Revised: [Equation]

Original:  The unknown parameter is the velocity ﬁeld η ( x, y ) .
Revised: ] 2 × [0 , T ] . The unknown parameter is the velocity ﬁeld η ( x, y ) .

Original: Its analytical value is given by
Revised: Its analytical value is given by

Original: [Equation]
Revised: [Equation]

Original: and the two initial conditions are
Revised: and the two initial conditions are

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Solutions for all time stepst are iteratively calculated using the ﬁnite differences scheme.
Revised: Solutions for all time steps are recursively calculated using the ﬁnite differences scheme using the correct η ( x, y ) .

Original: The velocity ﬁeld η ( x, y ) is extracted using three measurements with the following loss function
Revised: Given three consecutive values of u ( x, y, t ) the velocity ﬁeld η ( x, y ) is extracted by the minimization of equation 4 where

Original: [Equation]
Revised: [Equation]

Original: Reconstruction result is depicted in Figure 10 showing the advantage of the proposed method.
Revised: Reconstruction result is depicted in Figure 8 showing the advantage of the proposed method.

Original: Additional quantitave results of noisy measurements are  in lines 5 - 8 of Table 
Revised: Additional quantitave results of noisy measurements are given in rows 5 - 8 of Table 4.

Original: [Equation]
Revised: [Table]

Original:  I MPLEMENTATION D ETAILS
Revised:  I MPLEMENTATION D ETAILS

Original: The network architecture has 4 layers having 26 , 26 , 26 and 10 neurons.
Revised: The network architecture had 4 layers having 26 , 26 , 26 and 10 neurons.

Original: The algorithm was implemented by TensorFlow Ten (2015) using the ADAM optimizer which is a variant of the SGD algorithm.
Revised: The algorithm was implemented by TensorFlow Ten (2015) using the ADAM optimizer which is a variant of the SGD algorithm.

Original: We used  the same hyper parameters set inalmost all of our experiments.
Revised: We used almost the same hyper parameters set in our experiments.

Original: Batch size= 1000 , decaying learning rate starting at 5 e − 3 .
Revised: Batch size= 1000 , decaying learning rate starting at 5 e − 3 .

Original: The learning rate was factored by 0 .
Revised: The learning rate was factored by 0 .

Original: 8 every 200 epochs,
Revised: 8 every 200 epochs,

Original: Ns = 80000 , N b = 1200 , λ = 0 .
Revised: N s = 80000 , N b = 1200 , λ = 0 .

Original: 01 , α = 1 e − 8 , K = 40 , µ = 0 .
Revised: 01 , α = 1 e − 8 , K = 40 , µ = 0 .

Original: 01 , and β = 0 .
Revised: 01 , and β = 0 .

Original: In the diffusion 
Revised: In the diffusion and wave equation case we set λ

Original: [Table]
Revised: = 1 , β = 1 e − 4 , µ = 1 e − 4 and α = 1 e − 5 . Running time, for the EIT, on Intel i7-8650u CPU was about 15 minutes for the forward problem and 13 minutes for the generalized inverse problem. For the diffusion and wave equation the running time was aboutminutes.

Original: Figure 11 presents the sensitivity  of the algorithm  to the number of sampled points.
Revised: Finally, we performed a sensitivity analysis of the algorithm with respect to the number of sample points N s . Figure 9 shows the sensitivity of the forward problem algorithm.

Original: As expected, with large number of points we obtain a plateau while with small and intermediate number (up to 50000)
Revised: As expected, with large number of points we obtain a plateau while with small and intermediate number (up to 50000 )

Original: the error is not monotonic due to local minima.
Revised: the error is not monotonic due to local minima.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: Deep networks by their nature use compositions of simple functions such as matrix multiplication and non-linear activations like sigmoid or tanh.
Revised: Deep networks by their nature use compositions of simple functions such as matrix multiplication and non-linear activations like sigmoid or tanh.

Original: This structure (i) enables the approximation of an arbitrary function Hornik (1989) and (ii) is inherently differentiable.
Revised: This structure (i) enables the approximation of an arbitrary function Hornik (1989) and (ii) is inherently differentiable.

Original: The network architecture dictates the number of degrees of freedom which in turn enables the expressibility of complex functions.
Revised: The network architecture dictates the number of degrees of freedom which in turn enables the expressibility of complex functions.

Original: In this work we present a uniﬁed framework for the solution of forward and inverse problems in partial differential equations in an arbitrary domain.
Revised: In this work we present a uniﬁed framework for the solution of forward and inverse problems in partial differential equations in an arbitrary domain.

Original: The algorithm relies on direct approximation of the unknown function by a neural network which yields an analytical smooth solution.
Revised: The algorithm relies on direct approximation of the unknown function by a neural network which yields an analytical smooth solution.

Original: The network is trained to satisfy the PDE and boundary conditions in an unsupervised fashion by the minimization of a cost function.
Revised: The network is trained to satisfy the PDE and boundary conditions in an unsupervised fashion by the minimization of a cost function.

Original: The optimization procedure depends on random points set within the domain and its boundary.
Revised: The optimization procedure depends on random points set within the domain and its boundary.

Original: The problem is therefore mesh free with free-form domain.
Revised: The problem is therefore mesh free with free-form domain.

Original: We introduce a cost function which is composed of both L 2 and  L ∞ ﬁdelity terms, boundary conditions constraints and additional regularizers.
Revised: While recent works formulate the cost function in both forward and inverse problems in the L 2 sense, we introduce a broadened framework.

Original: The mathematical rationale of the L ∞ norm which controls possible outliers was veriﬁed in our numerical experiments.
Revised: The relaxed L ∞ ﬁdelity term was designed to approximate a strong solution and control possible discrepancies between the weak and the strong solutions. The L 1 norm on the boundary conditions is more robust to measurement noise than the L 2 norm. The additional regularizer (total variation in our examples) has a signiﬁcant role in the optimization process where known physical properties like smoothness are imposed on the solution. Lastly, the network regularizer controls the network weights and prevents over ﬁtting.

Revised: These additional terms make the proposed algorithm more accurate and robust compared to other learning-based methods.

Original: We also stress the robustness of our approach exempliﬁed by having  the same hyper parameters set for all our experiments.
Revised: We also stress the robustness of our approach exempliﬁed by having almost the same hyper parameters set in our experiments.

Original: This framework alleviates several problems of the ﬁnite differences and ﬁnite elements methods.
Revised: This framework alleviates several problems of the ﬁnite differences and ﬁnite elements methods.

Original: In particular meshing, discretization and derivatives approximation are solved in a simple and natural way.
Revised: In particular meshing, discretization and derivatives approximation are solved in a simple and natural way.

Original: Numerical solutions of PDEs in an arbitrary domain are of extreme importance, in particular in medical imaging applications.
Revised: Numerical solutions of PDEs in an arbitrary domain are of extreme importance, in particular in medical imaging applications.

Original: The framework is demonstrated by an elliptic system in R 2 applied to Electrical Impedance Tomography for both forward and inverse problems.
Revised: The framework is demonstrated by an elliptic system in R 2 applied to Electrical Impedance Tomography for both forward and inverse problems, diffusion and wave equations.

Original: Promising results were achieved for complex and non monotonic functions.
Revised: Promising results were achieved for complex and non monotonic functions.

Original: Rigorous analysis of the approximation error and its relation to the network architecture and design are under current study and will be published elsewhere.
Revised: Rigorous analysis of the approximation error and its relation to the network architecture and design are under current study.

Original: Future research includes also full tomography solution, higher dimensional problems and other classes of PDEs such as dynamic non-linear equations (Burgers, Navier-Stokes etc.)
Revised: Future research includes also full tomography solution, higher dimensional problems and other classes of PDEs such as dynamic non-linear equations (Burgers, Navier-Stokes etc.)