Original: M ESH -F REE U NSUPERVISED L EARNING -B ASED PDE S OLVER OF F ORWARD AND I NVERSE PROBLEMS
Revised: M ESH -F REE U NSUPERVISED L EARNING -B ASED PDE S OLVER OF F ORWARD AND I NVERSE PROBLEMS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We introduce a novel neural network-based partial differential equations solver for forward and inverse problems.
Revised: We introduce a novel neural network-based partial differential equations solver for forward and inverse problems.

Original: The solver is grid free, mesh free and shape free, and the solution is approximated by a neural network.
Revised: The solver is grid free, mesh free and shape free, and the solution is approximated by a neural network.

Original: We employ an unsupervised approach such that the input to the network is a points set in an arbitrary domain, and the output is the set of the corresponding function values.
Revised: We employ an unsupervised approach such that the input to the network is a points set in an arbitrary domain, and the output is the set of the corresponding function values.

Original: The network is trained to minimize deviations of the learned function from the  PDE solution and satisfy the boundary conditions.
Revised: The network is trained to minimize deviations of the learned function from the strong PDE solution and satisfy the boundary conditions.

Original: The resulting solution in turn is an explicit smooth differentiable function with a known analytical form.
Revised: The resulting solution in turn is an explicit smooth differentiable function with a known analytical form.

Original: Unlike other numerical methods such as ﬁnite differences and ﬁnite elements, the derivatives of the desired function can be analytically calculated to any order.
Revised: Unlike other numerical methods such as ﬁnite differences and ﬁnite elements, the derivatives of the desired function can be analytically calculated to any order.

Original: This framework therefore, enables the solution of high order non-linear PDEs.
Revised: This framework therefore, enables the solution of high order non-linear PDEs.

Original: The proposed algorithm is a uniﬁed formulation of both forward and inverse problems where the optimized loss function consists of few elements: ﬁdelity terms of Land L ∞ norms, boundary conditions constraints and additional regularizers.
Revised: The proposed algorithm is a uniﬁed formulation of both forward and inverse problems where the optimized loss function consists of few elements: ﬁdelity terms of Land L ∞ norms that unlike previous methods promote a strong solution. Robust boundary conditions constraints and additional regularizers are included as well.

Original: This setting is ﬂexible in the sense that regularizers can be tailored to speciﬁc problems.
Revised: This setting is ﬂexible in the sense that regularizers can be tailored to speciﬁc problems.

Original: We demonstrate our method on several free shape 2D second order systems with application to Electrical Impedance Tomography (EIT), diffusion and wave equations.
Revised: We demonstrate our method on several free shape 2D second order systems with application to Electrical Impedance Tomography (EIT), diffusion and wave equations.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Partial differential equations are fundamental in science and mathematics with wide applications in medical imaging, signal processing, computer vision, remote sensing, electromagnetism, economics and more.
Revised: Partial differential equations are fundamental in science and mathematics with wide applications in medical imaging, signal processing, computer vision, remote sensing, electromagnetism, economics and more.

Original: Classical methods such as ﬁnite differences, ﬁnite volume and ﬁnite elements are numerical discretization-based methods where the domain is divided into a uniform grid or polygon mesh.
Revised: Classical methods such as ﬁnite differences, ﬁnite volume and ﬁnite elements are numerical discretization-based methods where the domain is divided into a uniform grid or polygon mesh.

Original: The differential equation is then reduced to a system of algebraic equations.
Revised: The differential equation is then reduced to a system of algebraic equations.

Original: These methods may have some limitations: the solution is numeric and may suffer from high condition number, highly dependent on the discretization and even the second derivative is sensitive to noise, see for example
Revised: These methods may have some limitations: the solution is numeric and may suffer from high condition number, highly dependent on the discretization and even the second derivative is sensitive to noise, see for example

Original: Garcia (2017); LeVeque (2007); Thomas (1995); Reddy (2005) and references therein.
Revised: Garcia (2017); LeVeque (2007); Thomas (1995); Reddy (2005) and references therein.

Original: In the last few years, deep learning and neural network-based algorithms are extensively used in pattern recognition, image processing, computer vision and more.
Revised: In the last few years, deep learning and neural network-based algorithms are extensively used in pattern recognition, image processing, computer vision and more.

Original: Recently, the deep learning approach had been adopted to the ﬁeld of PDEs as well by converting the problem into a machine learning one.
Revised: Recently, the deep learning approach had been adopted to the ﬁeld of PDEs as well by converting the problem into a machine learning one.

Original: In Supervised learning , the network maps an input to an output based on example input-output pairs.
Revised: In Supervised learning , the network maps an input to an output based on example input-output pairs.

Original: This strategy is used in inverse problems, where the input to the network is a set of observations/measurements (e.g. x-ray tomography, ultrasound) and the output is the set of parameters of interest (tissue density etc.)
Revised: This strategy is used in inverse problems, where the input to the network is a set of observations/measurements (e.g. x-ray tomography, ultrasound) and the output is the set of parameters of interest (tissue density etc.)

Original: Feigin et al.
Revised: Feigin et al.

Original: ; Lucas et al.
Revised: ; Lucas et al.

Original: ; McCann et al.
Revised: ; McCann et al.

Original: Seo et al.
Revised: Seo et al.

Original: Unsupervised learning on the other hand is a self-learning mechanism where the natural structure present within a set of data points is inferred.
Revised: Unsupervised learning on the other hand is a self-learning mechanism where the natural structure present within a set of data points is inferred.

Original: Algorithms for forward and inverse problems in partial differential equations via unsupervised learning were recently introduced.
Revised: Algorithms for forward and inverse problems in partial differential equations via unsupervised learning were recently introduced.

Original: The indirect approach utilizes a neural network as a component in the solution.
Revised: The indirect approach utilizes a neural network as a component in the solution.

Original: Li et al.
Revised: Li et al.

Original: (2018) for example, proposed the NETT (Network Tikhonov) approach to inverse problems.
Revised: (2018) for example, proposed the NETT (Network Tikhonov) approach to inverse problems.

Original: NETT considers regularized solutions having a small value of a regularizer deﬁned by a trained neural network.
Revised: NETT considers regularized solutions having a small value of a regularizer deﬁned by a trained neural network.

Original: Khoo & Ying (2018) introduced a novel neural network architecture, SwitchNet, for solving the wave equation based inverse scattering problems via providing maps between the scatterers and the scattered ﬁeld.
Revised: Khoo & Ying (2018) introduced a novel neural network architecture, SwitchNet, for solving the wave equation based inverse scattering problems via providing maps between the scatterers and the scattered ﬁeld.

Original: Han et al.
Revised: Han et al.

Original: (2018) developed a deep learning-based approach that can handle general high-dimensional parabolic PDEs.
Revised: (2018) developed a deep learning-based approach that can handle general high-dimensional parabolic PDEs.

Original: To this end, the PDEs are reformulated using backward stochastic differential equations.
Revised: To this end, the PDEs are reformulated using backward stochastic differential equations.

Original: The latter is solved by a temporal discretization and the gradient of the unknown solution at each time step is approximated by a neural network.
Revised: The latter is solved by a temporal discretization and the gradient of the unknown solution at each time step is approximated by a neural network.

Original: Li et al.
Revised: Li et al.

Original: (2019) approximate the inverse solution map of linear and nonlinear problems directly by a deep network.
Revised: (2019) approximate the inverse solution map of linear and nonlinear problems directly by a deep network.

Original: Direct algorithms solve the forward and inverse problem PDEs by directly approximating the solution with a deep neural network.
Revised: Direct algorithms solve the forward and inverse problem PDEs by directly approximating the solution with a deep neural network.

Original: The network parameters are determined by the optimization of a cost function such that the optimal solution satisﬁes the PDE, boundary conditions and initial conditions.
Revised: The network parameters are determined by the optimization of a cost function such that the optimal solution satisﬁes the PDE, boundary conditions and initial conditions.

Original: Chiaramonte & Kiener (2017) addressed the forward problem by constructing a one layer network which satisﬁes the PDE within the domain.
Revised: Chiaramonte & Kiener (2017) addressed the forward problem by constructing a one layer network which satisﬁes the PDE within the domain.

Original: The boundary conditions were analytically integrated in the cost function.
Revised: The boundary conditions were analytically integrated in the cost function.

Original: They demonstrated their algorithm on the Laplace and hyperbolic conservation law PDEs.
Revised: They demonstrated their algorithm on the Laplace and hyperbolic conservation law PDEs.

Original: Sirignano & Spiliopoulos (2017) proposed a deep learning forward problem solver for high dimensional PDEs.
Revised: Sirignano & Spiliopoulos (2017) proposed a deep learning forward problem solver for high dimensional PDEs.

Original: Their algorithm was demonstrated on the American option freeboundary equation.
Revised: Their algorithm was demonstrated on the American option freeboundary equation.

Original: Raissi et al.
Revised: Raissi et al.

Original: (2017) focused on continuous time models and solved the Burgers and Schr¨odinger equations, and Xu & Darve (2019) introduced a novel direct method for the inverse problem and demonstrated their algorithm on various PDEs.
Revised: (2017) focused on continuous time models and solved the Burgers and Schr¨odinger equations, and Xu & Darve (2019) introduced a novel direct method for the inverse problem and demonstrated their algorithm on various PDEs.

Original: In this work we address the forward and inverse PDE problems via a direct unsupervised method.
Revised: In this work we address the forward and inverse PDE problems via a direct unsupervised method.

Original: Our key contributions are four fold: (1) inverse problems can be solved in the same framework as the forward problems.
Revised: Our key contributions are four fold: (1) inverse problems can be solved in the same framework as the forward problems.

Original: In the both forward and inverse parts we extend the standard L 2 -based ﬁdelity term in the cost function by adding L ∞ -like norm.
Revised: In the both forward and inverse parts we extend the standard L 2 -based ﬁdelity term in the cost function by adding L ∞ -like norm.

Original: Moreover, (3) some regularization terms which impose a-priori knowledge on the solution can be easily incorporated.
Revised: Moreover, (3) some regularization terms which impose a-priori knowledge on the solution can be easily incorporated.

Original: (4) Our construction exempliﬁes the ability to handle free-form domain in a mesh free manner.
Revised: (4) Our construction exempliﬁes the ability to handle free-form domain in a mesh free manner.

Original: Point (1) is essential for full tomography-like solutions as will be explained in the sequel.
Revised: Point (1) is essential for full tomography-like solutions as will be explained in the sequel.

Original: The extension of the loss function by the L ∞ -like norm is fundamental.
Revised: The extension of the loss function by the L ∞ -like norm is fundamental.

Original: This term promotes a strong solution of the PDE.
Revised: This term promotes a strong solution of the PDE.

Original: The L 2 term, used in recent studies, aims only for weak solutions of the PDE.
Revised: The L 2 term, used in recent studies, aims only for weak solutions of the PDE.

Original: Weak solutions may differ from the strong solutions by a set of isolated points where the function is not continuous.
Revised: Weak solutions may differ from the strong solutions by a set of isolated points where the function is not continuous.

Original: This is not a merely theoretical issue but strongly affects the quality of the result as we empirically demonstrate.
Revised: This is not a merely theoretical issue but strongly affects the quality of the result as we empirically demonstrate.

Original: In unsupervised learning of ill-posed problems regularization is crucial.
Revised: In unsupervised learning of ill-posed problems regularization is crucial.

Original: Choosing the right regularizer and the ability to incorporate it in the formulation is of prime importance.
Revised: Choosing the right regularizer and the ability to incorporate it in the formulation is of prime importance.

Original: Our formalism integrates such regularizations in a natural way.
Revised: Our formalism integrates such regularizations in a natural way.

Original: We demonstrate our algorithm by a second order elliptic equation, in particular the Electrical Impedance Tomography (EIT) application on a circular and three other arbitrary domains.
Revised: We demonstrate our algorithm by a second order elliptic equation, in particular the Electrical Impedance Tomography (EIT) application on a circular and three other arbitrary domains.

Original: We additionally solve the inverse problem of diffusion and wave equations.
Revised: We additionally solve the inverse problem of diffusion and wave equations.

Original:  M ATHEMATICAL F ORMULATION
Revised:  M ATHEMATICAL F ORMULATION

Original: Let Ω be a bounded open and connected subset of R d where d is the dimension.
Revised: Let Ω be a bounded open and connected subset of R d where d is the dimension.

Original: A differential operator L acting on a function u ( x ) : R d → R is deﬁned as
Revised: A differential operator L acting on a function u ( x ) : R d → R is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where D n is the nth order d-dimensional derivative and a 0 ( x ) , . .
Revised: where D n is the nth order d-dimensional derivative and a 0 ( x ) , . .

Original: . , a n ( x ) are the coefﬁcients.
Revised: . , a n ( x ) are the coefﬁcients.

Original: Consider the partial differential problem with Dirichlet boundary conditions
Revised: Consider the partial differential problem with Dirichlet boundary conditions

Original: [Equation]
Revised: [Equation]

Original:  where f ( x ) : R d → R is a given function.
Revised: 2) where f ( x ) : R d → R is a given function.

Original: The forward problem solves u ( x ) given the coefﬁcients θ :=
Revised: The forward problem solves u ( x ) given the coefﬁcients θ :=

Original: { a 0 ( x ) , . .
Revised: { a 0 ( x ) , . .

Original: . , a n ( x ) } while the inverse problem determines the coefﬁcients set θ given u ( x ) .
Revised: . , a n ( x ) } while the inverse problem determines the coefﬁcients set θ given u ( x ) .

Original: The proposed algorithm approximates the solutions in both problems by neural networks u ( x ; w u ) , { a j ( x ; w a j ) } such that the networks are parameterized by w u , { w a j } , and the input to the network is x ∈ R d .
Revised: The proposed algorithm approximates the solutions in both problems by neural networks u ( x ; w u ) , { a j ( x ; w a j ) } such that the networks are parameterized by w u , { w a j } , and the input to the network is x ∈ R d .

Original: Figure 1 depicts a network architecture of u in R 2 .
Revised: Figure 1 depicts a network architecture of u in R 2 .

Original: The network consists of few fully connected layers with tanh activation and linear sum in the last layer.
Revised: The network consists of few fully connected layers with tanh activation and linear sum in the last layer.

Original: The network is trained to satisfy the PDE with boundary conditions by minimizing a cost function.
Revised: The network is trained to satisfy the PDE with boundary conditions by minimizing a cost function.

Original: In the forward problem
Revised: In the forward problem

Original: [Equation]
Revised: [Equation]

Original: and in the inverse problem
Revised: and in the inverse problem

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst two terms enforce the solution to satisfy the equation.
Revised: The ﬁrst two terms enforce the solution to satisfy the equation.

Original: The ﬁrst term minimizes the error in the L 2 sense while the second term minimizes the maximal error.
Revised: The ﬁrst term minimizes the error in the L 2 sense while the second term minimizes the maximal error.

Original: Ideally, the solution to the PDE has to be satisﬁed at every point x .
Revised: Ideally, the solution to the PDE has to be satisﬁed at every point x .

Original: This is considered as a strong solution.
Revised: This is considered as a strong solution.

Original: In the deep learning framework the optimization is carried out on mini batches of points.
Revised: In the deep learning framework the optimization is carried out on mini batches of points.

Original: The ﬁrst term minimizes the equation in the integral or average sense and is therefore insensitive to a point jump in the value of the integrand.
Revised: The ﬁrst term minimizes the equation in the integral or average sense and is therefore insensitive to a point jump in the value of the integrand.

Original: It therefore yields a weak solution.
Revised: It therefore yields a weak solution.

Original: This manifests itself by point discontinuities of u ( x ) that are empirically seen as well.
Revised: This manifests itself by point discontinuities of u ( x ) that are empirically seen as well.

Original: The L ∞ term promotes a strong solution since it handles possible discontinuous points where the value of the week solution is different than the strong one.
Revised: The L ∞ term promotes a strong solution since it handles possible discontinuous points where the value of the week solution is different than the strong one.

Original: We consider this discrepancy as an error.
Revised: We consider this discrepancy as an error.

Original: Practically, we use a relaxed version of the L ∞ norm where we take the top- K errors.
Revised: Practically, we use a relaxed version of the L ∞ norm where we take the top- K errors.

Original: Then, at every mini batch the training procedure minimizes the equation at
Revised: Then, at every mini batch the training procedure minimizes the equation at

Original: K different points having the largest errors.
Revised: K different points having the largest errors.

Original: The third term imposes boundary conditions where uand θ 0 are the boundary values of u ( x ) and θ ( x ) .
Revised: The third term imposes boundary conditions where uand θ 0 are the boundary values of u ( x ) and θ ( x ) .

Original: We used the L 1 norm on the boundary so that the solution is robust to measurements noise.
Revised: We used the L 1 norm on the boundary so that the solution is robust to measurements noise.

Original: The last term is a regularizer which can be tailored to the application.
Revised: The last term is a regularizer which can be tailored to the application.

Original: There are few advantages of this setting.
Revised: There are few advantages of this setting.

Original: First, the solution is a network and therefore an explicit function.
Revised: First, the solution is a network and therefore an explicit function.

Original: It is a smooth analytic function and is therefore analytically differentiable .
Revised: It is a smooth analytic function and is therefore analytically differentiable .

Original: It was recently proven that for such functions, e.g. Lipchitz with constant ν , the approximation of the function by a fully connected network Γ is given by
Revised: It was recently proven that for such functions, e.g. Lipchitz with constant ν , the approximation of the function by a fully connected network Γ is given by

Original: [Equation]
Revised: [Equation]

Original: where p ∈ [0 , ∞ ) , N number of neurons in a layer, and L number of layers Shen et al.
Revised: where p ∈ [0 , ∞ ) , N number of neurons in a layer, and L number of layers Shen et al.

Original: In addition, the proposed framework enables setting a prior physical knowledge on the solution by designing the regularizers R F and R I .
Revised: In addition, the proposed framework enables setting a prior physical knowledge on the solution by designing the regularizers R F and R I .

Original: Moreover, the training procedure is mesh free.
Revised: Moreover, the training procedure is mesh free.

Original: Unlike ﬁnite differences or ﬁnite elements methods, we use random points in the domain and its boundary in the course of the optimization of equation 3 and equation 4, see Figure 2.
Revised: Unlike ﬁnite differences or ﬁnite elements methods, we use random points in the domain and its boundary in the course of the optimization of equation 3 and equation 4, see Figure 2.

Original: This means that the solution does not depend upon a coordinate mesh and we can also deﬁne an arbitrary regular domain Ω .
Revised: This means that the solution does not depend upon a coordinate mesh and we can also deﬁne an arbitrary regular domain Ω .

Original:  A PPLICATION TO E LECTRICAL I MPEDANCE T OMOGRAPHY
Revised:  A PPLICATION TO E LECTRICAL I MPEDANCE T OMOGRAPHY

Original: Consider the following elliptic equation which is a special case of equation 1,
Revised: Consider the following elliptic equation which is a special case of equation 1,

Original: [Equation]
Revised: [Equation]

Original: where ∇ := ( ∂/∂ x , ∂/∂ y ) .
Revised: where ∇ := ( ∂/∂ x , ∂/∂ y ) .

Original: We assume that 0 < σ ( x ) ∈ C 1 (Ω) , which guarantees existence and uniqueness of a solution u ∈ C 2 (Ω) Evans (2010).
Revised: We assume that 0 < σ ( x ) ∈ C 1 (Ω) , which guarantees existence and uniqueness of a solution u ∈ C 2 (Ω) Evans (2010).

Original: The elliptical system equation 5 was addressed by Siltanen et al.
Revised: The elliptical system equation 5 was addressed by Siltanen et al.

Original: (2000) in the context of Electrical
Revised: (2000) in the context of Electrical

Original: Impedance Tomography (EIT) which is a reconstruction method for the inverse conductivity problem.
Revised: Impedance Tomography (EIT) which is a reconstruction method for the inverse conductivity problem.

Original: The function σ indicates the electrical conductivity density, and u is the electrical potential.
Revised: The function σ indicates the electrical conductivity density, and u is the electrical potential.

Original: An electrical current
Revised: An electrical current

Original: [Equation]
Revised: [Equation]

Original: is applied on electrodes on the surface ∂ Ω , where κ is the angle in polar coordinate system along the domain boundary, n is the current frequency, ϕ is the phase and ν is the normal unit.
Revised: is applied on electrodes on the surface ∂ Ω , where κ is the angle in polar coordinate system along the domain boundary, n is the current frequency, ϕ is the phase and ν is the normal unit.

Original: The resulting voltage u | ∂ Ω = u 0 is measured through the electrodes.
Revised: The resulting voltage u | ∂ Ω = u 0 is measured through the electrodes.

Original: The conductivity σ is determined from the knowledge of the Dirichlet-to-Neumann map or voltage-to-current map
Revised: The conductivity σ is determined from the knowledge of the Dirichlet-to-Neumann map or voltage-to-current map

Original: [Equation]
Revised: [Equation]

Original: Mueller & Siltanen (2012); Alsaker & Mueller (2018); Fan & Ying (2019).
Revised: Mueller & Siltanen (2012); Alsaker & Mueller (2018); Fan & Ying (2019).

Original: We demonstrate our framework by solving the forward and inverse problem of equation 5 which is a ﬁrst step towards a full tomography.
Revised: We demonstrate our framework by solving the forward and inverse problem of equation 5 which is a ﬁrst step towards a full tomography.

Original: Following Mueller & Siltanen (2012), we simulate the voltage measurement u | ∂ Ω by the Finite Elements Method (FEM) given three variants of a conductivity phantom σ ( x ) depicted in Figure 3.
Revised: Following Mueller & Siltanen (2012), we simulate the voltage measurement u | ∂ Ω by the Finite Elements Method (FEM) given three variants of a conductivity phantom σ ( x ) depicted in Figure 3.

Original: We calculate the FEM solution with different triangle mesh densities and select as ground truth the one such that ﬁner meshes do not improve the numerical solution.
Revised: We calculate the FEM solution with different triangle mesh densities and select as ground truth the one such that ﬁner meshes do not improve the numerical solution.

Original: With our suggested method, the forward problem determines the electrical potential u in the whole domain Ω given σ , while the inverse problem uses the approximated u and calculates the conductivity σ given that σ | ∂ Ω = σ 0 .
Revised: With our suggested method, the forward problem determines the electrical potential u in the whole domain Ω given σ , while the inverse problem uses the approximated u and calculates the conductivity σ given that σ | ∂ Ω = σ 0 .

Original:  F ORWARD P ROBLEM
Revised:  F ORWARD P ROBLEM

Original: In the forward problem the conductivity σ ( x i ) and boundary conditions u 0 ( x b ) are given for random points set { x i } ∈ Ω ⊂ R 2 , { x b } ∈ ∂ Ω ⊂ R 2 with sets size of N s and N b respectively.
Revised: In the forward problem the conductivity σ ( x i ) and boundary conditions u 0 ( x b ) are given for random points set { x i } ∈ Ω ⊂ R 2 , { x b } ∈ ∂ Ω ⊂ R 2 with sets size of N s and N b respectively.

Original: A neural network having the architecture shown in Figure 1 approximates u ( x ) .
Revised: A neural network having the architecture shown in Figure 1 approximates u ( x ) .

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: The cost function equation 3 is then rewritten as
Revised: The cost function equation 3 is then rewritten as

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term is the L 2 norm of the differential operator, the second term is a relaxed version of the inﬁnity norm where we take the mean value of the top-K values of |L i | .
Revised: The ﬁrst term is the L 2 norm of the differential operator, the second term is a relaxed version of the inﬁnity norm where we take the mean value of the top-K values of |L i | .

Original: There is some balance between small K which accounts for few error points and can have negligible effect in the training procedure, and a large K which resembles the effect of the L 2 norm.
Revised: There is some balance between small K which accounts for few error points and can have negligible effect in the training procedure, and a large K which resembles the effect of the L 2 norm.

Original: In all our experiments we empirically selected K = 40 as an intermediate value.
Revised: In all our experiments we empirically selected K = 40 as an intermediate value.

Original: The third term imposes the boundary conditions and the last term serves as a regularizer of the network parameters.
Revised: The third term imposes the boundary conditions and the last term serves as a regularizer of the network parameters.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: The ﬁrst phantom is shown in Figure 3 left.
Revised: The ﬁrst phantom is shown in Figure 3 left.

Original: The impedance values associated with the background ellipses and circle were 1 , 5 and 2 respectively.
Revised: The impedance values associated with the background ellipses and circle were 1 , 5 and 2 respectively.

Original: The original piecewise constant function σ was slightly smoothed by a Gaussian kernel.
Revised: The original piecewise constant function σ was slightly smoothed by a Gaussian kernel.

Original: Figure 10 in the Appendix shows the forward problem results for current ψ with n = 1 and ϕ = π/ 8 .
Revised: Figure 10 in the Appendix shows the forward problem results for current ψ with n = 1 and ϕ = π/ 8 .

Original: The left column is the FEM solution which is referred to as ground truth, where the top row indicates the solution u ( x ) and the bottom row the derivative of u ( x ) with respect to the ﬁrst coordinate x calculated as the ﬁnite difference approximation of the FEM result.
Revised: The left column is the FEM solution which is referred to as ground truth, where the top row indicates the solution u ( x ) and the bottom row the derivative of u ( x ) with respect to the ﬁrst coordinate x calculated as the ﬁnite difference approximation of the FEM result.

Original: The middle column depicts the outcome of the proposed method where ∂u/∂x is an analytical derivative of our result.
Revised: The middle column depicts the outcome of the proposed method where ∂u/∂x is an analytical derivative of our result.

Original: The right column shows the outcome of the DGM method Sirignano & Spiliopoulos (2017) which is a special case of equation 3 with λ = 1 , µ = 0 , L 2 norm of the boundary conditions constraint with no regularizers.
Revised: The right column shows the outcome of the DGM method Sirignano & Spiliopoulos (2017) which is a special case of equation 3 with λ = 1 , µ = 0 , L 2 norm of the boundary conditions constraint with no regularizers.

Original: Quantitative results of the mean square error (MSE) and peak signal-to-noise ratio (PSNR) are summarized in Table 1.
Revised: Quantitative results of the mean square error (MSE) and peak signal-to-noise ratio (PSNR) are summarized in Table 1.

Original: Clearly, the proposed method outperforms the DGM method since the weighting parameters, the L ∞ norm and the network weights regularization play a signiﬁcant role in the loss function.
Revised: Clearly, the proposed method outperforms the DGM method since the weighting parameters, the L ∞ norm and the network weights regularization play a signiﬁcant role in the loss function.

Original: [Table]
Revised: [Table]

Original: 9.17e37.
Revised: 9.17e37.

Original: 3 , Ω π/1.74e31.
Revised: 3 , Ω π/1.74e31.

Original: 2.24e41.
Revised: 2.24e41.

Original: [Table]
Revised: [Table]

Original: The forward problem was repeated using phantom 2 where the background and circle conductivities were 1 and 0 .
Revised: The forward problem was repeated using phantom 2 where the background and circle conductivities were 1 and 0 .

Original: 2 respectively (Figure 3, middle).
Revised: 2 respectively (Figure 3, middle).

Original: Four different current combinations were applied.
Revised: Four different current combinations were applied.

Original: Quantitative results are summarized in rows 1 - 4 of Table 2.
Revised: Quantitative results are summarized in rows 1 - 4 of Table 2.

Original: Figure 11 in the Appendix shows the results for both u and ∂u/∂x for n = 2 and ϕ = π/ 2 .
Revised: Figure 11 in the Appendix shows the results for both u and ∂u/∂x for n = 2 and ϕ = π/ 2 .

Original: The right column presents the relative error deﬁned as e ( x, y )
Revised: The right column presents the relative error deﬁned as e ( x, y )

Original: = ( u fem ( x, y ) − u ( x, y )) / max( u fem ) .
Revised: = ( u fem ( x, y ) − u ( x, y )) / max( u fem ) .

Original:  I NVERSE P ROBLEM FOR EIT
Revised:  I NVERSE P ROBLEM FOR EIT

Original: In the inverse problem, the electrical potential u ( x ) is known while σ ( x ) is unknown.
Revised: In the inverse problem, the electrical potential u ( x ) is known while σ ( x ) is unknown.

Original: Since we have a network which approximates u ( x ) , we can evaluate it at any point x .
Revised: Since we have a network which approximates u ( x ) , we can evaluate it at any point x .

Original: The objective function equation 4 then takes the form
Revised: The objective function equation 4 then takes the form

Original: [Equation]
Revised: [Equation]

Original: As in the forward problem, the ﬁrst two terms enforce σ to satisfy the PDE, where L i is deﬁned in equation 7.
Revised: As in the forward problem, the ﬁrst two terms enforce σ to satisfy the PDE, where L i is deﬁned in equation 7.

Original: The third term imposes the boundary conditions, and the fourth regularizes the network parameters.
Revised: The third term imposes the boundary conditions, and the fourth regularizes the network parameters.

Original: A physical a priori knowledge regarding σ is exploited in this application.
Revised: A physical a priori knowledge regarding σ is exploited in this application.

Original: The conductivity is assumed to have well deﬁned sub-regions.
Revised: The conductivity is assumed to have well deﬁned sub-regions.

Original: We therefore design the ﬁfth term to have sparse edges via the total variation regularization ( p = 1 ).
Revised: We therefore design the ﬁfth term to have sparse edges via the total variation regularization ( p = 1 ).

Original: Additional inverse problem generalization may exploit multiple u approximations for several currents ψ j .
Revised: Additional inverse problem generalization may exploit multiple u approximations for several currents ψ j .

Original: The σ calculation thus, simultaneously relies on all { u j } , resulting a more stable solution.
Revised: The σ calculation thus, simultaneously relies on all { u j } , resulting a more stable solution.

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: Then equation 9 is generalized to
Revised: Then equation 9 is generalized to

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Adequate reconstruction results by the generalized inverse problem are shown in Figure 4 and rows 1 - 2 in Table 3.
Revised: Adequate reconstruction results by the generalized inverse problem are shown in Figure 4 and rows 1 - 2 in Table 3.

Original: In both phantoms we used four current combinations ψ n,ϕ : (1 , 0) , (1 , π/ 4) , (2 , 0) , (2 , π/ 4) , and σ 0 = 1 .
Revised: In both phantoms we used four current combinations ψ n,ϕ : (1 , 0) , (1 , π/ 4) , (2 , 0) , (2 , π/ 4) , and σ 0 = 1 .

Original:  F REE S HAPE G EOMETRY
Revised:  F REE S HAPE G EOMETRY

Original: We applied the proposed method to arbitrary domains Ω 1 , Ω 2 and Ω 3 , see Figure 5.
Revised: We applied the proposed method to arbitrary domains Ω 1 , Ω 2 and Ω 3 , see Figure 5.

Original: The random sample points within the domain and along its boundary can be easily obtained as can be seen in Figure 5 right.
Revised: The random sample points within the domain and along its boundary can be easily obtained as can be seen in Figure 5 right.

Original: Forward problem results applied on phantom 3 are presented in rows 5 - 10 of
Revised: Forward problem results applied on phantom 3 are presented in rows 5 - 10 of

Original: Table 2.
Revised: Table 2.

Original: Figure 12 in the Appendix shows the results for n = 2 and ϕ =
Revised: Figure 12 in the Appendix shows the results for n = 2 and ϕ =

Original: π/ 4 for the three domains.
Revised: π/ 4 for the three domains.

Original: The outcome of the generalized inverse problem equation 12 is shown in Figure 6 and rows 3 - 5 of
Revised: The outcome of the generalized inverse problem equation 12 is shown in Figure 6 and rows 3 - 5 of

Original: Table 3.
Revised: Table 3.

Original:  A DDITIONAL I NVERSE P ROBLEM E XAMPLES
Revised:  A DDITIONAL I NVERSE P ROBLEM E XAMPLES

Original: In this section we exemplify the proposed algorithm with additional inverse problem setting introduced by Xu & Darve (2019).
Revised: In this section we exemplify the proposed algorithm with additional inverse problem setting introduced by Xu & Darve (2019).

Original: Unknown parameters which are approximated by neural networks are inferred out of few measurements.
Revised: Unknown parameters which are approximated by neural networks are inferred out of few measurements.

Original: The differential operators are approximated in ﬁnite differences schemes discretized by ∆ t and ∆ h , the temporal and spatial grid spacings.
Revised: The differential operators are approximated in ﬁnite differences schemes discretized by ∆ t and ∆ h , the temporal and spatial grid spacings.

Original: In the two following examples we applied the cost function equation 4 with total variation and network parameters regularizations.
Revised: In the two following examples we applied the cost function equation 4 with total variation and network parameters regularizations.

Original: 7. D IFFUSION E QUATION
Revised: 7. D IFFUSION E QUATION

Original: Diffusion PDE is given by
Revised: Diffusion PDE is given by

Original: [Equation]
Revised: [Equation]

Original: where t denotes the time and the conductivity C ( x ) is the unknown parameter.
Revised: where t denotes the time and the conductivity C ( x ) is the unknown parameter.

Original: In the following we solve the equation in 2D where x :=
Revised: In the following we solve the equation in 2D where x :=

Original: ( x, y ) .
Revised: ( x, y ) .

Original: Given two measurements at adjacent times u ( x, y, t 1 ) and u ( x, y, t 1 +∆ t ) , the conductivity C ( x, y ) is approximated as a neural network and is determined as the minimizer of equation 4 with
Revised: Given two measurements at adjacent times u ( x, y, t 1 ) and u ( x, y, t 1 +∆ t ) , the conductivity C ( x, y ) is approximated as a neural network and is determined as the minimizer of equation 4 with

Original: [Equation]
Revised: [Equation]

Original: where D 2 is the central scheme approximation for the Laplacian on 3x3 stencil.
Revised: where D 2 is the central scheme approximation for the Laplacian on 3x3 stencil.

Original: The analytical 2D conductivity function is given by
Revised: The analytical 2D conductivity function is given by

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: The measurements were taken at t 1 = 0 .
Revised: The measurements were taken at t 1 = 0 .

Original: 1 with ∆ t = 0 .
Revised: 1 with ∆ t = 0 .

Original: 001 and ∆ h = 0 .
Revised: 001 and ∆ h = 0 .

Original: The result is shown in
Revised: The result is shown in

Original: Figure 7.
Revised: Figure 7.

Original: As can be seen the smoothness regularizer and the L ∞ terms increase the reconstruction accuracy.
Revised: As can be seen the smoothness regularizer and the L ∞ terms increase the reconstruction accuracy.

Original: In addition, we added few Gaussian noise levels to the measurements.
Revised: In addition, we added few Gaussian noise levels to the measurements.

Original: Quantitative results are summarized in rows 1 − 4 of Table 4.
Revised: Quantitative results are summarized in rows 1 − 4 of Table 4.

Original: [Figure]
Revised: [Figure]

Original: The second example is the wave equation given by
Revised: The second example is the wave equation given by

Original: [Equation]
Revised: [Equation]

Original: ] 2 × [0 , T ] .
Revised: ] 2 × [0 , T ] .

Original: The unknown parameter is the velocity ﬁeld η ( x, y ) .
Revised: The unknown parameter is the velocity ﬁeld η ( x, y ) .

Original: Its analytical value is given by
Revised: Its analytical value is given by

Original: [Equation]
Revised: [Equation]

Original: and the two initial conditions are
Revised: and the two initial conditions are

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Solutions for all time steps are recursively calculated using the ﬁnite differences scheme using the correct η ( x, y ) .
Revised: Solutions for all time steps are recursively calculated using the ﬁnite differences scheme using the correct η ( x, y ) .

Original: Given three consecutive values of u ( x, y, t ) the velocity ﬁeld η ( x, y ) is extracted by the minimization of equation 4 where
Revised: Given three consecutive values of u ( x, y, t ) the velocity ﬁeld η ( x, y ) is extracted by the minimization of equation 4 where

Original: [Equation]
Revised: [Equation]

Original: Reconstruction result is depicted in Figure 8 showing the advantage of the proposed method.
Revised: Reconstruction result is depicted in Figure 8 showing the advantage of the proposed method.

Original: Additional quantitave results of noisy measurements are given in rows 5 - 8 of Table 4.
Revised: Additional quantitave results of noisy measurements are given in rows 5 - 8 of Table 4.

Original: [Table]
Revised: [Table]

Original:  I MPLEMENTATION D ETAILS
Revised:  I MPLEMENTATION D ETAILS

Original: The network architecture had 4 layers having 26 , 26 , 26 and 10 neurons.
Revised: The network architecture had 4 layers having 26 , 26 , 26 and 10 neurons.

Original: The algorithm was implemented by TensorFlow Ten (2015) using the ADAM optimizer which is a variant of the SGD algorithm.
Revised: The algorithm was implemented by TensorFlow Ten (2015) using the ADAM optimizer which is a variant of the SGD algorithm.

Original: We used almost the same hyper parameters set in our experiments.
Revised: We used almost the same hyper parameters set in our experiments.

Original: Batch size= 1000 , decaying learning rate starting at 5 e − 3 .
Revised: Batch size= 1000 , decaying learning rate starting at 5 e − 3 .

Original: The learning rate was factored by 0 .
Revised: The learning rate was factored by 0 .

Original: 8 every 200 epochs,
Revised: 8 every 200 epochs,

Original: N s = 80000 , N b = 1200 , λ = 0 .
Revised: N s = 80000 , N b = 1200 , λ = 0 .

Original: 01 , α = 1 e − 8 , K = 40 , µ = 0 .
Revised: 01 , α = 1 e − 8 , K = 40 , µ = 0 .

Original: 01 , and β = 0 .
Revised: 01 , and β = 0 .

Original: In the diffusion and wave equation case we set λ
Revised: In the diffusion and wave equation case we set λ

Original: = 1 , β
Revised: = 1 , β

Original: = 1 e − 4 , µ = 1 e − 4 and α = 1 e − 5 .
Revised: = 1 e − 4 , µ = 1 e − 4 and α = 1 e − 5 .

Original: Running time, for the EIT, on Intel i7-8650u CPU was about 15 minutes for the forward problem and 13 minutes for the generalized inverse problem.
Revised: Running time, for the EIT, on Intel i7-8650u CPU was about 15 minutes for the forward problem and 13 minutes for the generalized inverse problem.

Original: For the diffusion and wave equation the running time was aboutminutes.
Revised: For the diffusion and wave equation the running time was aboutminutes.

Original: Finally, we performed a sensitivity analysis of the algorithm with respect to the number of sample points N s .
Revised: Finally, we performed a sensitivity analysis of the algorithm with respect to the number of sample points N s .

Original: Figure 9 shows the sensitivity of the forward problem algorithm.
Revised: Figure 9 shows the sensitivity of the forward problem algorithm.

Original: As expected, with large number of points we obtain a plateau while with small and intermediate number (up to 50000 )
Revised: As expected, with large number of points we obtain a plateau while with small and intermediate number (up to 50000 )

Original: the error is not monotonic due to local minima.
Revised: the error is not monotonic due to local minima.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: Deep networks by their nature use compositions of simple functions such as matrix multiplication and non-linear activations like sigmoid or tanh.
Revised: Deep networks by their nature use compositions of simple functions such as matrix multiplication and non-linear activations like sigmoid or tanh.

Original: This structure (i) enables the approximation of an arbitrary function Hornik (1989) and (ii) is inherently differentiable.
Revised: This structure (i) enables the approximation of an arbitrary function Hornik (1989) and (ii) is inherently differentiable.

Original: The network architecture dictates the number of degrees of freedom which in turn enables the expressibility of complex functions.
Revised: The network architecture dictates the number of degrees of freedom which in turn enables the expressibility of complex functions.

Original: In this work we present a uniﬁed framework for the solution of forward and inverse problems in partial differential equations in an arbitrary domain.
Revised: In this work we present a uniﬁed framework for the solution of forward and inverse problems in partial differential equations in an arbitrary domain.

Original: The algorithm relies on direct approximation of the unknown function by a neural network which yields an analytical smooth solution.
Revised: The algorithm relies on direct approximation of the unknown function by a neural network which yields an analytical smooth solution.

Original: The network is trained to satisfy the PDE and boundary conditions in an unsupervised fashion by the minimization of a cost function.
Revised: The network is trained to satisfy the PDE and boundary conditions in an unsupervised fashion by the minimization of a cost function.

Original: The optimization procedure depends on random points set within the domain and its boundary.
Revised: The optimization procedure depends on random points set within the domain and its boundary.

Original: The problem is therefore mesh free with free-form domain.
Revised: The problem is therefore mesh free with free-form domain.

Original: While recent works formulate the cost function in both forward and inverse problems in the L 2 sense, we introduce a broadened framework.
Revised: While recent works formulate the cost function in both forward and inverse problems in the L 2 sense, we introduce a broadened framework.

Original: The relaxed L ∞ ﬁdelity term was designed to approximate a strong solution and control possible discrepancies between the weak and the strong solutions.
Revised: The relaxed L ∞ ﬁdelity term was designed to approximate a strong solution and control possible discrepancies between the weak and the strong solutions.

Original: The L 1 norm on the boundary conditions is more robust to measurement noise than the L 2 norm.
Revised: The L 1 norm on the boundary conditions is more robust to measurement noise than the L 2 norm.

Original: The additional regularizer (total variation in our examples) has a signiﬁcant role in the optimization process where known physical properties like smoothness are imposed on the solution.
Revised: The additional regularizer (total variation in our examples) has a signiﬁcant role in the optimization process where known physical properties like smoothness are imposed on the solution.

Original: Lastly, the network regularizer controls the network weights and prevents over ﬁtting.
Revised: Lastly, the network regularizer controls the network weights and prevents over ﬁtting.

Original: These additional terms make the proposed algorithm more accurate and robust compared to other learning-based methods.
Revised: These additional terms make the proposed algorithm more accurate and robust compared to other learning-based methods.

Original: We also stress the robustness of our approach exempliﬁed by having almost the same hyper parameters set in our experiments.
Revised: We also stress the robustness of our approach exempliﬁed by having almost the same hyper parameters set in our experiments.

Original: This framework alleviates several problems of the ﬁnite differences and ﬁnite elements methods.
Revised: This framework alleviates several problems of the ﬁnite differences and ﬁnite elements methods.

Original: In particular meshing, discretization and derivatives approximation are solved in a simple and natural way.
Revised: In particular meshing, discretization and derivatives approximation are solved in a simple and natural way.

Original: Numerical solutions of PDEs in an arbitrary domain are of extreme importance, in particular in medical imaging applications.
Revised: Numerical solutions of PDEs in an arbitrary domain are of extreme importance, in particular in medical imaging applications.

Original: The framework is demonstrated by an elliptic system in R 2 applied to Electrical Impedance Tomography for both forward and inverse problems, diffusion and wave equations.
Revised: The framework is demonstrated by an elliptic system in R 2 applied to Electrical Impedance Tomography for both forward and inverse problems, diffusion and wave equations.

Original: Promising results were achieved for complex and non monotonic functions.
Revised: Promising results were achieved for complex and non monotonic functions.

Original: Rigorous analysis of the approximation error and its relation to the network architecture and design are under current study.
Revised: Rigorous analysis of the approximation error and its relation to the network architecture and design are under current study.

Original: Future research includes also full tomography solution, higher dimensional problems and other classes of PDEs such as dynamic non-linear equations (Burgers, Navier-Stokes etc.)
Revised: Future research includes also full tomography solution, higher dimensional problems and other classes of PDEs such as dynamic non-linear equations (Burgers, Navier-Stokes etc.)