Original: M ESH -F REE U NSUPERVISED L EARNING -B ASED PDE S OLVER OF F ORWARD AND I NVERSE PROBLEMS
Revised: M ESH -F REE U NSUPERVISED L EARNING -B ASED PDE S OLVER OF F ORWARD AND I NVERSE PROBLEMS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We introduce a novel neural network-based partial differential equations solver for forward and inverse problems.
Revised: We introduce a novel neural network-based partial differential equations solver for forward and inverse problems.

Original: The solver is grid free, mesh free and shape free, and the solution is approximated by a neural network.
Revised: The solver is grid free, mesh free and shape free, and the solution is approximated by a neural network.

Original: We employ an unsupervised approach such that the input to the network is a points set in an arbitrary domain, and the output is the set of the corresponding function values.
Revised: We employ an unsupervised approach such that the input to the network is a points set in an arbitrary domain, and the output is the set of the corresponding function values.

Original: The network is trained to minimize deviations of the learned function from the PDE solution and satisfy the boundary conditions.
Revised: The network is trained to minimize deviations of the learned function from the PDE solution and satisfy the boundary conditions.

Original: The resulting solution in turn is an explicit smooth differentiable function with a known analytical form.
Revised: The resulting solution in turn is an explicit smooth differentiable function with a known analytical form.

Original: Unlike other numerical methods such as ﬁnite differences and ﬁnite elements, the derivatives of the desired function can be analytically calculated to any order.
Revised: Unlike other numerical methods such as ﬁnite differences and ﬁnite elements, the derivatives of the desired function can be analytically calculated to any order.

Original: This framework therefore, enables the solution of high order non-linear PDEs.
Revised: This framework therefore, enables the solution of high order non-linear PDEs.

Original: The proposed algorithm is a uniﬁed formulation of both forward and inverse problems where the optimized loss function consists of few elements: ﬁdelity terms of Land L ∞ norms, boundary conditions constraints and additional regularizers.
Revised: The proposed algorithm is a uniﬁed formulation of both forward and inverse problems where the optimized loss function consists of few elements: ﬁdelity terms of Land L ∞ norms, boundary conditions constraints and additional regularizers.

Original: This setting is ﬂexible in the sense that regularizers can be tailored to speciﬁc problems.
Revised: This setting is ﬂexible in the sense that regularizers can be tailored to speciﬁc problems.

Original: We demonstrate our method on a free shape 2D second order elliptical system with application to Electrical Impedance Tomography (EIT).
Revised: We demonstrate our method on a free shape 2D second order elliptical system with application to Electrical Impedance Tomography (EIT).

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Partial differential equations are fundamental in science and mathematics with wide applications in medical imaging, signal processing, computer vision, remote sensing, electromagnetism and more.
Revised: Partial differential equations are fundamental in science and mathematics with wide applications in medical imaging, signal processing, computer vision, remote sensing, electromagnetism and more.

Original: Classical methods such as ﬁnite differences, ﬁnite volume and ﬁnite elements are numerical discretization-based methods where the domain is divided into a uniform grid or polygon mesh.
Revised: Classical methods such as ﬁnite differences, ﬁnite volume and ﬁnite elements are numerical discretization-based methods where the domain is divided into a uniform grid or polygon mesh.

Original: The differential equation is then reduced to a system of algebraic equations.
Revised: The differential equation is then reduced to a system of algebraic equations.

Original: These methods may have some limitations: the solution is numeric and may suffer from high condition number, highly dependent on the discretization and even the second derivative is sensitive to noise.
Revised: These methods may have some limitations: the solution is numeric and may suffer from high condition number, highly dependent on the discretization and even the second derivative is sensitive to noise.

Original: In the last few years, deep learning and neural network-based algorithms are extensively used in pattern recognition, image processing, computer vision and more.
Revised: In the last few years, deep learning and neural network-based algorithms are extensively used in pattern recognition, image processing, computer vision and more.

Original: Recently, the deep learning approach had been adopted to the ﬁeld of PDEs as well by converting the problem into a machine learning one.
Revised: Recently, the deep learning approach had been adopted to the ﬁeld of PDEs as well by converting the problem into a machine learning one.

Original: In Supervised learning , the network maps an input to an output based on example input-output pairs.
Revised: In Supervised learning , the network maps an input to an output based on example input-output pairs.

Original: This strategy is used in inverse problems, where the input to the network is a set of observations/measurements (e.g. x-ray tomography, ultrasound) and the output is the set of parameters of interest (tissue density etc.)
Revised: This strategy is used in inverse problems, where the input to the network is a set of observations/measurements (e.g. x-ray tomography, ultrasound) and the output is the set of parameters of interest (tissue density etc.)

Original: Feigin et al.
Revised: Feigin et al.

Original: ; Lucas et al.
Revised: ; Lucas et al.

Original: ; McCann et al.
Revised: ; McCann et al.

Original: Seo et al.
Revised: Seo et al.

Original: Unsupervised learning on the other hand is a self-learning mechanism where the natural structure presents within a set of data points is inferred.
Revised: Unsupervised learning on the other hand is a self-learning mechanism where the natural structure presents within a set of data points is inferred.

Original: Algorithms for forward and inverse problems in partial differential equations via unsupervised learning were recently introduced.
Revised: Algorithms for forward and inverse problems in partial differential equations via unsupervised learning were recently introduced.

Original: The indirect approach utilizes a neural network as a component in the solution.
Revised: The indirect approach utilizes a neural network as a component in the solution.

Original: Li et al.
Revised: Li et al.

Original: (2018) for example, proposed the NETT (Network Tikhonov) approach to inverse problems.
Revised: (2018) for example, proposed the NETT (Network Tikhonov) approach to inverse problems.

Original: NETT considers regularized solutions having a small value of a regularizer deﬁned by a trained neural network.
Revised: NETT considers regularized solutions having a small value of a regularizer deﬁned by a trained neural network.

Original: Khoo & Ying (2018) introduced a novel neural network architecture, SwitchNet, for solving the wave equation based inverse scattering problems via providing maps between the scatterers and the scattered ﬁeld.
Revised: Khoo & Ying (2018) introduced a novel neural network architecture, SwitchNet, for solving the wave equation based inverse scattering problems via providing maps between the scatterers and the scattered ﬁeld.

Original: Han et al.
Revised: Han et al.

Original: (2018) developed a deep learning-based approach that can handle general high-dimensional parabolic PDEs.
Revised: (2018) developed a deep learning-based approach that can handle general high-dimensional parabolic PDEs.

Original: To this end, the PDEs are reformulated using backward stochastic differential equations.
Revised: To this end, the PDEs are reformulated using backward stochastic differential equations.

Original: The latter is solved by a temporal discretization and the gradient of the unknown solution at each time step is approximated by neural network.
Revised: The latter is solved by a temporal discretization and the gradient of the unknown solution at each time step is approximated by neural network.

Original: Li et al.
Revised: Li et al.

Original: (2019) approximate the solution map of linear and nonlinear problems by a deep network.
Revised: (2019) approximate the solution map of linear and nonlinear problems by a deep network.

Original: Direct algorithms solve the forward problem PDEs by directly approximating the solution with a deep neural network.
Revised: Direct algorithms solve the forward problem PDEs by directly approximating the solution with a deep neural network.

Original: The network parameters are determined by the optimization of a cost function such that the optimal solution satisﬁes the PDE, boundary conditions and initial conditions.
Revised: The network parameters are determined by the optimization of a cost function such that the optimal solution satisﬁes the PDE, boundary conditions and initial conditions.

Original: Chiaramonte & Kiener (2017) addressed the forward problem by constructing a one layer network which satisﬁes the PDE within the domain.
Revised: Chiaramonte & Kiener (2017) addressed the forward problem by constructing a one layer network which satisﬁes the PDE within the domain.

Original: The boundary conditions were analytically integrated in the cost function.
Revised: The boundary conditions were analytically integrated in the cost function.

Original: They demonstrated their algorithm on the Laplace and hyperbolic conservation law
Revised: They demonstrated their algorithm on the Laplace and hyperbolic conservation law

Original: PDEs.
Revised: PDEs.

Original: Sirignano & Spiliopoulos (2017) proposed a deep learning forward problem solver for high dimensional PDEs.
Revised: Sirignano & Spiliopoulos (2017) proposed a deep learning forward problem solver for high dimensional PDEs.

Original: Their algorithm was demonstrated on the American option free-boundary equation.
Revised: Their algorithm was demonstrated on the American option free-boundary equation.

Original: Raissi et al.
Revised: Raissi et al.

Original: (2017) focused on continuous time models and solved the Burgers and Shr¨odinger equations.
Revised: (2017) focused on continuous time models and solved the Burgers and Shr¨odinger equations.

Original: In this work we address the forward and inverse PDE problems via a direct unsupervised method.
Revised: In this work we address the forward and inverse PDE problems via a direct unsupervised method.

Original: Our key contributions are four fold: (1) inverse problems can be solved in the same framework as the forward problems.
Revised: Our key contributions are four fold: (1) inverse problems can be solved in the same framework as the forward problems.

Original: In the forward part we extend the standard L 2 -based ﬁdelity term in the cost function by adding L ∞ -like norm.
Revised: In the forward part we extend the standard L 2 -based ﬁdelity term in the cost function by adding L ∞ -like norm.

Original: Moreover, (3) some regularization terms which impose a-priori knowledge on the solution can be easily incorporated.
Revised: Moreover, (3) some regularization terms which impose a-priori knowledge on the solution can be easily incorporated.

Original: (4) An important feature of our construction is the ability to handle free-form domain in a mesh free manner.
Revised: (4) An important feature of our construction is the ability to handle free-form domain in a mesh free manner.

Original: We demonstrate our algorithm by a second order elliptic equation, in particular the Electrical Impedance Tomography (EIT) application on circular and three other arbitrary domains.
Revised: We demonstrate our algorithm by a second order elliptic equation, in particular the Electrical Impedance Tomography (EIT) application on circular and three other arbitrary domains.

Original:  M ATHEMATICAL F ORMULATION
Revised:  M ATHEMATICAL F ORMULATION

Original: Let Ω be a bounded open and connected subset of R d , and
Revised: Let Ω be a bounded open and connected subset of R d , and

Original: A =
Revised: A =

Original: A ( x ) =
Revised: A ( x ) =

Original: ( a ij ( x )) be any given d × d symmetric positive deﬁnite matrix of functions for 1 ≤ i, j ≤ d .
Revised: ( a ij ( x )) be any given d × d symmetric positive deﬁnite matrix of functions for 1 ≤ i, j ≤ d .

Original: Let b
Revised: Let b

Original: = b ( x ) =
Revised: = b ( x ) =

Original: ( b j ( x )) be any given n-tuple of functions and let c
Revised: ( b j ( x )) be any given n-tuple of functions and let c

Original: = c ( x ) be any given function.
Revised: = c ( x ) be any given function.

Original: A second order operator L is said to be in divergence form, if L acting on some u has the form
Revised: A second order operator L is said to be in divergence form, if L acting on some u has the form

Original: [Equation]
Revised: [Equation]

Original: where we use the Einstein summation convention.
Revised: where we use the Einstein summation convention.

Original: Consider the partial differential problem with
Revised: Consider the partial differential problem with

Original: Dirichlet boundary conditions
Revised: Dirichlet boundary conditions

Original: [Equation]
Revised: [Equation]

Original: The forward problem solves u given the coefﬁcients θ := { a ij ( x ) , b j ( x ) , c ( x ) } while the inverse problem determines the coefﬁcients set θ given u .
Revised: The forward problem solves u given the coefﬁcients θ := { a ij ( x ) , b j ( x ) , c ( x ) } while the inverse problem determines the coefﬁcients set θ given u .

Original: The proposed algorithm approximates the solutions in both problems by neural networks u ( x ; w u ) , a ij ( x ; w ij ) , b j ( x ; w b ) , c ( x ; w c ) such that the networks are parameterized by w u , w ij , w b , w c , and the input to the network is x ∈ R d .
Revised: The proposed algorithm approximates the solutions in both problems by neural networks u ( x ; w u ) , a ij ( x ; w ij ) , b j ( x ; w b ) , c ( x ; w c ) such that the networks are parameterized by w u , w ij , w b , w c , and the input to the network is x ∈ R d .

Original: Figure 1 depicts a network architecture of u in R 2 .
Revised: Figure 1 depicts a network architecture of u in R 2 .

Original: The network consists of few fully connected layers with tanh activation and linear sum in the last layer.
Revised: The network consists of few fully connected layers with tanh activation and linear sum in the last layer.

Original: The network is trained to satisfy the PDE with boundary conditions by minimizing a cost function.
Revised: The network is trained to satisfy the PDE with boundary conditions by minimizing a cost function.

Original: In the forward problem
Revised: In the forward problem

Original: [Equation]
Revised: [Equation]

Original: and in the inverse problem
Revised: and in the inverse problem

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: The ﬁrst two terms enforce the solution to satisfy the equation.
Revised: The ﬁrst two terms enforce the solution to satisfy the equation.

Original: The ﬁrst term minimizes the error in L 2 sense while the second term minimizes the maximal error.
Revised: The ﬁrst term minimizes the error in L 2 sense while the second term minimizes the maximal error.

Original: The second term is important since the L 2 term only forces the equation up to a set of zero measure.
Revised: Ideally, the solution to the PDE has to be satisﬁed at every point x . This is considered as a strong solution. In the deep learning framework the optimization is carried out on mini batches of points. The ﬁrst term minimizes the operator in the integral or average sense and is therefore insensitive to a point jump in the value of the integrand which yields a weak solution.

Revised: This manifest itself by point discontinuities u ( x ) that are seen empirically as well.

Original: The L ∞ term  takes care of possible outliers.
Revised: The L ∞ term promotes a strong solution since it takes care of possible error points.

Original: The third term imposes boundary conditions  and  the  last term is a regularizer which can be tailored to the application.
Revised: The third term imposes boundary conditions where u 0 and θ 0 are the boundary values of u ( x ) and θ ( x ) . The last term is a regularizer which can be tailored to the application.

Original: There are few advantages of this setting.
Revised: There are few advantages of this setting.

Original: First, the solutions are smooth analytic functions and are therefore analytically differentiable .
Revised: First, the solutions are smooth analytic functions and are therefore analytically differentiable .

Revised: It was recently proven that for such functions, e.g. Lipchitz with constant ν , the approximation of the function by a fully connected network Θ is given by

Revised: [Equation]

Revised: where p ∈ [0 , ∞ ) , N number of neurons in a layer and L number of layers Shen et al.

Original: In addition, this framework enables setting a prior knowledge on the solution by designing the regularizers R F and R I .
Revised: In addition, this framework enables setting a prior knowledge on the solution by designing the regularizers R F and R I .

Original: Moreover, the training procedure is mesh free.
Revised: Moreover, the training procedure is mesh free.

Original: Unlike ﬁnite differences or ﬁnite elements methods, we use random points in the domain and its boundary in the course of the optimization of equation 3 and equation 4, see Figure 2.
Revised: Unlike ﬁnite differences or ﬁnite elements methods, we use random points in the domain and its boundary in the course of the optimization of equation 3 and equation 4, see Figure 2.

Original: This means that the solution does not depend upon a coordinate mesh and we can also deﬁne an arbitrary regular domain Ω .
Revised: This means that the solution does not depend upon a coordinate mesh and we can also deﬁne an arbitrary regular domain Ω .

Original: Suppose u ( x ) is Lipchitz with constant ν , then the approximation of the network Θ is given by

Original: [Equation]

Original: where p ∈ [0 , ∞ ) , N number of neurons in a layer and L number of layers Shen et al.

Original:  A PPLICATION TO E LECTRICAL I MPEDANCE T OMOGRAPHY
Revised:  A PPLICATION TO E LECTRICAL I MPEDANCE T OMOGRAPHY

Original: Let us address a special case of equation 1,
Revised: Let us address a special case of equation 1,

Original: [Equation]
Revised: [Equation]

Original: We assume that 0 < σ ( x ) ∈ C 1 (Ω) , which guarantees existence and uniqueness of a solution u ∈ C 2 (Ω) Evans (2010).
Revised: We assume that 0 < σ ( x ) ∈ C 1 (Ω) , which guarantees existence and uniqueness of a solution u ∈ C 2 (Ω) Evans (2010).

Original: The elliptical system equation 5 was addressed by Siltanen et al.
Revised: The elliptical system equation 5 was addressed by Siltanen et al.

Original: (2000) in the context of Electrical
Revised: (2000) in the context of Electrical

Original: Impedance Tomography (EIT) which is a reconstruction method for the inverse conductivity problem.
Revised: Impedance Tomography (EIT) which is a reconstruction method for the inverse conductivity problem.

Original: The function σ indicates the electrical conductivity density, and u is the electrical potential.
Revised: The function σ indicates the electrical conductivity density, and u is the electrical potential.

Original: An electrical current
Revised: An electrical current

Original: [Equation]
Revised: [Equation]

Original: is applied on electrodes on the surface ∂ Ω , where κ is the angle in polar coordinate system along the domain boundary, ϕ is the phase and ν is the normal unit.
Revised: is applied on electrodes on the surface ∂ Ω , where κ is the angle in polar coordinate system along the domain boundary, ϕ is the phase and ν is the normal unit.

Original: The resulting voltage u | ∂ Ω = uis measured through the electrodes.
Revised: The resulting voltage u | ∂ Ω = u is measured through the electrodes.

Original: The conductivity σ is determined from the knowledge of the
Revised: The conductivity σ is determined from the knowledge of the

Original: Dirichlet-to-Neumann map or voltage-to-current map
Revised: Dirichlet-to-Neumann map or voltage-to-current map

Original: [Equation]
Revised: [Equation]

Original: Mueller & Siltanen (2012); Alsaker & Mueller (2018); Bera et al.
Revised: Mueller & Siltanen (2012); Alsaker & Mueller (2018) [Equation] Bera et al.

Original: ; Fan & Ying (2019).
Revised: [Equation] Fan & Ying (2019).

Original: We demonstrate our framework by solving the forward and inverse problem of equation 5 which is a ﬁrst step towards a full tomography.
Revised: We demonstrate our framework by solving the forward and inverse problem of equation 5 which is a ﬁrst step towards a full tomography.

Original: Following Mueller & Siltanen (2012), we simulate the voltage measurement u | ∂ Ω by the Finite Elements Method (FEM) given three variants of a conductivity phantom σ ( x ) depicted in Figure 3.
Revised: Following Mueller & Siltanen (2012), we simulate the voltage measurement u | ∂ Ω by the Finite Elements Method (FEM) given three variants of a conductivity phantom σ ( x ) depicted in Figure 3.

Original: We calculate the FEM solution with different triangle mesh densities and select as ground truth the one such that ﬁner meshes do not improve the numerical solution.
Revised: We calculate the FEM solution with different triangle mesh densities and select as ground truth the one such that ﬁner meshes do not improve the numerical solution.

Original: With our suggested method, the forward problem determines the electrical potential u in the whole domain Ω given σ , while the inverse problem uses the approximated u and calculates the conductivity σ given that σ | ∂ Ω = σ 0 .
Revised: With our suggested method, the forward problem determines the electrical potential u in the whole domain Ω given σ , while the inverse problem uses the approximated u and calculates the conductivity σ given that σ | ∂ Ω = σ 0 .

Original:  F ORWARD P ROBLEM
Revised:  F ORWARD P ROBLEM

Original: In the forward problem the conductivity σ ( x i ) and boundary conditions u 0 ( x b ) are given for random points set { x i } ∈ Ω ⊂ R 2 , { x b } ∈ ∂ Ω ⊂ R 2 with sets size of N s and N b respectively.
Revised: In the forward problem the conductivity σ ( x i ) and boundary conditions u 0 ( x b ) are given for random points set { x i } ∈ Ω ⊂ R 2 , { x b } ∈ ∂ Ω ⊂ R 2 with sets size of N s and N b respectively.

Original: A neural network having the architecture shown in Figure 1 approximates u ( x ) .
Revised: A neural network having the architecture shown in Figure 1 approximates u ( x ) .

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: The cost function equation 3 is then rewritten as
Revised: The cost function equation 3 is then rewritten as

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term is the L 2 norm of the differential operator, the second term is a relaxed version of the inﬁnity norm where we take the mean value of the top-K values of |L i | .
Revised: The ﬁrst term is the L 2 norm of the differential operator, the second term is a relaxed version of the inﬁnity norm where we take the mean value of the top-K values of |L i | .

Original: The third term imposes the boundary conditions and the last term serves as a regularizer of the network parameters.
Revised: The third term imposes the boundary conditions and the last term serves as a regularizer of the network parameters.

Original: The ﬁrst phantom is shown in Figure 3 left.
Revised: The ﬁrst phantom is shown in Figure 3 left.

Original: The impedance values associated with the background ellipses and circle were 1 , 5 and 2 respectively.
Revised: The impedance values associated with the background ellipses and circle were 1 , 5 and 2 respectively.

Original: The original piecewise constant function σ was slightly smoothed by a Gaussian kernel.
Revised: The original piecewise constant function σ was slightly smoothed by a Gaussian kernel.

Original: Figure 4 shows the forward problem results for current ψ with n = 1 and ϕ = π/ 8 .
Revised: Figure 4 shows the forward problem results for current ψ with n = 1 and ϕ = π/ 8 .

Original: The left column is the FEM solution which is referred to as ground truth, where the top row indicates the solution u and the bottom row the derivative of u with respect to x calculated as the ﬁnite difference approximation of the FEM result.
Revised: The left column is the FEM solution which is referred to as ground truth, where the top row indicates the solution u and the bottom row the derivative of u with respect to x calculated as the ﬁnite difference approximation of the FEM result.

Original: The middle column depicts the outcome of the proposed method where ∂u/∂x is an analytical derivative of our result.
Revised: The middle column depicts the outcome of the proposed method where ∂u/∂x is an analytical derivative of our result.

Original: The right column shows the outcome of the
Revised: The right column shows the outcome of the

Original: DGM method Sirignano & Spiliopoulos (2017) which is a special case of equation 3 with λ = 1 and µ = 0 .
Revised: DGM method Sirignano & Spiliopoulos (2017) which is a special case of equation 3 with λ = 1 and µ = 0 .

Original: Quantitative results of the Mean square error (MSE) and PSNR are summarized in Table 1.
Revised: Quantitative results of the Mean square error (MSE) and PSNR are summarized in Table 1.

Original: Clearly, the proposed method outperforms the DGM method since the weighting parameters and the
Revised: Clearly, the proposed method outperforms the DGM method since the weighting parameters and the

Original: L ∞ norm play a signiﬁcant role in the loss function. 
Revised: L ∞ norm play a signiﬁcant role in the loss function. [Figure]

Original: The forward problem was repeated using phantom 2 where the background and circle conductivities were 1 and 0 .
Revised: MSE and PSNR are noted in line 2 of Table [Figure] [Table] [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: The forward problem was repeated using phantom 2 where the background and circle conductivities were 1 and 0 .

Original: 2 respectively (Figure 3, middle).
Revised: 2 respectively (Figure 3, middle).

Original: Four different current combinations were applied.
Revised: Four different current combinations were applied.

Original: Quantitative results are summarized in lines 1 - 4 of Table 2.
Revised: Quantitative results are summarized in lines 1 - 4 of Table 2.

Original: Figure 5 shows the results for both u and ∂u/∂x for n = 2 and ϕ = π/ 2 .
Revised: Figure 5 shows the results for both u and ∂u/∂x for n = 2 and ϕ = π/ 2 .

Original: The right column presents the relative error deﬁned as e ( x, y )
Revised: The right column presents the relative error deﬁned as e ( x, y )

Original: = ( u fem ( x, y ) − u ( x, y )) / max( u fem ) .
Revised: = ( u fem ( x, y ) − u ( x, y )) / max( u fem ) .

Original: [Figure]

Original: MSE and PSNR are noted in line 2 of Table

Original: [Figure]

Original: [Table]

Original: [Table]

Original: 1.30e45.

Original: 8.63e37.

Original: 3 , Ω π/6.42e47.

Original: 5.64e39.

Original: 3 , Ω π/1.08e34.

Original: 2.61e41.

Original: 3 , Ω π/1.08e44.

Original: 4.51e40.

Original: 3 , Ω π/5.93e36.

Original: 2.74e41.

Original: 3 , Ω π/1.22e44.

Original: 9.17e37.

Original: 3 , Ω π/1.74e31.

Original: 2.24e41.  I NVERSE P ROBLEM 
Revised:  I NVERSE P ROBLEM FOR EIT

Original: In the inverse problem, the electrical potential u ( x ) is known while σ ( x ) is unknown.
Revised: In the inverse problem, the electrical potential u ( x ) is known while σ ( x ) is unknown.

Original: Since we have a network which approximates u ( x ) , we can evaluate it at any point x .
Revised: Since we have a network which approximates u ( x ) , we can evaluate it at any point x .

Original: The objective function equation 4 then takes the form
Revised: The objective function equation 4 then takes the form

Original: [Equation]
Revised: [Equation]

Original: As in the forward problem, the ﬁrst two terms enforce σ to satisfy the PDE, where L i is deﬁned in equation 6.
Revised: As in the forward problem, the ﬁrst two terms enforce σ to satisfy the PDE, where L i is deﬁned in equation 6.

Original: The third term imposes the boundary conditions, and the fourth regularizes the network parameters.
Revised: The third term imposes the boundary conditions, and the fourth regularizes the network parameters.

Original: The last term is the total variation regularization ( p = 1 ) which promotes the solution towards a piecewise constant function.[Table]
Revised: The last term is the total variation regularization ( p = 1 ) which promotes the solution towards a piecewise constant function.

Original: MSE and PSNR are summed up in lines 1 - 2 of Table 3.

Original: Additional inverse problem generalization may exploit multiple u approximations for several currents ψ j .
Revised: Additional inverse problem generalization may exploit multiple u approximations for several currents ψ j .

Original: The σ calculation thus, simultaneously relies on all { u j } , resulting a more stable solution,
Revised: The σ calculation thus, simultaneously relies on all { u j } , resulting a more stable solution,

Original: [Equation]
Revised: [Equation]

Revised: MSE and PSNR are summed up in lines 1 - 2 of Table 3.

Revised: [Table]

Original: Then equation 8 is generalized to
Revised: Then equation 8 is generalized to

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: I 
Revised: I (cid:16) σ

Original: [Equation]
Revised: [Equation]

Original: σ
Revised: x

Original: [Equation]
Revised: [Equation]

Original: Reconstruction results by the generalized inverse problem are shown in Figure 6 and lines 1 - 2 in
Revised: Reconstruction results by the generalized inverse problem are shown in Figure 6 and lines 1 - 2 in

Original: Table 3.
Revised: Table 3.

Original: In both phantoms we used four current combinations ψ n,ϕ : (1 , 0) , (1 , π/ 4) , (2 , 0) , (2 , π/ 4) , and σ 0 = 1 .
Revised: In both phantoms we used four current combinations ψ n,ϕ : (1 , 0) , (1 , π/ 4) , (2 , 0) , (2 , π/ 4) , and σ 0 = 1 .

Original:  F REE S HAPE G EOMETRY
Revised:  F REE S HAPE G EOMETRY

Original: We applied the proposed method to arbitrary domains Ω 1 , Ω 2 and Ω 3 , see Figure 7.
Revised: We applied the proposed method to arbitrary domains Ω 1 , Ω 2 and Ω 3 , see Figure 7.

Original: The random sample points within the domain and along its boundary can be easily obtained as can be seen in
Revised: The random sample points within the domain and along its boundary can be easily obtained as can be seen in

Original: Figure 7 right.
Revised: Figure 7 right.

Original: The forward problem results applied on phantom 3 are presented in lines 5 - 10 of
Revised: The forward problem results applied on phantom 3 are presented in lines 5 - 10 of

Original: Table 2.
Revised: Table 2.

Original: Figure 8 shows the results for n = 2 and ϕ =
Revised: Figure 8 shows the results for n = 2 and ϕ =

Original: π/ 4 for the three domains.
Revised: π/ 4 for the three domains.

Original: The outcome of the generalized inverse problem equation 11 is shown in Figure 12 and lines 3 - 5 of Table 3. 
Revised: The outcome of the generalized inverse problem equation 11 is shown in Figure 12 and lines 3 - 5 of Table 3. [Figure]

Revised:  A DDITIONAL I NVERSE P ROBLEM E XAMPLES

Revised: In this section we exemplify the proposed algorithm with additional inverse problem setting introduced by Xu & Darve (2019).

Revised: Unknown parameters which are parameterized by neural networks are inferred out of few measurements.

Revised: The differential operators are approximated in ﬁnite differences schemes discretized by ∆ t and ∆ h , the temporal and spatial grid spacings.

Original: A DDITIONAL I NVERSE P ROBLEM E XAMPLES 7. D IFFUSION E QUATION
Revised: 7. D IFFUSION E QUATION

Revised: Diffusion PDE is given by

Original: [Equation]
Revised: [Equation]

Revised: where the conductivity C ( x ) is the unknown parameter.

Revised: In the following we solve the equation in 2D where x :=

Revised: ( x, y ) .

Revised: Given two measurements at adjacent times u ( x, y, t 1 ) and u ( x, y, t 1 + ∆ t ) , the loss function then takes the form

Revised: [Equation]

Revised: where D 2 is the central scheme approximation for the Laplacian on 3x3 stencil.

Revised: The analytical 2D conductivity function is given by

Revised: [Equation]

Revised: −

Revised: [Equation]

Revised: xy )

Revised: [Equation]

Revised: , and

Revised: [Equation]

Revised: The measurements were taken at t 1 = 0 .

Revised: 1 with ∆ t = 0 .

Revised: 001 and ∆ h = 0 .

Original: Result is shown in Figure 9.
Revised: The result is shown in Figure 9.

Original: [Figure]
Revised: As can be seen the smoothness regularizer and the L ∞ terms increase the reconstruction accuracy. In addition, we added few Gaussian noise levels to the measurements. Quantitative results are summarized in lines 1 − 4 of Table 4.

Original: 7. W AVE E QUATION
Revised: W AVE E QUATION

Revised: The second example is the wave equation given by

Revised: [Equation]

Revised: The unknown parameter is the velocity ﬁeld η ( x, y ) .

Revised: Its analytical value is given by

Original: [Equation]
Revised: [Equation]

Revised: and the two initial conditions are

Revised: [Equation]

Revised: and

Revised: [Equation]

Revised: Solutions for all time steps t are iteratively calculated using the ﬁnite differences scheme.

Revised: The velocity ﬁeld η ( x, y ) is extracted using three measurements with the following loss function

Original: Result is shown in Figure 10.
Revised: [Equation] Reconstruction result is depicted in Figure 10 showing the advantage of the proposed method. Additional quantitave results of noisy measurements are in lines 5 - 8 of Table [Equation]

Original:  I MPLEMENTATION D ETAILS
Revised:  I MPLEMENTATION D ETAILS

Original: The network architecture has 4 layers having 26 , 26 , 26 and 10 neurons.
Revised: The network architecture has 4 layers having 26 , 26 , 26 and 10 neurons.

Original: The algorithm was implemented by TensorFlow Ten (2015) using the ADAM optimizer which is a variant of the SGD algorithm.
Revised: The algorithm was implemented by TensorFlow Ten (2015) using the ADAM optimizer which is a variant of the SGD algorithm.

Original: We used the same hyper parameters set in  all  our experiments.
Revised: We used the same hyper parameters set in almost all of our experiments.

Original: Batch size= 1000 , decaying learning rate starting at 5 e − 3 .
Revised: Batch size= 1000 , decaying learning rate starting at 5 e − 3 .

Original: The learning rate was factored by 0 .
Revised: The learning rate was factored by 0 .

Original: 8 every 200 epochs,
Revised: 8 every 200 epochs,

Original: Ns = 80000 , N b = 1200 , λ = 0 .
Revised: Ns = 80000 , N b = 1200 , λ = 0 .

Original: 01 , α = 1 e − 8 , K = 40 , µ = 0 .
Revised: 01 , α = 1 e − 8 , K = 40 , µ = 0 .

Original: 01 , and β = 0 .
Revised: 01 , and β = 0 .

Original: Running time on Intel i7-8650u CPU was about 15 minutes for the forward problem and 13 minutes for the generalized inverse problem.
Revised: In the diffusion

Original: [Table]
Revised: [Table]

Original: Figure 11 presents the sensitivity of the algorithm to the number of sampled points.
Revised: Figure 11 presents the sensitivity of the algorithm to the number of sampled points.

Original: As expected, with large number of points we obtain a plateau while with small and intermediate number (up to 50000)
Revised: As expected, with large number of points we obtain a plateau while with small and intermediate number (up to 50000)

Original: the error is not monotonic due to local minima
Revised: the error is not monotonic due to local minima.

Original: [Table]

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: Deep networks by their nature use compositions of simple functions such as matrix multiplication and non-linear activations like sigmoid or tanh.
Revised: Deep networks by their nature use compositions of simple functions such as matrix multiplication and non-linear activations like sigmoid or tanh.

Original: This structure (i) enables the approximation of an arbitrary function Hornik (1989) and (ii) is inherently differentiable.
Revised: This structure (i) enables the approximation of an arbitrary function Hornik (1989) and (ii) is inherently differentiable.

Original: The network architecture dictates the number of degrees of freedom which in turn enables the expressibility of complex functions.
Revised: The network architecture dictates the number of degrees of freedom which in turn enables the expressibility of complex functions.

Original: In this work we present a uniﬁed framework for the solution of forward and inverse problems in partial differential equations in an arbitrary domain.
Revised: In this work we present a uniﬁed framework for the solution of forward and inverse problems in partial differential equations in an arbitrary domain.

Original: The algorithm relies on direct approximation of the unknown function by a neural network which yields an analytical smooth solution.
Revised: The algorithm relies on direct approximation of the unknown function by a neural network which yields an analytical smooth solution.

Original: The network is trained to satisfy the PDE and boundary conditions in an unsupervised fashion by the minimization of a cost function.
Revised: The network is trained to satisfy the PDE and boundary conditions in an unsupervised fashion by the minimization of a cost function.

Original: The optimization procedure depends on random points set within the domain and its boundary.
Revised: The optimization procedure depends on random points set within the domain and its boundary.

Original: The problem is therefore mesh free with free-form domain.
Revised: The problem is therefore mesh free with free-form domain.

Original: We introduce a cost function which is composed of both L 2 and L ∞ ﬁdelity terms, boundary conditions constraints and additional regularizers.
Revised: We introduce a cost function which is composed of both L 2 and L ∞ ﬁdelity terms, boundary conditions constraints and additional regularizers.

Original: The mathematical rationale of the L ∞ norm which controls possible outliers was veriﬁed in our numerical experiments.
Revised: The mathematical rationale of the L ∞ norm which controls possible outliers was veriﬁed in our numerical experiments.

Original: We also stress the robustness of our approach exempliﬁed by having the same hyper parameters set for all our experiments.
Revised: We also stress the robustness of our approach exempliﬁed by having the same hyper parameters set for all our experiments.

Original: This framework alleviates several problems of the ﬁnite differences and ﬁnite elements methods.
Revised: This framework alleviates several problems of the ﬁnite differences and ﬁnite elements methods.

Original: In particular meshing, discretization and derivatives approximation are solved in a simple and natural way.
Revised: In particular meshing, discretization and derivatives approximation are solved in a simple and natural way.

Original: Numerical solutions of PDEs in an arbitrary domain are of extreme importance, in particular in medical imaging applications.
Revised: Numerical solutions of PDEs in an arbitrary domain are of extreme importance, in particular in medical imaging applications.

Original: The framework is demonstrated by an elliptic system in R 2 applied to
Revised: The framework is demonstrated by an elliptic system in R 2 applied to

Original: Electrical Impedance Tomography for both forward and inverse problems.
Revised: Electrical Impedance Tomography for both forward and inverse problems.

Original: Promising results were achieved for complex and non monotonic functions.
Revised: Promising results were achieved for complex and non monotonic functions.

Original: Rigorous analysis of the approximation error and its relation to the network architecture and design are under current study and will be published elsewhere.
Revised: Rigorous analysis of the approximation error and its relation to the network architecture and design are under current study and will be published elsewhere.

Original: Future research includes also full tomography solution, higher dimensional problems and other classes of PDEs such as dynamic non-linear equations (Burgers, Navier-Stokes etc.)
Revised: Future research includes also full tomography solution, higher dimensional problems and other classes of PDEs such as dynamic non-linear equations (Burgers, Navier-Stokes etc.)