Proceedings of the 7th World Congress on Mechanical, Chemical, and Material Engineering (MCM'21)
Prague, Czech Republic Virtual Conference – August, 2021
Paper No. HTFF 113
DOI: 10.11159/htff21.113

Physics-Informed Neural Network with Variable Initial Conditions
Yo Nakamura1, Suguru Shiratori1, Hideaki Nagano1, Kenjiro Shimano1
Mechanical Systems Engineering, Tokyo City University
1-28-1 Tamazutsumi, Setagaya-ku, Tokyo, Japan
g2081038@tcu.ac.jp (Y. Nakamura); sshrator@tcu.ac.jp (S. Shiratori)
1

Abstract – This work addresses an extension of Physics-Informed Neural Network (PINN), which learns solutions of partial differentialequations (PDE). In the originally proposed PINN [1], the initial conditions are kept constant. In this study, we extend the PINN so thatthe initial conditions can be varied by means of low-dimensional identifiers which represent the initial conditions. A validity of theproposed method is confirmed through the PDE for the liquid film flow. The solutions of the PDE predicted by the PINN showed good
agreement with those obtained by the finite difference method, and the dependence of the initial conditions are also correctly reproduced.
As requirements for the low-dimensional identifier of the initial conditions, it is suggested that continuity and uniqueness are necessary
condition and the linearity is sufficient condition.
Keywords: 4 - 8 keywords

Machine learning, Numerical simulation, Liquid film flow, Partial differential equations

1. Introduction

In the manufacturing process of semiconductors and MEMS devices, a resist film consisting of a resin and a solvent iscoated onto a substrate, and it is known that various thickness undulations appear in the final resist films [2, 3]. Althoughnumerical simulations have been used to predict such thickness undulations, it is difficult to optimize the coating conditions
from an inverse problem perspective because the time evolution of the governing equations takes a long calculation time. In
the general optimization procedure, it is necessary to evaluate the objective function and its gradient with respect to theparameters in coating conditions. Both take much computational cost in the conventional numerical method based on thetime evolution calculation.
Physics-Informed Neural Network (PINN), which has been proposed by Raissi et al. [1], can be considered as a way toresolve such a situation. The PINN learns the solutions of a partial differential equation (PDE) for given dataset of the input
variables. In the training process of PINN, a loss function is defined as the mean square error of the predicted solutions ofthe PDE. To evaluate the loss function, the temporal and spatial derivatives of the unknowns are calculated by the automaticdifferentiation (AD) which is implemented in the neural network (NN) framework. Once a PINN has been trained, thesolutions for any time instance can be directly calculated without time integration by forward calculations of the NN. Inaddition, the gradient of the solution with respect to the input variable can be efficiently calculated by the AD. The authors'
research group has studied to apply the PINN to the PDE for the liquid film flow in order to finally achieve the optimizationof the coating conditions which can avoid or suppress the thickness undulations. In the problem of the liquid film flow
including the surface tension (Laplace pressure) effect, the PDE involves a fourth-order spatial derivative and the fourthorder nonlinearity of unknowns. Even for such a type of PDE, the PINN was found to correctly predict the solutions [4].
In the original PINN, the initial conditions are fixed during training. For the different initial conditions, the PINN has tobe retrained. Due to this restriction, the it is difficult to calculate the gradient of the solution with respect to the initialconditions, thus the PINN cannot be applied to the optimization of the initial conditions of the coating process. The PINN isexpected to also be applied to the data assimilation, in which the uncertain parameters for the simulation are optimized tominimize the difference between simulation and observation [5]. Regarding the application of the PINN to the data
assimilation, the restriction of initial conditions becomes to be serious drawback.
Therefore, the purpose of this study is to extend the original PINN so that the initial conditions be variable. A way torealize this is described in the next section, then the proposed method is validated by comparing the results with thoseobtained by the finite difference method.

HTFF 113-1

2. Approach

2.1. Overview
Figure 1 shows how we make the initial condition be variable in the PINN. We express the initial conditions with
low-dimensional identifiers. These identifiers are obtained by certain mapping from the initial conditions. Then, theseidentifiers are taken as input of the PINN. During training of the PINN, the loss function is evaluated using not the lowdimensional identifier but the original initial condition itself. Although, in principle the discretized initial conditions canbe directly put into the PINN, the number discretized points are likely be too huge for the input of the PINN. We consider
making the low-dimensional identifiers in order to reduce the input dimension of the PINN.

Fig. 1: PINN with a low-dimensional identifier added to the input

2.2. Low-Dimensional Identifiers
In this study, we considered four types of low-dimensional identifiers; LIN, CAE, WAV and DIS, as summarizedin Table 1. The LIN is the identity mapping, where the initial conditions are expressed as parametric analytic function,and its parameters are directly used as the identifier. This LIN is regarded as essential verification whether the difference
in initial conditions can be reflected in the PINN. In the CAE, the given dataset of initial conditions is mapped onto theidentifiers by the Convolutional Autoencoder (CAE) [6]. Through the CAE network, the data dimension of the initialcondition can be highly reduced. In the CAE, we examine the extensibility of this method to general-purpose input.
WAV creates a non-linear low-dimensional identifier and examines the effect of the degree of non-linearity. The WAV
is a nonlinear mapping with a wavy function Through this WAV, we investigate the effect of nonlinearity in the accuracyof the PINN. Finally, the DIS is the discontinuous mapping. The function used as DIS is discontinuous but unique (nofolding). Through this DIS, we will investigate whether the continuity is necessary for the identifiers. Figure 2 shows
the detailed functions used as mapping of the identifiers.
Table 1: Low-dimensional identifiers considered

Identifier

Description

Function

LIN

Identity mapping

CAE

Encoded by CAE

𝑓𝑓(𝑥𝑥) = 𝑥𝑥

WAV

Nonlinear mapping

DIS

Discontinuous mapping

𝑓𝑓(𝑥𝑥) = CAE(𝑥𝑥)𝑓𝑓(𝑥𝑥) = �HTFF 113-2

1sin(6𝜋𝜋𝜋𝜋)20(𝑥𝑥 ≤ 1/8) or (1/4 < 𝑥𝑥 ≤ 3/8)
else𝑓𝑓(𝑥𝑥) = 𝑥𝑥 +1 − 𝑥𝑥
𝑥𝑥 + 1/8

2.00

0.50

(a) LIN

0.25

1.50
g

g

1.75

1.25(b) CAE

1.00
0.00
0.00

0.75
0.00

0.50

0.25

x0
0.50

0.50

(c) WAV

0.25

g

g

0.50

0.25

x0

0.00
0.00

0.50

0.25

0.25

(d) DIS

0.00
0.00

0.50

0.25
x0x0Fig. 2: Profiles of mapping functions for low-dimensional identifiers. (a) linear function: LIN, (b) encoded by CAE, (c) wavy function:
WAV, and (d) discontinue function: DIS.

3. Validation

3.1. Problem formulation: Governing equation and boundary conditions
In order to validate the proposed method, it is applied to a liquid film flow problem. As the governing equations, weconsider the following equations, where only the relaxation term due to Laplace pressure is considered.

𝜕𝜕ℎ 𝜕𝜕𝜕𝜕 3 ℎ3𝐹𝐹:+�ℎ
�=0𝜕𝜕𝜕𝜕 𝜕𝜕𝜕𝜕𝜕𝜕𝑥𝑥 3(1)This partial differential equation describes the time evolution of the film thickness distribution due to surface tension.
ℎ, 𝑡𝑡, and 𝑥𝑥 are the film thickness, time, and coordinates, respectively. The equation is nondimensionalized using the scales
listed in the Table 2, where ℎ0 is the base thickness of the film, 𝐿𝐿 is the representative length, 𝜇𝜇 is the viscosity coefficientand 𝜎𝜎 is the surface tension.
Table 2: Scales used for the nondimensionalization.

Lateral length

Thickness

TimePressure𝐿𝐿ℎ03𝜇𝜇𝐿𝐿4 /(𝜎𝜎ℎ03 )ℎ0 𝜎𝜎/𝐿𝐿2

HTFF 113-3

The following boundary conditions were applied so that the gradient of the thickness and the curvature becomeszero at the end of the solution domain.

𝐵𝐵:3.2. Initial Conditions

𝜕𝜕ℎ 𝜕𝜕 3 ℎ==0𝜕𝜕𝜕𝜕 𝜕𝜕𝑥𝑥 3

at𝑥𝑥 = 𝑥𝑥min ，𝑥𝑥max(2)

The initial conditions are expressed in the following functional form.

𝑆𝑆: ℎ(𝑥𝑥, 𝑡𝑡 = 0) = (ℎ1 − ℎ0 ) �1 − �𝑥𝑥 −(𝑥𝑥 − 𝑥𝑥0 )2𝑥𝑥0 2
� � exp �� + ℎ0(2𝜆𝜆2 )𝜆𝜆(3)Using Eq. (3) and the parameters listed in the Table 3, the dataset of the initial conditions are generated. The numberof discrete points in the initial condition is selected as 𝑁𝑁𝑥𝑥 = 256.
Table 3: Dataset of initial condition parameters

Parameter𝑥𝑥0

Value range𝜆𝜆0.0~0.5ℎ00.71ℎ133.3. Loss Function of PINN

The loss function of PINN is defined as follows.

𝐽𝐽𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 = 𝐽𝐽𝐹𝐹 + 𝐽𝐽𝑂𝑂 + 𝐽𝐽𝑃𝑃 + 𝐽𝐽𝐿𝐿2𝑁𝑁𝑓𝑓

𝐽𝐽𝐹𝐹 = �[𝐹𝐹(ℎ𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 (𝑥𝑥𝑖𝑖 , 𝑡𝑡𝑖𝑖 , 𝒈𝒈𝑖𝑖 )]2𝑁𝑁𝑖𝑖𝑖𝑖𝑖𝑖

𝑁𝑁𝐵𝐵

𝐽𝐽𝐵𝐵 = � �𝑖𝑖𝑖𝑖

𝐽𝐽𝑂𝑂 = �[ℎ𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 (𝑥𝑥𝑖𝑖 , 𝑡𝑡𝑖𝑖 , 𝒈𝒈𝑖𝑖 ) − ℎ𝑖𝑖𝑖𝑖𝑖𝑖 (𝑥𝑥𝑖𝑖 )]2𝑖𝑖

2𝑁𝑁𝐵𝐵32𝜕𝜕ℎ𝜕𝜕 ℎ�(𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚,𝑚𝑚𝑚𝑚𝑚𝑚 , 𝑡𝑡𝑖𝑖 , 𝒈𝒈𝑖𝑖 )� + � � 3 �(𝑥𝑥, 𝑡𝑡 , 𝒈𝒈 )�
𝜕𝜕𝜕𝜕 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝜕𝜕𝑥𝑥 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 𝑚𝑚𝑚𝑚𝑚𝑚,𝑚𝑚𝑚𝑚𝑚𝑚 𝑖𝑖 𝑖𝑖
𝑁𝑁𝑓𝑓𝑖𝑖

𝐽𝐽𝑃𝑃 = � max( 0, exp�−ℎ𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 (𝑥𝑥𝑖𝑖 , 𝑡𝑡𝑖𝑖 , 𝒈𝒈𝑖𝑖 )� − 1)𝑖𝑖

𝐽𝐽𝐹𝐹 is the mean squared error of the governing equation, 𝐽𝐽𝑂𝑂 and 𝐽𝐽𝐵𝐵 are the initial and boundary conditions,
respectively, 𝐽𝐽𝑃𝑃 is the penalty function for the film thickness ℎ to impose a positivity constraint, and 𝐽𝐽𝐿𝐿2 is the L2
normalization term. 𝒈𝒈 is low-dimensional identifier for the initial conditions. In the present study, the network iscomposed by 8 hidden dense layers with 20 neurons per layer. The training dataset was generated by randomly sampled
𝑁𝑁𝑓𝑓 = 3000 points from a set of discrete points equally spaced on a linear scale in the 𝑥𝑥 direction and on a logarithmic
scale in the 𝑡𝑡 direction.

3.4. Finite Differences

In order to validate the proposed method, the results obtained by the PINN was compared with those obtained by
the finite difference method (FDM). In the FDM, the governing equation Eq. (1) is discretized by the finite differences
on a equidistant grid of 𝑁𝑁𝑥𝑥 points with an interval Δ𝑥𝑥. The positivity-preserving scheme is applied to ℎ3 of the Laplace
HTFF 113-4(4)

pressure term in Eq. (1), because the standard finite difference schemes do not necessarily preserve a positivity of ℎ, and theoccurrence of nonpositive solution introduces artificial instability [7]. Due to the fourth-order spatial derivative of ℎ in Eq.
(1), a time step restriction for the numerical stability requires Δ𝑡𝑡 < 𝑂𝑂(Δ𝑥𝑥 4 ), which is a quite severe condition in practice. Inorder to cope with this restriction, the fully implicit time integration of Newton-Kantrovich method is applied with CrankNicholson, according to Diez and Kondic [8].

4. Results

Fig.3 shows the contours of the film thickness in logarithmic scale log ℎ in the plane of horizontal axis is time 𝑡𝑡 and thevertical axis is coordinate 𝑥𝑥. Fig.3(a) shows the result of FDM, and Fig.3(b) of PINN for the case of the LIN as the lowdimensional identifier. From Fig.3, it can be confirmed that the relaxed shape of the initial condition can be reproduced even
if the low-dimensional identifier is used for input.

Fig. 3: Prediction of film thickness ℎ(𝑡𝑡, 𝑥𝑥) in space-time (𝑡𝑡, 𝑥𝑥).
(a) FDM (b) PINN with low-dimensional identifier by LIN

For a more detailed comparison, snapshots of the film thickness distributions for the initial state (𝑡𝑡 = 0) and after time
(𝑡𝑡 = 0.5) are shown in Fig.4 and Fig.5. Table 4 shows the RMS average values at the initial state (𝑡𝑡 = 0) and at thesufficiently later time instance (𝑡𝑡 = 0.5). From Fig.4, it was found that the difference of the initial conditions is successfully
reproduced for the cases of LIN, CAE, and WAV. However, DIS did not reflect the difference in the initial conditions. For
the cases of LIN, CAE and WAV, the error between the PINN and the FDM are found at the location where the curvature ofthe thickness is large. The RMS average in the DIS is one order of magnitude larger than those in other three cases.
Table 4: The RMS average values in the initial state (𝑡𝑡 = 0) and after the passage of time (𝑡𝑡 = 0.5)

Identifier
LINCAEWAVDIS

Initial state𝑥𝑥0 = 0

4.4 × 10−2
4.1 × 10−2
4.8 × 10−2
1.4 × 10−1

Initial state𝑥𝑥0 = 0.3

4.4 × 10−2
4.2 × 10−2
4.8 × 10−2
2.0 × 10−1

HTFF 113-5

End state𝑥𝑥0 = 08.8 × 10−3
9.8 × 10−3
1.4 × 10−2
1.2 × 10−2

End state𝑥𝑥0 = 0.3

9.6 × 10−3
9.7 × 10−3
1.7 × 10−2
2.4 × 10−2

Fig. 4: Snapshots of the thickness distribution at the initial state (𝑡𝑡 = 0)
when low-dimensional identifiers of LIN, CAE, WAV, DIS and FDM.

Fig. 5: Snapshots of the thickness distribution at the end state (𝑡𝑡 = 0.5)
when low-dimensional identifiers of LIN, CAE, WAV, DIS and FDM.

HTFF 113-6

5. Discussion

From the result shown in Fig.4, Fig.5, and Table 4, it was found that the variation of the initial condition is successfully
successfully reproduced by the PINN by the method proposed in the present work; low-dimensional identifiers. However,
the accuracy was different depending on the mapping function used. Therefore, we consider what the conditions are required
required for low-dimensional identifiers. First, if there are duplicate values in the low-dimensional identifier, they will have
have the same low-dimensional identifier value for different initial conditions, and it will not correspond to the difference in
the initial conditions, so there should be no duplication (uniqueness of the identifier). From Fig.2, if the discontinuity isinvolved in the mapping function, the difference in the initial conditions cannot be correctly reproduced (case DIS). Therefore,
continuity can be regarded as necessary condition for the low-dimensional identifier. Among the three cases where the initialconditions can be reflected, the difference was the degree of non-linearity. The case LIN is linear, the CAE is slightlynonlinear, and the WAV is highly nonlinear. The prediction error in the case WAV is significant. Although the RMS error
are almost the same in cases LIN and CAE, the CAE is more applicability for the general initial conditions.

6. Conclusions

In the present study, the PINN is extended so that the initial condition can be varied by the low-dimensional identifiers.
Through the validation of the four-types of the identifier, we concluded that continuity and uniqueness are necessary
condition and the linearity is sufficient condition for the low-dimensional identifier of the initial conditions. Even if theserequirements for the identifier are satisfied, the accuracy of the PINN is depending on the degree of the nonlinearity of themapping functions.
By use of the proposed method, it can be realized that the gradient of the solutions with respect to the initial conditions.
Consequently, the optimization of the process conditions in the film-coating is expected to be realized.

7. References[1] M. Raissi, P. Perdikaris and G.E. Karniadakis “Physics-informed neural networks: A deep learning framework forsolving forward and inverse problems involving nonlinear partial differential equations,” J. Comp. Phys. vol. 378 686707, 2019.
[2] S. Shiratori, D. Kato, K. Sugasawa, H. Nagano and K. Shimano “Spatio-temporal thickness variation and transient
Marangoni number in striations during spin coating,” Int. J. Heat Mass Trans., vol. 154 119678, 2020.
[3] S. Shiratori and T. Kubokawa “Double-peaked edge-bead in drying film of solvent-resin mixtures,” Phys. Fluids vol.
27, 102105, 2015.
[4] Y. Nakamura, S. Shiratori, R. Takagi, M. Sutoh, H. Nagano and K. Shimano “Physics-informed neural network applied
to 4th order partial differential equation for liquid film flows,” Comp. Fluids. (to appear), 2021.
[5] W. Lahoz, B. Khattatov and R. Ménard “Data Assimilation: making sense of observations”, Springer, 2010.
[6] W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu and F.E. Alssadi “A survey of deep neural network architectures and theirapplications”, Neurocomputing., vol. 234 11-26, 2017.
[7] J.A. Diez, L. Kondic and A. Bertozzi “Global models for moving contact lines,” Phys. Rev. E Vol. 63, 011208, 2000.
[8] J.A. Diez and L. Kondic “Computing three-dimensional thin film flows including contact lines,” J. Comp. Phys. vol.
183, 274-306, 2002.

HTFF 113-7