Original: How PINNs cheat: Predicting chaotic motion of a double pendulum
Revised: How PINNs cheat: Predicting chaotic motion of a double pendulum

Original: Abstract
Revised: Abstract

Original: Despite extensive research, physics-informed neural networks (PINNs) are stilldifficult to train, especially when the optimization relies heavily on the physicsloss term.
Revised: Despite extensive research, physics-informed neural networks (PINNs) are still difficult to train, especially when the optimization relies heavily on the physics loss term.

Original: Convergence problems frequently occur when simulating dynamicalsystems with high-frequency components, chaotic or turbulent behavior.
Revised: Convergence problems frequently occur when simulating dynamical systems with high-frequency components, chaotic or turbulent behavior.

Original: In thiswork, we discuss whether the traditional PINN framework is able to predict chaoticmotion by conducting experiments on the undamped double pendulum.
Revised: In this work, we discuss whether the traditional PINN framework is able to predict chaotic motion by conducting experiments on the undamped double pendulum.

Original: Our resultsdemonstrate that PINNs do not exhibit any sensitivity to perturbations in the initialcondition.
Revised: Our results demonstrate that PINNs do not exhibit any sensitivity to perturbations in the initial condition.

Original: Instead, the PINN optimization consistently converges to physicallycorrect solutions that violate the initial condition only marginally, but divergesignificantly from the desired solution due to the chaotic nature of the system.
Revised: Instead, the PINN optimization consistently converges to physically correct solutions that violate the initial condition only marginally, but diverge significantly from the desired solution due to the chaotic nature of the system.

Original: Infact, the PINN predictions primarily exhibit low-frequency components with asmaller magnitude of higher-order derivatives, which favors lower physics lossvalues compared to the desired solution.
Revised: In fact, the PINN predictions primarily exhibit low-frequency components with a smaller magnitude of higher-order derivatives, which favors lower physics loss values compared to the desired solution.

Original: We thus hypothesize that the PINNs“cheat” by shifting the initial conditions to values that correspond to physicallycorrect solutions that are easier to learn.
Revised: We thus hypothesize that the PINNs “cheat” by shifting the initial conditions to values that correspond to physically correct solutions that are easier to learn.

Original: Initial experiments suggest that domaindecomposition combined with an appropriate loss weighting scheme mitigates thiseffect and allows convergence to the desired solution.
Revised: Initial experiments suggest that domain decomposition combined with an appropriate loss weighting scheme mitigates this effect and allows convergence to the desired solution.

Original:  Introduction
Revised:  Introduction

Original: Physics-informed neural networks (PINNs)
Revised: Physics-informed neural networks (PINNs)

Original: [1, 2] are an emerging class of physics-enhanced machinelearning techniques that provide a mesh-free and time-continuous approach to solving forward andinverse problems governed by differential equations.
Revised: [1, 2] are an emerging class of physics-enhanced machine learning techniques that provide a mesh-free and time-continuous approach to solving forward and inverse problems governed by differential equations.

Original: PINNs seamlessly integrate knowledge abouta physical system into a neural network model by incorporating the residuals of the governingdifferential equations via an additional physics-based loss function.
Revised: PINNs seamlessly integrate knowledge about a physical system into a neural network model by incorporating the residuals of the governing differential equations via an additional physics-based loss function.

Original: The physics loss acts as aregularization term, penalizing solutions that violate the physical laws of the underlying system.
Revised: The physics loss acts as a regularization term, penalizing solutions that violate the physical laws of the underlying system.

Original: Thisapproach has been proven successful in numerous applications [3, 4, 5, 6, 7, 8, 9, 10, 9, 11], includingthe simulation of dynamical systems.
Revised: This approach has been proven successful in numerous applications [3, 4, 5, 6, 7, 8, 9, 10, 9, 11], including the simulation of dynamical systems.

Original: However, the prediction accuracy of PINNs relies on the optimization of the multi-objective costfunction consisting of the data and physics-based loss term which encode the initial conditions (ICs)and governing differential equations, respectively.
Revised: However, the prediction accuracy of PINNs relies on the optimization of the multi-objective cost function consisting of the data and physics-based loss term which encode the initial conditions (ICs) and governing differential equations, respectively.

Original: Several studies have identified these competingloss terms as one cause of severe convergence problems [12, 13, 14], proposing various manualor adaptive loss weighting schemes.
Revised: Several studies have identified these competing loss terms as one cause of severe convergence problems [12, 13, 14], proposing various manual or adaptive loss weighting schemes.

Original: Still, PINNs frequently struggle to solve problems for largecomputational domains [15, 16, 14, 17].
Revised: Still, PINNs frequently struggle to solve problems for large computational domains [15, 16, 14, 17].

Original: Additionally, the spectral bias [18, 19, 20, 21] was shownto cause neural networks to learn low-frequency functions faster and to impede the convergenceof PINNs for large computational domains [22].
Revised: Additionally, the spectral bias [18, 19, 20, 21] was shown to cause neural networks to learn low-frequency functions faster and to impede the convergence of PINNs for large computational domains [22].

Original: As the input variable of the network is typicallynormalized to a fixed range in a preprocessing step, low frequencies on large computational domainsbecome high frequency components.
Revised: As the input variable of the network is typically normalized to a fixed range in a preprocessing step, low frequencies on large computational domains become high frequency components.

Original: At the same time, a large computational domain results in a more complicated landscape for PINN optimization, where nonphysical solutions may even correspond tobetter optima of the multi-objective problem than the desired physical solutions [23].
Revised: At the same time, a large computational domain results in a more complicated landscape for PINN optimization, where nonphysical solutions may even correspond to better optima of the multi-objective problem than the desired physical solutions [23].

Original: Overall, the properties and limitations of PINNs are still poorly understood.
Revised: Overall, the properties and limitations of PINNs are still poorly understood.

Original: Thus, the aim of thiswork is to gain a deeper understanding of PINNs through experiments on a nonlinear dynamicalsystem, the undamped double pendulum.
Revised: Thus, the aim of this work is to gain a deeper understanding of PINNs through experiments on a nonlinear dynamical system, the undamped double pendulum.

Original: Our main contributions comprise the following:
Revised: Our main contributions comprise the following:

Original: • We empirically demonstrate the difficulty of predicting chaotic motions using the traditionalPINN framework on a double pendulum.
Revised: • We empirically demonstrate the difficulty of predicting chaotic motions using the traditional PINN framework on a double pendulum.

Original: We observe that the predicted trajectories violatethe ICs marginally, but deviate significantly from the reference.•
Revised: We observe that the predicted trajectories violate the ICs marginally, but deviate significantly from the reference.

Original:  We show that the PINN exhibits no sensitivity to perturbations of the given IC and prefersconvergence to the same undesired solution characterized by a significantly lower physicsloss compared to the reference trajectory.•
Revised: • We show that the PINN exhibits no sensitivity to perturbations of the given IC and prefers convergence to the same undesired solution characterized by a significantly lower physics loss compared to the reference trajectory.

Original:  We claim that the convergence to such incorrect attractive solutions is related to the spectralpower distribution and magnitude of higher-order derivatives.
Revised: • We claim that the convergence to such incorrect attractive solutions is related to the spectral power distribution and magnitude of higher-order derivatives.

Original: In other words, the PINN“cheats” by choosing an IC that corresponds to a simpler solution.• We note that a reduced computational domain decreases the sensitivity to the IC with respectto the aforementioned measures.
Revised: In other words, the PINN “cheats” by choosing an IC that corresponds to a simpler solution. • We note that a reduced computational domain decreases the sensitivity to the IC with respect to the aforementioned measures.

Original: In combination with a suitable loss weighting scheme, thedesired solution becomes the dominant one.
Revised: In combination with a suitable loss weighting scheme, the desired solution becomes the dominant one.

Original:  Methods
Revised:  Methods

Original: Double pendulum The planar double pendulum is a nonlinear dynamical system that exhibitsharmonic and chaotic behavior, based on its initial displacement and velocity.
Revised: Double pendulum The planar double pendulum is a nonlinear dynamical system that exhibits harmonic and chaotic behavior, based on its initial displacement and velocity.

Original: It consists of twopoint mass pendulums with masses m 1 , m 2 and rod length L 1 , L 2 attached to each other.
Revised: It consists of two point mass pendulums with masses m 1 , m 2 and rod length L 1 , L 2 attached to each other.

Original: The anglebetween the two pendulum rods and the vertical axis (i.e., θ 1 , θ 2 ) is governed by two nonlinearsecond-order coupled ordinary differential equations (ODEs) of the form
Revised: The angle between the two pendulum rods and the vertical axis (i.e., θ 1 , θ 2 ) is governed by two nonlinear second-order coupled ordinary differential equations (ODEs) of the form

Original: [Equation]
Revised: [Equation]

Original: For a detailed description, see Appendix A.1.
Revised: For a detailed description, see Appendix A.1.

Original: Physics-informed neural networks The unknown solution y is approximated by a fully-connecteddeep neural network y w with w describing all trainable weights and biases of the network.
Revised: Physics-informed neural networks The unknown solution y is approximated by a fully-connected deep neural network y w with w describing all trainable weights and biases of the network.

Original: The heartof PINNs [1] consists of combining data- and physics-based constraints into a single loss function
Revised: The heart of PINNs [1] consists of combining data- and physics-based constraints into a single loss function

Original: L ( w ) =
Revised: L ( w ) =

Original: αL IC ( w ) + (1 − α )
Revised: αL IC ( w ) + (1 − α )

Original: L F ( w ) via multi-objective optimization.
Revised: L F ( w ) via multi-objective optimization.

Original: The data loss term L ICreflects the deviation from the given initial condition, while the physical term L F incorporates the
Revised: The data loss term L IC reflects the deviation from the given initial condition, while the physical term L F incorporates the

Original: ODE residuals of the underlying physical system.
Revised: ODE residuals of the underlying physical system.

Original: Thus, the performance of PINNs depends stronglyon the ODE system itself.
Revised: Thus, the performance of PINNs depends strongly on the ODE system itself.

Original: For a detailed description of each loss term, see Appendix A.2.
Revised: For a detailed description of each loss term, see Appendix A.2.

Original: Analysis of solution trajectories We analyze two measures that describe properties of the predictedsolution trajectories compared to the desired one.
Revised: Analysis of solution trajectories We analyze two measures that describe properties of the predicted solution trajectories compared to the desired one.

Original: The aim is to improve our understanding of thedifficulties in training PINNs for the simulation of chaotic motion.
Revised: The aim is to improve our understanding of the difficulties in training PINNs for the simulation of chaotic motion.

Original: First, we compare the distribution of the solutions’ frequency components, namely, the amount oflow-frequency components.
Revised: First, we compare the distribution of the solutions’ frequency components, namely, the amount of low-frequency components.

Original: We compute this measure using the relative fraction of total signal powerabove a certain frequency f 0 , given by
Revised: We compute this measure using the relative fraction of total signal power above a certain frequency f 0 , given by

Original: [Equation]
Revised: [Equation]

Original: where X k denotes the discrete Fourier transform (DFT) of a signal x ( t n ) (in our experiments, θ 1 , θ 2 )and k f 0 denotes the DFT index corresponding to the boundary frequency f 0 .
Revised: where X k denotes the discrete Fourier transform (DFT) of a signal x ( t n ) (in our experiments, θ 1 , θ 2 ) and k f 0 denotes the DFT index corresponding to the boundary frequency f 0 .

Original: As a second measure, we consider the average magnitude of the second order derivatives, which areapproximated using the central finite difference method with a step size of h = 0 .
Revised: As a second measure, we consider the average magnitude of the second order derivatives, which are approximated using the central finite difference method with a step size of h = 0 .

Original: [Equation]
Revised: [Equation]

Original: The derivatives θ ′′ 1 , θ ′′ 2 are used in the computation of the physics loss, summarized in Appendix A.2.
Revised: The derivatives θ ′′ 1 , θ ′′ 2 are used in the computation of the physics loss, summarized in Appendix A.2.

Original: As we consider the absolute error for the physics loss, smaller values of the derivatives also contributeto smaller values of the resulting physics loss.
Revised: As we consider the absolute error for the physics loss, smaller values of the derivatives also contribute to smaller values of the resulting physics loss.

Original:  Experiments
Revised:  Experiments

Original: For all experiments, the system parameters of the double pendulum ( m 1 , m 2 , L 1 , L 2 ) are fixed toone.
Revised: For all experiments, the system parameters of the double pendulum ( m 1 , m 2 , L 1 , L 2 ) are fixed to one.

Original: The initial angles of the pendulums are set to θ 1 ( t 0 )
Revised: The initial angles of the pendulums are set to θ 1 ( t 0 )

Original: = θ 2 ( t 0 )
Revised: = θ 2 ( t 0 )

Original: =
Revised: =

Original: θ 0 with zero initial velocitiesθ ′ 1 ( t 0 )
Revised: θ 0 with zero initial velocities θ ′ 1 ( t 0 )

Original: = θ ′ 2 ( t 0 )
Revised: = θ ′ 2 ( t 0 )

Original: = 0 .
Revised: = 0 .

Original: We use a fourth-order Runge-Kutta (RK45) scheme with step size h = 0 .
Revised: We use a fourth-order Runge-Kutta (RK45) scheme with step size h = 0 .to

Original: 005to obtain a reference solution.
Revised: obtain a reference solution.

Original: All experiments are conducted with the following network specifications and hyperparameters:network type (6x30 fully-connected neural network), network activation function (swish), outputactivation function (linear), weight initialization (Glorot uniform [24]), optimizer (Adam [25], defaultsettings of Tensorflow [26]), learning rate (0.01), number of epochs (25000).
Revised: All experiments are conducted with the following network specifications and hyperparameters: network type (6x30 fully-connected neural network), network activation function (swish), output activation function (linear), weight initialization (Glorot uniform [24]), optimizer (Adam [25], default settings of Tensorflow [26]), learning rate (0.01), number of epochs (25000).

Original: Additionally, the inputof the network (time t ) is normalized to a fixed domain of [ − 5 , 5] .
Revised: Additionally, the input of the network (time t ) is normalized to a fixed domain of [ − 5 , 5] .

Original: 3.1 Results
Revised: 3.1 Results

Original: First, the initial angles of the double pendulum are selected as θ 0 = 150 ◦ ≈ 2 .
Revised: First, the initial angles of the double pendulum are selected as θ 0 = 150 ◦ ≈ 2 .

Original: 62 rad, resultingin chaotic motion.
Revised: 62 rad, resulting in chaotic motion.

Original: We train ten PINN instances with the same IC and network specifications, butdifferent seeds for the weight initialization.
Revised: We train ten PINN instances with the same IC and network specifications, but different seeds for the weight initialization.

Original: As the traditional PINN framework incorporates theoptimization of the IC as a soft constraint, deviations from the true value are allowed.
Revised: As the traditional PINN framework incorporates the optimization of the IC as a soft constraint, deviations from the true value are allowed.

Original: The predictions obtained by minimizing the unweighted loss function with α = 0 .
Revised: The predictions obtained by minimizing the unweighted loss function with α = 0 .

Original: 5 differ heavilyfrom the reference solution, as illustrated in Figure 1 (a).
Revised: 5 differ heavily from the reference solution, as illustrated in Figure 1 (a).

Original: Despite this deviation, the predictedtrajectories can be shown to correspond to physically correct solutions.
Revised: Despite this deviation, the predicted trajectories can be shown to correspond to physically correct solutions.

Original: Indeed, repeating the RKsimulation with the shifted ICs that where chosen by the PINN instances from the previous experiment,we observe that in the initial phase ( t < 2 ), the RK solution follows the PINN predictions exactly(green lines in Figure 1).
Revised: Indeed, repeating the RK simulation with the shifted ICs that where chosen by the PINN instances from the previous experiment, we observe that in the initial phase ( t < 2 ), the RK solution follows the PINN predictions exactly (green lines in Figure 1).

Original: Due to the chaotic nature of the pendulum, the RK trajectories begin todiverge for t > 2 due to minor differences in the imposed ICs.
Revised: Due to the chaotic nature of the pendulum, the RK trajectories begin to diverge for t > 2 due to minor differences in the imposed ICs.

Original: Continuing the RK simulation ata later time step, just before the trajectories begin to diverge, again yields an exact match between
Revised: Continuing the RK simulation at a later time step, just before the trajectories begin to diverge, again yields an exact match between

Original: PINN and RK (not shown here).
Revised: PINN and RK (not shown here).

Original: Interestingly, looking at the physics loss averaged over the ten PINNinstances we obtain a value of L F,α =0 .
Revised: Interestingly, looking at the physics loss averaged over the ten PINN instances we obtain a value of L F,α =0 .

Original: 5 = 1 . 79 · 10 − 3 , with a standard deviation of 6 . 44 · 10 − 4 , thatis significantly lower compared to the reference solution with L F,RK = 5 .
Revised: 5 = 1 . 79 · 10 − 3 , with a standard deviation of 6 . 44 · 10 − 4 , that is significantly lower compared to the reference solution with L F,RK = 5 .

Original: 74 · 10 − 1 .
Revised: 74 · 10 − 1 .

Original: From theseresults, we find that the PINNs achieve a solution with an overall lower physics loss by, first, shiftingthe IC accordingly and, second, violating the physical constraints slightly in the initial phase.
Revised: From these results, we find that the PINNs achieve a solution with an overall lower physics loss by, first, shifting the IC accordingly and, second, violating the physical constraints slightly in the initial phase.

Original: A stronger emphasis on the IC ( α = 0 . 99 ) does not improve the prediction, as can be seen in Figure 1(b).
Revised: A stronger emphasis on the IC ( α = 0 . 99 ) does not improve the prediction, as can be seen in Figure(b).

Original: Although the IC is fulfilled more precisely, the shape of the trajectories still coincides well withthe results of the unweighted PINN ( α = 0 . 5 ).
Revised: Although the IC is fulfilled more precisely, the shape of the trajectories still coincides well with the results of the unweighted PINN ( α = 0 . 5 ).

Original: This causes the RK solution for the shifted ICs todeviate more from the PINN predictions.
Revised: This causes the RK solution for the shifted ICs to deviate more from the PINN predictions.

Original: Instead of converging to the solution that matches theshifted ICs, the PINNs choose to neglect the physical constraints in the initial phase and thus fail toexhibit the sensitivity to ICs characteristic for chaotic systems.
Revised: Instead of converging to the solution that matches the shifted ICs, the PINNs choose to neglect the physical constraints in the initial phase and thus fail to exhibit the sensitivity to ICs characteristic for chaotic systems.

Original: Compared to the unweighted case,the results for α = 0 .
Revised: Compared to the unweighted case, the results for α = 0 .

Original: 99 agree more accurately with the IC, but consequently need to violate thephysics more severely in order to reach this solution trajectory.
Revised: 99 agree more accurately with the IC, but consequently need to violate the physics more severely in order to reach this solution trajectory.

Original: Despite the violation of the physicalconstraints, the average physics loss over the ten PINN instances is still in a similar range to thereference with L F,α =0 .
Revised: Despite the violation of the physical constraints, the average physics loss over the ten PINN instances is still in a similar range to the reference with L F,α =0 .

Original: 99 = 2 .
Revised: 99 = 2 .

Original: 70 · 10 − 1 and a standard deviation of 4 . 7 · 10 − 1 .
Revised: 70 · 10 − 1 and a standard deviation of 4 . 7 · 10 − 1 .

Original: In a next step, we confirm that the convergence in the chaotic regime is strongly affected by the lengthof the computational domain and an appropriate choice of α .
Revised: In a next step, we confirm that the convergence in the chaotic regime is strongly affected by the length of the computational domain and an appropriate choice of α .

Original: Only after decreasing the computationaldomain and increasing α , the PINN converges to the correct solution, shown in Figure 1 (c) and (d).
Revised: Only after decreasing the computational domain and increasing α , the PINN converges to the correct solution, shown in Figure 1 (c) and (d).

Original: Lastly, the experiment is extended to initial angles of θ 0 = { 141 ◦ , . . .
Revised: Lastly, the experiment is extended to initial angles of θ 0 = { 141 ◦ , . . .

Original: , 156 ◦ } , focusing on theunweighted case ( α = 0 . 5 ) and one PINN instance with a fixed seed for the initial weights.
Revised: , 156 ◦ } , focusing on the unweighted case ( α = 0 . 5 ) and one PINN instance with a fixed seed for the initial weights.

Original: In theprevious experiment we observed that all PINNs equally shifted the IC to reach a solution with asignificantly lower physics loss.
Revised: In the previous experiment we observed that all PINNs equally shifted the IC to reach a solution with a significantly lower physics loss.

Original: Now, we aim to assess why the shifted solution is more attractivecompared to the reference solution.
Revised: Now, we aim to assess why the shifted solution is more attractive compared to the reference solution.

Original: To do so, we compare the spectral power ratio P f 0 above thefrequencies f 0 = 0 .
Revised: To do so, we compare the spectral power ratio P f 0 above the frequencies f 0 = 0 .

Original: 5 Hz , and the average magnitude of the derivatives | θ ′′ 1 | + | θ ′′ 2 | , shown in Figure 2.
Revised: 5 Hz , and the average magnitude of the derivatives | θ ′′ [Equation] | + | θ [Equation]

Revised: 2 | , shown in Figure 2.

Original: Both measures are computed using RK.
Revised: Both measures are computed using RK.

Original: Indicated arrows point from the true IC θ 0 to the shifted
Revised: Indicated arrows point from the true IC θ 0 to the shifted

Original: IC of the PINN ( α = 0 .
Revised: IC of the PINN ( α = 0 .

Original: 5 ) prediction.
Revised: 5 ) prediction.

Original: From Figure 2 (a) we can observe that the PINN shifts the
Revised: From Figure 2 (a) we can observe that the PINN shifts the

Original: IC to solutions with primarily low-frequency components.
Revised: IC to solutions with primarily low-frequency components.

Original: Figure 2 (b) indicates that the shiftedsolutions also exhibit smaller magnitudes of the second-order derivatives, which lead to smallerphysics loss values.
Revised: Figure 2 (b) indicates that the shifted solutions also exhibit smaller magnitudes of the second-order derivatives, which lead to smaller physics loss values.

Original: In particular, this effect is prominent for a computational domain of T = 2 ,where the PINN predictions lie in a valley of small values for both measures.
Revised: In particular, this effect is prominent for a computational domain of T = 2 , where the PINN predictions lie in a valley of small values for both measures.

Original: This valley is also
Revised: This valley is also

Original: 4 t − 2 .0 .2 .θ 1 , θθ 1 ( t 0 )
Revised: 4 t − 2 .0 .2 .θ 1 , θθ 1 ( t 0 )

Original: = 2 .
Revised: = 2 .

Original: 75 , θ 2 ( t 0 ) = 2 .
Revised: 75 , θ 2 ( t 0 ) = 2 .

Original: RK
Revised: RK

Original: PINN
Revised: PINN

Original: RK
Revised: RK

Original: [Figure]
Revised: [Figure]

Original: 4 t −   ω 1 , ωω 1 ( t 0 )
Revised: 4 t −   ω 1 , ωω 1 ( t 0 )

Original: = 0 .
Revised: = 0 .

Original: 03 , ω 2 ( t 0 )
Revised: 03 , ω 2 ( t 0 )

Original: = − 0 .(a)
Revised: = − 0 .(a)

Original: α = 0 .
Revised: α = 0 .

Original:  4 t − 2 .0 .2 .θ
Revised:  4 t − 2 .0 .2 .θ

Original: 1 , θθ
Revised: 1 , θθ

Original: [Figure]
Revised: [Figure]

Original: ( t 0 )
Revised: ( t 0 )

Original: = 2 .
Revised: = 2 .

Original: 71 , θ
Revised: 71 , θ

Original: [Figure]
Revised: [Figure]

Original: ( t
Revised: ( t

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: PINN
Revised: PINN

Original: RK
Revised: RK

Original: [Figure]
Revised: [Figure]

Original: 4 t
Revised: 4 t

Original: [Figure]
Revised: [Figure]

Original: ω 1 , ωω 1 ( t 0 )
Revised: ω 1 , ωω 1 ( t 0 )

Original: = 0 .
Revised: = 0 .

Original: 02 , ω 2 ( t 0 )
Revised: 02 , ω 2 ( t 0 )

Original: = − 0 .
Revised: = − 0 .

Original: [Figure]
Revised: [Figure]

Original: α = 0 .
Revised: α = 0 .

Original: [Figure]
Revised: [Figure]

Original: (d) α
Revised: (d) α

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: t 0 )
Revised: t 0 )

Original: [deg]  θ 2 ( t)
Revised: [deg]  θ 2 ( t)

Original: [ d e g ] 0.53 0.55 0.
Revised: [ d e g ] 0.53 0.55 0.

Original: P f(a)
Revised: P f(a)

Original: T =  θ 1 ( t 0 )
Revised: T =  θ 1 ( t 0 )

Original: [deg]  θ 2 ( t)
Revised: [deg]  θ 2 ( t)

Original: [ d e g ] 0.25 0.
Revised: [ d e g ] 0.25 0.

Original: P f(b)
Revised: P f(b)

Original: T =  θ 1 ( t 0 )
Revised: T =  θ 1 ( t 0 )

Original: [deg]  θ 2 ( t)
Revised: [deg]  θ 2 ( t)

Original: [ d e g ]  | θ 00 1 | + | θ 00 2 | (c)
Revised: [ d e g ]  | θ 00 1 | + | θ 00 2 | (c)

Original: T =  θ 1 ( t 0 )
Revised: T =  θ 1 ( t 0 )

Original: [deg]  θ 2 ( t)
Revised: [deg]  θ 2 ( t)

Original: [ d e g ]  | θ 00 1 | + | θ 00 2 | (d)
Revised: [ d e g ]  | θ 00 1 | + | θ 00 2 | (d)

Original: T = visible for the computational domain of T = 5 , although being more narrow.
Revised: T = visible for the computational domain of T = 5 , although being more narrow.

Original: Moreover, for T = 5the chaotic property of the double pendulum becomes evident by the strong influence of the IC on thevalues of the two measures we considered.
Revised: Moreover, for T =the chaotic property of the double pendulum becomes evident by the strong influence of the IC on the values of the two measures we considered.

Original:  Discussion and limitations
Revised:  Discussion and limitations

Original: Our experiments revealed that the traditional PINN framework is not suitable for simulating chaoticmotion.
Revised: Our experiments revealed that the traditional PINN framework is not suitable for simulating chaotic motion.

Original: As the goal of the PINN optimization is solely to minimize the combined loss functionof data and physics loss term, it is able to “cheat” by either shifting the initial condition to reach asimpler function or by violating the physics in a small domain to again reach the simpler function.
Revised: As the goal of the PINN optimization is solely to minimize the combined loss function of data and physics loss term, it is able to “cheat” by either shifting the initial condition to reach a simpler function or by violating the physics in a small domain to again reach the simpler function.

Original: Inboth cases, the trajectories resulted in lower physics loss values compared to the reference solution.
Revised: In both cases, the trajectories resulted in lower physics loss values compared to the reference solution.

Original: We argue that these convergence problems stem, on the one hand, from the spectral bias that causes
Revised: We argue that these convergence problems stem, on the one hand, from the spectral bias that causes

Original: PINNs to converge to low-frequency solutions first.
Revised: PINNs to converge to low-frequency solutions first.

Original: On the other hand, we observed that thosesolutions also exhibit lower magnitudes of the derivatives that comprise the calculation of the physicsloss and hence explain low physics loss values.
Revised: On the other hand, we observed that those solutions also exhibit lower magnitudes of the derivatives that comprise the calculation of the physics loss and hence explain low physics loss values.

Original: We find that reducing the computational domaincombined with an appropriate loss weighting scheme allows convergence to the correct solution.
Revised: We find that reducing the computational domain combined with an appropriate loss weighting scheme allows convergence to the correct solution.

Original: In future research, we intend to extend our experiments to additional ODE-systems in order tosubstantiate our claims presented in this work.
Revised: In future research, we intend to extend our experiments to additional ODE-systems in order to substantiate our claims presented in this work.

Original: A AppendixA.1 Double pendulum
Revised: A Appendix A.1 Double pendulum

Original: The motion of the planar double pendulum (see Figure 3) is described by the following nonlinearinitial-value problem with y =
Revised: The motion of the planar double pendulum (see Figure 3) is described by the following nonlinear initial-value problem with y =

Original: [ θ 1 , θ 2 ] T
Revised: [ θ 1 , θ 2 ] T

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation][Figure] [Equation] [Figure]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: 4) subject to the initial conditions
Revised: subject to the initial conditions

Original: [Equation]
Revised: [Equation]

Original: where θ ′ 1 = ω 1 , θ ′ 2 = ω 2 , ∆ θ = θ 1 − θ 2 , and g = 9 .
Revised: where θ ′ 1 = ω 1 , θ ′ 2 = ω 2 , ∆ θ = θ 1 − θ 2 , and g = 9 .

Original: 81 is the gravitational acceleration.
Revised: 81 is the gravitational acceleration.

Original: [Figure]
Revised: [Figure]

Original: A.2 Physics-informed neural networks
Revised: A.2 Physics-informed neural networks

Original: The unknown solution y is approximated by a fully-connected deep neural network y w with wdescribing all trainable weights and biases of the network.
Revised: The unknown solution y is approximated by a fully-connected deep neural network y w with w describing all trainable weights and biases of the network.

Original: For simulating the motion of the doublependulum, the PINN [1] is trained by minimizing the following combined loss function
Revised: For simulating the motion of the double pendulum, the PINN [1] is trained by minimizing the following combined loss function

Original: [Equation]
Revised: [Equation]

Original: (6) with the data-based loss
Revised: (6) with the data-based loss

Original: [Equation]
Revised: [Equation]

Original: and the physics-based loss consisting of the ODE-residuals of the physical system
Revised: and the physics-based loss consisting of the ODE-residuals of the physical system

Original: [Equation]
Revised: [Equation]

Original: where { t i } N col i =1 denotes the collocation points where the physics loss is evaluated.
Revised: where { t i } N col i =1 denotes the collocation points where the physics loss is evaluated.

Original: These collocation292 points can be chosen arbitrarily within the computational domain, either by using fixed locationsor by adaptive resampling [27, 3, 28].
Revised: These collocation points can be chosen arbitrarily within the computational domain, either by using fixed locations or by adaptive resampling [27, 3, 28].

Original: In this work, N col = 1024 collocation points are sampleduniformly in each training epoch.
Revised: In this work, N col = 1024 collocation points are sampled uniformly in each training epoch.