Original: B R GAN S : S TABILIZING GAN S ’ T RAINING P ROCESS WITH B ROWNIAN M OTION C ONTROL
Revised: B R GAN S : S TABILIZING GAN S ’ T RAINING P ROCESS WITH B ROWNIAN M OTION C ONTROL

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The training process of generative adversarial networks (GANs) is unstable and does not converge globally.
Revised: The training process of generative adversarial networks (GANs) is unstable and does not converge globally.

Original: In this paper, we propose a universal higher-order noise-based controller called Brownian Motion Controller (BMC) that is invariant to GANs’ frameworks so that the training process of GANs is stabilized.
Revised: In this paper, we propose a universal higher-order noise-based controller called Brownian Motion Controller (BMC) that is invariant to GANs’ frameworks so that the training process of GANs is stabilized.

Original: Speciﬁcally, starting with the prototypical case of Dirac-GANs, we design a BMC and propose Dirac-BrGANs, which retrieve exactly the same but reachable optimal equilibrium regardless of GANs’ framework.
Revised: Speciﬁcally, starting with the prototypical case of Dirac-GANs, we design a BMC and propose Dirac-BrGANs, which retrieve exactly the same but reachable optimal equilibrium regardless of GANs’ framework.

Original: The optimal equilibrium of our Dirac-BrGANs’ training system is globally unique and always exists.
Revised: The optimal equilibrium of our Dirac-BrGANs’ training system is globally unique and always exists.

Original: Furthermore, we give theoretical proof that the training process of Dirac-BrGANs achieves exponential stability almost surely for any arbitrary initial value and derive bounds for the rate of convergence.
Revised: Furthermore, we give theoretical proof that the training process of Dirac-BrGANs achieves exponential stability almost surely for any arbitrary initial value and derive bounds for the rate of convergence.

Original: Then we extend our BMC to normal
Revised: Then we extend our BMC to normal

Original: GANs and propose BrGANs.
Revised: GANs and propose BrGANs.

Original: We provide numerical experiments showing that our BrGANs effectively stabilize GANs’ training process and obtain state-of-theart performance in terms of FID and inception score compared to other stabilizing methods.
Revised: We provide numerical experiments showing that our BrGANs effectively stabilize GANs’ training process and obtain state-of-theart performance in terms of FID and inception score compared to other stabilizing methods.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) are popular deep learning based generative architecture.
Revised: Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) are popular deep learning based generative architecture.

Original: Given a multi-dimensional input dataset with unknown P real , GANs can obtain an estimated P model and produce new entries that are as close to indistinguishable as possible from the input entries.
Revised: Given a multi-dimensional input dataset with unknown P real , GANs can obtain an estimated P model and produce new entries that are as close to indistinguishable as possible from the input entries.

Original: For example, GANs can be used to generate images that look real to the human eyes (Wang et al., 2017).
Revised: For example, GANs can be used to generate images that look real to the human eyes (Wang et al., 2017).

Original: A GAN’s architecture consists of two neural networks: a generator and a discriminator .
Revised: A GAN’s architecture consists of two neural networks: a generator and a discriminator .

Original: The generator creates new elements that resemble the entries from the input dataset as closely as possible.
Revised: The generator creates new elements that resemble the entries from the input dataset as closely as possible.

Original: The discriminator, on the other hand, aims to distinguish the (counterfeit) entries produced by the generator from the original members of the input dataset.
Revised: The discriminator, on the other hand, aims to distinguish the (counterfeit) entries produced by the generator from the original members of the input dataset.

Original: The GAN’s two networks can be modeled as a minimax problem; they compete against one another while striving to reach a Nash-equilibrium , an optimal solution where the generator can produce fake entries that are, from the point of view of the discriminator, in all respects indistinguishable from real ones.
Revised: The GAN’s two networks can be modeled as a minimax problem; they compete against one another while striving to reach a Nash-equilibrium , an optimal solution where the generator can produce fake entries that are, from the point of view of the discriminator, in all respects indistinguishable from real ones.

Original: Unfortunately, training GANs often suffers from instabilities.
Revised: Unfortunately, training GANs often suffers from instabilities.

Original: Previously, theoretical analysis has been conducted on GAN’s training process.
Revised: Previously, theoretical analysis has been conducted on GAN’s training process.

Original: Fedus et al.
Revised: Fedus et al.

Original: (2018) argue that the traditional view of considering training GANs as minimizing the divergence of real distribution and model distribution is too restrictive and thus leads to instability.
Revised: (2018) argue that the traditional view of considering training GANs as minimizing the divergence of real distribution and model distribution is too restrictive and thus leads to instability.

Original: Arora et al.
Revised: Arora et al.

Original: (2018) show that GANs training process does not lead generator to the desired distribution.
Revised: (2018) show that GANs training process does not lead generator to the desired distribution.

Original: Farnia & Ozdaglar (2020) suggest that current training methods of GANs do not always have Nash equilibrium, and Heusel et al.
Revised: Farnia & Ozdaglar (2020) suggest that current training methods of GANs do not always have Nash equilibrium, and Heusel et al.

Original: (2017a) is able to push GANs to converge to local Nash equilibrium using a two-time scale update rule (TTUR).
Revised: (2017a) is able to push GANs to converge to local Nash equilibrium using a two-time scale update rule (TTUR).

Original: Many previous methods (Mescheder et al., 2018; Arjovsky & Bottou, 2017; Nagarajan & Kolter, 2017; Kodali et al., 2017) have investigated the causes of such instabilities and attempted to reduce them by introducing various changes to the GAN’s architecture.
Revised: Many previous methods (Mescheder et al., 2018; Arjovsky & Bottou, 2017; Nagarajan & Kolter, 2017; Kodali et al., 2017) have investigated the causes of such instabilities and attempted to reduce them by introducing various changes to the GAN’s architecture.

Original: However, as Mescheder et al.
Revised: However, as Mescheder et al.

Original: (2018) show in their study, where they analyze the convergence behavior of several GANs models, despite bringing signiﬁcant improvements, GANs and its variations are still far from achieving stability in the general case.
Revised: (2018) show in their study, where they analyze the convergence behavior of several GANs models, despite bringing signiﬁcant improvements, GANs and its variations are still far from achieving stability in the general case.

Original: To accomplish this goal, in our work, we design a Brownian Motion Control (BMC) using control theory and propose a universal model, BrGANs, to stabilize GANs’ training process.
Revised: To accomplish this goal, in our work, we design a Brownian Motion Control (BMC) using control theory and propose a universal model, BrGANs, to stabilize GANs’ training process.

Original: We start with the prototypical Dirac-GAN (Mescheder et al., 2018) and analyze its system of training dynamics.
Revised: We start with the prototypical Dirac-GAN (Mescheder et al., 2018) and analyze its system of training dynamics.

Original: We then design a Brownian motion controller (BMC) on the training dynamic of Dirac-GAN in order to stabilize this system over time domain t .
Revised: We then design a Brownian motion controller (BMC) on the training dynamic of Dirac-GAN in order to stabilize this system over time domain t .

Original: We generalize our BMC to normal GANs’ setting and propose BrGANs.
Revised: We generalize our BMC to normal GANs’ setting and propose BrGANs.

Original: S UMMARY OF OUR C ONTRIBUTION
Revised: S UMMARY OF OUR C ONTRIBUTION

Original: Compared with previous methods, we have the following contributions:
Revised: Compared with previous methods, we have the following contributions:

Original: • We design Brownian Motion Controller (BMC), a universal higher order noise-based controller for GANs’ training dynamics, which is compatible with all GANs’ frameworks, and we give both theoretical and empirical analysis showing BMC effectively stabilizes GANs’ training process.
Revised: • We design Brownian Motion Controller (BMC), a universal higher order noise-based controller for GANs’ training dynamics, which is compatible with all GANs’ frameworks, and we give both theoretical and empirical analysis showing BMC effectively stabilizes GANs’ training process.

Original: • Under Dirac-GANs’ setting, we propose Dirac-BrGANs and conduct theoretical stability analysis to derive bounds on the rate of convergence.
Revised: • Under Dirac-GANs’ setting, we propose Dirac-BrGANs and conduct theoretical stability analysis to derive bounds on the rate of convergence.

Original: Our proposed Dirac-BrGANs are able to converge globally with exponential stability.
Revised: Our proposed Dirac-BrGANs are able to converge globally with exponential stability.

Original: • We extend BMC to normal GANs’ settings and propose BrGANs.
Revised: • We extend BMC to normal GANs’ settings and propose BrGANs.

Original: In experiments, our BrGANs converge faster and perform better in terms of inception scores and FID scores on CIFAR-10 and CelebA datasets than previous baselines in various GANs models.
Revised: In experiments, our BrGANs converge faster and perform better in terms of inception scores and FID scores on CIFAR-10 and CelebA datasets than previous baselines in various GANs models.

Original: 1. R ELATED W ORK
Revised: 1. R ELATED W ORK

Original: To stabilize GANs training process, a lot of work has been done on modifying its training architecture.
Revised: To stabilize GANs training process, a lot of work has been done on modifying its training architecture.

Original: Karras et al.
Revised: Karras et al.

Original: (2018) train the generator and the discriminator progressively to stabilize the training process.
Revised: (2018) train the generator and the discriminator progressively to stabilize the training process.

Original: Wang et al.
Revised: Wang et al.

Original: (2021) observe that during training, the discriminator converges faster and dominates the dynamics.
Revised: (2021) observe that during training, the discriminator converges faster and dominates the dynamics.

Original: They produce an attention map from the discriminator and use it to improve the spatial awareness of the generator.
Revised: They produce an attention map from the discriminator and use it to improve the spatial awareness of the generator.

Original: In this way, they push GANs’ solution closer to the equilibrium.
Revised: In this way, they push GANs’ solution closer to the equilibrium.

Original: On the other hand, many work stabilizes GANs’ training process with modiﬁed objective functions.
Revised: On the other hand, many work stabilizes GANs’ training process with modiﬁed objective functions.

Original: Kodali et al.
Revised: Kodali et al.

Original: (2017) add gradient penalty to their objective function to avoid local equilibrium with their model called DRAGAN.
Revised: (2017) add gradient penalty to their objective function to avoid local equilibrium with their model called DRAGAN.

Original: This method has fewer mode collapses and can be applied to a lot of GANs’ frameworks.
Revised: This method has fewer mode collapses and can be applied to a lot of GANs’ frameworks.

Original: Other work, such as Generative Multi-Adversarial Network (GMAN) (Durugkar et al., 2017), packing GANs (PacGAN) (Lin et al., 2017) and energy-based GANs (Zhao et al., 2016), modiﬁes the discriminator to achieve better stability.
Revised: Other work, such as Generative Multi-Adversarial Network (GMAN) (Durugkar et al., 2017), packing GANs (PacGAN) (Lin et al., 2017) and energy-based GANs (Zhao et al., 2016), modiﬁes the discriminator to achieve better stability.

Original: Xu et al.
Revised: Xu et al.

Original: (2019) formulate GANs as a system of differential equations and add closed-loop control (CLC) on a few variations of the GANs to enforce stability.
Revised: (2019) formulate GANs as a system of differential equations and add closed-loop control (CLC) on a few variations of the GANs to enforce stability.

Original: However, the design of their controller depends on the objective function of the GANs models and does not work for all variations of the GANs models.
Revised: However, the design of their controller depends on the objective function of the GANs models and does not work for all variations of the GANs models.

Original: Motivated by them, we analyze GANs’ training process from control theory’s perspective and design an invariant Brownian Motion Controller (BMC) to stabilize GANs training process.
Revised: Motivated by them, we analyze GANs’ training process from control theory’s perspective and design an invariant Brownian Motion Controller (BMC) to stabilize GANs training process.

Original: Compared with Xu et al.
Revised: Compared with Xu et al.

Original: (2019), our proposed BrGANs converge faster, perform better, and do not rely on any speciﬁc GANs’ architecture.
Revised: (2019), our proposed BrGANs converge faster, perform better, and do not rely on any speciﬁc GANs’ architecture.

Original:  C ONTROLLING D IRAC -GAN WITH B ROWNIAN MOTION
Revised:  C ONTROLLING D IRAC -GAN WITH B ROWNIAN MOTION

Original: In this section, we come up with the BMC, a higher order noise-based controller, as a universal control function that is invariant to objective functions of various GANs models.
Revised: In this section, we come up with the BMC, a higher order noise-based controller, as a universal control function that is invariant to objective functions of various GANs models.

Original: In addition, we prove that Dirac-GAN with BMC is exponentially stable and we derive bounds on its converge rate.
Revised: In addition, we prove that Dirac-GAN with BMC is exponentially stable and we derive bounds on its converge rate.

Original: 2. D YNAMIC S YSTEM OF D IRAC -GAN S
Revised: 2. D YNAMIC S YSTEM OF D IRAC -GAN S

Original: In Dirac-GANs’ settings, the distribution of generator G follows p G ( x ) =
Revised: In Dirac-GANs’ settings, the distribution of generator G follows p G ( x ) =

Original: δ ( x − θ ) and the discriminator is linear D φ ( x )
Revised: δ ( x − θ ) and the discriminator is linear D φ ( x )

Original: = φx .
Revised: = φx .

Original: The true data distribution is given by p D ( x ) =
Revised: The true data distribution is given by p D ( x ) =

Revised: δ ( x − c ) with a constant c. (Notice that for Dirac distribution, δ can be considered as an impulse on the origin

Original:  δ ( x− c ) with a constant c. The objective functions of Dirac-GANs can be written as:
Revised: such that δ ( x ) = 1 at the origin and 0 otherwise.) The objective functions of Dirac-GANs can be written as:

Original: [Equation]
Revised: [Equation]

Original: (1) where h 1 ( · ) and h 3 ( · ) are increasing functions and h 2 ( · ) is a decreasing function around zero (Xu et al., 2019).
Revised: (1) where h 1 ( · ) and h 3 ( · ) are increasing functions and h 2 ( · ) is a decreasing function around zero (Xu et al., 2019).

Original: The training process of Dirac-GAN can be modelled as a system of differential equations.
Revised: The training process of Dirac-GAN can be modelled as a system of differential equations.

Original: Following Mescheder et al.
Revised: Following Mescheder et al.

Original: (2018) and Xu et al.
Revised: (2018) and Xu et al.

Original: (2019), the training dynamical system of Dirac-GAN is formulated as:
Revised: (2019), the training dynamical system of Dirac-GAN is formulated as:

Original: [Equation]
Revised: [Equation]

Original: This system has a constant nontrivial solution ( θ, φ ) =
Revised: This system has a constant nontrivial solution ( θ, φ ) =

Original: ( c, 0) .
Revised: ( c, 0) .

Original: Let ˜ θ ( t )
Revised: Let ˜ θ ( t )

Original: = θ ( t ) − c , and convert the original system (2) to:
Revised: = θ ( t ) − c , and convert the original system (2) to:

Original: [Equation]
Revised: [Equation]

Original: At this time, the equilibrium of system (3) is (0 , 0) .
Revised: At this time, the equilibrium of system (3) is (0 , 0) .

Original: Deﬁne X ( t ) =
Revised: Deﬁne X ( t ) =

Original: ( φ ( t ) ,˜ θ ( t ))
Revised: ( φ ( t ) ,

Original: [Equation]
Revised: [Equation]

Original: , f ( X ( t )) =

Original:  (h
Revised: θ (

Original: [Equation]
Revised: [Equation]

Original: 1 ( φ (
Revised: )) (cid:62) , f

Original: [Equation]
Revised: [Equation]

Original: ) c ) c
Revised: X ( t )) =

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original:  +
Revised: 1 ( φ ( t ) c ) c +

Original: [Equation]
Revised: [Equation]

Original:  + h (cid:48) 2 ( φ ( t )(˜ θ ( t ) + c ))
Revised: 2 ( φ ( t )(˜ θ ( t ) + c )) ˜ θ ( t ) + h (cid:48) 2 ( φ ( t )(˜ θ ( t ) + c ))

Original: c, h (cid:48) 3 ( φ ( t )(
Revised: c, h (cid:48) 3 ( φ ( t )(

Original: ˜ θ ( t ) + c )) φ ( t )) (cid:62) .
Revised: ˜ θ ( t ) + c )) φ ( t )) (cid:62) .

Original: Then system (3) can be rewritten as:
Revised: Then system (3) can be rewritten as:

Original: [Equation]
Revised: [Equation]

Original: 2. D ESIGNING B ROWNIAN M OTION C ONTROLLER FOR D IRAC -GAN
Revised: 2. D ESIGNING B ROWNIAN M OTION C ONTROLLER FOR D IRAC -GAN

Original: Brownian motion is a natural phenomenon that captures the random displacements of particles in d -dimensional space.
Revised: Brownian motion is a natural phenomenon that captures the random displacements of particles in d -dimensional space.

Original: At each time step, the displacement B t is an independent, identical random variable ranging in R d .
Revised: At each time step, the displacement B t is an independent, identical random variable ranging in R d .

Original: The distribution of B t is normally characterized by a multivariate Gaussian distribution.
Revised: The distribution of B t is normally characterized by a multivariate Gaussian distribution.

Original: Denote the position of a particle at initial time 0 as X (0) .
Revised: Denote the position of a particle at initial time 0 as X (0) .

Original: Then at time T , this particle’s position is given as
Revised: Then at time T , this particle’s position is given as

Original: [Equation]
Revised: [Equation]

Original: In control theory, noise-based controllers like our Brownian motion controller (BMC) are a useful tool to stabilize dynamical systems and push the solution towards the optimal value over time domain t (Mao et al., 2002).
Revised: In control theory, noise-based controllers like our Brownian motion controller (BMC) are a useful tool to stabilize dynamical systems and push the solution towards the optimal value over time domain t (Mao et al., 2002).

Original: In this section, we design a BMC on Dirac-GAN’s training dynamic to improve stability.
Revised: In this section, we design a BMC on Dirac-GAN’s training dynamic to improve stability.

Original: To stabilize system (4), we propose the following higher order noise-based controller:
Revised: To stabilize system (4), we propose the following higher order noise-based controller:

Original: [Equation]
Revised: [Equation]

Original: where B 1 ( t ) and B 2 ( t ) are independent one-dimensional Brownian motions, β > 1 , (cid:37) 1 and (cid:37) 2 are non-negative constants.
Revised: where B 1 ( t ) and B 2 ( t ) are independent one-dimensional Brownian motions, β > 1 , (cid:37) 1 and (cid:37) 2 are non-negative constants.

Original: Incorporating BMC (6), the controlled system is given as
Revised: Incorporating BMC (6), the controlled system is given as

Original: [Equation]
Revised: [Equation]

Original: 2. D IRAC -B R GAN WITH EXPONENTIAL STABILITY
Revised: 2. D IRAC -B R GAN WITH EXPONENTIAL STABILITY

Original: In this section, we derive the existence of unique global solution and stability of system (7).
Revised: In this section, we derive the existence of unique global solution and stability of system (7).

Original: The equilibrium point of system (3) is
Revised: The equilibrium point of system (3) is

Original: [Equation]
Revised: [Equation]

Original: Without the BMC, the training of a regular GAN or a WGAN (Arjovsky et al., 2017) is unstable and it oscillates around the equilibrium point (0 , 0) (cid:62) .
Revised: Without the BMC, the training of a regular GAN or a WGAN (Arjovsky et al., 2017) is unstable and it oscillates around the equilibrium point (0 , 0) (cid:62) .

Original: Figure 1 illustrates the gradient maps of θ against φ and convergence behavior over time domain t .
Revised: Figure 1 illustrates the gradient maps of θ against φ and convergence behavior over time domain t .

Original: We can see that the gradients of both the generator and the discriminator are oscillating around the equilibrium point, but they never converge to it.
Revised: We can see that the gradients of both the generator and the discriminator are oscillating around the equilibrium point, but they never converge to it.

Original: For the stability analysis, we impose the following assumption on the smoothness of functions h 1 , h 2 , h 3 in system (2).
Revised: For the stability analysis, we impose the following assumption on the smoothness of functions h 1 , h 2 , h 3 in system (2).

Original: Assumption 1.
Revised: Assumption 1.

Original: There exist positive constants α 1 , α 2 , α 3 such that for any x, y ∈ R n ,
Revised: There exist positive constants α 1 , α 2 , α 3 such that for any x, y ∈ R n ,

Original: [Equation]
Revised: [Equation]

Original: In what follows, we ﬁrst prove that the BMC from equation (6) yields a unique global solution in
Revised: In what follows, we ﬁrst prove that the BMC from equation (6) yields a unique global solution in

Original: Theorem 1.
Revised: Theorem 1.

Original: That is, no matter which initial point X (0) we are starting from, system (7) a.s. will have a unique solution X ( t ) as t goes to inﬁnity.
Revised: That is, no matter which initial point X (0) we are starting from, system (7) a.s. will have a unique solution X ( t ) as t goes to inﬁnity.

Original: Then, in Theorem 2, we show that this unique global solution exponentially converges to the equilibrium point a.s. with bounds on the hyperparameters (cid:37) 1 , (cid:37) 2 and β which in turn affect the rate of convergence.
Revised: Then, in Theorem 2, we show that this unique global solution exponentially converges to the equilibrium point a.s. with bounds on the hyperparameters (cid:37) 1 , (cid:37) 2 and β which in turn affect the rate of convergence.

Original: Combining Theorem 1 and
Revised: Combining Theorem 1 and

Original: Theorem 2, we claim that system (7) is stable and thus the Dirac-BrGAN is stable and converges to the optimal solution as required.
Revised: Theorem 2, we claim that system (7) is stable and thus the Dirac-BrGAN is stable and converges to the optimal solution as required.

Original: Theorem 1.
Revised: Theorem 1.

Original: (Proof in Appendix A)
Revised: (Proof in Appendix A)

Original: Under Assumption 1, for any initial value X (0) =
Revised: Under Assumption 1, for any initial value X (0) =

Original: ξ ∈ R 2 , if (cid:37) 2 (cid:54) = 0 and β > 1 , then there a.s. exists a unique global solution X ( t ) to system (7) on t ∈ [0 , ∞ ) .
Revised: ξ ∈ R 2 , if (cid:37) 2 (cid:54) = 0 and β > 1 , then there a.s. exists a unique global solution X ( t ) to system (7) on t ∈ [0 , ∞ ) .

Original: Theorem 2.
Revised: Theorem 2.

Original: (Proof in Appendix B) Let Assumption 1 hold.
Revised: (Proof in Appendix B) Let Assumption 1 hold.

Original: Assume that (cid:37) 2 (cid:54) = 0 and β > 1 .
Revised: Assume that (cid:37) 2 (cid:54) = 0 and β > 1 .

Original: If
Revised: If

Original: [Equation]
Revised: [Equation]

Original: where ϕ takes the value of
Revised: where ϕ takes the value of

Original: [Equation]
Revised: [Equation]

Revised: x ≥

Revised: [Equation]

Revised: [Equation]

Original: then for any X (0) =
Revised: then for any X (0) =

Original: ξ with sufﬁciently small constant (cid:15) ∈ (0 , (cid:37) 21 / 2 − ϕ ) , the global solution X ( t ) of system (7) has the property that 
Revised: ξ with sufﬁciently small constant (cid:15) ∈ (0 , (cid:37) 21 / 2 − ϕ ) , the global solution X ( t ) of system (7) has the property that [Equation]

Original: [Equation]
Revised: [Equation]

Original: that is, the solution of system (7) is a.s. exponentially stable.
Revised: that is, the solution of system (7) is a.s. exponentially stable.

Original: Here since (cid:37) 21 2 − ϕ > 0 and (cid:15) is a sufﬁciently small constant, then when Eq.
Revised: Here since (cid:37) 21 2 − ϕ > 0 and (cid:15) is a sufﬁciently small constant, then when Eq.

Original: (9) is satisﬁed, we have
Revised: (9) is satisﬁed, we have

Original: [Equation]
Revised: [Equation]

Original: for some positive constant λ .
Revised: for some positive constant λ .

Original: Rearranging we get
Revised: Rearranging we get

Original: [Equation]
Revised: [Equation]

Original: which implies
Revised: which implies

Original: [Equation]
Revised: [Equation]

Original: as required.
Revised: as required.

Original: Notice that the rate of convergence depends only on constant λ , which in turn depends on (cid:37) 1 and ϕ .
Revised: Notice that the rate of convergence depends only on constant λ , which in turn depends on (cid:37) 1 and ϕ .

Original: It means that the convergence rate is decided by the choice of hyper-parameters (cid:37) 1 , (cid:37) 2 , and β .
Revised: It means that the convergence rate is decided by the choice of hyper-parameters (cid:37) 1 , (cid:37) 2 , and β .

Original: In practice, we can tune these three variables as desired, as long as they satisfy the constraint from equation 9.
Revised: In practice, we can tune these three variables as desired, as long as they satisfy the constraint from equation 9.

Original: Notice that our Dirac-BrGANs works for any h 1 , h 2 and h 3 as long as they satisfy the smoothness condition under assumption 1.
Revised: Notice that our Dirac-BrGANs works for any h 1 , h 2 and h 3 as long as they satisfy the smoothness condition under assumption 1.

Original: In other words, we have proven that the Dirac-BrGANs are stable regardless of the GANs’ architecture, and we have given theoretical bounds on the rate of convergence.
Revised: In other words, we have proven that the Dirac-BrGANs are stable regardless of the GANs’ architecture, and we have given theoretical bounds on the rate of convergence.

Original: In Figure 1, we present visual proof that the Dirac-BrGANs are stable and converge to the optimal equilibrum as required.
Revised: In Figure 1, we present visual proof that the Dirac-BrGANs are stable and converge to the optimal equilibrum as required.

Original:  G ENERALIZATION OF BMC TO GAN S
Revised:  G ENERALIZATION OF BMC TO GAN S

Original: In section 2 we designed a universal BMC for Dirac-GANs and proved that the Dirac-BrGANs are globally exponentially stable.
Revised: In section 2 we designed a universal BMC for Dirac-GANs and proved that the Dirac-BrGANs are globally exponentially stable.

Original: In this section, we are going to generalize the BMC for normal GANs (i.e., GANs other than the Dirac-GAN).
Revised: In this section, we are going to generalize the BMC for normal GANs (i.e., GANs other than the Dirac-GAN).

Original: We consider any GANs where the Generator ( G ) and the
Revised: We consider any GANs where the Generator ( G ) and the

Original: Discriminator ( D ) are neural networks in their respective function spaces.
Revised: Discriminator ( D ) are neural networks in their respective function spaces.

Original: 3. M ODELLING D YNAMICS OF GAN S
Revised: 3. M ODELLING D YNAMICS OF GAN S

Original: Analogously to the Dirac-GAN, the training dynamics of normal GANs can be formulated as a system of differential equations.
Revised: Analogously to the Dirac-GAN, the training dynamics of normal GANs can be formulated as a system of differential equations.

Original: Instead of θ and φ , we directly start with G ( z, t ) and D ( x, t ) to represent, respectively, the generator and the discriminator.
Revised: Instead of θ and φ , we directly start with G ( z, t ) and D ( x, t ) to represent, respectively, the generator and the discriminator.

Original: The objective functions of GANs can be written as:
Revised: The objective functions of GANs can be written as:

Original: [Equation]
Revised: [Equation]

Original: Following Xu et al.
Revised: Following Xu et al.

Original: (2019)’s notation, the training dynamic for the generator and discriminator over the time domain t can be transformed to:
Revised: (2019)’s notation, the training dynamic for the generator and discriminator over the time domain t can be transformed to:

Original: [Equation]
Revised: [Equation]

Original: Now we deﬁne X ( t ) =
Revised: Now we deﬁne X ( t ) =

Original: ( D ( x, t ) , G ( z, t )) (cid:62) and f
Revised: ( D ( x, t ) , G ( z, t )) (cid:62) and f

Original: [Equation]
Revised: [Equation]

Original: X
Revised: X

Original: [Equation]
Revised: [Equation]

Original: Now we convert system (14) to
Revised: Now we convert system (14) to

Original: [Equation]
Revised: [Equation]

Original: 3. B R GAN: S TABILIZED GAN S WITH BMC
Revised: 3. B R GAN: S TABILIZED GAN S WITH BMC

Original: The optimal solution of a normal GAN is achieved when D ( x ) = 0 for the discriminator and p G ( x ) = p ( x ) for the generator.
Revised: The optimal solution of a normal GAN is achieved when D ( x ) = 0 for the discriminator and p G ( x ) = p ( x ) for the generator.

Original: In control theory, when we design a controller for a dynamical system, we need to know the optimal solution so that we can use the controller to push this dynamical system to its equilibrium point without changing it.
Revised: In control theory, when we design a controller for a dynamical system, we need to know the optimal solution so that we can use the controller to push this dynamical system to its equilibrium point without changing it.

Original: With normal GANs, we only have information on the optimal solution for the discriminator, so we are going to impose the BMC only on the discriminator this time.
Revised: With normal GANs, we only have information on the optimal solution for the discriminator, so we are going to impose the BMC only on the discriminator this time.

Original: Notice that in Dirac-GAN, we impose the BMC u ( t ) =
Revised: Notice that in Dirac-GAN, we impose the BMC u ( t ) =

Original: (cid:37) 1 X ( t ) ˙
Revised: (cid:37) 1 X ( t ) ˙

Original: B 1 ( t ) + (cid:37) 2 | X ( t ) | β X ( t ) ˙
Revised: B 1 ( t ) + (cid:37) 2 | X ( t ) | β X ( t ) ˙

Original: B 2 ( t ) on
Revised: B 2 ( t ) on

Original: X ( t ) , which is for both the generator and discriminator.
Revised: X ( t ) , which is for both the generator and discriminator.

Original: Since now we are going to impose BMC only on the discriminator, without losing information from the generator, here we slightly modify
Revised: Since now we are going to impose BMC only on the discriminator, without losing information from the generator, here we slightly modify

Original: Eq.
Revised: Eq.

Original: (6) so that
Revised: (6) so that

Original: [Equation]
Revised: [Equation]

Original: 17) where B 1 ( t ) and B 2 ( t ) are independent one-dimensional Brownian motions, (cid:37) 1 and (cid:37) 2 are nonnegative constants
Revised: 17) where B 1 ( t ) and B 2 ( t ) are independent one-dimensional Brownian motions, (cid:37) 1 and (cid:37) 2 are nonnegative constants

Original: [Equation]
Revised: [Equation]

Original: Since we are going to implement BrGANs through gradient descent, our BMC can be reﬂected on the discriminator’s objective function with the derivative being Eq.
Revised: Since we are going to implement BrGANs through gradient descent, our BMC can be reﬂected on the discriminator’s objective function with the derivative being Eq.

Original: We thus take integration of
Revised: We thus take integration of

Original: Eq.
Revised: Eq.

Original: and modify the objective function of the discriminator in (13) to:
Revised: and modify the objective function of the discriminator in (13) to:

Original: [Equation]
Revised: [Equation]

Original: We implement our designed objective functions in section 4.
Revised: We implement our designed objective functions in section 4.

Original: Our numerical experiments show that BrGANs successfully stabilize GANs models and are able to generate images with promising quality.
Revised: Our numerical experiments show that BrGANs successfully stabilize GANs models and are able to generate images with promising quality.

Original:  E VALUATION
Revised:  E VALUATION

Original: In this section, we show the effectiveness of BMC by providing both quantitive and qualititive results.
Revised: In this section, we show the effectiveness of BMC by providing both quantitive and qualititive results.

Original: 4. E XEPERIMENTAL SETTING
Revised: 4. E XEPERIMENTAL SETTING

Original: Dataset: We evaluate our proposed BrGANs on well-established CIFAR10 (Krizhevsky et al., 2009) and CelebA datasets (Liu et al., 2015).
Revised: Dataset: We evaluate our proposed BrGANs on well-established CIFAR10 (Krizhevsky et al., 2009) and CelebA datasets (Liu et al., 2015).

Original: The CIFAR-10 dataset consists of 60000 32 × 32 color images in 10 classes, with 6000 images per class.
Revised: The CIFAR-10 dataset consists of 60000 32 × 32 color images in 10 classes, with 6000 images per class.

Original: There are 50000 training images and 10000 test images.
Revised: There are 50000 training images and 10000 test images.

Original: This dataset can be used for both conditional image generation and unconditional image generation.
Revised: This dataset can be used for both conditional image generation and unconditional image generation.

Original: In order to compare our training method fairly with the solutions from the related works, we use a batch size of 64, the same generator, and discriminator architecture under the same codebase.
Revised: In order to compare our training method fairly with the solutions from the related works, we use a batch size of 64, the same generator, and discriminator architecture under the same codebase.

Original: CelebA contains 202,599 face images of size 64 × 64, which has diverse facial features.
Revised: CelebA contains 202,599 face images of size 64 × 64, which has diverse facial features.

Original: Implementation details: Both our generator and discriminator are composed of convolutional layers, batch normalization, and activation layers.
Revised: Implementation details: Both our generator and discriminator are composed of convolutional layers, batch normalization, and activation layers.

Original: The generator uses four layers of transposed convolutional layers to convert 1 × 100 latent vector to a 3 × 32 × 32 image.
Revised: The generator uses four layers of transposed convolutional layers to convert 1 × 100 latent vector to a 3 × 32 × 32 image.

Original: Batch normalization and ReLU activation are followed by each layer.
Revised: Batch normalization and ReLU activation are followed by each layer.

Original: For the discriminator, ﬁrst, we use three layers of convolutional layer to obtain 1024 × 4 × 4 image
Revised: For the discriminator, ﬁrst, we use three layers of convolutional layer to obtain 1024 × 4 × 4 image

Original: and then feed this image to an MLP to get a single value.
Revised: and then feed this image to an MLP to get a single value.

Original: Each of our models is trained on an Nvidia 2080TI GPU.
Revised: Each of our models is trained on an Nvidia 2080TI GPU.

Original: The batch size is 64, while the generator is trained for 50000 iterations, and the discriminator is trained for 250000 iterations.
Revised: The batch size is 64, while the generator is trained for 50000 iterations, and the discriminator is trained for 250000 iterations.

Original: We use Adam with 1e-4 learning rate as an optimizer to train our model.
Revised: We use Adam with 1e-4 learning rate as an optimizer to train our model.

Original: Evaluation Metric:
Revised: Evaluation Metric:

Original: For the Dirac-GAN problem, the optimal solution is known to us so we can measure the convergence speed and draw the gradient map for different training algorithms.
Revised: For the Dirac-GAN problem, the optimal solution is known to us so we can measure the convergence speed and draw the gradient map for different training algorithms.

Original: For
Revised: For

Original: CIFAR 10, we use the FID score (Heusel et al., 2017b) and the inception score(Barratt & Sharma, 2018) to measure the quality of the generated images.
Revised: CIFAR 10, we use the FID score (Heusel et al., 2017b) and the inception score(Barratt & Sharma, 2018) to measure the quality of the generated images.

Original: We also compare the FID and inception scores across different timestamps to show the convergence speed.
Revised: We also compare the FID and inception scores across different timestamps to show the convergence speed.

Original: The FID score is used to measure the distance between our generated data distribution and the ground truth data distribution.

Original: The idea behind the FID score is simple.

Original: If our generated image has high ﬁdelity and diversity, then we feed this image into the inception net and the last layer should have the same distribution as a normal image.

Original: Thus, we take the 1024 dimension feature from the inception net and calculate Fr´echet distance as Eq.

Original: (19) between a generated distribution and the ground truth distribution:

Original: [Equation]

Original: [Equation]

Original: The Inception score is another performance measure for GAN.

Original: It reﬂects generated image quality by passing the images to a classiﬁer.

Original: For instance, if we want to generate a cat image and our generated image can be classiﬁed as a cat, we can regard this image as high ﬁdelity sample.

Original: We can calculate inception score (IS) following Eq.

Original: [Equation]

Original: 4. C ONVERGENCE OF D IRAC -B R GAN
Revised: 4. C ONVERGENCE OF D IRAC -B R GAN

Original: The gradient map and the convergence curve are presented in Figure 1.
Revised: The gradient map and the convergence curve are presented in Figure 1.

Original: These results show that
Revised: These results show that

Original: our
Revised: our

Original: Dirac-BrGANs have better convergence patterns and speed than Dirac-GANs.
Revised: Dirac-BrGANs have better convergence patterns and speed than Dirac-GANs.

Original: Without adding BMC to the training objective, Dirac-GANs cannot reach equilibrium.
Revised: Without adding BMC to the training objective, Dirac-GANs cannot reach equilibrium.

Original: The parameters of the generator and the discriminator are oscillating in a circle as shown in ﬁgure 1.
Revised: The parameters of the generator and the discriminator are oscillating in a circle as shown in ﬁgure 1.

Original: However, the parameters of the
Revised: However, the parameters of the

Original: Dirac-BrGANs only oscillate in the ﬁrst 500 iterations and soon converge in 800 iterations.
Revised: Dirac-BrGANs only oscillate in the ﬁrst 500 iterations and soon converge in 800 iterations.

Original: We also study different combinations of (cid:37) 1 and (cid:37) 2 under β = 1 and β = 2 .
Revised: We also study different combinations of (cid:37) 1 and (cid:37) 2 under β = 1 and β = 2 .

Original: As shown in tableand table 2, Dirac-BrGANs converge better when we set (cid:37) 1 = 0 .
Revised: As shown in tableand table 2, Dirac-BrGANs converge better when we set (cid:37) 1 = 0 .

Original: 1 and (cid:37) 2 = 0 .
Revised: 1 and (cid:37) 2 = 0 .

Original: Generally, a larger (cid:37) will lead to a faster convergence rate but when (cid:37) is large enough, the effect of increasing (cid:37) will become saturated.
Revised: Generally, a larger (cid:37) will lead to a faster convergence rate but when (cid:37) is large enough, the effect of increasing (cid:37) will become saturated.

Original: On the other hand, when (cid:37) is too small, Dirac-BrGANs will take more than 100000 iterations to converge.
Revised: On the other hand, when (cid:37) is too small, Dirac-BrGANs will take more than 100000 iterations to converge.

Original: [Table] [Table]
Revised: [Table] [Table]

Revised: = 0 .>>

Original: 4. S TABILIZED B R GAN S : C ONVERGE FASTER AND PERFORM BETTER
Revised: 4. S TABILIZED B R GAN S : C ONVERGE FASTER AND PERFORM BETTER

Original: We demonstrate that BrGANs converge fast and generate higher ﬁdelity image than Wasserstein
Revised: We demonstrate that BrGANs converge fast and generate higher ﬁdelity image than Wasserstein

Original: GANs (WGAN) (Arjovsky et al., 2017), WGAN with weight clipping (WGAN-CP), WGAN with gredient penlty (WGAN-GP) (Gulrajani et al., 2017), GAN with closed loop control (WGANCLC)
Revised: GANs (WGAN) (Arjovsky et al., 2017), WGAN with weight clipping (WGAN-CP), WGAN with gredient penlty (WGAN-GP) (Gulrajani et al., 2017), GAN with closed loop control (WGANCLC)

Original: (Xu et al., 2019) and their combinations.
Revised: (Xu et al., 2019) and their combinations.

Original: From the results in Table
Revised: From the results in Table

Original: [Table]
Revised: [Table]

Original: we can observe that our proposed WGAN-BR-GP achieves 22.10 FID and 5.42 inception score on CIFAR10 dataset, which is the best result among all other GANs.
Revised: we can observe that our proposed WGAN-BR-GP achieves 22.10 FID and 5.42 inception score on CIFAR10 dataset, which is the best result among all other GANs.

Original: Speciﬁcally, WGAN-BR out performs WGAN,
Revised: Speciﬁcally, WGAN-BR out performs WGAN,

Original: WGAN-BR-CP outperforms WGAN-CP and WGAN-CLC-CP, and WGAN-BR-GP outperforms
Revised: WGAN-BR-CP outperforms WGAN-CP and WGAN-CLC-CP, and WGAN-BR-GP outperforms

Original: WGAN-GP and WGAN-CLC-GP.
Revised: WGAN-GP and WGAN-CLC-GP.

Original: From Table 4, we can observe similar trends
Revised: From Table [Table] we can observe similar trends

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Convergence iterations of GANs is presented in Fig.
Revised: Convergence iterations of GANs is presented in Fig.

Original: 2 and Fig.
Revised: 2 and Fig.

Original: 3, measured by FID score and inception score, respectively.
Revised: 3, measured by FID score and inception score, respectively.

Original: It is readily observed that our proposed BrGANs achieves better FID and inception scores given the same training iteration.
Revised: It is readily observed that our proposed BrGANs achieves better FID and inception scores given the same training iteration.

Original: Xu et al.
Revised: Xu et al.

Original: (2019) add a L2 regularization (CLC) on the objective function on discriminator.
Revised: (2019) add a L2 regularization (CLC) on the objective function on discriminator.

Original: In our BrGANs, we incorporate both information from the generator and discriminator to our controller, so that the discriminator and generator to make sure the discriminator does not dominate the training process.
Revised: In our BrGANs, we incorporate both information from the generator and discriminator to our controller, so that the discriminator and generator to make sure the discriminator does not dominate the training process.

Original: Our BrGANs also compute faster than Xu et al.
Revised: Our BrGANs also compute faster than Xu et al.

Original: (2019) since we do not need to keep a buffer and update accordingly during training process.
Revised: (2019) since we do not need to keep a buffer and update accordingly during training process.

Original: 4. Q UALITATIVE RESULTS
Revised: 4. Q UALITATIVE RESULTS

Original: We provide qualitative results on CIFAR and CelebA datasets.
Revised: We provide qualitative results on CIFAR and CelebA datasets.

Original: In Fig.
Revised: In Fig.

Original: 5 and Fig.
Revised: 5 and Fig.

Original: 4, these images are generated by WGAN, WGAN-GP, WGAN-GP-BR, and WGAN-GP-CLC, from top left to bottom right respectively.
Revised: 4, these images are generated by WGAN, WGAN-GP, WGAN-GP-BR, and WGAN-GP-CLC, from top left to bottom right respectively.

Original: It can be observed that our WGAN-GP-BR can generate images with higher ﬁdelity and more reasonable in visual perception.
Revised: It can be observed that our WGAN-GP-BR can generate images with higher ﬁdelity and more reasonable in visual perception.

Original:  C ONCLUSION AND D ISCUSSION
Revised:  C ONCLUSION AND D ISCUSSION

Original: In this paper, we revisit GANs’ instability problem from the perspective of control theory.
Revised: In this paper, we revisit GANs’ instability problem from the perspective of control theory.

Original: Our work novelty incorporates a higher order non-linear controller and modify the objective function of the discriminator to stabilize GANs models.
Revised: Our work novelty incorporates a higher order non-linear controller and modify the objective function of the discriminator to stabilize GANs models.

Original: We innovatively design a universal noise-based control method called Brownian Motion Control (BMC) and propose BrGANs to achieve exponential stability.
Revised: We innovatively design a universal noise-based control method called Brownian Motion Control (BMC) and propose BrGANs to achieve exponential stability.

Original: Notably, our BMC is compatible with all GANs variations.
Revised: Notably, our BMC is compatible with all GANs variations.

Original: Experimental results demonstrate that our BrGANs converge faster and and perform better in terms of FID and inception scores on
Revised: Experimental results demonstrate that our BrGANs converge faster and and perform better in terms of FID and inception scores on

Original: CIFAR-10 and CelebA.
Revised: CIFAR-10 and CelebA.

Original: In our paper, theoretical analysis has been done under Dirac-GANs’ setting and we are able to stabilize both generator and discriminator simultaneously.
Revised: In our paper, theoretical analysis has been done under Dirac-GANs’ setting and we are able to stabilize both generator and discriminator simultaneously.

Original: However, under normal GANs’ settings, we only design BMC for the discriminator and stabilize the discriminator, then force the stability of the generator.
Revised: However, under normal GANs’ settings, we only design BMC for the discriminator and stabilize the discriminator, then force the stability of the generator.

Original: Additionally, our BMC is derived under continuous setting, but GANs’ training process is considered as discrete time steps.
Revised: Additionally, our BMC is derived under continuous setting, but GANs’ training process is considered as discrete time steps.

Original: To resolve these two problems, further work can be done on estimating the generator’s equilibrium at each time step and imposing a controller on both generators and discriminators simultaneously.
Revised: To resolve these two problems, further work can be done on estimating the generator’s equilibrium at each time step and imposing a controller on both generators and discriminators simultaneously.

Original: A PPENDIX A: P ROOF OF T HEOREM
Revised: A PPENDIX A: P ROOF OF T HEOREM

Original: Under Assumption 1, for any initial value X (0) = ξ ∈ R 2 , if (cid:37) 2 (cid:54) = 0 and β > 1 , then there a.s. exists a unique global solution X ( t ) to system (7) on t ∈ [0 , ∞ ) .
Revised: Under Assumption 1, for any initial value X (0) = ξ ∈ R 2 , if (cid:37) 2 (cid:54) = 0 and β > 1 , then there a.s. exists a unique global solution X ( t ) to system (7) on t ∈ [0 , ∞ ) .

Original: Proof.
Revised: Proof.

Original: Under Assumption 1, then, we can calculate that
Revised: Under Assumption then we can calculate that

Original: [Equation]
Revised: [Equation]

Original: For any bounded initial value X (0) ∈ R n , there exists a unique maximal local strong solution X ( t ) of system (7) on t ∈ [0 , τ e ) , where τ e is the explosion time.
Revised: For any bounded initial value X (0) ∈ R n , there exists a unique maximal local strong solution X ( t ) of system (7) on t ∈ [0 , τ e ) , where τ e is the explosion time.

Original: To show that the solution is actually global, we only need to prove that τ e = ∞
Revised: To show that the solution is actually global, we only need to prove that τ e = ∞

Original: a.s.
Revised: a.s.

Original: Let k 0 be a sufﬁciently large positive number such that | X (0)
Revised: Let k 0 be a sufﬁciently large positive number such that | X (0)

Original: | < k 0 .
Revised: | < k 0 .

Original: For each integer k ≥ k 0 , deﬁne the stopping time
Revised: For each integer k ≥ k 0 , deﬁne the stopping time

Original: [Equation]
Revised: [Equation]

Original: with the traditional setting inf ∅ = ∞ , where ∅ denotes the empty set.
Revised: with the traditional setting inf ∅ = ∞ , where ∅ denotes the empty set.

Original: Clearly, τ k is increasing as k → ∞ and τ k → τ ∞ ≤ τ e a.s.
Revised: Clearly, τ k is increasing as k → ∞ and τ k → τ ∞ ≤ τ e a.s.

Original: If we can show that τ ∞ = ∞ , then τ e = ∞ a.s., which implies the desired result.
Revised: If we can show that τ ∞ = ∞ , then τ e = ∞ a.s., which implies the desired result.

Original: This is also equivalent to prove that, for any t > 0 , P ( τ k ≤ t ) → 0 as k → ∞ .
Revised: This is also equivalent to prove that, for any t > 0 , P ( τ k ≤ t ) → 0 as k → ∞ .

Original: To prove this statement, for any p ∈ (0 , 1) , deﬁne a C 2 -function
Revised: To prove this statement, for any p ∈ (0 , 1) , deﬁne a C 2 -function

Original: [Equation]
Revised: [Equation]

Original: One can obtain that X ( t ) (cid:54) = 0 for all 0 ≤ t ≤ τ e a.s.
Revised: One can obtain that X ( t ) (cid:54) = 0 for all 0 ≤ t ≤ τ e a.s.

Original: Thus, one can apply the It ˆo formula to show that for any t ∈ [0 , τ e ) ,
Revised: Thus, one can apply the It ˆo formula to show that for any t ∈ [0 , τ e ) ,

Original: [Equation]
Revised: [Equation]

Original: where LV is deﬁned as
Revised: where LV is deﬁned as

Original: [Equation]
Revised: [Equation]

Revised: | X | p −

Revised: [Equation]

Revised: X

Revised: [Equation]

Revised: f ( X

Revised: [Equation]

Revised: t ))

Revised: + p ( p

Revised: [Equation]

Revised: [Equation]

Revised: 21| X | p + p ( p − 1) (cid:37) | X | 2 β + p

Original: By Assumption 1, we therefore have
Revised: By Assumption 1, we therefore have

Original: [Equation]
Revised: [Equation]

Original: Noting that p ∈ (0 , 1) and β > 1 and (cid:37) 2 (cid:54) = 0 , by the boundedness of polynomial functions, there exists a positive constant ¯ H such that LV ( x ) ≤ ¯ H .
Revised: Noting that p ∈ (0 , 1) and β > 1 and (cid:37) 2 (cid:54) = 0 , by the boundedness of polynomial functions, there exists a positive constant ¯ H such that LV ( x ) ≤ ¯ H .

Original: We therefore have
Revised: We therefore have

Original: [Equation]
Revised: [Equation]

Original: where ¯ H t is independent of k .
Revised: where ¯ H t is independent of k .

Original: By the deﬁnition of τ k , | X ( τ k ) | =
Revised: By the deﬁnition of τ k , | X ( τ k ) | =

Original: k , so
Revised: k , so

Revised: P

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which implies that
Revised: which implies that

Original: [Equation]
Revised: [Equation]

Original: as required.
Revised: as required.

Original: A PPENDIX B: P ROOF OF T HEOREM
Revised: A PPENDIX B: P ROOF OF T HEOREM

Original: Let Assumption 1 hold.
Revised: Let Assumption 1 hold.

Original: Assume that (cid:37) 2 (cid:54) = 0 and β > 1 .
Revised: Assume that (cid:37) 2 (cid:54) = 0 and β > 1 .

Original: If (cid:37)2 − ϕ > 0 , where
Revised: If [Equation] where

Original: [Equation]
Revised: [Equation]

Original: then for any X (0) =
Revised: then for any X (0) =

Original: ξ with sufﬁciently small constant (cid:15) ∈ (0 , (cid:37) 21 / 2 − ϕ ) , the global solution X ( t ) of system (7) has the property that
Revised: ξ with sufﬁciently small constant (cid:15) ∈ (0 , (cid:37) 21 / 2 − ϕ ) , the global solution X ( t ) of system (7) has the property that

Original: [Equation]
Revised: [Equation]

Original: that is, the solution of system (7) is a.s. exponentially stable.
Revised: that is, the solution of system (7) is a.s. exponentially stable.

Original: Proof.
Revised: Proof.

Original: Applying It ˆo formula to log | X ( t ) | yields
Revised: Applying It ˆo formula to log | X ( t ) | yields

Original: [Equation]
Revised: [Equation]

Original: Letting M ( t ) =
Revised: Letting M ( t ) =

Original: (cid:37) 2 (cid:82) t 

Revised: (cid:37) 2 (cid:82) t

Original: | X ( s ) | β d B 2 ( s ) , clearly M ( t ) is a continuous local martingale with the quadratic variation
Revised: | X ( s ) | β d B 2 ( s ) , clearly M ( t ) is a continuous local martingale with the quadratic variation

Original: [Equation]
Revised: [Equation]

Original: For any ε ∈ (0 , 1) , choose θ > 0 such that θε > 1 .
Revised: For any ε ∈ (0 , 1) , choose θ > 0 such that θε > 1 .

Original: Then for each integer m > 0 , the exponential martingale inequality gives
Revised: Then for each integer m > 0 , the exponential martingale inequality gives

Original: [Equation]
Revised: [Equation]

Original: Since (cid:80) ∞ m =1 m − θε < ∞ , by the well-known Borel-Cantelli lemma, there exists an ¯Ω 0 ⊆ Ω with
Revised: Since (cid:80) ∞ m =1 m − θε < ∞ , by the well-known Borel-Cantelli lemma, there exists an ¯Ω 0 ⊆ Ω with

Original: P (¯Ω 0 )
Revised: P (¯Ω 0 )

Original: = 1 such that for any ω ∈ ¯Ω 0 , there exists an integer ¯ m ( ω ) , when m > ¯ m ( ω ) , and m − 1 ≤ t ≤ m ,
Revised: = 1 such that for any ω ∈ ¯Ω 0 , there exists an integer ¯ m ( ω ) , when m > ¯ m ( ω ) , and m − 1 ≤ t ≤ m ,

Original: [Equation]
Revised: [Equation]

Original: This, together with Assumption 1, yields
Revised: This, together with Assumption 1, yields

Original: [Equation]
Revised: [Equation]

Original: Letting (cid:15) be sufﬁciently small, by the deﬁnition of ϕ in (9), for sufﬁciently small (cid:15) ∈ (0 , (cid:37) 2 1 / 2 − ϕ ) , we have
Revised: Letting (cid:15) be sufﬁciently small, by the deﬁnition of ϕ in (9), for sufﬁciently small (cid:15) ∈ (0 , (cid:37) 21 / 2 − ϕ ) , we have

Original: [Equation]
Revised: [Equation]

Original: Applying the strong law of large number, we therefore have
Revised: Applying the strong law of large number, we therefore have

Original: [Equation]
Revised: [Equation]