# Multiplicative score-based generative models for fluid dynamics

Merveille Talla, Valentin Resseguier, Robert Gruhlke, Dominique Heitz, Etienne Mémin

► To cite this version:

Merveille Talla, Valentin Resseguier, Robert Gruhlke, Dominique Heitz, Etienne Mémin. Multiplicative score-based generative models for fluid dynamics. CSE 2025 - SIAM Computational Science and Engineering, Mar 2025, Fort Worth, Texas, United States. hal-05035629

HAL Id: hal-05035629

https://hal.science/hal-05035629v1

Submitted on 15 Apr 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Multiplicative score-based generative models for fluid dynamics

Merveille TALLA

PhD Student, INRAE - Inria, France

<figure><img src="image_3.png" /></figure>

Valentin RESSEGUIER, INRAE France

Robert GRUHLKE, Freie Universität Berlin, Allemagne

Dominique HEITZ, INRAE France

Etienne MEMIN, Inria France

<figure><img src="image_4.png" /></figure>
# Data Assimilation and Generative models
# Data Assimilation and Generative models

* Data assimilation
# Data Assimilation and Generative models

* Data assimilation

<figure><img src="image_5.png" /></figure>

Carrassi, A. & Bocquet, Marc & Bertino, Laurent & Evensen, Geir. (2017). **Data Assimilation in the Geosciences - An overview on methods, issues and perspectives**. Wiley Interdisciplinary Reviews: Climate Change. 9. 10.1002/wcc.535.
# Data Assimilation and Generative models

* Data assimilation

<figure><img src="image_6.png" /></figure>

Carrassi, A. & Bocquet, Marc & Bertino, Laurent & Evensen, Geir. (2017). **Data Assimilation in the Geosciences - An overview on methods, issues and perspectives**. Wiley Interdisciplinary Reviews: Climate Change. 9. 10.1002/wcc.535.
# Data Assimilation and Generative models

* Data assimilation

<figure><img src="image_7.png" /></figure>

Carrassi, A. & Bocquet, Marc & Bertino, Laurent & Evensen, Geir. (2017). **Data Assimilation in the Geosciences - An overview on methods, issues and perspectives**. Wiley Interdisciplinary Reviews: Climate Change. 9. 10.1002/wcc.535.
# Data Assimilation and Generative models

* Data assimilation

<figure><img src="image_8.png" /></figure>

Carrassi, A. & Bocquet, Marc & Bertino, Laurent & Evensen, Geir. (2017). **Data Assimilation in the Geosciences - An overview on methods, issues and perspectives**. Wiley Interdisciplinary Reviews: Climate Change. 9. 10.1002/wcc.535.
# Data Assimilation and Generative models

* Data assimilation

<figure><img src="image_9.png" /></figure>

Carrassi, A. & Bocquet, Marc & Bertino, Laurent & Evensen, Geir. (2017). **Data Assimilation in the Geosciences - An overview on methods, issues and perspectives**. Wiley Interdisciplinary Reviews: Climate Change. 9. 10.1002/wcc.535.

* Generative Models-IA
# Data Assimilation and Generative models

* Data assimilation

<figure><img src="image_10.png" /></figure>

Carrassi, A. & Bocquet, Marc & Bertino, Laurent & Evensen, Geir. (2017). Data Assimilation in the Geosciences - An overview on methods, issues and perspectives. Wiley Interdisciplinary Reviews: Climate Change. 9. 10.1002/wcc.535.

* Generative Models-IA

<figure><img src="image_11.png" /></figure>
# Data Assimilation and Generative models

* Data assimilation

<figure><img src="image_12.png" /></figure>

Carrassi, A. & Bocquet, Marc & Bertino, Laurent & Evensen, Geir. (2017). **Data Assimilation in the Geosciences - An overview on methods, issues and perspectives**. Wiley Interdisciplinary Reviews: Climate Change. 9. 10.1002/wcc.535.

* Generative Models-IA

<figure><img src="image_13.png" /></figure>

Generate complex forecast distribution of the
physical states from training data
# Data Assimilation and Generative models

* Data assimilation

<figure><img src="image_14.png" /></figure>

Carrassi, A. & Bocquet, Marc & Bertino, Laurent & Evensen, Geir. (2017). **Data Assimilation in the Geosciences - An overview on methods, issues and perspectives**. Wiley Interdisciplinary Reviews: Climate Change. 9. 10.1002/wcc.535.

* Generative Models-IA

<figure><img src="image_15.png" /></figure>
# Data Assimilation and Generative models

## Data assimilation

<figure><img src="image_16.png" /></figure>

Carrassi, A. & Bocquet, Marc & Bertino, Laurent & Evensen, Geir. (2017). **Data Assimilation in the Geosciences - An overview on methods, issues and perspectives**. Wiley Interdisciplinary Reviews: Climate Change. 9. 10.1002/wcc.535.

## Generative Models-IA

<figure><img src="image_17.png" /></figure>

Finn, T. S., Disson, L., Farchi, A., Bocquet, M., & Durand, C. (2024). Representation learning with unconditional denoising diffusion models for dynamical systems. Nonlinear Processes in Geophysics, 31(3), 409–431. https://doi.org/10.5194/npg-31-409-2024
# Data Assimilation and Generative models

* Data assimilation

<figure><img src="image_18.png" /></figure>

Carrassi, A. & Bocquet, Marc & Bertino, Laurent & Evensen, Geir. (2017). **Data Assimilation in the Geosciences - An overview on methods, issues and perspectives**. Wiley Interdisciplinary Reviews: Climate Change. 9. 10.1002/wcc.535.

* Generative Models-IA

<figure><img src="image_19.png" /></figure>
# Outline of the talk

1. Score-Based Generative Models – Additive Noise

2. Score-Based Generative Models – Multiplicative Noise

3. Results

4. Conclusion and perspectives
# 1. Score-Based Generative Models – Additive Noise

**Goal:** learn and sample the probability distribution of a given set of observations $x$ from an easy to sample distribution
# Map Gaussian white noise from data
Map Gaussian white noise from data

<figure><img src="image_20.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970
# Map Gaussian white noise from data

* Generalization through SDE: (Guth, F et al. 2022)

<figure><img src="image_21.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970
# Map Gaussian white noise from data

* Generalization through SDE: (Guth, F et al. 2022)

$$d\vec{x}_t = -\vec{x}_t dt + \sqrt{2} d\vec{B}_t$$

Where $\vec{B}_t$ is the standard wiener process.

<figure><img src="image_22.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970
# Map Gaussian white noise from data

* Generalization through SDE: (Guth, F et al. 2022)

$$d\vec{x}_t = -\vec{x}_t dt + \sqrt{2} d\vec{B}_t$$

Where $\vec{B}_t$ is the standard wiener process.

The solution is an **Ornstein-Uhlenbeck** process :
for any $t \ge 0$,

$$\vec{x}_t = e^{-t}x_0 + (1 + e^{-2t})\frac{1}{2}z \\ z \sim \mathcal{N}(0, I)$$

<figure><img src="image_23.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970

<figure><img src="image_24.png" /></figure>

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456
Map the distribution of data from the Gaussian white noise

<figure><img src="image_25.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970

<figure><img src="image_26.png" /></figure>

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021).
*Score-Based Generative Modeling through Stochastic Differential Equations*.
arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456
# Map the distribution of data from the Gaussian white noise

* Generalisation through Time-reversed SDE: (Song, Y et al. 2021)

<figure><img src="image_27.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970

<figure><img src="image_28.png" /></figure>

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021).
*Score-Based Generative Modeling through Stochastic Differential Equations*.
arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456
# Map the distribution of data from the Gaussian white noise

* Generalisation through Time-reversed SDE: (Song, Y et al. 2021)

$$d\tilde{x}_{T-t} = \{\tilde{x}_{T-t} + 2\nabla\log p_{T-t}(\tilde{x}_{T-t})\}dt + \sqrt{2}d\tilde{B}_t$$

Where:

* $\tilde{B}_t$ is a standard wiener process

* $p_t$ is the probability density of $x_t$

* Starting sample from $x_T$ to obtain $x_0$

<figure><img src="image_29.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970

<figure><img src="image_30.png" /></figure>

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456
# Map the distribution of data from the Gaussian white noise

* Generalisation through Time-reversed SDE: (Song, Y et al. 2021)

$$d\tilde{x}_{T-t} = \{\tilde{x}_{T-t} + 2\nabla\log p_{T-t}(\tilde{x}_{T-t})\}dt + \sqrt{2}d\tilde{B}_t$$

Stein Score

Where:

* $\tilde{B}_t$ is a standard wiener process

* $p_t$ is the probability density of $x_t$

* Starting sample from $x_T$ to obtain $x_0$

<figure><img src="image_31.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970

<figure><img src="image_32.png" /></figure>

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456
# Score-Based Generative Models modeling

* Generalisation through Time-reversed SDE:

$$d\tilde{x}_{T-t} = \{\tilde{x}_{T-t} + 2s_{\theta^*}(\tilde{x}_{T-t}, T-t)\}dt + \sqrt{2}d\tilde{B}_t$$

Where:

<figure><img src="image_33.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970

<figure><img src="image_34.png" /></figure>

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456
# Score-Based Generative Models modeling

* Generalisation through Time-reversed SDE:

$$d\tilde{x}_{T-t} = \{\tilde{x}_{T-t} + 2s_{\theta^*}(\tilde{x}_{T-t}, T-t)\}dt + \sqrt{2}d\tilde{B}_t$$
Score-based model

Where:

<figure><img src="image_35.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970

<figure><img src="image_36.png" /></figure>

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456
# Score-Based Generative Models modeling

* Generalisation through Time-reversed SDE:

$$d\tilde{x}_{T-t} = \{\tilde{x}_{T-t} + 2s_{\theta^*}(\tilde{x}_{T-t}, T-t)\}dt + \sqrt{2}d\tilde{B}_t$$
Score-based model

Where:

$$\theta^* = \operatorname*{argmin} \frac{1}{2} \int_{0}^{T} \mathbb{E}_{\tilde{x}_t} \left[ \| s_{\theta}(\tilde{x}_t, t) - \nabla_{\tilde{x}_t} \log p(\tilde{x}_t | \tilde{x}_0) \|^2 \right] dt$$

<figure><img src="image_37.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970

<figure><img src="image_38.png" /></figure>

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456
# Score-Based Generative Models modeling

* Generalisation through Time-reversed SDE:

$$d\tilde{x}_{T-t} = \{\tilde{x}_{T-t} + 2s_{\theta^*}(\tilde{x}_{T-t}, T-t)\}dt + \sqrt{2}d\tilde{B}_t$$
Score-based model

Where:

$$\theta^* = \underset{T}{\operatorname{argmin}} \frac{1}{2} \int_{0}^{T} \mathbb{E}_{\tilde{x}_t} \left[ \left\| s_{\theta}(\tilde{x}_t, t) - \nabla_{\tilde{x}_t} \log p(\tilde{x}_t | \tilde{x}_0) \right\|^2 \right] dt$$

Denoising score matching Loss (Vincent et al. 2011)

<figure><img src="image_39.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970

<figure><img src="image_40.png" /></figure>

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456
# Score-Based Generative Models modeling

* Generalisation through Time-reversed SDE:

$$d\tilde{x}_{T-t} = \{\tilde{x}_{T-t} + 2s_{\theta^*}(\tilde{x}_{T-t}, T-t)\}dt + \sqrt{2}d\tilde{B}_t$$
Score-based model

Where:

$$\theta^* = \underset{T}{\operatorname{argmin}} \frac{1}{2} \int_{0}^{T} \mathbb{E}_{\tilde{x}_t} \left[ \left\| s_{\theta}(\tilde{x}_t, t) - \nabla_{\tilde{x}_t} \log p(\tilde{x}_t | \tilde{x}_0) \right\|^2 \right] dt$$

Denoising score matching Loss (Vincent et al. 2011)

<figure><img src="image_41.png" /></figure>

Luo, C. (2022). Understanding Diffusion Models: A Unified Perspective. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970

<figure><img src="image_42.png" /></figure>

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456

**Others score matching loss:**
Explicit score matching Loss (Song et al. 2021)
Implicit score matching Loss (Hyvärinen & Dayan, 2005)
Sliced score matching Loss (Song, Y et al., 2020)
SGM-super resolution (Talla, M et al. 2025+)

Low resolution image

Generate image (super resolution)

Test Image (super resolution)
# SGM-super resolution (Talla, M et al. 2025+)

<figure><img src="image_43.png" /><figcaption>Generate image (super resolution)</figcaption></figure>

<figure><img src="image_44.png" /><figcaption>Test Image (super resolution)</figcaption></figure>
# SGM-super resolution (Talla, M et al. 2025+)

<figure><img src="image_45.png" /><figcaption>Low resolution image</figcaption></figure>

<figure><img src="image_46.png" /><figcaption>Generate image (super resolution)</figcaption></figure>

<figure><img src="image_47.png" /></figure>

<figure><img src="image_48.png" /><figcaption>Test Image (super resolution)</figcaption></figure>
# SGM-super resolution (Talla, M et al. 2025+)

<figure><img src="image_49.png" /><figcaption>## Generate image (super resolution)</figcaption></figure>

<figure><img src="image_50.png" /><figcaption>## Test Image (super resolution)</figcaption></figure>

<figure><img src="image_51.png" /></figure>
## 2. Score-Based Generative models – Multiplicative Noise

* Motivations

* Forward diffusion

* Sampling the latent space

* Backward diffusion
# Motivations
Motivations

$$
dx_t = -Ax_t dt + g(t) dB_t
$$

$$
dx_t = -Ax_t dt + G(x_t) dB_t
$$
# Motivations

$$dx_t = -Ax_t dt + g(t) dB_t$$

<figure><img src="image_52.png" /></figure>

$$dx_t = -Ax_t dt + G(x_t) dB_t$$
# Motivations

$$dx_t = -Ax_t dt + g(t) dB_t$$

<figure><img src="image_53.png" /></figure>

$$dx_t = -Ax_t dt + G(x_t) dB_t$$
# Motivations

$$dx_t = -Ax_t dt + g(t) dB_t$$

<figure><img src="image_54.png" /></figure>

$$dx_t = -Ax_t dt + G(x_t) dB_t$$

## ☐Limits of additive noise:

* With Gaussian latent space, it may be difficult to represent extreme event (Lafon et al. 2023)
# Motivations

$$dx_t = -Ax_t dt + g(t) dB_t$$

<figure><img src="image_55.png" /></figure>

$$dx_t = -Ax_t dt + G(x_t) dB_t$$

## ☐Limits of additive noise:

* With Gaussian latent space, it may be difficult to represent extreme event (Lafon et al. 2023)

## ☐Advantages of multiplicative noise:

* More faithful representation of physical phenomena (LU (Memin, E 2014), SALT (Holm, D 2015), Kraichnan (1968), MTV (Majda, A et al. 1999), MQG (Sapsis, P et al. 2013))

* Better preserves local relationships and correlated structures in the data (energy conservation)
Forward diffusion
# Forward diffusion

Our forward SDE (Stratonovich form):

$$d\vec{X}_t = \underbrace{G(\vec{X}_t) \circ d\vec{B}_t}_{\text{linear in } \vec{X}_t}$$
# Forward diffusion

Our forward SDE (Stratonovich form):

$$d\vec{X}_t = \underbrace{G(\vec{X}_t) \circ d\vec{B}_t}_{\text{linear in } \vec{X}_t}$$

$$d\vec{X}_t = \frac{1}{2}(\nabla \cdot \Sigma)(\vec{X}_t, t)dt + G(\vec{X}_t)d\vec{B}_t$$

Itô form
# Forward diffusion

Our forward SDE (Stratonovich form):

$$d\vec{X}_t = \underbrace{G(\vec{X}_t) \circ d\vec{B}_t}_{\text{linear in } \vec{X}_t}$$

Rewrite as:

$$d(\vec{X}_t)_i = \sum_{j=1}^{d} \left( \sum_{k=1}^{d} G_{ij}^k \circ d\vec{B}_t^k \right) (\vec{X}_t)_j$$
# Forward diffusion

Our forward SDE (Stratonovich form):

$$d\vec{X}_t = \underbrace{G(\vec{X}_t) \circ d\vec{B}_t}_{\text{linear in } \vec{X}_t}$$

□ Distribution In the latent space:

Rewrite as:

$$d(\vec{X}_t)_i = \sum_{j=1}^{d} \left( \sum_{k=1}^{d} G_{ij}^k \circ d\vec{B}_t^k \right) (\vec{X}_t)_j$$
<figure><img src="image_56.png" /></figure>

Sampling the latent space

<figure><img src="image_57.png" /></figure>
Sampling the latent space

**Corollary** (Gruhlke, Resseguier, Talla 2025 +)

• $\vec{X}_T^n = \frac{\vec{X}_T}{\|\vec{X}_T\|} \xrightarrow{\mathcal{L}} U^n \sim U(\mathbb{S}^{d-1})$ when $T \to +\infty$

• $\|\vec{X}_T\| = \|\vec{X}_0\|$

• $\|\vec{X}_T\|$ and $\vec{X}_T^n$ are asymptotically independent for $T \to +\infty$

<figure><img src="image_58.png" /></figure>
# Sampling the latent space

**Corollary** (Gruhlke, Resseguier, Talla 2025 +)

* $\vec{X}_T^n = \frac{\vec{X}_T}{\|\vec{X}_T\|} \xrightarrow{\mathcal{L}} U^n \sim U(\mathbb{S}^{d-1})$ when $T \to +\infty$

* $\|\vec{X}_T\| = \|\vec{X}_0\|$

* $\|\vec{X}_T\|$ and $\vec{X}_T^n$ are asymptotically independent for $T \to +\infty$

In latent space, given a parametric distribution of the **one-dimensional variable** $\|\vec{X}_0\|$, we sample $\vec{X}_T$ as:

<figure><img src="image_59.png" /></figure>
# Backward diffusion

The reverse-time SDE is given by :

$$d\tilde{X}_{T-t} = \left\{ -G(\tilde{X}_{T-t})G(\tilde{X}_{T-t})^T \nabla_{\tilde{X}_{T-t}} \log p_{T-t}(\tilde{X}_{T-t}) \right\} dt + G(\tilde{X}_{T-t}) \circ d\tilde{B}_t$$

in Stratonovich form (Song, Y et al. 2021)
# Backward diffusion

The reverse-time SDE is given by :

$$d\tilde{X}_t = \{-G(\tilde{X}_t)G(\tilde{X}_t)^{\top}s_{\theta}(\tilde{X}_t, t)\}dt + G(\tilde{X}_t) \circ d\tilde{B}_t$$

in Stratonovich form

**Theorem: Sliced score matching Loss** (Gruhlke, Resseguier, Talla 2025 +)

$$\mathcal{L}_{SSM} = \int_{0}^{T} \mathbb{E}_{v} \mathbb{E}_{\tilde{X}_{t}} \left[ \frac{1}{2} \left\| s_{\theta}(\tilde{X}_{t}, t) \right\|_{GG^{\top}}^{2} + v^{\top} \nabla_{\tilde{X}_{t}} \left( G(\tilde{X}_{t}, t) G(\tilde{X}_{t}, t)^{\top} s_{\theta}(\tilde{X}_{t}, t) \right) v \right] dt$$
To summarise score-based generative models with
multiplicative noise
3. Results
# Generating swissroll (d=2)

SGM – additive noise

SGM – multiplicative noise

## Parameters:

- 10000 learning iterations

- 256 batch size

- 10 steps backward diffusion

"A Variational Perspective on Diffusion-Based Generative Models and Score Matching" by Chin-Wei Huang, Jae Hyun Lim and Aaron Courville [arXiv]
# Generating swissroll (d=2)

## SGM – additive noise

## SGM – multiplicative noise

### Parameters:

- 10000 learning iterations

- 256 batch size

- 10 steps backward diffusion

<figure><img src="image_60.png" /></figure>

<figure><img src="image_61.png" /></figure>

<figure><img src="image_62.png" /></figure>

Density

0.25
0.00

<figure><img src="image_63.png" /></figure>

<figure><img src="image_64.png" /></figure>

<figure><img src="image_65.png" /></figure>

<figure><img src="image_66.png" /></figure>

Density

0.25
0.00

<figure><img src="image_67.png" /></figure>

"A Variational Perspective on Diffusion-Based Generative Models and Score Matching" by Chin-Wei Huang, Jae Hyun Lim and Aaron Courville [arXiv]
# Generating swissroll (d=2)

## SGM – additive noise

### Parameters:

* 10000 learning iterations

* 256 batch size

* 10 steps backward diffusion

<figure><img src="image_68.png" /></figure>

<figure><img src="image_69.png" /></figure>

## SGM – multiplicative noise

<figure><img src="image_70.png" /></figure>

<figure><img src="image_71.png" /></figure>

"A Variational Perspective on Diffusion-Based Generative Models and Score Matching" by Chin-Wei Huang, Jae Hyun Lim and Aaron Courville [arXiv]
# Generating swissroll (d=2)

## SGM – additive noise

## SGM – multiplicative noise

### Parameters:

* 10000 learning iterations

* 256 batch size

* 10 steps backward diffusion

<figure><img src="image_72.png" /></figure>

<figure><img src="image_73.png" /></figure>
# Generating swissroll (d=2)

## SGM – additive noise

### Parameters:

* 10000 learning iterations

* 256 batch size

* 10 steps backward diffusion

<figure><img src="image_74.png" /></figure>

## SGM – multiplicative noise

<figure><img src="image_75.png" /></figure>

### Parameters:

* 10000 learning iterations

* 256 batch size

* 100 steps backward diffusion

"A Variational Perspective on Diffusion-Based Generative Models and Score Matching" by Chin-Wei Huang, Jae Hyun Lim and Aaron Courville [arXiv]
# Generating swissroll (d=2)

## SGM – additive noise

### Parameters:

* 10000 learning iterations

* 256 batch size

* 10 steps backward diffusion

<figure><img src="image_76.png" /></figure>

## SGM – multiplicative noise

<figure><img src="image_77.png" /></figure>

### Parameters:

* 10000 learning iterations

* 256 batch size

* 100 steps backward diffusion

<figure><img src="image_78.png" /></figure>

"A Variational Perspective on Diffusion-Based Generative Models and Score Matching" by Chin-Wei Huang, Jae Hyun Lim and Aaron Courville [arXiv]
# Generating swissroll (d=2)

## SGM – additive noise

### Parameters:

* 10000 learning iterations

* 256 batch size

* 10 steps backward diffusion

<figure><img src="image_79.png" /></figure>

## SGM – multiplicative noise

<figure><img src="image_80.png" /></figure>

### Parameters:

* 10000 learning iterations

* 256 batch size

* 100 steps backward diffusion

<figure><img src="image_81.png" /></figure>

* Less steps needed for the backward diffusion

"A Variational Perspective on Diffusion-Based Generative Models and Score Matching" by Chin-Wei Huang, Jae Hyun Lim and Aaron Courville [arXiv]
# Generation from real high-dimensional dataset from weather station (d=30)

## SGM – additive noise

<figure><img src="image_82.png" /></figure>

## Parameters:

- 10000 learning iterations

- 256 batch size

- 100 steps backward diffusion

## SGM – multiplicative noise

<figure><img src="image_83.png" /></figure>
# Generation from real high-dimensional dataset from weather station (d=30)

## SGM – additive noise

<figure><img src="image_84.png" /></figure>

## SGM – multiplicative noise

<figure><img src="image_85.png" /></figure>

## Parameters:

- 10000 learning iterations

- 256 batch size

- 100 steps backward diffusion
Generation from real high-dimensional
dataset from weather station (d=30)

SGM – additive noise

<figure><img src="image_86.png" /></figure>

Parameters:

- 10000 learning iterations

- 256 batch size

- 100 steps backward
diffusion

SGM – multiplicative noise

<figure><img src="image_87.png" /></figure>
4. Conclusion and perspectives
Conclusion: about multiplicative score-based generative model

* $d\vec{X}_t = G(\vec{X}_t) \circ d\vec{B}_t$ with **skew symmetric** noise

* The distribution in latent space is in general non gaussian but parametrized by a **one-dimensional distribution** and **easy to sample**

* Score function is learned by sliced score matching

* Less steps needed for the backward diffusion

* More accurate estimation of fat-tail distributions (real high-dimensional dataset from weather station)

* Preserves Physical structure of the state (Conservation law, etc...)

<figure><img src="image_88.png" /></figure>
Conclusion: about multiplicative score-based generative model

* $d\vec{X}_t = G(\vec{X}_t) \circ d\vec{B}_t$ with **skew symmetric** noise

* The distribution in latent space is in general non gaussian but parametrized by a **one-dimensional distribution** and **easy to sample**

* Score function is learned by sliced score matching

* Less steps needed for the backward diffusion

* More accurate estimation of fat-tail distributions (real high-dimensional dataset from weather station)

* Preserves Physical structure of the state (Conservation law, etc...)

Perspectives:

* Learn and sample the model on high dimension with around $d = O(10^6)$ degrees of freedom

* Learn and sample the model for dynamic system

* Data assimilation

* Faster time integration of the forward SDE

<figure><img src="image_89.png" /></figure>
Conclusion: about multiplicative score-based generative model

• $d\vec{X}_t = G(\vec{X}_t) \circ d\vec{B}_t$ with **skew symmetric** noise

• The distribution in latent space is in general non gaussian
but parametrized by a **one-dimensional distribution** and **easy to sample**

• Score function is learned by sliced score matching

• Less steps needed for the backward diffusion

• More accurate estimation of fat-tail distributions
(real high-dimensional dataset from weather station)

• Preserves Physical structure of the state (Conservation law, etc...)

Perspectives:

• Learn and sample the model on high dimension with around $d = O(10^6)$ degrees of freedom

• Learn and sample the model for dynamic system

• Data assimilation

• Faster time integration of the forward SDE

<figure><img src="image_90.png" /></figure>
References

1. Carrassi, A. & Bocquet, Marc & Bertino, Laurent & Evensen, Geir. (2017). Data Assimilation in the Geosciences - An overview on methods, issues and perspectives. Wiley Interdisciplinary Reviews: Climate Change. 9. 10.1002/wcc.535.

2. Finn, T. S., Disson, L., Farchi, A., Bocquet, M., & Durand, C. (2024). *Representation learning with unconditional denoising diffusion models for dynamical systems*. Nonlinear Processes in Geophysics, 31(3), 409–431. https://doi.org/10.5194/npg-31-409-2024

3. Guth, F., Coste, S., De Bortoli, V., & Mallat, S. (2022). *Wavelet Score-Based Generative Modeling*. arXiv. https://arxiv.org/abs/2208.05003

4. Huang, C.-W., Lim, J. H., & Courville, A. (2021). *A Variational Perspective on Diffusion-Based Generative Models and Score Matching*. arXiv preprint arXiv:2106.02808. https://arxiv.org/abs/2106.02808

5. Luo, C. (2022). *Understanding Diffusion Models: A Unified Perspective*. arXiv preprint arXiv:2208.11970. https://arxiv.org/abs/2208.11970

6. Majda, A. J., Timofeyev, I., & Vanden Eijnden, E. (1999). Models for stochastic climate prediction. *Proceedings of the National Academy of Sciences*, 96(26), 14687–14691. https://doi.org/10.1073/pnas.96.26.14687

7. Mémin, E. (2014). Fluid flow dynamics under location uncertainty. *Geophysical & Astrophysical Fluid Dynamics*, 108(2), 119–146. https://doi.org/10.1080/03091929.2013.836832

8. Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). *Score-Based Generative Modeling through Stochastic Differential Equations*. arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456

9. Resseguier, V., Picard, A. M., Memin, E., Chapron, B. (2021). Quantifying truncation-related uncertainties in unsteady fluid dynamics reduced order models. *SIAM/ASA Journal on Uncertainty Quantification*, 9(3), 1152–1183. https://doi.org/10.1137/19M1354819

10. Gruhlke, R., Resseguier, V., Talla, M. (2025+). Multiplicative Score-based Generative Model. To be submitted in 2025

11. Talla, M., Resseguier, V., Heitz, D., Memin, E. (2025+). Super resolution with score-based generative model (with additive noise). To be submitted in 2025

12. Sapsis, T. P., & Majda, A. J. (2013). A statistically accurate modified quasilinear Gaussian closure for uncertainty quantification in turbulent dynamical systems. *Physica D: Nonlinear Phenomena*, 252, 34–45. https://doi.org/10.1016/j.physd.2013.02.009
# Estimation of the Stein score $\nabla \log p(\vec{x}_t)$

Generate distribution of data from the prior distribution is possible if:

* We can discretize the SDE without introducing large errors

* We can estimate an accurate approximation of the stein score at each time t.

The score of the distribution can be estimated by training a time-dependent score - based model $s_{\theta}(x, t)$ which minimize these following score matching losses:

<figure><img src="image_91.png" /></figure>

Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. arXiv preprint arXiv:2011.13456. https://arxiv.org/abs/2011.13456

**Explicit score matching Loss :** Song et al. (2021)

$$ \mathcal{L}_{ESM} = \int_{0}^{T} \mathbb{E}_{\vec{x}_t} \left[ \frac{1}{2} \|s_{\theta}(\vec{x}_t, t) - \nabla_{\vec{x}_t} \log p(\vec{x}_t)\|^2 \right] dt $$

**Denoising score matching Loss :** Vincent, 2011

$$ \mathcal{L}_{DSM} = \frac{1}{2} \int_{0}^{T} \mathbb{E}_{\vec{x}_t} \left[ \|s_{\theta}(\vec{x}_t, t) - \nabla_{\vec{x}_t} \log p(\vec{x}_t | \vec{x}_0)\|^2 \right] dt $$

**Implicit score matching Loss :** Hyvärinen & Dayan, 2005

$$ \mathcal{L}_{ISM} = \int_{0}^{T} \mathbb{E}_{\vec{x}_t} \left[ \frac{1}{2} \|s_{\theta}(\vec{x}_t, t)\|^2 + \nabla_{\vec{x}_t} \cdot (s_{\theta}(\vec{x}_t, t)) \right] dt $$

**Sliced score matching Loss :** Song et al., 2020

$$ \mathcal{L}_{SSM} = \int_{0}^{T} \mathbb{E}_{v} \mathbb{E}_{\vec{x}_t} \left[ \frac{1}{2} \|s_{\theta}(\vec{x}_t, t)\|^2 + v^{\top} \nabla_{\vec{x}_t} (s_{\theta}(\vec{x}_t, t) \cdot v) \right] dt $$

$$ \mathcal{L}_{ESM} - \frac{1}{2} \mathbb{E}_{\vec{x}_t} \left[ \| \nabla_{\vec{x}_t} \log p(\vec{x}_t) \|^2 \right] = \mathcal{L}_{ISM} = \mathcal{L}_{SSM} = \mathcal{L}_{DSM} - \frac{1}{2} \mathbb{E}_{\vec{x}_0} \left[ \| \nabla_{\vec{x}_t} \log p(\vec{x}_t | \vec{x}_0) \|^2 \right] $$

Huang, C.-W., Lim, J. H., & Courville, A. (2021). A Variational Perspective on Diffusion-Based Generative Models and Score Matching. arXiv preprint arXiv:2106.02808. https://arxiv.org/abs/2106.02808
<figure><img src="image_92.png" /></figure>

## ☐ Sampling the latent space

Under the previous constraints:

* $\vec{X}_T^n = \frac{\vec{X}_T}{\|\vec{X}_T\|} \xrightarrow{\mathcal{L}} U^n \sim U(\mathcal{S}^{d-1})$ when $T \to +\infty$

* $\|\vec{X}_T\|$ and $\vec{X}_T^n$ are asymptotically independent for $T \to +\infty$

$$ \tilde{F}^{-1}: \mathbb{R}^d \to \mathbb{R}^d \\ \tilde{X}_T^N \to (\tilde{F}_{\cdot|\cdot}^{-1} \circ F_{\chi^2}) (\|\tilde{X}_T^N\|^2) \frac{\tilde{X}_T^N}{\|\tilde{X}_T^N\|} $$

* $F_{\chi^2}$ is the cdf of the $\chi^2$ distribution with $d-1$ degrees of freedom

* $\tilde{F}_{\cdot|\cdot}: \mathbb{R}^{d+1} \to [0,1] \\ r \to \frac{1}{N} \sum_{i=1}^{N} \mathbb{1}_{\{r \ge \|\tilde{X}_T^{(i)}\|\}}$ the empirical cumulative distribution function (eCDf)

$$ \tilde{X}_T^N \sim \mathcal{N}(0, Id), \quad \tilde{X}_T = \tilde{F}^{-1}(\tilde{X}_T) $$

## ☐ Backward diffusion :

The reverse-time Itô SDE is given by :

$$ d\tilde{x}_t = \left\{ -\frac{1}{2} \nabla \cdot [G(\tilde{x}_t, t)G(\tilde{x}_t, t)^T] - G(\tilde{x}_t, t) G(\tilde{x}_t, t)^T \nabla_{\tilde{x}_t} \log p_t(\tilde{x}_t) \right\} dt + G(\tilde{x}_t, t) d\tilde{B}_t $$

In Itô form and

$$ d\tilde{x}_t = \left\{ -G(\tilde{x}_t, t) G(\tilde{x}_t, t)^T \nabla_{\tilde{x}_t} \log p_t(\tilde{x}_t) \right\} dt + G(\tilde{x}_t, t) \circ d\tilde{B}_t $$

In Stratonovich form

## ☐ Score matching:

One can work on

$$ a_{\theta}(\tilde{X}_t, t) = G(\tilde{x}_t, t)^{\top} s_{\theta}(\tilde{x}_t, t) $$

With minimise the scores matching loss by replacing $\|\cdot\|$ with $\|\cdot\|_{GG^T}$

## Sliced score matching Loss : Song et al., 2020

$$ \mathcal{L}_{SSM} = \int_{0}^{T} \mathbb{E}_{v} \mathbb{E}_{\tilde{x}_{t}} \left[ \frac{1}{2} \|s_{\theta}(\tilde{x}_{t}, t)\|_{GG^{T}}^{2} + v^{\top} \nabla_{\tilde{x}_{t}} (G(\tilde{x}_{t}, t) G(\tilde{x}_{t}, t)^{\top} s_{\theta}(\tilde{x}_{t}, t)) v \right] dt \\ = \int_{0}^{T} \mathbb{E}_{v} \mathbb{E}_{\tilde{x}_{t}} \left[ \frac{1}{2} \|a_{\theta}(\tilde{x}_{t}, t)\|_{G^{T}}^{2} + v^{\top} \nabla_{\tilde{x}_{t}} (G(\tilde{x}_{t}, t) a_{\theta}(\tilde{x}_{t}, t)) v \right] dt $$