<figure><img src="image_1.png" /></figure>

HAL
open science

# When to Learn: Conformal Scores as Online Update Criteria

Ben Gao, Jordan Patracone, Stéphane Chrétien, Olivier Alata

► To cite this version:

Ben Gao, Jordan Patracone, Stéphane Chrétien, Olivier Alata. When to Learn: Conformal Scores as Online Update Criteria. Workshop on Predictions and Uncertainty at COLT 2025, Jun 2025, Lyon, France. hal-05224472

HAL Id: hal-05224472

https://inria.hal.science/hal-05224472v1

Submitted on 26 Aug 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

## Key Elements

We introduce a lightweight and novel framework designed for adaptive model updates in streaming environments: **Conformal Online Learning (COL)**, inspired by conformal prediction but repurposed for learning. We demonstrate the effectiveness of COL through its application to online learning of Koopman linear embeddings for non-linear dynamical systems

### Koopman Operator [1]

Instead of directly modeling the nonlinear dynamics $x_{t+1} = T x_t$, Koopman operator theory considers the evolution of lifted states through a linear operator:

$$ \Phi_{\theta}(x_{t+1}) \approx K \Phi_{\theta}(x_t). $$

This approach enables the use of linear spectral analysis tools.

### Conformal Online Learning of Koopman embeddings

The goal is to incrementally $\Phi_{\theta}$ and $K$ from sequentially observed dynamics. At each time step $t$, the parameters $(\theta_t, K_t)$ are updated by minimizing the multi-step prediction loss within $\mathcal{D}_t = \{x_{t-w}, \dots, x_t\}$. The score function at time step $t$ is defined as

$$ s_t(x_t, (\theta_t, K_t)) := \sum_{\tau=1}^{w} \|\Phi_{\theta_t}(x_t) - K_t^{\tau} \Phi_{\theta_t}(x_{t-\tau})\|^2 $$

to evaluate the consistency of $(\theta_t, K_t)$.

### Prediction Score Set

Given a newly observed state $x_t$ and a conformity threshold $q_t > 0$, the prediction score set at time $t$ is defined as

$$ S_t = s_t(x_t, \text{Param}_t), $$

where $\text{Param}_t = \{(\theta, K) \text{ such that } s = s_t(x_t, (\theta, K)) \le q_t\}$. This set contains all prediction scores attainable at $x_t$ by Koopman models that satisfy the current calibration constraint.

### Dynamic threshold by Conformal PI Control [2]

Rather than fixing the quantile threshold $q_t$ in advance, the method updates it online in response to conformity violations. After observing whether $s_t \in S_t$, a binary error signal $e_t = \mathbf{1}\{s_t \notin S_t\}$ is computed, and the threshold $q_t$ is adjusted via:

$$ q_{t+1} = q_t + \underbrace{\gamma(e_t - \alpha)}_{\text{Proportional term (P)}} + r_t \underbrace{\left(\sum_{i=1}^{t} (e_i - \alpha)\right)}_{\text{Integral term (I)}} $$

### Dynamic Regret Theorem

Let $(\theta_t, K_t)$ be the parameters produced by COL and let $(\theta_t^*, K_t^*) \in \text{argmin}_{(\theta,K)} \mathcal{L}_t(\theta, K)$ denote any time-dependent optimal model minimizing the loss at step $t$. Further assume:

(A1) Each $\mathcal{L}_t$ is L-smooth with $\|\nabla \mathcal{L}_t(\theta, K)\| \le B$;

(A2) Bounded total variation and squared variation for oracle path: $V_T := \sum_{t=1}^{T} \|(\theta_{t+1}^*, K_{t+1}^*) - (\theta_t^*, K_t^*)\| < \infty$, $S_T := \sum_{t=1}^{T} \|(\theta_{t+1}^*, K_{t+1}^*) - (\theta_t^*, K_t^*)\|^2 < \infty$;

(A3) $\sum_{t=1}^{T} q_t \le \mathcal{O}(\alpha h(T))$ for some sublinear, nonnegative, nondecreasing function $h$;

Then the dynamic regret satisfies:

$$ \sum_{t=1}^{T} [\mathcal{L}_t(\theta_t, K_t) - \mathcal{L}_t(\theta_t^*, K_t^*)] \le \mathcal{O}(\alpha h(T) + V_T + S_T). $$

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

Take a picture to download the full paper

Shifting from producing conformal prediction sets to assessing the model's consistency on new observations, model updates are triggered only when its conformal score $s_t(\text{model}_t)$ is greater than a dynamic threshold $q_t$, and continue until $s_t(\text{model}_t) \le q_t$.

<figure><img src="image_6.png" /></figure>

## Compare to baselines

Models are trained using the training trajectories while computing the online prediction error. After the training, models are evaluated on the held-out trajectories to compute generalization error. We report averages across five splits, along with the standard deviation of the averages.

<table><thead><tr><th></th><th>ODMD</th><th>OEDMD</th><th>OnlineAE</th><th>OLoKe</th><th>COLoKe</th></tr></thead><tbody><tr><td>Single attractor</td><td>1.1 · 10<sup>-3</sup><br>(±3.6 · 10<sup>-5</sup>)</td><td>2.5 · 10<sup>-2</sup><br>(±2.8 · 10<sup>-4</sup>)</td><td>1.0 · 10<sup>-2</sup><br>(±7.7 · 10<sup>-4</sup>)</td><td>2.1 · 10<sup>-6</sup><br>(±6.6 · 10<sup>-7</sup>)</td><td><b>2.4 · 10<sup>-7</sup></b><br>(±3.6 · 10<sup>-8</sup>)</td></tr><tr><td>Duffing oscillator</td><td>4.6 · 10<sup>-5</sup><br>(±7.3 · 10<sup>-7</sup>)</td><td>1.5 · 10<sup>-2</sup><br>(±4.7 · 10<sup>-4</sup>)</td><td>7.4 · 10<sup>-5</sup><br>(±2.8 · 10<sup>-5</sup>)</td><td>7.5 · 10<sup>-6</sup><br>(±2.5 · 10<sup>-6</sup>)</td><td><b>7.6 · 10<sup>-7</sup></b><br>(±9.6 · 10<sup>-8</sup>)</td></tr><tr><td>VdP oscillator</td><td>2.5 · 10<sup>-4</sup><br>(±7.8 · 10<sup>-6</sup>)</td><td>6.8 · 10<sup>-3</sup><br>(±4.5 · 10<sup>-3</sup>)</td><td>8.7 · 10<sup>-3</sup><br>(±2.5 · 10<sup>-3</sup>)</td><td>5.5 · 10<sup>-5</sup><br>(±1.0 · 10<sup>-5</sup>)</td><td><b>3.1 · 10<sup>-6</sup></b><br>(±2.3 · 10<sup>-7</sup>)</td></tr><tr><td>Lorenz system</td><td>1.9 · 10<sup>-4</sup><br>(±1.5 · 10<sup>-6</sup>)</td><td>3.8 · 10<sup>-3</sup><br>(±3.3 · 10<sup>-4</sup>)</td><td>2.0 · 10<sup>-3</sup><br>(±6.2 · 10<sup>-4</sup>)</td><td>2.3 · 10<sup>-4</sup><br>(±4.0 · 10<sup>-5</sup>)</td><td><b>7.3 · 10<sup>-5</sup></b><br>(±1.9 · 10<sup>-5</sup>)</td></tr><tr><td>VdP oscillator</td><td>2.1 · 10<sup>-3</sup><br>(±3.6 · 10<sup>-5</sup>)</td><td>2.1 · 10<sup>-3</sup><br>(±3.2 · 10<sup>-5</sup>)</td><td>1.7 · 10<sup>-2</sup><br>(±3.0 · 10<sup>-3</sup>)</td><td>6.6 · 10<sup>-4</sup><br>(±1.5 · 10<sup>-4</sup>)</td><td><b>3.8 · 10<sup>-4</sup></b><br>(±1.2 · 10<sup>-5</sup>)</td></tr><tr><td>Lorenz system</td><td>1.1 · 10<sup>-3</sup><br>(±4.7 · 10<sup>-6</sup>)</td><td>1.1 · 10<sup>-3</sup><br>(±7.8 · 10<sup>-6</sup>)</td><td>3.8 · 10<sup>-3</sup><br>(±1.0 · 10<sup>-3</sup>)</td><td>9.2 · 10<sup>-4</sup><br>(±3.0 · 10<sup>-4</sup>)</td><td><b>6.0 · 10<sup>-4</sup></b><br>(±1.4 · 10<sup>-4</sup>)</td></tr><tr><td>ETD *</td><td>2.7 · 10<sup>-1</sup><br>(±1.3 · 10<sup>-3</sup>)</td><td>5.5 · 10<sup>-1</sup><br>(±2.2 · 10<sup>-2</sup>)</td><td>5.9 · 10<sup>-1</sup><br>(±8.4 · 10<sup>-2</sup>)</td><td>7.6 · 10<sup>-3</sup><br>(±1.8 · 10<sup>-4</sup>)</td><td><b>6.5 · 10<sup>-3</sup></b><br>(±1.0 · 10<sup>-4</sup>)</td></tr><tr><td>ETD *</td><td>1.0 · 10<sup>-1</sup><br>(±5.8 · 10<sup>-4</sup>)</td><td>2.7 · 10<sup>-1</sup><br>(±3.1 · 10<sup>-2</sup>)</td><td>3.8 · 10<sup>-2</sup><br>(±2.6 · 10<sup>-3</sup>)</td><td>4.7 · 10<sup>-3</sup><br>(±3.0 · 10<sup>-4</sup>)</td><td><b>3.3 · 10<sup>-3</sup></b><br>(±1.1 · 10<sup>-4</sup>)</td></tr><tr><td>ETD *</td><td>1.4 · 10<sup>-1</sup><br>(±2.9 · 10<sup>-1</sup>)</td><td>2.7 · 10<sup>-1</sup><br>(±3.0 · 10<sup>-1</sup>)</td><td>2.1 · 10<sup>-1</sup><br>(±3.4 · 10<sup>-1</sup>)</td><td>2.1 · 10<sup>-1</sup><br>(±3.4 · 10<sup>-2</sup>)</td><td><b>2.1 · 10<sup>-1</sup></b><br>(±8.6 · 10<sup>-2</sup>)</td></tr><tr><td>ETD *</td><td>1.2 · 10<sup>-1</sup><br>(±2.9 · 10<sup>-1</sup>)</td><td>1.5 · 10<sup>-1</sup><br>(±2.6 · 10<sup>-1</sup>)</td><td>7.9 · 10<sup>-2</sup><br>(±7.3 · 10<sup>-2</sup>)</td><td>9.7 · 10<sup>-2</sup><br>(±8.5 · 10<sup>-2</sup>)</td><td><b>7.3 · 10<sup>-2</sup></b><br>(±6.3 · 10<sup>-2</sup>)</td></tr></tbody></table>

## Adaptive update behavior

<figure><img src="image_7.png" /></figure>

## Learning efficiency

<figure><img src="image_8.png" /></figure>

## Convergence of eigen-properties

<figure><img src="image_9.png" /></figure>

## Acknowledgements

This work was sponsored by a public grant overseen by Auvergne-Rhône-Alpes region, Grenoble Alpes Metropole and BPIFrance, as part of project I-Démo Région "Green AI".

## References

1. S. L. Brunton, M. Budišić, E. Kaiser, and J. N. Kutz, "Modern Koopman Theory for Dynamical Systems," *SIAM Review*, vol. 64, no. 2, pp. 229–340, 2022.

2. A. Angelopoulos, E. Candès, and R. J. Tibshirani, "Conformal PID Control for Time Series Prediction," *Advances in Neural Information Processing Systems*, vol. 36, pp. 23047–23074, 2023.