# RoseCDL: Robust and scalable convolutional dictionary learning for rare-event detection

Jad Yehya, Mansour Benbakoura, Cédric Allain, Benoît Malezieux, Matthieu Kowalski, Thomas Moreau

## ► To cite this version:

Jad Yehya, Mansour Benbakoura, Cédric Allain, Benoît Malezieux, Matthieu Kowalski, et al. RoseCDL: Robust and scalable convolutional dictionary learning for rare-event detection. 2025. hal-05250429

HAL Id: hal-05250429

https://hal.science/hal-05250429v1

Preprint submitted on 11 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
RoseCDL: Robust and Scalable Convolutional
Dictionary Learning for Rare-event Detection

Jad Yehya¹

Mansour Benbakoura¹

Cédric Allain¹

Benoît Malezieux¹

Matthieu Kowalski²

Thomas Moreau¹

¹Inria Saclay, MIND Team, Université Paris-Saclay

²Inria Saclay, Tau Team, Université Paris-Saclay

Abstract

Identifying recurring patterns and rare events in large-scale signals is a fundamental challenge in fields such as astronomy, physical simulations, and biomedical science. Convolutional Dictionary Learning (CDL) offers a powerful framework for modeling local structures in signals, but its use for detecting rare or anomalous events remains largely unexplored. In particular, CDL faces two key challenges in this setting: high computational cost and sensitivity to artifacts and outliers. In this paper, we introduce RoseCDL, a scalable and robust CDL algorithm designed for unsupervised rare event detection in long signals. RoseCDL combines stochastic windowing for efficient training on large datasets with inline outlier detection to enhance robustness and isolate anomalous patterns. This reframes CDL as a practical tool for event discovery and characterization in real-world signals, extending its role beyond traditional tasks like compression or denoising.

# 1 Introduction

Identifying recurring patterns and rare events in a signal is a crucial task in many scientific fields, from finding QRS complex – a.k.a. heartbeats – in ECG [1] to detecting blood-cells in biological images [2] or particular celestial objects in astronomical images [3]. When working with sets of large physical signals, such a process needs to be automated. Supervised methods have been developed in each domain to address these tasks, often relying on large annotated datasets and deep learning models [4–6]. However, the reliance on labeled data can be a significant limitation, as obtaining annotations is often time-consuming, expensive, or even infeasible when the patterns are hard to characterize or happen with a very low probability.

To overcome this limitation, there is a need for unsupervised methods that can automatically learn to identify and characterize patterns in large datasets. Convolutional Dictionary Learning (CDL; [7]) is a powerful method for modeling local structures in signals, with applications ranging from audio classification and neuroscience [8] to image processing (see [9] and references therein). Its core principle is representing an observed signal as the convolution of a learned dictionary of patterns and their corresponding sparse activation vector. By considering a large set of signals and sparse enough activations, CDL can be expressed as finding a model that best reconstructs the signal in expectation, therefore allowing the identification of the presence of recurring patterns in the data. However, the use of CDL has been limited to single sample or small-scale datasets and artifact-free signals due to two main challenges: its sensitivity to anomalies in the data and its limited scalability with increasing dataset size. Moreover, it is ill-suited for detecting rare events by design, as their low probability of occurrence makes them negligible in the overall reconstruction error.
The scalability of CDL has been a key research focus since its inception, with most works addressing the computational cost of the sparse coding phase through deterministic approaches [see the survey by 10]. Another direction has been to leverage local computation to reduce the problem by solving the sparse coding problem on a window [7, 9, 11, 12]. However, these methods typically assume a fixed partitioning of the data and aim to recover the full signal by integrating the results of the local computations, limiting the computational gain of the approach. Approximate methods like learned sparse coding [13] and algorithm unrolling [14–16] have also been explored to reduce the cost of sparse coding and improve the gradient updates, with limited success in the context of pattern discovery [17]. Finally, the use of online algorithms has been proposed to reduce the computational cost of CDL by updating the dictionary while only using a small fraction of the dataset [18–21].

The robustness of CDL to artifacts and outliers has received much less attention in the recent literature. From an intuitive point of view, the sensitivity of CDL to outliers arises from the inherently sparse nature of anomalies and their potentially significant impact on the data-fitting term. This sparsity may lead the algorithm to mistake abnormal segments for meaningful patterns. Gribonval et al. [22] carried out a thorough theoretical study of this phenomenon in the context of classical dictionary learning, showing the conditions under which the optimization problem admits a local minimum close to the generating dictionary, with the presence of outliers. From a practical point of view, only a few approaches have been proposed. Mairal et al. [18] considered using an Elastic Net regularization to limit the size of the coefficient, while Jas et al. [23] proposed using an alpha-stable data fitting term, which discards large outliers but increases the computational cost.

The broader outlier detection (OD) literature has increasingly turned to reconstruction-based frame-
works to mitigate similar challenges. These methods rely on training models to reconstruct normal
data patterns while inherently failing to reconstruct anomalies accurately due to their sparse and
irregular nature. Consequently, outliers can be detected and discarded by identifying data points
with high reconstruction errors, as they deviate significantly from the learned structure of the normal
data [24, 25]. In practice, the same principle is used to detect rare events [26].

A classical strategy for robust regression consists of discarding the most corrupted observations based on their residual magnitude. This is formalized in the *Least Trimmed Squares* (LTS) estimator [27, 28], which minimizes the sum of the smallest squared residuals, thereby achieving a high breakdown point. To handle high-dimensional settings where sparsity is required, *Sparse LTS* [29] extends this approach by adding an $l_1$ penalty. Importantly, in this model, trimming is applied only to the squared residuals, i.e., the selection of observations depends solely on the reconstruction error, not on the total cost, including the regularization term. Further refinements include modified trimming schemes for quantile regression to handle leverage points [30], and extensions to compositional high-dimensional data [31]. Other approaches integrate outlier modeling directly into the regression problem, using explicit sparse error vectors [32], or introduce non-convex formulations such as the *Trimmed Lasso* [33], where both coefficients and samples are trimmed jointly within the objective.

This paper introduces a novel CDL algorithm called RObust and Scalable CDL (RoseCDL) that addresses the two challenges mentioned above: scalability and robustness to outliers. First, we propose an efficient and scalable algorithm for CDL that targets learning a model for the local structure that generalizes well to large datasets. This algorithm relies on a stochastic windowing approach, where the sparse coding problem is solved approximately on a small data window, and the dictionary is updated using the results of the local computations. Second, we introduce an outlier detection method based on the CDL model’s local reconstruction error, which allows for identifying data patches not well represented by the learned dictionary. This approach is used both during the training phase to improve the model’s robustness and during the testing phase to identify rare events in the data. The resulting algorithm is able to learn a dictionary that captures the local structure of the data while being robust to outliers and can be applied to large datasets in a scalable manner. We demonstrate our approach’s effectiveness on various datasets, including synthetic data and real-world applications.

2 Finding common and rare patterns in signals: the RoseCDL algorithm

Let $\mathbf{x} \in \mathbb{R}^T$ be a univariate signal with length $T$. Convolutional Dictionary Learning (CDL) consists of finding a dictionary: $D = (\mathbf{d}_k)_{k \in [1,K]} \in \mathbb{R}^{K \times L}$ of $K$ patterns of length $L \ll T$, and corresponding activation vectors $Z = (\mathbf{z}_k)_{k \in [1,K]} \in \mathbb{R}^{K \times (T-L+1)}$, that minimize the distance

<footnote n="1">¹https://tommoral.github.io/dicode/auto_examples/plot_gait.html</footnote>
<figure><img src="image_2.png" /><figcaption>Figure 1: Schematic operation of the CDL 1D univariate signal, adapted from the dicodile package example¹. The output of the CDL is composed of a set of atoms (here 1) alongside their respective activations.</figcaption></figure>

between $\mathbf{x}$ and $\hat{\mathbf{x}} = D * Z = \sum_k \mathbf{d}_k * \mathbf{z}_k$, where * denotes the convolution. This is achieved by solving the following optimization problem:

$$ \min_{D,Z} F(D, Z; \mathbf{x}) = \underbrace{\frac{1}{2} \|\mathbf{x} - D * Z\|_2^2 + \lambda \|Z\|_1}_{f_{\mathbf{x}}(Z)}, \quad \text{s.t.} \quad \|\mathbf{d}_k\|_2^2 \le 1, \forall k \in [1, K], \quad (1) $$

where $\|Z\|_1 = \sum_k \|\mathbf{z}_k\|_1$. Fig. 1 illustrates the CDL model for an univariate signal. This problem can easily be extended to multivariate and multidimensional signals such as images by adapting the convolution operator to work with tensors.

When working with large signal databases, the primary interest is not finding the best reconstruction of the signal used to learn the dictionary but instead finding the patterns that best model the population. Therefore, this paper focuses on **characterizing the distribution** of the signals $\mathbf{x}$, with the following optimization problem:

$$ \min_{D} \mathbb{E}_{\mathbf{x}} \left[ \min_{Z} F(D, Z; \mathbf{x}) \right], \quad \text{s.t.} \quad \|\mathbf{d}_k\|_2^2 \le 1, \forall k \in [1, K]. \quad (2) $$

While this formulation departs from classical optimization-based literature for CDL [10], it is often used with deep CDL approaches to learn denoisers that generalize to unseen images using super learning losses [34–36]. Here, we consider the unsupervised loss $F$ to learn the dictionary of event patterns directly from the signals.

For both formulation (1) and (2), classical solvers rely on an iterative optimization algorithm to solve the sparse coding problem in $Z$ for fixed $D$, and alternate it with a dictionary update step. The sparse coding step is typically solved using iterated soft-thresholding iterations (e.g., FISTA [37, 38]), which reads

$$ \begin{gathered} Z_0 = Y_0 \in \mathbb{R}^{K \times T - L + 1}, t_0 = 1; \quad Z_{k+1} = \text{St}(Y_k - \nu \nabla_Z f_{\mathbf{x}}(Z), \nu \lambda) \\ t_{k+1} = \frac{1}{2}(1 + \sqrt{1 + 4t_k^2}) \quad Y_{k+1} = Z_k + \frac{t_k - 1}{t_{k+1}}(Z_{k+1} - Z_k) \end{gathered} \quad (3) $$

with $\text{St}(Z) = \text{sign}(Z)(|Z| - \lambda)_+$ and $\nu$ the step size. The dictionary update may rely on projected gradient descent constrained to the unit $\ell_2$-ball. However, these methods become computationally expensive on long signals or large datasets, as both the sparse codes and gradients must be computed over all the signals. Online methods mitigate the need to consider all signals, but at the expense of increased memory usage and still require solving the sparse coding problem on full signals. This motivates using stochastic or localized approximations, especially in settings where robustness to artifacts or outliers is required.

## 2.1 Stochastic windowing

Due to the convolutional structure of the CDL model, points in the far-apart signal are only weakly dependent [11]. Indeed, the value of the sparse code at time $t$ is seldom impacted by the values at time $t + s$, with $s$ larger than the size of the dictionary $L$. By considering windows of the signal which are large enough, the problem (2) can be approximated by

$$ \min_{D} \mathbb{E}_{\tau} \left[ \min_{Z} F(D, Z; \mathbf{x}_{\tau}) \right], \quad \text{s.t.} \quad \|\mathbf{d}_k\|_2^2 \le 1, \forall k \in [1, K], \quad (4) $$
**Algorithm 1** CDL with stochastic windowing.

**input** X, N<sub>iter</sub>, N<sub>W</sub>, N<sub>FISTA</sub>

Initialize D<sup>(0)</sup>

**for** 0 ≤ i ≤ N<sub>iter</sub> − 1 **do**

Sample N<sub>W</sub> windows in the dataset: (X<sub>w</sub>)<sub>w∈[1,N<sub>W</sub>]</sub>

**for** 1 ≤ w ≤ N<sub>W</sub> **do**

Compute the approximate sparse code

$$Z_w^{N_{\text{FISTA}}} \approx Z_w^*(D^{(i)}; X_w)$$

Compute an outlier mask (cf. Sec. 2.2)

Compute the loss F and its gradient ∇<sub>D</sub>F outside the outlier mask

**end for**

Compute best step size α<sub>i</sub> with SLS

$$D^{(i+1)} \leftarrow D^{(i)} - \alpha_i \nabla_D \sum_w F_w(D^{(i)}, Z_w^{N_{\text{FISTA}}}; X_w)$$

**end for**

**output** D<sup>(N<sub>iter</sub>)</sup>

where x<sub>τ</sub> are windows of the signal, starting at τ ∈ [1, T − W<sub>win</sub> + 1] and of size W<sub>win</sub> such that L ≤ W<sub>win</sub> ≪ T. This formulation is only approximate as it produces inexact sparse code for the windows, due to border effect. But with the weak spatial dependence of the model, if the window is large and the activation are sparse, these effects are negligible. Moreover, by considering all windows with overlap, we limit the impact of specific border effects in the algorithm. To minimize (4), we propose RoseCDL, a stochastic gradient descent algorithm aimed to characterize the distribution of patches in the signal.

We process as follows: At each step of the CDL outer problem, we begin by sampling N<sub>W</sub> windows (x<sub>w</sub>)<sub>1≤w≤N<sub>W</sub></sub> from a uniform distribution. The windows are selected with overlap to limit the bias due to border effects. We then compute an approximate sparse code Z<sup>*</sup><sub>w</sub> for each subproblem by minimizing the sparse coding over the window. Following [17, 16], whose results suggested that optimizing over D did not require precise sparse coding at each step, we use an approximation Z<sup>N<sub>FISTA</sub></sup> of Z<sup>*</sup>(D; x), where Z<sup>N<sub>FISTA</sub></sup> is given by N<sub>FISTA</sub> iterations of the FISTA algorithm (3).

Using these per-window approximated activation vectors, we can then update the dictionary D. Due to the stochastic nature of the algorithm, we depart from traditional alternate minimization strategies and perform only one gradient step. This single-step update strategy is further justified by the inherent noise in the gradient estimate, which arises from both the stochastic sampling of windows and the use of approximate sparse codes. In such settings, performing multiple gradient steps per batch does not significantly reduce the update variance and may even amplify the impact of the approximation error in the sparse coding stage. This strategy is in-line with the deep CDL algorithm, but we do not backpropagate the gradient through the sparse code approximation Z<sup>N<sub>FISTA</sub></sup>, as it leads to unstable Jacobian estimation for the original problem (4), as described in [17]. To stabilize the learning and reduce the number of parameters, we compute the optimal stepsize of the dictionary update with the Stochastic Line Search algorithm [SLS; 39]. This algorithm can be efficiently implemented using deep learning frameworks and can leverage GPU acceleration. The complete procedure is summarized in Alg. 1.

## 2.2 Inline outlier detection

The second component of RoseCDL is its inline outlier detection framework. Let x be a signal of the form:

$$x = d_a * z_a + d_b * z_b + n, \qquad (5)$$

where d<sub>a</sub> is a common pattern in the signal, d<sub>b</sub> is a rare pattern, and n is an abnormal pollution (e.g., artifacts, sudden spikes in the signal). Traditional CDL algorithms are likely to struggle to recover d<sub>a</sub> for two main reasons. First, artifacts in n often have a significant variance, which can distract the CDL from the significant signal. Second, even when the anomalies in n are well discarded, the rare pattern d<sub>b</sub> acts as a pollution that prevents the algorithm from learning the atom d<sub>a</sub>. While effects of n and d<sub>b</sub> are often discarded through preprocessing [8], it is often very difficult to use this reliably on large population datasets.
<figure><img src="image_3.png" /><figcaption>Figure 2: Raw signal X, reconstruction error, threshold and learned outlier mask on subject a02 (minute 56) of Physionet Apnea-ECG data set. Detection method is based on modified z-score (MAD), with $\alpha = 3.5$. The method correctly identifies outliers blocks.</figcaption></figure>

The intuition behind our approach is that if $d_a$ is sufficiently represented in $x$, then most of the patches of $x$ should contain information relative to this pattern. Consequently, these patches should be the best reconstructed ones. In comparison, the patches containing non-zero values of $n$ are expected to have a high reconstruction error due to the unpredictable nature of anomalies and artifacts. Finally, given a dictionary $d$ that is more correlated with $d_a$ than $d_b$, the patches containing chunks of $d_b$ are expected to have a higher reconstruction error than those with $d_a$.

To summarize, the distribution of patch reconstruction errors $F(D, Z; x_\tau)_\tau$ is expected to have three modes:

1. The main, low-value mode corresponding to chunks of $x_a = d_a * z_a$,

2. A secondary mode with a slightly higher value corresponding to chunks of $x_b = d_b * z_b$,

3. A high-value mode corresponding to chunks of $n$.

Consequently, the reconstruction error of a patch can be used as an indicator to determine whether it contains information about the relevant signal. We leverage this intuition to inflect the learning trajectory of $d$ towards $d_a$. We define the set $P_\beta = \{\tau | F(D, Z; x_\tau) < \beta\}$, where $\beta \in [0, 1]$ is a threshold selecting the proportion of outliers. With a well chosen $\beta$, $P_\beta$ indicates the patches that coincide with realizations of $x_a$. Therefore, to make the CDL converge towards $d_a$, we change the objective value for the dictionary updates for its trimmed version:

$$ \tilde{F}(Z, D; \mathbf{x}) = \frac{1}{W_{\text{patch}}} \sum_{\tau \in P_{\beta}} F(Z, D; \mathbf{x}_{\tau}). \quad (6) $$

Using this trimmed loss, we can show that in simple settings, the RoseCDL algorithm is more robust to the presence of outliers.

**Proposition 2.1 (Stability of the common pattern).** Consider a population of signals $X$ composed of two patterns $d_a$ and $d_b$, with activations such that the patterns do not overlap in the signal, and corrupted by an additive Gaussian noise $\epsilon \sim N(0, \sigma I_d)$. Introduce $c = d_a^\top d_b$ and $\rho$ the proportion of rare-event pattern $d_b$ activations in the population. Then, in the noiseless setting,

i. $d_a$ is a fixed point of the classical CDL algorithm for $K = 1$ if $c \le \lambda$

ii. $d_b$ is a fixed point of RoseCDL algorithm for $K = 1$ if $c \le \lambda$ or the RoseCDL algorithm is used with an outlier threshold trimming a proportion of windows greater than $\rho$.

The proof is deferred in App. A. This demonstrates that even in ideal conditions (no overlap and starting with the right pattern), classical CDL fails to recover the common pattern due to interference from rare events, while RoseCDL corrects this via trimming. However, a critical design choice is the selection of the statistic $\beta$ used as the threshold, discussed below.

**Threshold selection.** From the distribution of patch reconstruction errors $(\epsilon_\tau)_\tau = (F(D, Z; x_\tau))_\tau$, one needs to compute a threshold $\beta$ which separates the outlier patches in $x$ from the normal ones. The goal is to detect extreme points in the distribution, which are too large compared to the population of patch errors. The outlier detection literature provides four main methods relevant in this case:
1. the **method of quantiles**, where $\beta = Q_{\varepsilon, (1-\alpha)}$ is the quantile of order $(1 - \alpha)$ of the set $\varepsilon$,

2. the **z-score method** [40], where each error $\varepsilon_{\tau}$ has an associated score $z_{\tau} = (\varepsilon_{\tau} - \mu_{\varepsilon})/\sigma_{\varepsilon}$, with $\mu_{\varepsilon}$ and $\sigma_{\varepsilon}$ denoting respectively the mean and standard deviation of the distribution of errors, and where outliers are defined as observations such that $|z_{\tau}| > \alpha$, generally $\alpha = 2$ or 3, thus having $\beta = \mu_{\varepsilon} + \alpha \sigma_{\varepsilon}$,

3. the **modified z-score** (MAD; Iglewicz and Hoaglin 40), where, similarly as the z-score, each error point is associated with a score based on the median, and the resulting threshold is $\beta = \text{Med}_{\varepsilon} + (\alpha \text{Mad}_{\varepsilon})/0.6745$, with $\text{Mad}_{\varepsilon} = \text{Med}(|\varepsilon - \text{Med}_{\varepsilon}|)$, where $\text{Med}$ denotes the median operator, and generally $\alpha = 3.5$.

These methods are initially bilateral, but only the upper bound is considered in this work because we aim to detect outliers with large reconstruction errors. Fig. 2 illustrates the outlier detection method on real ECG data, with the outliers mask computed with the reconstruction error and the threshold.

**Role of the outlier mask for rare-event detection.** The inline outlier detection module is a central component of our algorithm. It enables the computation of an outlier mask during training, serving two key purposes: (1) it excludes identified outliers from the loss function, thereby improving the robustness of dictionary learning; (2) it enables the unsupervised detection of rare events in the signal by interpreting the outlier mask as a detection map. Indeed, provided a signal $x$ in the form of Eq. (5), we have shown that RoseCDL is able to extract the contribution of the common pattern $d_a$. Consequently, the corresponding outlier mask can be used to select the residual signal $x' = d_b * z_b + n$. Running another instance of RoseCDL on $x'$ then allows one to recover the pattern $d_b$ and localize its occurrences $z_b$.

## 3 Numerical experiments

We now present numerical results that demonstrate our implementation's scalability, robustness, and stabilization behavior, evaluated on both simulated and real-world datasets. RoseCDL is implemented using the Pytorch framework [41]<fnref n="2" />. Our analysis highlights the method's computational efficiency and ability to capture essential data characteristics consistently under varying conditions. To evaluate our model's performance on synthetic data, we rely on the convolutional dictionary recovery metric proposed in [11] and detailed in App. C. This metric is based on the best assignment between the true dictionary used to generate the data and the estimated dictionary. To account for the convolutional nature of the dictionary, the similarity used for assignment is the convolutional cosine similarity.

<figure><img src="image_4.png" /><figcaption>Figure 3: Comparison of optimization runtime for **RoseCDL**, AlphaCSC, Sporco, and DeepCDL in 1D and 2D settings, highlighting the superior scalability and convergence speed of **RoseCDL**. The runtime plots show the evolution of test loss over time. The third subplot reports the dictionary recovery score at convergence for 2D data. As AlphaCSC does not 2D data, only results for Sporco, DeepCDL, and RoseCDL are shown.</figcaption></figure>

**On RoseCDL scalability.** In Fig. 3, we present a comprehensive comparison between RoseCDL and three CDL methods: AlphaCSC [8], Sporco [42], and DeepCDL, a variant of [16]. DeepCDL

<footnote n="2">²Code is available in the supplementary materials.</footnote>
represents the unrolled variant of RoseCDL, where gradients account for the Jacobian of the sparse code computed through backpropagation. In contrast, RoseCDL employs an alternating minimization scheme, decoupling the update steps.

We evaluate each method's computational cost and runtime on two large-scale datasets. The one-dimensional (1D) experiment is performed on a dataset of 20 multivariate signals containing 50,000 time samples with two channels (see App. B for details). The two-dimensional (2D) experiment is conducted on a semi-synthetic dataset of images of 2000 × 2000 pixels. The cost is evaluated as the value of the objective function $F(D, Z^{(D)}; x)$ at each iteration on a separate test set, with $Z^*$ computed to convergence, assessing the capacity of the solver to minimize (2).

The empirical results demonstrate that RoseCDL exhibits superior scalability attributable to several architectural advantages. First, GPU-optimized training yields substantial speedups when properly leveraged. Second, RoseCDL employs `fftconv` (Fast Fourier Transform-based convolution) instead of standard spatial convolutions for large kernels, significantly reducing computation time. The distinction from DeepCDL arises primarily from optimization strategy differences: DeepCDL's unrolled architecture requires gradient computation over substantially larger parameter sets, increasing computational overhead compared to RoseCDL's alternating minimization approach.

To thoroughly assess scalability, we conduct additional experiments varying both window sizes and signal lengths. For window size analysis on 1D signals with $T = 100,000$ and $\lambda = 0.8$, we evaluate both Adam and SLS optimizers across window sizes ranging from $10L$ to $100L$. To ensure full GPU utilization, we maintain a constant product window size × batch size. As shown in Tab. 1, RoseCDL consistently achieves validation losses within 4% of AlphaCSC performance across all configurations, while maintaining runtimes of 12–22% relative to AlphaCSC, corresponding to approximately 5× speedup.

<table><caption>Table 1: Comparison of Adam and SLS with different window sizes.</caption><thead><tr><th rowspan="2">optimizer</th><th colspan="4">Adam</th><th colspan="4">SLS</th></tr><tr><th>10L</th><th>20L</th><th>50L</th><th>100L</th><th>10L</th><th>20L</th><th>50L</th><th>100L</th></tr></thead><tbody><tr><td>window size</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>validation loss</td><td>3.0%</td><td>3.8%</td><td>2.7%</td><td>2.7%</td><td>-0.2%</td><td>-0.2%</td><td>-0.2%</td><td>-0.4%</td></tr><tr><td>runtime</td><td>15.4%</td><td>12.3%</td><td>12.5%</td><td>12.7%</td><td>21.7%</td><td>21.2%</td><td>16.3%</td><td>17.6%</td></tr></tbody></table>

Notably, the SLS optimizer demonstrates superior convergence properties with validation losses within 0.4% of AlphaCSC, though with slightly increased runtime compared to Adam. The results validate that border effects do not hinder convergence with our stochastic windowing approach, even for small window sizes (as low as $10L$).

For signal length scalability, we evaluate performance on signals ranging from 10k to 1M time samples. As detailed in Tab. 2, RoseCDL demonstrates sublinear scaling, with runtimes growing from 15.2 s (10k samples) to 202.8 s (1M samples), while AlphaCSC becomes computationally prohibitive beyond 100k samples. This superior scalability enables RoseCDL to process signals substantially larger than existing full-signal sparse coding methods.

<table><caption>Table 2: Signal length scaling experiments: runtime comparison between RoseCDL and AlphaCSC for varying signal sizes <i>T</i>.</caption><thead><tr><th><i>T</i></th><th>10k</th><th>30k</th><th>100k</th><th>300k</th><th>1M</th></tr></thead><tbody><tr><td>Runtime RoseCDL (s)</td><td>15.2</td><td>16.4</td><td>32.6</td><td>68.0</td><td>202.8</td></tr><tr><td>Runtime AlphaCSC (s)</td><td>67.5</td><td>102.8</td><td>198.5</td><td>N/A</td><td>N/A</td></tr></tbody></table>

It is important to note that AlphaCSC does not support two-dimensional data and was excluded from image-based experiments. Despite this limitation, RoseCDL achieves comparable or superior test costs across all evaluated configurations. Dictionary recovery performance evaluation on the 2D dataset confirms that RoseCDL achieves results comparable to DeepCDL and substantially outperforms Sporco, validating both computational efficiency and solution quality.
<figure><img src="image_5.png" /><figcaption>Figure 4: (a) Impact of the regularization on the recovery score over the epochs. (b) Atoms recovered at the end of training for different regularization values.</figcaption></figure>

**Parameter sensitivity.** A key parameter of RoseCDL is the regularization coefficient $\lambda$, introduced in Sec. 2, which controls the sparsity of the learned activation vector. We express $\lambda$ as a fraction of the maximum regularization value, $\lambda_{max}$, corresponding to the smallest regularization value for which the activation vector is entirely zero. Note that when using the trimmed objective, $\lambda_{max}$ computation is adapted to account for the modified loss. To analyze the effect of regularization on dictionary recovery, we compare recovery scores across different values of $\lambda$ using synthetic data. The results, presented in Fig. 4, demonstrate how varying $\lambda$ influences the quality of the recovered dictionary. Our findings indicate that the best dictionary recovery is achieved when setting $\lambda = 0.1\lambda_{max}$, effectively balancing sparsity and reconstruction accuracy.

**On rare event detection with synthetic 2D data.** To evaluate the performance of RoseCDL on 2D data, we constructed a semi-synthetic dataset of images by generating 5000 characters sampled from a set of four letters (R, O, S, and E) along with spaces.

These images emulate text-like documents composed of words formed from the selected characters. We added the letter Z in a small proportion to introduce rare events. Experiments were conducted with a 10% contamination rate. Consequently, for the inline outlier detection methods, we implemented the quantile detection method with $\alpha$ value: 10%, alongside with MAD and z-score methods as described in Sec. 2.2. To assess the efficiency of the inline outlier module on this dataset, we compared it to RoseCDL without the inline outlier module, where we computed the threshold by computing the reconstruction error after the learning part. We show the detection over the epochs using different outlier detection algorithms by computing the F1 score between the inline model mask for the *during* procedure and the mask after the reconstruction by a RoseCDL model without an inline outlier method for the *after* procedure in Fig. 5. The inline outlier detection module, particularly the MAD method, works by discarding the rare events from the dictionary learning part. These rare events are reconstructed with lower fidelity compared to the more prevalent

patterns. This selective degradation results in higher reconstruction errors that are sharply localized at the rare events' positions and improves their detection. In opposition to that, in the absence of this module, rare events are still reconstructed less accurately than common patterns; however, the

<figure><img src="image_6.png" /><figcaption>Figure 5: Comparison of the F1 score evolution of different methods over the epochs on rare event detection task on the (R, O, S, E, and Z).</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Figure 6: Evolution of median recovery scores for RoseCDL on 2D data across 20 independent runs. Comparison between RoseCDL without inline outlier detection (a) and with it (b). Shaded regions represent the range between 25th and 75th percentiles. The inline detection mechanism demonstrates enhanced training stability across different regularization parameters.</figcaption></figure>

contrast in reconstruction quality is less distinct. This diminishes the precision of rare event detection and ultimately leads to a lower F1 score. Additionally, in Fig. 6, we illustrate the impact of inline outlier detection on the dictionary recovery score. We conducted experiments using RoseCDL with and without the best-performing inline outlier detection method, as identified in Fig. 5, across 20 independent trials on the ROSE+Z dataset. We observe two important things. First, the inline outlier detection procedure stabilizes training, improves the quality of the learned dictionary, and reduces variability in its quality across trials. This is achieved by preventing rare events from being included in the training process, thereby allowing the model to focus on learning the underlying patterns of interest. Second, the method substantially enhances the algorithm's robustness in choosing the regularization parameter $\lambda$. To further support this observation, we visually compare the learned dictionary atoms corresponding to different recovery scores. For each method, we show the atoms that are the most similar to the true atoms based on the best assignment metric described at the beginning of Sec. 3.

**RoseCDL on real-world data** To assess the efficiency of outlier detection methods on real-world data, we utilized the Physionet Apnea-ECG dataset [43]. Notably, no preprocessing was applied to the signals, thereby minimizing manual interventions and ensuring the integrity of the raw data. A comprehensive description of this dataset is provided in [43]. Our experimental procedure entailed learning a dictionary composed of three atoms, each spanning 1 s, from a 10 min segment of ECG data interspersed with blocks of outliers, as shown in Fig. 2. The objective was to evaluate the model's ability to learn an effective dictionary from corrupted data and subsequently apply it for encoding signals free of outliers. Despite the inherent uncertainty in the actual proportion of outliers present in real-world data, the z-score-based detection method consistently yielded strong performance. Visualizations of the learned dictionaries, contrasting the cases with and without outlier detection, are provided in Fig. 7. We observe that, without any outlier detection mechanism and without careful parameter tuning, the model fails to recover meaningful atoms, instead converging to noise-like patterns. In contrast, incorporating a z-score-based

<figure><img src="image_8.png" /><figcaption>Figure 7: Learned atoms with and without outliers detection method, on 10 bad trials of subject a02 of dataset Physionet Apnea-ECG.</figcaption></figure>
outlier detection method enables the model to recover characteristic ECG patterns successfully. This phenomenon is attributable to the fact that outlier blocks, characterized by significantly higher variance than the rest of the signal, are preferentially addressed during the sparse coding phase to minimize reconstruction error, consequently leading to the neglect of non-outlier signal segments that contain relevant ECG patterns. However, it is important to note that although it is possible to recover the correct atoms through careful tuning of the regularization parameter when using no outlier detection mechanism, achieving this consistently requires a time-consuming trial-and-error process. Without such tuning, reliable recovery remains unlikely.

We evaluated RoseCDL against established outlier detection methods: Matrix Profile (MP), Autoencoder (AE), and One-Class SVM (OC-SVM) and on several datasets from the TSB-UAD benchmark [44]. Results reported in Table 3 show that RoseCDL consistently matches or outperforms competing methods in terms of AUC, while achieving significantly lower runtimes.

<table><caption>Table 3: AUC ROC scores of different anomaly detection methods across datasets.</caption><thead><tr><th></th><th>RoseCDL</th><th>AE</th><th>MP</th><th>OC-SVM</th></tr></thead><tbody><tr><td>Simulated</td><td>0.97</td><td>0.94</td><td>0.91</td><td>0.61</td></tr><tr><td>MITDB</td><td>0.81</td><td>0.59</td><td>0.76</td><td>0.66</td></tr><tr><td>MGAB</td><td>0.58</td><td>0.61</td><td>0.91</td><td>0.52</td></tr><tr><td>ECG</td><td>0.92</td><td>0.91</td><td>0.47</td><td>0.92</td></tr><tr><td>Avg Runtime (s)</td><td>9.71</td><td>307.97</td><td>196.47</td><td>3327.21</td></tr></tbody></table>

These experiments demonstrate the effectiveness of our approach in extracting meaningful patterns from corrupted real-world data using outlier detection methods without the need for explicit data preprocessing or prior knowledge of the outlier proportion. This is made possible by the inline outlier detection module, which enhances both training stability and robustness to hyperparameter choices.

## 4 Conclusion

In this study, we introduce RoseCDL, a robust and scalable approach to Convolutional Dictionary Learning (CDL) for unsupervised rare event detection in both 1D and 2D data. This algorithm relies on an intuition close to bootstrapping: to model the distribution of a population of signals, rather than looking for a model that reconstructs the training set well, we characterize the distribution of small subsets (patches) of the training samples. This approach supports the two main features of RoseCDL: stochastic windowing and inline outlier detection, which address the key challenges of CDL scalability and robustness to outliers, respectively. Furthermore, RoseCDL's ability to output outlier masks as a by-product of pattern detection can be associated with its speed in building an unsupervised rare-event detection pipeline: the principle is to first learn a dictionary of common patterns in a signal, then use it to extract these patterns, and finally, learn a dictionary of rare patterns on the residual signal.

A central concept of this paper is considering the CDL as a tool to characterize the distribution of patterns in the signal, which enables both stochastic updates on signal windows and outlier detection based on the local reconstruction loss.

These enhancements in speed and outlier resistance pave the way for future large-scale studies on real-world, high-dimensional, noisy datasets.

**Limitations.** The proposed approach relies on a non-convex objective, as with all CDL-based methods. However, our optimization scheme builds on well-established principles, for which convergence to meaningful local minima has been observed and studied in prior work [22, 17, 16]. Our trimming strategy intentionally excludes poorly reconstructed patches during training, which may raise bias concerns. Yet this behavior is by design: the method aims to localize and separate such regions rather than fit them and retain their information via a learned mask. While this mechanism confers robustness to outliers, it does not directly translate into precise anomaly detection: the model is conservative by design and may flag many segments as atypical, making it more suitable for robust representation learning and rare event detection than for high-precision anomaly scoring.
# 5 Acknoledgements

This project was supported by the French National Research Agency (ANR) through the BenchArk project (ANR-24-IAS2-0003).

Mansour Benbakoura was supported from a national grant attributed to the ExaDoST project of the NumPEx PEPR program, under the reference ANR-22-EXNU-0004

This work was performed using HPC resources from GENCI-IDRIS (Grant 2025-AD011015308R1).

# References

[1] Luz Luz, Eduardo José da S, William Robson Schwartz, Guillermo Cámara-Chávez, and David Menotti. ECG-based heartbeat classification for arrhythmia detection: A survey. *Computer Methods and Programs in Biomedicine*, 127:144–164, 2016. 1

[2] Florence Yellin, Benjamin D. Haeffele, and René Vidal. Blood cell detection and counting in holographic lens-free imaging by convolutional sparse dictionary learning and coding. In *IEEE International Symposium on Biomedical Imaging (ISBI)*, Melbourne, Australia, 2017. 1

[3] M. Giavalisco and the GOODS Teams. The Great Observatories Origins Deep Survey: Initial Results From Optical and Near-Infrared Imaging. *The Astrophysical Journal Letters*, 600(2): L93, 2004. 1

[4] Fatma Murat, Ozal Yildirim, Muhammed Talo, Ulas Baran Baloglu, Yakup Demir, and U. Rajendra Acharya. Application of deep learning techniques for heartbeats detection using ECG signals-analysis and review. *Computers in Biology and Medicine*, 120:103726, May 2020. 1

[5] Shilpa Choudhary, Sandeep Kumar, Pammi Sri Siddhaarth, and Guntu Charitasri. Transforming Blood Cell Detection and Classification with Advanced Deep Learning Models: A Comparative Study, October 2024.

[6] D. Cornu, P. Salomé, B. Semelin, A. Marchal, J. Freundlich, S. Aicardi, X. Lu, G. Sainton, F. Mertens, F. Combes, and C. Tasse. YOLO-CIANNA: Galaxy detection with deep learning in radio data - I. A new YOLO-inspired source detection method applied to the SKAO SDC1. *Astronomy & Astrophysics*, 690:A211, October 2024. 1

[7] Roger Grosse, Rajat Raina, Helen Kwong, and Andrew Y. Ng. Shift-Invariant Sparse Coding for Audio Classification. *Cortex*, 8:9, 2007. 1, 2

[8] Tom Dupré la Tour, Thomas Moreau, Mainak Jas, and Alexandre Gramfort. Multivariate convolutional sparse coding for electromagnetic brain signals. *Advances in Neural Information Processing Systems*, 31:3292–3302, 2018. 1, 4, 6

[9] Vardan Papyan, Yaniv Romano, Jeremias Sulam, and Michael Elad. Convolutional dictionary learning via local processing. In *Proceedings of the IEEE International Conference on Computer Vision (ICCV)*, Oct 2017. 1, 2

[10] Brendt Wohlberg. Efficient algorithms for convolutional sparse representations. *IEEE Transactions on Image Processing*, 25(1):301–315, 2015. 2, 3

[11] Thomas Moreau and Alexandre Gramfort. DiCoDiLe: Distributed Convolutional Dictionary Learning. *IEEE Transactions on Pattern Analysis and Machine Intelligence*, 2020. 2, 3, 6, 16

[12] Laurent Dragoni, Rémi Flamary, Karim Lounici, and Patricia Reynaud-Bouret. Sliding window strategy for convolutional spike sorting with lasso: Algorithm, theoretical guarantees and complexity. *Acta Appl. Math.*, 179(1), June 2022. ISSN 0167-8019. doi: 10.1007/s10440-022-00494-x. URL https://doi.org/10.1007/s10440-022-00494-x. 2

[13] Karol Gregor and Yann LeCun. Learning fast approximations of sparse coding. In *Proceedings of the 27th International Conference on International Conference on Machine Learning*, ICML'10, page 399–406, Madison, WI, USA, 2010. Omnipress. ISBN 9781605589077. 2
[14] Hao Tang, Hong Liu, Wei Xiao, and Nicu Sebe. When Dictionary Learning Meets Deep Learning: Deep Dictionary Learning and Coding Network for Image Recognition With Limited Data. *IEEE Transactions on Neural Networks and Learning Systems*, 32(5):2129–2141, May 2021. 2

[15] Bahareh Tolooshams, Sourav Dey, and Demba Ba. Scalable convolutional dictionary learning with constrained recurrent sparse auto-encoders. In *2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP)*, pages 1–6, 2018. doi: 10.1109/MLSP.2018.8516996.

[16] Bahareh Tolooshams and Demba E. Ba. Stable and interpretable unrolled dictionary learning. *Transactions on Machine Learning Research*, 2022. ISSN 2835-8856. URL https://openreview.net/forum?id=e3S0B12R08. 2, 4, 6, 10

[17] Benoît Malézieux, Thomas Moreau, and Matthieu Kowalski. Understanding approximate and unrolled dictionary learning for pattern recovery. *International Conference on Learning Representations*, 2022. 2, 4, 10

[18] Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro. Online learning for matrix factorization and sparse coding. *Journal of Machine Learning Research*, 11(1), 2010. 2

[19] Arthur Mensch, Julien Mairal, Bertrand Thirion, and Gaël Varoquaux. Dictionary learning for massive matrix factorization. In *International Conference on Machine Learning*, pages 1737–1746. PMLR, 2016.

[20] Jialin Liu, Cristina Garcia-Cardona, Brendt Wohlberg, and Wotao Yin. Online Convolutional Dictionary Learning. In *International Conference on Image Processing (ICIP)*, pages 1707–1711, Beijing, China, 2017.

[21] Yijie Zeng, Jichao Chen, and Guang-Bin Huang. Slice-Based Online Convolutional Dictionary Learning. *IEEE Transactions on Cybernetics*, pages 1–14, 2019. 2

[22] Rémi Gribonval, Rodolphe Jenatton, and Francis Bach. Sparse and spurious: Dictionary learning with noise and outliers. *IEEE Transactions on Information Theory*, 61(11):6298–6319, 2015. doi: 10.1109/TIT.2015.2472522. 2, 10

[23] Mainak Jas, Tom Dupré La Tour, Umut Şimşekli, and Alexandre Gramfort. Learning the morphology of brain signals using alpha-stable convolutional sparse coding. *Advances in Neural Information Processing Systems*, pages 1099–1108, 2017. 2

[24] Lukas Ruff, Jacob R. Kauffmann, Robert A. Vandermeulen, Gregoire Montavon, Wojciech Samek, Marius Kloft, Thomas G. Dietterich, and Klaus-Robert Muller. A Unifying Review of Deep and Shallow Anomaly Detection. *Proceedings of the IEEE*, 109(5):756–795, May 2021. ISSN 0018-9219, 1558-2256. doi: 10.1109/JPROC.2021.3052449. URL https://ieeexplore.ieee.org/document/9347460/. 2

[25] Sebastian Schmidl, Phillip Wenig, and Thorsten Papenbrock. Anomaly detection in time series: a comprehensive evaluation. *Proceedings of the VLDB Endowment*, 15(9):1779–1797, May 2022. ISSN 2150-8097. doi: 10.14778/3538598.3538602. URL https://dl.acm.org/doi/10.14778/3538598.3538602. 2

[26] Chathurangi Shyalika, Ruwan Wickramarachchi, and Amit Sheth. A comprehensive survey on rare event prediction, 2024. URL https://arxiv.org/abs/2309.11356. 2

[27] Peter J Rousseeuw. Least median of squares regression. *Journal of the American statistical association*, 79(388):871–880, 1984. 2

[28] Peter J Rousseeuw and Annick M Leroy. *Robust regression and outlier detection*. John wiley & sons, 2005. 2

[29] Andreas Alfons, Christophe Croux, and Sarah Gelper. Sparse least trimmed squares regression for analyzing high-dimensional large data sets. *The Annals of Applied Statistics*, pages 226–248, 2013. 2
[30] Habshah Midi, Taha Alshaybawee, and Mohammed Alguraibawi. Modified least trimmed quan-
tile regression to overcome effects of leverage points. Mathematical Problems in Engineering,
2020:1–13, 2020. 2

[31] Gianna Serafina Monti and Peter Filzmoser. Sparse least trimmed squares regression with compositional covariates for high-dimensional data. *Bioinformatics*, 37(21):3805–3814, 2021. 2

[32] Hansheng Wang, Guodong Li, and Guohua Jiang. Robust regression shrinkage and consistent variable selection through the lad-lasso. *Journal of Business & Economic Statistics*, 25(3): 347–355, 2007. 2

[33] Dimitris Bertsimas, Martin S Copenhaver, and Rahul Mazumder. The trimmed lasso: Sparsity and robustness. *arXiv preprint arXiv:1708.04527*, 2017. 2

[34] Meyer Scetbon, Michael Elad, and Peyman Milanfar. Deep k-svd denoising. *IEEE Transactions on Image Processing*, 30:5944–5955, 2021. 3

[35] Hongyi Zheng, Hongwei Yong, and Lei Zhang. Deep convolutional dictionary learning for image denoising. In *IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pages 630–641, 2021.

[36] Xin Deng, Jingyi Xu, Fangyuan Gao, Xiancheng Sun, and Mai Xu. Deep M²cdl: Deep multi-scale multi-modal convolutional dictionary learning network. *IEEE transactions on pattern analysis and machine intelligence*, 46(5):2770–2787, 2023. 3

[37] Amir Beck and Marc Teboulle. A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems. *SIAM Journal on Imaging Sciences*, 2(1):183–202, January 2009. ISSN 1936-4954. doi: 10.1137/080716542. URL https://epubs.siam.org/doi/10.1137/080716542. 3

[38] Antonin Chambolle and Ch Dossal. On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. *Journal of Optimization theory and Applications*, 166: 968–982, 2015. 3

[39] Sharan Vaswani, Aaron Mishkin, Issam Laradji, Mark Schmidt, Gauthier Gidel, and Simon Lacoste-Julien. Painless stochastic gradient: Interpolation, line-search, and convergence rates. *Advances in neural information processing systems*, 32:3732–3745, 2019. 4

[40] Boris Iglewicz and David C Hoaglin. *Volume 16: how to detect and handle outliers*. Quality Press, 1993. 6

[41] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in PyTorch. 2017. 6

[42] Brendt Wohlberg. SPORCO: A Python package for standard and convolutional sparse representations. In *Proceedings of the 15th Python in Science Conference*, pages 1–8, Austin, TX, USA, July 2017. doi: 10.25080/shinma-7f4c6e7-001. 6

[43] Thomas Penzel, George B Moody, Roger G Mark, Ary L Goldberger, and J Hermann Peter. The apnea-ecg database. In *Computers in Cardiology 2000. Vol. 27 (Cat. 00CH37163)*, pages 255–258. IEEE, 2000. 9

[44] John Paparrizos, Yuhao Kang, Paul Boniol, Ruey S Tsay, Themis Palpanas, and Michael J Franklin. Tsb-uad: an end-to-end benchmark suite for univariate time-series anomaly detection. *Proceedings of the VLDB Endowment*, 15(8):1697–1711, 2022. 10

[45] David F Crouse. On implementing 2d rectangular assignment algorithms. *IEEE Transactions on Aerospace and Electronic Systems*, 52(4):1679–1696, 2016. 16
A Analytical study

**Proposition 2.1** (Stability of the common pattern). Consider a population of signals X composed of two patterns d_a and d_b, with activations such that the patterns do not overlap in the signal, and corrupted by an additive Gaussian noise $\epsilon \sim N(0, \sigma^2 I)$. Introduce $c = d_a^\top d_b$ and $\rho$ the proportion of rare-event pattern d_b activations in the population. Then, in the noiseless setting,

i. $d_a$ is a fixed point of the classical CDL algorithm for $K = 1$ if $c \le \lambda$

ii. $d_b$ is a fixed point of RoseCDL algorithm for $K = 1$ if $c \le \lambda$ or the RoseCDL algorithm is used with an outlier threshold trimming a proportion of windows greater than $\rho$.

*Proof.* We consider a dictionary $D \in \mathbb{R}^{1 \times L}$ with a single atom $\mathbf{d}$. As we consider signals composed of patterns with no overlap, we can separate each segment and we have a population of signals $X = zd_i + \epsilon$, with $z \in \mathbb{R}$, $\epsilon \sim N(0, \sigma^2 I)$ and $d_i = \mathbf{d}_a$ with probability $1 - \rho$ and $\mathbf{d}_b$ with probability $\rho$, with $\|\mathbf{d}_a\|_2 = \|\mathbf{d}_b\|_2 = 1$. We consider all atoms $\mathbf{d}$, $\mathbf{d}_a$, $\mathbf{d}_b$ to be unit norm. Wlog, we can consider $z = 1$, as this amounts to rescaling the value of $\lambda_{max}$, and we consider that $c_a = \mathbf{d}_a^\top \mathbf{d}_a$ and $c_b = \mathbf{d}_b^\top \mathbf{d}_b$ are positive, as we can consider $-\mathbf{d}$ otherwise. We also consider that the noise level is small enough such that $\sigma^2 < c_j$.

This model is a simplified model in which we have a population of signals where we want to identify the pattern of an event $d_a$ from the pattern of a rare event $d_b$.

In this setting, if we further have that the auto-correlation of $\mathbf{d}$ with $d_a$ and $d_b$ is maximal when they are aligned, then the sparse coding of a signal $X$ can be computed with the following formula:

$$z^*(X, \mathbf{d}) = \begin{cases} 0 & \text{if } c + \epsilon^\top \mathbf{d} \le \lambda \\ c + \epsilon^\top \mathbf{d} - \lambda & \text{otherwise} \end{cases} \quad (7)$$

with $c = \mathbf{d}_a^\top d_i$, which has value $c_a$ with probability $1 - \rho$ and $c_b$ otherwise.

We can compute the loss value for this $z^*(X, \mathbf{d})$ for $X$ where $z^*$ is non-zero:

$$F(\mathbf{d}, z^*; X) = \frac{1}{2} \|X - z^* \mathbf{d}\|_2^2 + \lambda \|z^*\|_1 \quad (8)$$

$$= \frac{1}{2} (\|X\|_2^2 - 2(c + \epsilon^\top \mathbf{d} - \lambda)(c + \epsilon^\top \mathbf{d}) + \|(c + \epsilon^\top \mathbf{d} - \lambda)\mathbf{d}\|_2^2) + \lambda|c + \epsilon^\top \mathbf{d} - \lambda| \quad (9)$$

$$= \frac{1}{2} (\|X\|_2^2 - 2(c + \epsilon^\top \mathbf{d} - \lambda)(c + \epsilon^\top \mathbf{d}) + (c + \epsilon^\top \mathbf{d} - \lambda)^2 + 2\lambda(c + \epsilon^\top \mathbf{d} - \lambda)) \quad (10)$$

$$= \frac{1}{2} (\|X\|_2^2 - 2(c + \epsilon^\top \mathbf{d} - \lambda)(c + \epsilon^\top \mathbf{d} - \lambda) + (c + \epsilon^\top \mathbf{d} - \lambda)^2) \quad (11)$$

$$= \frac{1}{2} (\|X\|_2^2 - (c + \epsilon^\top \mathbf{d} - \lambda)^2) \quad (12)$$

$$= \frac{1}{2} (\|d_i\|_2^2 - (c - \lambda)^2 + \|\epsilon\|_2^2 - (\epsilon^\top \mathbf{d})^2 - 2(1 - (c - \lambda))\epsilon^\top \mathbf{d}) \quad (13)$$

(14)

Taking the expectation over the noise yields:

$$\mathbb{E}_{\epsilon} [F(\mathbf{d}, z^*; X)] = \frac{1}{2} (1 - (c - \lambda)^2 + (L - 1)\sigma^2) \quad (15)$$

For $c$ between $\lambda$ and $1$, this function is decreasing in $c$, meaning that for two samples constructed with $d_a$ and $d_b$, if the correlation $c_0 = \mathbf{d}_a^\top d_a$ is larger than the correlation $c_b = \mathbf{d}_b^\top d_b$, then the reconstruction loss for sample 0 is smaller in expectation than the reconstruction loss for a sample 1.

We can also compute the gradient of this function with respect to $\mathbf{d}$. Note that with the KKT condition defining $z^*$, we have that $\nabla_z F(\mathbf{d}, z^*; X) = 0$, and thus we do not need to compute the Jacobian
<figure><img src="image_9.png" /><figcaption>Figure B.8: True dictionary in experiments on synthetic data.</figcaption></figure>

of $z^*$ when computing the derivative of $F$ with respect to $\mathbf{d}$. The gradient reads:

$$ \nabla_{\mathbf{d}} F(\mathbf{d}, z^*; X) = z^*(z^*\mathbf{d} - X) \quad (16) $$

$$ = (z^*)^2 \mathbf{d} - z^* X \quad (17) $$

$$ = (c + \epsilon^\top \mathbf{d} - \lambda)^2 \mathbf{d} - (c + \epsilon^\top \mathbf{d} - \lambda)(d_i + \epsilon) \quad (18) $$

Taking the expectation over the noise yields:

$$ \mathbb{E}_{\epsilon} [\nabla_{\mathbf{d}} F(\mathbf{d}, z^*; X)] = ((c - \lambda)^2 + \sigma^2) \mathbf{d} - (c - \lambda)d_i + \underbrace{\mathbb{E}_{\epsilon} [\epsilon^\top \mathbf{d} \epsilon]}_{\sigma^2 \mathbf{d}} \quad (19) $$

$$ = ((c - \lambda)^2 + 2\sigma^2) \mathbf{d} - (c - \lambda)d_i \quad (20) $$

This yields

$$ \mathbb{E}[\nabla_{\mathbf{d}} F(\mathbf{d}, z^*; X)] = ((1 - \rho)(c_a - \lambda)^2 + \rho(c_b - \lambda)^2 + 2\sigma^2) \mathbf{d} - (1 - \rho)(c_a - \lambda)\mathbf{d}_a - \rho(c_b - \lambda)\mathbf{d}_b $$

In the noiseless case, if $\mathbf{d} = \mathbf{d}_a$, and $\lambda \le c_b = (\mathbf{d}_b)^\top \mathbf{d}_a < 1$, with the classical algorithm, the expected gradient reads

$$ \mathbb{E}_X [\nabla_{\mathbf{d}} F(\mathbf{d}_a, z^*; X)] = -(1 - \rho)\lambda(1 - \lambda)\mathbf{d}_a + \rho((c_a - \lambda)^2 \mathbf{d}_a - (c_b - \lambda)\mathbf{d}_b) \quad (21) $$

$$ = (\rho(c_a - \lambda)^2 - (1 - \rho)\lambda(1 - \lambda)) \mathbf{d}_a - \rho(c_b - \lambda)\mathbf{d}_b \quad (22) $$

This gradient is not colinear with $\mathbf{d}_a$, showing that $\mathbf{d}_a$ is not a fixed point of the projected gradient descent algorithm in this context. Even in a noiseless and very simple setting, the $\mathbf{d}_a$ is not a solution of the Classical CDL algorithm.

In contrast, when using the least trimmed square procedure with a trimming threshold rejecting a proportion $\rho$ of the samples, we can show that $\mathbf{d}_a$ is a fixed point in the noiseless setting. As seen in (15), the loss for samples $X$ associated with $\mathbf{d}_b$ is smaller than the loss for samples associated with $\mathbf{d}_a$, and therefore rejecting $\rho$ samples from the gradient computation leads to:

$$ \mathbb{E}_X [\nabla_{\mathbf{d}} F(\mathbf{d}_a, z^*; X)] = -(1 - \rho)\lambda(1 - \lambda)\mathbf{d}_a \quad (23) $$

as the gradient is colinear with $\mathbf{d}_a$, thus $\mathbf{d}_a$ is a fixed point of the projected gradient descent and of the learning procedure. $\square$

## B Data simulation

The synthetic multivariate 1D signals $X \in \mathbb{R}^{P \times T}$ used in Sect. 3 are generated from a dictionary $D \in \mathbb{R}^{K \times P \times L}$, a sparse activation vector $Z \in \mathbb{R}^{K \times (T-L+1)}$, and a random Gaussian noise $\varepsilon \sim N(0, \sigma^2)$ as $X = D * Z + \varepsilon$. In this definition,

* $P$ is the number of channels,
* T is the length of the signal,

* K is the number of atoms,

* L is the length of the atoms.

In the experiments conducted in Sect. 3, we generated signals of length $T = 50 000$ with $P = 2$ channels from dictionaries with $K = 2$ atoms of length $L = 64$. The atoms were generated from sine and gaussian waveforms, as illustrated in Fig. B.8. The activations $Z$ were randomly generated sparse Dirac combs with sparsity 0.4 % and the noise level was set to $\sigma = 0.1$.

## C Dictionary evaluation

In our methodology, we evaluate the effectiveness of a learned dictionary, denoted as $\hat{D} \in \mathbb{R}^{K' \times P \times L'}$, by comparing it against a set of true dictionary patterns, represented as $D \in \mathbb{R}^{K \times P \times L}$ and computing a “recovery score”, using the convolutional cosine similarity following optimal assignment, as defined by [11]. The learned dictionary and the true patterns are structured as three-dimensional arrays, where dimensions correspond to the number of atoms, channels, and atoms’ duration. The learned dictionary may differ from the true dictionary in terms of the number of atoms and the length of time atoms, typically featuring more atoms and extended durations.

The evaluation process involves a computational step known as multi-channel correlation. In this step, each atom of the learned dictionary is systematically compared with each pattern in the true dictionary. This comparison is carried out channel by channel, aggregating the results to capture the overall similarity between the dictionary atom and the pattern.

After performing these comparisons for all combinations of atoms and patterns, we create a matrix that represents the correlation strengths between each pair. To objectively assess the quality of the learned dictionary, we use an optimization technique called the Hungarian algorithm. This algorithm finds the best possible “matching” between the learned dictionary atoms and the true patterns, aiming to maximize the overall correlation.

The final score, which quantifies the performance of the learned dictionary, is derived by averaging the values of these optimal matchings. This score is scaled between 0 and 1, where 1 represents the best possible performance. A higher score indicates that the learned dictionary more accurately represents the true dictionary patterns, providing a measure of its quality and effectiveness in capturing the essential features of the data.

Mathematically, the recovery score between the dictionaries $\hat{D}$ and $D$ can be expressed as follow:

$$ score = \frac{1}{K} \sum_{i=1}^{K} C_{i,j^*(i)}, \qquad (24) $$

where $j^*(i)$, $i = 1, \dots, K$ denote the results of the linear sum assignment problem [45]$^3$ on correlation matrix $C := \text{Corr}(D, \hat{D}) \in \mathbb{R}^{K \times K'}$, with $\forall i \in [1, K]$, $\forall j \in [1, K']$,

$$ C_{i,j} = \max_{l=1, \dots, L+L'-1} \text{Corr}_{2D}(D_i, \hat{D}_j)[l] \in \mathbb{R}, \qquad (25) $$

where $D_i \in \mathbb{R}^{P \times L}$ and $\hat{D}_j \in \mathbb{R}^{P \times L'}$. The multivariate “2D” correlation between the two matrices $D$ and $\hat{D}$ is defined as follow:

$$ \text{Corr}_{2D}(D, \hat{D}) = \sum_{p=1}^{P} \text{Corr}_{1D}(d_p, \hat{d}_p) \in \mathbb{R}^{L+L'-1}, \qquad (26) $$

where $d_p \in \mathbb{R}^L$ and $\hat{d}_p \in \mathbb{R}^{L'}$. The 1D “full” correlation between the two vectors $d$ and $\hat{d}$ is defined as follow, $\forall t \in [1, L+L'-1]$:

$$ \text{Corr}_{1D}(d, \hat{d})[t] = (d * \hat{d})[t-T+1] = \sum_{l=1}^{L} d[l]\hat{d}[l-t+T] \in \mathbb{R}, \qquad (27) $$

where $T := \max(L, L')$.

<footnote>$^3$We use the SciPy’s implementation.</footnote>
D Experiments setup

<table><thead><tr><th>Experiment</th><th>Data size</th><th>Number of runs</th><th>Hardware</th></tr></thead><tbody><tr><td>Runtime 1D</td><td>20 signals, 50,000 data points, 2 channels</td><td>50</td><td>GPU NVIDIA A40, 30 CPUs</td></tr><tr><td>Runtime 2D</td><td>2000 × 2000 grayscale images</td><td>20</td><td>GPU NVIDIA A40</td></tr><tr><td>Regularization impact 1D</td><td>10 signals, 30,000 data points, 1 channel</td><td>10</td><td>GPU NVIDIA A40</td></tr><tr><td>Regularization impact 2D</td><td>2000 × 2000 grayscale images</td><td>10</td><td>GPU NVIDIA A40</td></tr><tr><td>Inline vs After</td><td>2D image data</td><td>10</td><td>GPU NVIDIA A40</td></tr><tr><td>Physionet</td><td>10 trials of subject a02</td><td>10</td><td>GPU NVIDIA A40 and CPUs</td></tr></tbody></table>