# Deep Unrolled Architecture for Fast and Accurate Gaussian Independent Vector Analysis

Gaspard Blaise, Clément Cosserat, Emilie Chouzenoux, Jean-Christophe

Pesquet, Tülay Adali

► To cite this version:

Gaspard Blaise, Clément Cosserat, Emilie Chouzenoux, Jean-Christophe Pesquet, Tülay Adali. Deep Unrolled Architecture for Fast and Accurate Gaussian Independent Vector Analysis. SSP 2025 - IEEE Statistical Signal Processing Workshop, Jun 2025, Edimbourg (Ecosse), United Kingdom. hal-05056650

HAL Id: hal-05056650

https://inria.hal.science/hal-05056650v1

Submitted on 5 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Deep Unrolled Architecture for Fast and Accurate Gaussian Independent Vector Analysis

Gaspard Blaise, Clément Cosserat, Emilie Chouzenoux, Jean-Christophe Pesquet

CVN, CentraleSupélec, Inria Saclay, Univ. Paris-Saclay, Gif-sur-Yvette, France

Tülay Adali

UMBC, Baltimore, MD21250, USA

***Abstract***—Joint blind source separation (JBSS) is an inverse problem arising in engineering, particularly in medical imaging, where multiple signal datasets must be factorized simultaneously. A powerful approach to JBSS is Gaussian independent vector analysis (IVA-G), which models source datasets as independent Gaussian vectors and estimates both precision and demixing matrices. Recently, we introduced PALM-IVA-G, an iterative algorithm derived from the proximal alternating linearized minimization (PALM) framework, to solve IVA-G by minimizing a cost function derived from a maximum-likelihood estimator with provable convergence. However, its computational cost increases with the number of datasets and sources, and it requires careful hyperparameter tuning. To address these challenges, we propose U-PALM-IVA-G, an unrolled version of PALM-IVA-G that leverages deep unfolding to enhance efficiency. Experiments on six synthetic datasets of varying size and complexity demonstrate that U-PALM-IVA-G achieves significant speed improvements and enhanced solution quality compared to PALM-IVA-G.

***Index Terms***—Blind source separation, independent vector analysis, deep unrolling, proximal alternating algorithm.

## I. INTRODUCTION

The joint blind source separation (JBSS) [1] problem aims at simultaneously analyzing several datasets while leveraging the information they share. The goal is to extract latent features by factorizing data matrices as a product of mixing matrices and source matrices. JBSS problem has been successfully applied to brain imaging [2], [3], among many other application areas.

Independent vector analysis (IVA) [4] is a family of methods to tackle JBSS which introduces mutually independent random vectors called source component vectors (SCVs) to model the statistical dependence across the datasets. IVA methods typically seek to maximize the likelihood of the data to compute the demixing matrices, and finally use them to estimate the sources [3], [5]. In this paper, we assume the Gaussianity of SCVs, placing ourselves in the specific important case of IVA-G [6]. This version of IVA simply makes use of second-order statistics of the data, and minimizes the data negative log-likelihood with respect to both the demixing matrices and precision matrices of the sources.

The algorithm PALM-IVA-G recently introduced by the authors [7], [8], efficiently solves IVA-G problem using the PALM framework [9], and has provable convergence guarantees to a critical point of the cost function. It relies on a proximal alternating minimization scheme, iteratively updating the estimates for the precision matrices and the demixing matrices, through proximal gradient updates [10], [11], until a convergence criterion is met. Extensive experiments on synthetic datasets in [7] have shown that PALM-IVA-G yields

good results in terms of joint intersymbol interference (jISI) — which measures the correspondence between the mixing matrices and their estimates — but might have reduced efficiency when the problem dimensions grow. Moreover, as most advanced optimization methods, PALM-IVA-G requires the tuning of hyper-parameters which might be cumbersome.

In the last years, a new paradigm has emerged, for the resolution of signal/image inverse problems, based on the principle of deep unrolling (or unfolding) [12]–[14]. Starting from a sound iterative optimization method, its iterations are untied and re-interpreted as the layers of a neural network with an interpretable architecture. The hyper-parameters are learned in a supervised fashion, so as to optimize the performance on a task-oriented loss. Many recent works have demonstrated the great capability of deep unrolled approaches, with gains in computational time as well as performance [15]–[17]. In this paper, we propose an unrolling of PALM-IVA-G, called U-PALM-IVA-G. By untying the algorithm’s iterations, we allow a different set of hyper-parameters for each layer, learned so as to optimize the jISI loss function, aligning with the final goal of IVA-G. Our experimental results, compared with PALM-IVA-G, show improvements in terms of jISI score, and a considerable gain in computational execution, at test time.

The paper is organized as follows. In Section II, we formulate the IVA-G problem, and describe the PALM-IVA-G algorithm. In Section III, we present our unrolled architecture U-PALM-IVA-G and the training approach, and Section IV is dedicated to experimental results.

## II. PROBLEM PRESENTATION

### A. Notation

Matrices are represented by bold uppercase letters, while column or row vectors are denoted by bold lowercase letters. Tensors of order three or higher are indicated by calligraphic uppercase symbols, and scalars are shown using regular lowercase symbols. Deterministic quantities are written in italic font, whereas random quantities are presented in regular font. For any integer $N$, the subset of non-singular (resp. symmetric) matrices of $\mathbb{R}^{N \times N}$ is denoted $GL_N(\mathbb{R})$ (resp. $S_N$). We also denote $S_+^N$ the convex cone of positive semi-definite matrices of size $N$. Finally, we note $I_N$ the identity matrix of size $N$.

### B. Definition of IVA-G problem

Let $K, N, V$ be positive integers. For all $k \in \{1, \dots, K\}$, we are given the dataset $X^{[k]} \in \mathbb{R}^{N \times V}$, described by the mixing equations $X^{[k]} = A^{[k]}S^{[k]}$, where $A^{[k]} \in \mathbb{R}^{N \times N}$ is an unknown non-singular mixing matrix and $S^{[k]} \in \mathbb{R}^{N \times V}$ is a latent source matrix. The IVA assumption states that, for all $k \in \{1, \dots, K\}$, the $V$ columns of $S^{[k]}$ are independently sampled from an $\mathbb{R}^N$-valued random vector $s^{[k]} = [s_1^{[k]}, \dots, s_N^{[k]}]^T$. Consequently, the $K$ datasets can be statistically related to

G.B., C.C., and É.C. acknowledge support from the European Research Council Starting Grant MAJORIS ERC-2019-STG850925, and T.A. from the US National Science Foundation Grant NSF 2316420.
the latent vectors as follows: for all $n \in \{1, \dots, N\}$, the $\mathbb{R}^K$-valued random vector $\mathbf{s}_n = [\mathbf{s}_n^{[1]}, \dots, \mathbf{s}_n^{[K]}]^\top$, obtained by stacking the $n$-th component of each $\mathbf{s}^{[k]}$, has internal dependence, whereas the $\mathbf{s}_n$ are assumed to be mutually independent. The $(\mathbf{s}_n)_{1 \le n \le N}$ are called source component vectors (SCVs). In the specific context of IVA-G [6], also considered in this work, the SCVs are modeled as non-degenerate zero-mean Gaussian vectors. The generative model of the data is thus entirely determined by the mixing matrices and the covariance matrices of the SCVs, $(\Sigma_n)_{1 \le n \le N} \in \mathcal{GL}_K(\mathbb{R})^N$.

### C. The PALM-IVA-G approach

In [7], [8], we introduced the PALM-IVA-G method. In terms of modeling, the PALM-IVA-G aims to reconstruct sources from the data $\mathcal{X} = (\mathbf{X}^{[k]})_{1 \le k \le K} \in \mathbb{R}^{N \times V \times K}$ by jointly estimating the demixing matrices $\mathcal{W} = (\mathbf{W}^{[k]})_{1 \le k \le K} \in \mathbb{R}^{N \times N \times K}$ and the precision matrices $\mathcal{C} = (\mathbf{C}_n)_{1 \le n \le N} \in \mathbb{R}^{K \times K \times N}$ that approximate the inverses of the $\mathbf{A}^{[k]}$'s and the $\Sigma_n$'s, respectively. This estimation amounts to searching $(\mathcal{W}, \mathcal{C})$ that maximizes the likelihood of $\mathcal{X}$, that can be recast as the minimization of the following cost function:

$$ \mathcal{J}_{\text{IVA-G}}^{\text{Reg}}(\mathcal{W}, \mathcal{C}) = h(\mathcal{W}, \mathcal{C}) + f(\mathcal{W}) + g(\mathcal{C}), \quad (1) $$

with, for all $\mathcal{W} \in \mathbb{R}^{N \times N \times K}$ and $\mathcal{C} \in \mathbb{R}^{K \times K \times N}$,

$$ h(\mathcal{W}, \mathcal{C}) = \frac{1}{2} \sum_{n=1}^{N} \text{tr}(\mathbf{C}_n \mathbf{W}_n \hat{\mathbf{R}}_x \mathbf{W}_n^\top) + \alpha \| \text{diag}(\mathbf{C}_n) - \mathbf{1}_K \|^2, \quad (2) $$

$$ f(\mathcal{W}) = \begin{cases} - \sum_{k=1}^{K} \log |\det \mathbf{W}^{[k]}| & \text{if } (\forall k \in \{1, \dots, K\}) \mathbf{W}^{[k]} \in \mathcal{GL}_N(\mathbb{R}) \\ + \infty & \text{otherwise,} \end{cases} \quad (3) $$

$$ g(\mathcal{C}) = \begin{cases} -\frac{1}{2} \sum_{n=1}^{N} \log \det \mathbf{C}_n & \text{if } (\forall n \in \{1, \dots, N\}) \mathbf{C}_n - \epsilon \mathbf{I}_K \in \mathcal{S}_K^{+} \\ + \infty & \text{otherwise.} \end{cases} \quad (4) $$

In (2), we denote $\hat{\mathbf{R}}_x = \frac{1}{\sqrt{V}} \mathbf{X} \mathbf{X}^\top \in \mathbb{R}^{KN \times KN}$ the empirical covariance matrix of the data, gathered in the matrix

$$ \mathbf{X} = \begin{pmatrix} \mathbf{X}^{[1]} \\ \vdots \\ \mathbf{X}^{[K]} \end{pmatrix} \in \mathbb{R}^{KN \times V}. \quad \text{Note that the second term in} $$

(2) is a quadratic penalty that mitigates the scaling ambiguity of the model [6], [18] by promoting precision matrices with diagonal coefficients equal to 1, with penalty parameter $\alpha > 0$. Moreover, $\epsilon > 0$ in (4) is a constant ensuring the non-degeneracy of the precision matrices, typically taken very small.

Given the above model, the PALM-IVA-G algorithm was proposed in [7] to minimize function (1). The method relies on the methodology presented in [9], which takes advantage of the structure of $\mathcal{J}_{\text{IVA-G}}^{\text{Reg}}(\mathcal{W}, \mathcal{C})$. It consists of an iterative loop alternating proximal gradient updates of the variable blocks $(\mathcal{W}, \mathcal{C})$ with an essentially cyclic scheme [19], until convergence. PALM-IVA-G builds a sequence $(\mathcal{W}^{(i)}, \mathcal{C}^{(i)})_{i \in \mathbb{N}}$, following the main iteration described in Algorithm 1 below, where prox denotes the proximal operator as introduced in [10], and $(\nabla_W h, \nabla_C h)$ are the partial derivatives of the two-variables function $h$.

### Algorithm 1 PALM-IVA-G i-th iteration

1: $\mathcal{W}^{(i,0)} = \mathcal{W}^{(i)}$

2: For $j = 0, \dots, n_{\mathcal{W}}^{(i)} - 1$:

$$ \mathcal{W}^{(i,j+1)} = \text{prox}_{c_{\mathcal{W}}^{(i)} f} (\mathcal{W}^{(i,j)} - c_{\mathcal{W}}^{(i)} \nabla_{\mathcal{W}} h(\mathcal{W}^{(i,j)}, \mathcal{C}^{(i)})) $$

3: $\mathcal{W}^{(i+1)} = \mathcal{W}^{(i, n_{\mathcal{W}}^{(i)})}$

4: $\mathcal{C}^{(i+1)} = \text{prox}_{c_{\mathcal{C}}^{(i)} g} (\mathcal{C}^{(i)} - c_{\mathcal{C}}^{(i)} \nabla_C h(\mathcal{C}^{(i)}, \mathcal{W}^{(i+1)}))$

For every iteration $i \in \mathbb{N}$, PALM-IVA-G requires the setting of $n_{\mathcal{W}}^{(i)} > 0$, a positive integer that depends on an internal convergence criterion, the stepsizes $c_{\mathcal{W}}^{(i)} = \frac{\gamma_{\mathcal{W}}}{L_{\mathcal{W}}(\mathcal{C}^{(i)})}$ and $c_{\mathcal{C}}^{(i)} = \frac{\gamma_{\mathcal{C}}}{L_{\mathcal{C}}(\mathcal{W}^{(i+1)})}$, where $L_{\mathcal{W}}(\mathcal{C}^{(i)})$ and $L_{\mathcal{C}}(\mathcal{W}^{(i+1)})$ are the Lipschitz moduli of $\nabla_{\mathcal{W}} h(\cdot, \mathcal{C}^{(i)})$ and $\nabla_{\mathcal{C}} h(\mathcal{W}^{(i+1)}, \cdot)$, made explicit in [7, Sec.III], and $\gamma_{\mathcal{W}} \in (0, 1)$, $\gamma_{\mathcal{C}} \in (0, 2)$ are hyper-parameters. The PALM-IVA-G algorithm is such that $(\mathcal{W}^{(i)}, \mathcal{C}^{(i)})_{i \in \mathbb{N}}$ converges to a critical point of the non-convex function $\mathcal{J}_{\text{IVA-G}}^{\text{Reg}}(\mathcal{W}, \mathcal{C})$ [7, Theo.2].

### D. Performance assessment

The performance in an IVA task is measured through the jISI score [6], [20] between the generated ground truth mixing matrices $\mathcal{A} = [\mathbf{A}^{[1]}, \dots, \mathbf{A}^{[K]}]$ and the demixing matrices $\mathcal{W} = [\mathbf{W}^{[1]}, \dots, \mathbf{W}^{[K]}]$ that are recovered from the data $\mathcal{X}$ obtained by mixing the source datasets with the $(\mathbf{A}^{[k]})_{1 \le k \le K}$. For all $k \in \{1, \dots, K\}$, let us note $\mathbf{G}^{[k]} = \mathbf{W}^{[k]} \mathbf{A}^{[k]}$ and, for all $(m, n) \in \{1, \dots, N\}^2$, $\bar{g}_{n,m} = \sum_{k=1}^{K} |g_{n,m}^{[k]}|$. The jISI score of the pair $(\mathcal{W}, \mathcal{A})$ is

$$ \text{jISI}(\mathcal{W}, \mathcal{A}) = \frac{1}{2N(N-1)} \left[ \sum_{n=1}^{N} \left( \sum_{m=1}^{N} \frac{\bar{g}_{n,m}}{\max_p \bar{g}_{n,p}} - 1 \right) + \sum_{m=1}^{N} \left( \sum_{n=1}^{N} \frac{\bar{g}_{n,m}}{\max_p \bar{g}_{p,m}} - 1 \right) \right]. \quad (5) $$

This score is a real number between 0 and 1 with $\text{jISI}(\mathcal{W}, \mathcal{A}) = 0$ if and only if a perfect separation is achieved (up to a scale and permutation ambiguity).

PALM-IVA-G algorithm has been shown to achieve satisfying performance for source separation, on multiple synthetic datasets [7], compared to state-of-the-art methods. However, its computational cost increases significantly with the data dimensions. Furthermore, the setting of the stepsize hyper-parameters $(\gamma_{\mathcal{W}}, \gamma_{\mathcal{C}})$, and the penalty weight $\alpha$ impacts the algorithm behaviour. Finally, relying on a maximum likelihood estimation, this approach does not incorporate explicitly nor implicitly, any general prior knowledge one could get by inspecting a large quantity of available data. These drawbacks are addressed in this paper, by adopting the unrolling methodology, used for the first time in [21].

## III. PROPOSED U-PALM-IVA-G APPROACH

### A. Unrolled architecture

The deep unrolling paradigm consists in setting a number $L \in \mathbb{N} \setminus \{0\}$ of iterations of an iterative algorithm, and modeling each update operation as the layer of a neural network [14]. In the case of U-PALM-IVA-G (see Fig. 1), we set, $(\forall \ell \in \{0, \dots, L-1\})$ $(\mathcal{W}^{(\ell+1)}, \mathcal{C}^{(\ell+1)}, \hat{\mathbf{R}}_x) = \mathcal{L}^{(\ell)}(\mathcal{W}^{(\ell)}, \mathcal{C}^{(\ell)}, \hat{\mathbf{R}}_x)$. Instead of stopping the sequence when
a precision criterion is met, the result of the method is simply the output of the last layer of the network, given by

$$ (W^{(L)}, C^{(L)}, \hat{\mathbf{R}}_{\mathbf{x}}) = \mathcal{L}^{(L-1)} \circ \dots \circ \mathcal{L}^{(0)}(W^{(0)}, C^{(0)}, \hat{\mathbf{R}}_{\mathbf{x}}), \quad (6) $$

where we propose to intialize (i.e., define the network inputs) as $(\forall n \in \{1, \dots, N\}, C_n^{(0)} = I_K$ and $(\forall k \in \{1, \dots, K\}, W^{(0)[k]} = I_N$. The L layers $(\mathcal{L}^{(\ell)})_{0 \le \ell \le L-1}$ are parameterized by weights $\theta^{(\ell)} = (\alpha^{(\ell)}, \gamma_W^{(\ell)}, \gamma_C^{(\ell)})$, hence the proposed unrolled network is similar to PALM-IVA-G except that each layer is now seen as an independent operator rather than a single iterator applied many times within an algorithmic loop.

<figure><img src="image_2.png" /><figcaption>Fig. 1: U-PALM-IVA-G architecture</figcaption></figure>

Let us now describe the structure of each layer of the network. Following Algorithm 1, a layer $\mathcal{L}^{(\ell)}$ is composed of $n_W^{(\ell)}$ sublayers $(\tilde{\mathcal{L}}_W^{(\ell,j)})_{0 \le j \le n_W^{(\ell)-1}}$ accounting for the update of the block $W$, followed by a single sublayer $\tilde{\mathcal{L}}_C^{(\ell)}$ accounting for the update of $C$. Each $n_W^{(\ell)} \in \mathbb{N} \setminus \{0\}$ is a hyperparameter that we set all equal to $n_W = 10$. Thus, we have, for all $\ell \in \{0 \dots L-1\}$,

$$ \mathcal{L}^{(\ell)} = \tilde{\mathcal{L}}_C^{(\ell)} \circ \tilde{\mathcal{L}}_W^{(\ell, n_W-1)} \circ \dots \circ \tilde{\mathcal{L}}_W^{(\ell,0)} $$

with $\forall j \in \{1, \dots, \ell-1\}, \forall (W, C) \in \mathbb{R}^{N \times N \times K} \times \mathbb{R}^{K \times K \times N}$,

$$ \left\{ \begin{aligned} \mathcal{L}_W^{(\ell,j)}(W, C, \hat{\mathbf{R}}_x) &= \operatorname{prox}_{c_W^{(\ell)} f} (W - c_W^{(\ell)} \nabla_W h(W, C)) \\ \tilde{\mathcal{L}}_W^{(\ell,j)}(W, C, \hat{\mathbf{R}}_x) &= (\mathcal{L}_W^{(\ell,j)}(W, C, \hat{\mathbf{R}}_x), C, \hat{\mathbf{R}}_x), \\ \mathcal{L}_C^{(\ell)}(W, C, \hat{\mathbf{R}}_x) &= \operatorname{prox}_{c_C^{(\ell)} f} (C - c_C^{(\ell)} \nabla_C h(W, C)) \\ \tilde{\mathcal{L}}_C^{(\ell)}(W, C, \hat{\mathbf{R}}_x) &= (W, \mathcal{L}_C^{(\ell)}(W, C, \hat{\mathbf{R}}_x), \hat{\mathbf{R}}_x). \end{aligned} \right. $$

The stepsizes $(c_W^{(\ell)}, c_C^{(\ell)})$, computed following the formula in [7, Algorithm 1] depend on the learnable network parameters $\theta^{(\ell)}$. These relations are expressed concisely as: $\forall \ell \in \{1, \dots, L\}$,

$$ c_W^{(\ell)} = \mathcal{K}_W^{(\ell)}(C^{(\ell)}, \gamma_W^{(\ell)}, \hat{\mathbf{R}}_x) $$

$$ c_C^{(\ell)} = \mathcal{K}_C^{(\ell)}(\alpha^{(\ell)}, \gamma_C^{(\ell)}). $$

In order to control stability and preserve the network interpretability, we propose to set the parameters as

* $\alpha^{(\ell)} = \text{SoftPlus}(\tilde{\alpha}^{(\ell)}) = \log(1 + \exp(\tilde{\alpha}^{(\ell)}))$

* $\gamma_W^{(\ell)} = 0.3 + 5 \cdot (\tanh(\tilde{\gamma}_W^{(\ell)}) + 1)$

* $\gamma_C^{(\ell)} = 0.3 + 5 \cdot (\tanh(\tilde{\gamma}_C^{(\ell)}) + 1)$.

As such, the new real-valued parameters $(\tilde{\alpha}^{(\ell)}, \tilde{\gamma}_W^{(\ell)}, \tilde{\gamma}_C^{(\ell)})$ are learned, while the above expressions constrain $\alpha^{(\ell)}$ to positive

values via the SoftPlus activation, while $\gamma_W^{(\ell)}$ and $\gamma_C^{(\ell)}$ are mapped to the interval $[0.3, 10.3]$, so defining a reasonable value range for the stepsizes. The schematic description of a sublayer of U-PALM-IVA-G is provided in Fig. 2.

<figure><img src="image_3.png" /><figcaption>Fig. 2: Sublayer $\mathcal{L}^{(\ell)}$ of U-PALM-IVA-G</figcaption></figure>

## B. Training strategy

The deep unrolling paradigm assumes a supervised training of the parameters, relying on a dedicated, task-oriented, loss function. Here, we proposed to train U-PALM-IVA-G on a synthetically generated labeled dataset $D = \{(\mathcal{X}_p, A_p)\}_{1 \le p \le P}$ where for each $p$, $A_p$ contains randomly generated ground truth mixing matrices, and $\mathcal{X}_p$ contains datasets mixed from source datasets using $A_p$. The loss function used to train the network is the average, over index $p$ in the training set, of

$$ \mathcal{J}_{\text{U-PALM-IVA-G}} (W^{(0)}, C^{(0)}, \mathcal{X}_p, A_p) = \text{jISI}(W_p^{(L)}, A_p), \quad (7) $$

where $(W_p^{(L)}, C_p^{(L)}, \hat{\mathbf{R}}_{x,p})$ is the output of U-PALM-IVA-G for an input $(W^{(0)}, C^{(0)}, \hat{\mathbf{R}}_{x,p})$ and $\hat{\mathbf{R}}_{x,p}$ is the covariance matrix of the data $\mathcal{X}_p$.

# IV. NUMERICAL EXPERIMENTS

## A. Settings and datasets

For two problem sizes, namely $(N, K) \in \{(10, 10), (20, 20)\}$, we generate several synthetic datasets made of $P = 1200$ pairs $(\mathcal{X}_p, A_p)$. We assign 1000 of those pairs to the training set, 100 to the validation set, and 100 to the testing set. Each data pair $(\mathcal{X}, A)$ is obtained by drawing the entries of $\mathcal{A}$ independently from $\mathcal{N}(0, 1)$ (except otherwise stated), and $\mathcal{X}$ following the IVA-G model (see Sec. II-B) with ground truth covariance $\Sigma = [\Sigma_1, \dots, \Sigma_N]$. We set

$$ (\forall n \in \{1, \dots, N\}) \quad \Sigma_n = \rho_n \mathbf{11}^{\top} + \frac{\lambda}{R} \mathbf{Q} \mathbf{Q}^{\top} + \eta_n \mathbf{I}_K \quad (8) $$

with $(\rho_n, \eta_n, \lambda) \in [0, 1]^3$ such that $\rho_n + \eta_n + \lambda = 1$. Matrix $\mathbf{Q}$ in (8) is randomly sampled from $\mathbb{R}^{K \times R}$ with elements $q_{i,j} \sim \mathcal{N}(0, 1)$ being mutually independent, where $R$ is an integer that we took equal to $K + 10$. Parameters $\rho_n$ and $\lambda$ control the level of correlation of the $n$-th SCV between the datasets and the variability in the coefficients of $\Sigma_n$, respectively. We set $\rho_n = 0.2 + \frac{n-1}{N-1}0.5$, and we consider two scenarios for $\lambda$, namely the high variability scenario with $\lambda = 0.25$ which makes the source separation relatively easy [7] (Dataset $D_1$), and the low variability scenario, with $\lambda = 0.04$, making it harder (Dataset $D_2$). We also consider a third case where the mixing matrices in $\mathcal{A}$ also follow (8) with low variability settings. This scenario, yielding our Dataset $D_3$,
aims at verifying if the training can leverage a specific dataset structure.

All the computations were run on NVIDIA Tesla V100-SXM2 GPUs (32GB, CUDA 11.0, Driver 450.80.02) with Python 3.8.12 and PyTorch 1.11.

## B. Training and benchmarking

Several training strategies have been considered in the literature of deep unrolling, based for instance on greedy training, end-to-end training, or deep equilibrium. The number of layers $L$, set separately for each dataset, plays also a role in the obtained results. Here, we propose to leverage the possibility of tuning each layer independently to increase the space of possible settings and thus improve the practical performance, using an end-to-end training of the network. For each dataset and each $(K, N)$, we set the depth $L$ aiming to the smallest possible number such that the mean jISI cannot be further improved. The optimizer used in this training was Adam [22], with a learning rate of 0.01 and a batch size of 64. For all $\ell \in \{1, \dots, L\}$, $\alpha^{(\ell)}$, $\gamma_w^{(\ell)}$, and $\gamma_c^{(\ell)}$ are initialized with values close to 1.

Figure 3 displays an example of training and validation loss of U-PALM-IVA-G, showing good stability and absence of overfitting. Here, we stopped training the weights after 30 epochs.

<figure><img src="image_4.png" /><figcaption>Fig. 3: $\mathcal{D}_3$ and $(K, N) = (10, 10)$: The training and validation losses decrease sharply and stabilize around a jISI of $10^{-2}$ after about 15 epochs.</figcaption></figure>

Once $L$ is fixed and U-PALM-IVA-G is trained, we compare the mean and standard deviation of its jISI score $\mu_{jISI} \pm \sigma_{jISI}$, and mean computational time $\mu_T$, with the iterative version PALM-IVA-G (run until convergence), averaged on the testing set (i.e., 100 runs). To assess the benefits of untying the weights, we also include in our benchmark the results when training a tied version of the network called U-PALM-IVA-G-tied, where the weights are constrained to stay identical across layers.

## C. Results and Analysis

The results in Table I illustrate the acceleration, in inference time, of U-PALM-IVA-G (tied or untied), over PALM-IVA-G, with a gain of a factor 10 to 20 in the most challenging scenarios. Moreover, the speed-up increases with the dimension, showing great potential for applications on real datasets. U-PALM-IVA-G also yields a slight and sometimes significant improvement of the jISI score, compared to PALM-IVA-G and U-PALM-IVA-G-tied, showing the advantage of supervised learning of adaptive step sizes and regularization weights along layers. For the low variability scenario $(\mathcal{D}_2)$ with $(K, N) = (20, 20)$, the untied U-PALM-IVA-G achieves an improved, and also more stable jISI score, with standard deviations reduced by a factor 2 (resp. 5), compared to PALM-IVA-G (resp. U-PALM-IVA-G-tied).

<table><caption>TABLE I: Performance results for proposed U-PALM-IVA-G and the benchmarks. For $(K, N) = (10, 10)$, we set $L = 50$ for $\mathcal{D}_1$, $L = 100$ for $\mathcal{D}_2$, and $L = 50$ for $\mathcal{D}_3$. For $(K, N) = (20, 20)$, we set $L = 100$ for $\mathcal{D}_1$, $L = 120$ for $\mathcal{D}_2$, and $L = 90$ for $\mathcal{D}_3$.</caption><thead><tr><th rowspan="2">Dataset</th><th rowspan="2">Algorithm</th><th colspan="2">(K, N) = (10, 10)</th><th colspan="2">(K, N) = (20, 20)</th></tr><tr><th>μ<sub>jISI</sub> ± σ<sub>jISI</sub></th><th>μ<sub>T</sub>(s)</th><th>μ<sub>jISI</sub> ± σ<sub>jISI</sub></th><th>μ<sub>T</sub>(s)</th></tr></thead><tbody><tr><td rowspan="3">D<sub>1</sub></td><td>PALM-IVA-G</td><td>8.45E-03 ± 0.21E-03</td><td>3.52</td><td>7.01E-03 ± 0.11E-03</td><td>7.79</td></tr><tr><td>U-PALM-IVA-G-tied</td><td>9.41E-03 ± 0.23E-03</td><td><b>0.54</b></td><td>8.06E-03 ± 0.53E-03</td><td>1.13</td></tr><tr><td>U-PALM-IVA-G</td><td><b>8.41E-03 ± 0.12E-03</b></td><td><b>0.54</b></td><td><b>6.97E-03 ± 0.45E-03</b></td><td><b>1.05</b></td></tr><tr><td rowspan="3">D<sub>2</sub></td><td>PALM-IVA-G</td><td>2.36E-02 ± 0.96E-02</td><td>11.45</td><td>2.58E-02 ± 0.42E-02</td><td>29.77</td></tr><tr><td>U-PALM-IVA-G-tied</td><td>2.45E-02 ± 0.81E-02</td><td><b>1.04</b></td><td>2.95E-02 ± 0.20E-02</td><td>1.46</td></tr><tr><td>U-PALM-IVA-G</td><td><b>2.25E-02 ± 0.51E-02</b></td><td>1.06</td><td><b>2.36E-02 ± 0.12E-02</b></td><td><b>1.39</b></td></tr><tr><td rowspan="3">D<sub>3</sub></td><td>PALM-IVA-G</td><td>8.58E-03 ± 0.50E-03</td><td>3.78</td><td>7.61E-03 ± 0.60E-03</td><td>7.43</td></tr><tr><td>U-PALM-IVA-G-tied</td><td>8.84E-03 ± 0.81E-03</td><td>0.45</td><td>8.18E-03 ± 0.73E-03</td><td>0.98</td></tr><tr><td>U-PALM-IVA-G</td><td><b>8.51E-03 ± 0.51E-03</b></td><td><b>0.44</b></td><td><b>7.40E-03 ± 0.41E-03</b></td><td><b>0.96</b></td></tr></tbody></table>

<figure><img src="image_5.png" /><figcaption>Fig. 4: Dataset $\mathcal{D}_3$ with $(K, N) = (10, 10)$: U-PALM-IVA-G learned parameters across layers.</figcaption></figure>

## D. Evolution of the weights

Figure 4 illustrates the evolution of the learned parameters $\gamma_w$, $\gamma_c$, and $\alpha$ across the layers. The parameters $\gamma_w$ and $\gamma_c$, which are linked to the step sizes of the updates, increase rapidly in the first layers before stabilizing after approximately 20 layers. This suggests that U-PALM-IVA-G adaptively tunes the step size to boost convergence speed near the limit. The regularization parameter $\alpha$, which controls the strength of the penalty on the precision matrices, initially decreases before stabilizing around 0.68. This indicates that strong regularization might be necessary in the early layers to mitigate scaling ambiguity, while the later layers benefit from more flexibility.

## V. CONCLUSION

In this paper, we have presented U-PALM-IVA-G, an unrolled deep architecture that enhances PALM-IVA-G for joint blind source separation. Using deep unrolling, this approach automates hyperparameter tuning and significantly reduce computational time. Experiments demonstrate improved jISI scores, including if the mixing matrices have a special structure, showing the superiority of the proposed approach over the basic iterative method. The results also highlight the benefits of adaptive step sizes and layer-specific parameters, ensuring better convergence and stability. Future work may extend this methodology to more sophisticated neural architectures, and explore its application to real-world datasets, particularly in medical imaging, where JBSS methods have proven particularly useful.
REFERENCES

[1] C. Jutten and J. Herault, "Blind separation of sources, part i: An adaptive algorithm based on neuromimetic architecture," *Signal Processing*, vol. 24, no. 1, pp. 1-10, 1991.

[2] T. Adali, F. Kantar, M. A. B. S. Akhonda, S. Strother, V. D. Calhoun, and E. Acar, "Reproducibility in Matrix and Tensor Decompositions: Focus on Model Match, Interpretability, and Uniqueness," *IEEE Signal Processing Magazine*, vol. 39, no. 4, pp. 8-24, Jul. 2022.

[3] T. Adali, M. Anderson, and G.-S. Fu, "Diversity in independent component and vector analyses: Identifiability, algorithms, and applications in medical imaging," *IEEE Signal Processing Magazine*, vol. 31, no. 3, pp. 18-33, May 2014.

[4] T. Kim, I. Lee, and T.-W. Lee, "Independent vector analysis: Definition and algorithms," in *Proceedings of the 40th Asilomar Conference on Signals, Systems, and Computers (ASILOMAR 2006)*, 2006, pp. 1393-1396.

[5] M. Anderson, G.-S. Fu, R. Phlypo, and T. Adali, "Independent vector analysis: Identification conditions and performance bounds," *IEEE Transactions on Signal Processing*, vol. 62, no. 17, pp. 4399-4410, 2014.

[6] M. Anderson, T. Adali, and X.-L. Li, "Joint Blind Source Separation With Multivariate Gaussian Model: Algorithms and Performance Analysis," *IEEE Transactions on Signal Processing*, vol. 60, no. 4, pp. 1672-1683, Apr. 2012.

[7] C. Cosserat, B. Gabrielson, E. Chouzenoux, J.-C. Pesquet, J.-C. Pesquet, and T. Adali, "An Effective Iterative Solution for Independent Vector Analysis with Convergence Guarantees," Tech. Rep., Nov. 2024, arviv 2411.12314.

[8] C. Cosserat, B. Gabrielson, E. Chouzenoux, J.-C. Pesquet, and T. Adali, "A proximal approach to IVA-G with convergence guarantees," in *Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023)*, 2023, pp. 1-5.

[9] J. Bolte, S. Sabach, and M. Teboulle, "Proximal alternating linearized minimization for nonconvex and nonsmooth problems," *Mathematical Programming*, vol. 146, no. 1, pp. 459-494, 2014.

[10] P. Combettes and J.-C. Pesquet, *Proximal Splitting Methods in Signal Processing*. Springer-Verlag, New York, 2010, pp. 185-212.

[11] P. Combettes and V. Wajs, "Signal recovery by proximal forward-backward splitting," *Multiscale Modeling and Simulation*, vol. 4, no. 4, pp. 1168-1200, 2005.

[12] V. Monga, Y. Li, and Y. C. Eldar, "Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing," *IEEE Signal Processing Magazine*, vol. 38, no. 2, pp. 18-44, 2021.

[13] M.-C. Corbineau, C. Bertocchi, E. Chouzenoux, M. Prato, and J.-C. Pesquet, "Learned image deblurring by unfolding a proximal interior point algorithm," in *Proceedings of the IEEE International Conference on Image Processing (ICIP 2019)*, 2019, pp. 4664-4668.

[14] J. R. Hershey, J. L. Roux, and F. Weninger, "Deep unfolding: Model-based inspiration of novel deep architectures," *arXiv preprint arXiv:1409.2574*, 2014.

[15] M. Gharbi, S. Villa, E. Chouzenoux, J.-C. Pesquet, and L. Duval, "Unrolled deep networks for sparse signal restoration in analytical chemistry," in *Proceedings of the 34th IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2024)*, 2024, pp. 1-6.

[16] M. Savanier, E. Chouzenoux, J.-C. Pesquet, and C. Riddell, "Deep unfolding of the DBFB algorithm with application to roi ct imaging with limited angular density," *IEEE Transactions on Computational Imaging*, vol. 9, pp. 502-516, 2023.

[17] M. Nguyen and P. Weiss, "Comparing plug-and-play and unrolled networks," Tech. Rep., 2024, hal-04703008.

[18] P. Comon, "Independent component analysis, a new concept?" *Signal Processing*, vol. 36, no. 3, pp. 287-314, 1994.

[19] E. Chouzenoux, J.-C. Pesquet, and A. Repetti, "A block coordinate variable metric forward-backward algorithm," *Journal of Global Optimization*, vol. 66, no. 3, pp. 457-485, 2016.

[20] S.-i. Amari, A. Cichocki, and H. Yang, "A New Learning Algorithm for Blind Signal Separation," in *Advances in Neural Information Processing Systems*, vol. 8. MIT Press, 1995.

[21] K. Gregor and Y. LeCun, "Learning fast approximations of sparse coding," in *Proceedings of the 27th international conference on international conference on machine learning*, 2010, pp. 399-406.

[22] D. P. Kingma, "Adam: A method for stochastic optimization," *arXiv preprint arXiv:1412.6980*, 2014.