# Deep Equilibrium for Hyperparameter Estimation in Dynamic PET Reconstruction

Alix Chazottes, Emilie Chouzenoux, Jean-Christophe Pesquet, Florent Sureau

► To cite this version:

Alix Chazottes, Emilie Chouzenoux, Jean-Christophe Pesquet, Florent Sureau. Deep Equilibrium for Hyperparameter Estimation in Dynamic PET Reconstruction. MIC 2024 - IEEE Medical Imaging Conference, Oct 2024, Tampa (Florida), United States. 10.1109/NSS/MIC/RTSD57108.2024.10657967. [hal-04707560](https://HAL Id: hal-04707560)

https://inria.hal.science/hal-04707560v1

Submitted on 24 Sep 2024

**HAL** is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Deep Equilibrium for Hyperparameter Estimation in Dynamic PET Reconstruction

Alix Chazottes, Émilie Chouzenoux, Jean-Christophe Pesquet, and Florent Sureau

**Abstract**—Positron Emission Tomography (PET) is a functional quantitative medical imaging modality with main applications in oncology and treatment of neurodegenerative diseases. The reconstruction step solves a tomographic ill-posed inverse problem with Poisson noise, with particularly high intensity for dynamic protocols. The injected dose, radioactive tracer distribution, and noise level, are also varying across the exams, which requires to devise a sophisticated regularization scheme adapted to the protocol. In this work, we focus on constructing a parametric model of the dose-dependence with a large number of hyperparameters in regularized dynamic PET reconstruction. Our objective is to emphasize the interpretability of the learned parameters. Classical grid search becomes inefficient to explore such a high-dimensional space of hyperparameters. This work investigates a new approach based on the Deep Equilibrium (DEQ) framework. Starting from an iterative algorithm, DEQ performs learning on the fixed point of this algorithm seen as a (deep) sequence of parameterized iterations. The dose-dependence model is then built from a series of DEQ estimates obtained for a constant number of detected events. The proposed method is validated on a synthetic 2D+t database derived from acquired data of a dynamic PET protocol.

**Index Terms**—PET image reconstruction, dynamic PET, Deep Equilibrium

## I. INTRODUCTION

POSITRON Emission Tomography (PET) is a functional and quantitative imaging modality based on injecting a radiotracer to a patient to target a specific molecular pathway, with useful applications in oncology and in neurodegenerative diseases study. Reconstruction of images of this radiotracer distribution involves solving a tomographic ill-posed inverse problems with Poisson noise. In dynamic PET, the temporal evolution of the radiotracer distribution is estimated to derive physiological information. The exam is typically divided into several time frames to reconstruct, each with a low number of detected events. The reconstruction problem is particularly challenging as the signal to noise ratio is low, varies across frames and across exams (in particular due to the injected dose). This raises the need for constructing complex spatio-temporal regularizers adapted to the dynamic protocol.

Deep Learning techniques have shown great potential in medical imaging for a large number of tasks, including image reconstruction [5]. In particular, hybrid reconstruction techniques that combine learnt deep regularization operators and physics-informed direct models have been explored to provide accurate and explainable results.

When it comes to dynamic PET, various hybrid deep learning approaches were already proposed for reconstruction

[2]. This work investigates an alternative approach, namely the construction of a parametric model to capture the dose dependence of "optimal" hyperparameters of a spatio-temporal regularization. This is achieved by estimating hyperparameters at convergence for a given number of detected events. The model parameters are then estimated by varying the number of detected events. For the first step, estimating the relatively large number of "optimal" hyperparameters using a grid-search would be inefficient; we rather propose to use the framework of Deep Equilibrium (DEQ) models, developed for sequential data [3]. To the best of our knowledge, such models have not been used for dynamic PET reconstruction. We describe the bi-level problem for hyperparameter optimization in Section II, the parametric model and the data used on Section III, and present our results on Section IV.

## II. NOTATION AND BACKGROUND

### A. Regularized Dynamic PET reconstruction

We denote by $x \in \mathbb{R}^{NT} = (x_t)_{1 \le t \le T}$ a sequence of $T$ images in time, with associated acquired data $y \in \mathbb{R}^{MT} = (y_t)_{1 \le t \le T}$. Following the approach in [1], the dynamic PET reconstruction problem can then be formulated as

$$ \underset{x \in \mathbb{R}_+^{NT}}{\text{minimize}} \quad f(x) = L(x, y) + \eta \Phi(x) + \sum_{t=1}^{T} \lambda_t \Psi(x_t, \epsilon), \quad (1) $$

where $L$ is the Poisson negative log-likelihood and $\Phi$ (resp. $\Psi$) is a temporal (resp. spatial) regularization term which incorporates some prior knowledge on the images. These regularization terms are chosen as $\Phi(x) = \frac{1}{2} \sum_{t=2}^{T} \|x_t - x_{t-1}\|^2$ and $\Psi(x_t, \epsilon) = \sum_{n=1}^{N} \sqrt{([D_h x_t]_n)^2 + ([D_v x_t]_n)^2 + \epsilon^2}$ with $D_h \in \mathbb{R}^{N \times N}$ (resp. $D_v \in \mathbb{R}^{N \times N}$) the operator of horizontal (resp. vertical) discrete gradient. For a given dose, the hyper-parameters to estimate are the temporal parameter $\eta > 0$, the spatial parameters $\lambda_t > 0$ for each frame $t \in \{1, \dots, T\}$, and $\epsilon > 0$ a smoothing hyperparameter.

### B. Estimation of Regularization Hyperparameters

The estimation of hyperparameters $\eta$, $\epsilon$, and $\lambda = (\lambda_t)_{1 \le t \le T}$ in (1) with $T$ typically large (e.g., $T = 20$ in the dynamic protocol considered here), dependent on the injected dose $D$, can be formulated as a supervised learning problem:

$$ \underset{\eta_D \in \mathbb{R}_+^*, \epsilon_D \in \mathbb{R}_+^*, \lambda_D \in \mathbb{R}_+^T}{\text{minimize}} \quad \|\hat{x}(\eta_D, \epsilon_D, \lambda_D) - x\|^2, \quad (2) $$

where $x$ is a reference low-noise image and $\hat{x}(\eta, \epsilon, \lambda)$ a reconstructed image obtained by solving (1) with hyperparameters $(\eta, \epsilon, \lambda)$.

<footnote>A.C., E.C. and J-C.P. are with the Center for Numerical Vision, Centrale-Supélec, Université Paris-Saclay, Inria, Gif-sur-Yvette, France.</footnote>

<footnote>A.C. and F.S. are with BioMaps, Université Paris-Saclay, Orsay, France.</footnote>
# III. MATERIALS AND METHODS

## A. Proposed approach

To solve Problem (2), we trained a DEQ model [3], to learn the regularization hyperparameters, for a patient, and a constant number of detected coincidences corrected by the number of background events (given *D*). We then repeat this procedure for different values of *D* (taking into account proper scaling of background events). As a final step, we perform a fitting of the evolution of the estimated hyperparameters, as a function of *D*, with a parametric model.

The forward pass requires to solve (1) for a given hyperparameter set ($η, ε, λ$). To do so, we followed the majorization-minimization approach, using a logarithmic majorant surrogate derived from [1, Example 3]. This choice is theoretically expected not to impact the study [3]. The backward pass is performed using PyTorch auto-differentiator.

## B. Dataset

A synthetic dynamic database was derived from a 1-hour exam (decomposed into 5 × 60s, 5 × 120s, 2 × 150s and 8 × 300s frames) on 18 healthy volunteers injected with ¹¹C-PE2I, a selective dopamine transporter radioligand. The approach described in [6] was applied to the original time series to produce a series of dynamic piecewise constants phantoms based on PET measurements on 100 regions of interests. PET simulations for a Biograph 6 TruePoint TrueV PET system were then generated using an analytical simulator, by varying the total number of detections during the exam [4] in the same range as in the original database. The reference image in (2) was chosen as the noise-free phantom.

## C. Training the Deep Equilibrium model

For this study, the hyperparameters were learnt for a single PET exam and a single axial slice at a time (2D+t data). For a given PET image and a chosen dose, 96 noise realizations where generated and then split into a training and validation set of respectively 80 and 16 realizations. The loss function was chosen as the Mean Squared Error (MSE) between the original image and the reconstructed image, when solving (1) with the hyperparameter set. The smoothing hyperparameter $ε$ was fixed equals to 0.1, and we focus on learning $η$ and $λ$.

The DEQ model was implemented following the methodology described in [3], adapting the provided code and using the Anderson acceleration for fixed point iteration with usual convergence criteria based on relative difference (or maximum number of 500 iterations, not reached in practice). The model was trained on 50 epochs (in batches of 16) using the Adam optimizer, with a scheduled decreasing learning rate.

# IV. RESULTS

## A. DEQ approach compared to usual estimation methods

Table I shows some examples of final MSE values for the DEQ model, compared to two state-of-the-art hyperparameter estimation approaches where $λ$ was set as a scalar (i.e., not varying along time frame). The DEQ approach yields a much better reconstruction quality in the same computational time,

which is visible on the reconstructed images (see Figure 1), especially on frames with a low signal-to-noise ratio.

<table><caption>TABLE I: Comparison of the MSE for different estimation methods</caption><thead><tr><th></th><th>Learnt<br>params</th><th>Patient 1<br>Dose 1</th><th>Patient 1<br>Dose 2</th><th>Patient 2<br>Dose 1</th><th>Patient 2<br>Dose 2</th></tr></thead><tbody><tr><td>DEQ</td><td>21</td><td>5.26 × 10<sup>4</sup></td><td>2.38 × 10<sup>5</sup></td><td>4.03 × 10<sup>4</sup></td><td>2.03 × 10<sup>5</sup></td></tr><tr><td>DEQ</td><td>2</td><td>5.33 × 10<sup>4</sup></td><td>2.58 × 10<sup>5</sup></td><td>4.48 × 10<sup>4</sup></td><td>2.14 × 10<sup>5</sup></td></tr><tr><td>Grid search</td><td>2</td><td>5.33 × 10<sup>4</sup></td><td>2.58 × 10<sup>5</sup></td><td>4.51 × 10<sup>4</sup></td><td>2.18 × 10<sup>5</sup></td></tr><tr><td>TPE</td><td>2</td><td>5.38 × 10<sup>4</sup></td><td>3.94 × 10<sup>5</sup></td><td>4.51 × 10<sup>4</sup></td><td>3.79 × 10<sup>5</sup></td></tr></tbody></table>

Estimations based on Grid Search and a Tree Parzen Estimator (TPE) sampler (Optuna) set to have equivalent computational times as the DEQ method.

<figure><img src="image_2.png" /><figcaption>Fig. 1: Comparison of reference image (a) and absolute reconstruction errors for 3 different hyperparameter estimation methods (normalized colorscale).</figcaption></figure>

## B. Robust regression for $η$ and $λ$

Figure 2 shows an example of fitting of the learned value of $η_D$ for 6 doses, using four different parametric models. The double exponential model yielded the best fit for each patient with $R^2$ scores over 0.98. Interestingly, regression coefficients were found to be patient-robust. Double exponential models were also the best fit for the $λ_D$ dependency to dose.

<figure><img src="image_3.png" /><figcaption>Fig. 2: Example of fitting of the learned value $η_D$ for 6 dose values $D \in \{150, 250, 300, 400, 500, 750\}$ million of counts.</figcaption></figure>

# V. CONCLUSION

This work proposed the first use of the Deep Equilibrium framework to the estimation of a high number of hyperparameters in an iterative regularized TEP reconstruction approach. A generic model for the dose-dependent hyperparameters was learned, that could be applied for subsequent inference tasks. These promising results open a broad range of perspectives in the context of low-dose dynamic medical imaging.

# REFERENCES

[1] C. Rossignol et al., doi: 10.1109/ICIP46576.2022.9897619, 2022.

[2] K. Gong et al. doi: 10.1109/TMI.2021.3120913, 2022.

[3] S. Bai et al. doi: 10.48550/arXiv.1909.01377, 2019.

[4] C. Stute et al. doi: 10.1109/NSSMIC.2015.7582064, 2015.

[5] M. Savanier et al. doi: 10.1109/TCI.2023.3279053, 2023.

[6] F. Sureau et al. doi: 10.48550/arXiv.2310.04299, 2023.