# Implicit neural representations for end-to-end pet reconstruction

Younès Moussaoui, Diana Mateus, Nasrin Taheri, Saïd Moussaoui, Thomas Carlier, Simon Stute

► To cite this version:

Younès Moussaoui, Diana Mateus, Nasrin Taheri, Saïd Moussaoui, Thomas Carlier, et al. Implicit neural representations for end-to-end pet reconstruction. IEEE International Symposium on Biomedical Imaging, Apr 2025, Houston (Texas), United States. hal-04999198

HAL Id: hal-04999198

https://hal.science/hal-04999198v1

Submitted on 20 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
IMPLICIT NEURAL REPRESENTATIONS FOR END-TO-END PET RECONSTRUCTION

Younès Moussaoui¹,², Diana Mateus¹, Nasrin Taheri ²,³, Saïd Moussaoui¹,
Thomas Carlier²,³, Simon Stute ²,³

¹Nantes Université, École Centrale Nantes, LS2N, CNRS, UMR 6004, F-44300 Nantes, France

²Dépt. de Médecine Nucléaire, Centre Hospitalier Universitaire de Nantes, F-44000 Nantes, France

³CRCI2NA, INSERM, CNRS, Université d'Angers, Université de Nantes, F-44000 Nantes, France

ABSTRACT

Implicit neural representations (INRs) have demonstrated
strong capabilities in various medical imaging tasks, such as
denoising, registration, and segmentation, by representing im-
ages as continuous functions, allowing complex details to be
captured. For image reconstruction problems, INRs can also
reduce artifacts typically introduced by conventional recon-
struction algorithms. However, to the best of our knowledge,
INRs have not been studied in the context of PET recon-
struction. In this paper, we propose an unsupervised PET
image reconstruction method based on the implicit SIREN
neural network architecture using sinusoidal activation func-
tions. Our method incorporates a forward projection model
and a loss function adapted to perform PET image recon-
struction directly from sinograms, without the need for large
training datasets. The performance of the proposed approach
was compared with that of conventional penalized likelihood
methods and deep image prior (DIP) based reconstruction us-
ing brain phantom data and realistically simulated sinograms.
The results show that the INR-based approach can reconstruct
high-quality images with a simpler, more efficient model, of-
fering improvements in PET image reconstruction, particu-
larly in terms of contrast, activity recovery, and relative bias.

***Index Terms***— Implicit neural network, PET imaging, image reconstruction, SIREN

# 1. INTRODUCTION

Positron Emission Tomography (PET) is a non-invasive imag-
ing modality widely used in medical diagnostics since it
provides detailed metabolic and functional information, and
offers complementary insights beyond traditional anatomical
imaging. However, the raw data (sinograms) collected by
PET scanners requires adequate reconstruction algorithms to
produce meaningful images. Traditional PET reconstruction

methods rely heavily on iterative algorithms such as maxi-
mum likelihood expectation maximization (MLEM), which
minimizes the discrepancy between the acquired sinogram
and the predicted sinogram generated according to the acqui-
sition (projection) model applied to the estimated image [1].
Although effective, these methods can suffer from slow con-
vergence and noise amplification, particularly in low-count
scenarios where the signal-to-noise ratio (SNR) is low. To
alleviate these issues, post-processing techniques are often
used to enhance the reconstructed image quality [2]. For that
purpose, various regularization techniques have been pro-
posed and adopted to preserve image details and reduce noise
impact, through penalized maximum likelihood (PML) meth-
ods, such as the block Sequential Regularized Expectation
Maximization (BSREM) [3].

Recently, several deep learning approaches have emerged for PET image reconstruction. Zhu et al. [4] introduced AUTOMAP, a deep learning-based method that reconstructs high-quality PET images from sinograms, but it requires large datasets for training. This data-driven approach relies on a learned mapping from sinograms to images. Therefore, the adaptation of the resulting model to different acquisition protocols is not trivial. In contrast to data priors, methods like the deep image prior (DIP) rely on the deep network structure itself as a prior. Originally proposed by Ulyanov et al. [5], DIP fits the network weights to a single image, thus avoiding the need for training data. DIP has been successfully applied to low-dose PET denoising by leveraging the structure of a U-Net-like network as implicit prior knowledge [6]. DIP relies on the network to parameterize the image representation. In practice, DIP denoising takes advantage of CNNs' spectral bias, where these networks tend to capture low-frequency image components (such as smooth structures) before high-frequency details (like fine edges or noise), acting as inherent denoisers [7]. DIP can be used either as a post-processing step or integrated into iterative optimization updates for end-to-end reconstruction. Hashimoto et al. [8] extended this scheme by incorporating the forward projection into a unified DIP-based PET reconstruction. End-to-end approaches have the advantage of jointly optimizing both image quality and noise

© 2025 IEEE. Personal use of this material is permitted. Permission
from IEEE must be obtained for all other uses, in any current or future media,
including reprinting/republishing this material for advertising or promotional
purposes, creating new collective works, for resale or redistribution to servers
or lists, or reuse of any copyrighted component of this work in other works.
reduction.

Lately, Implicit Neural Representations (INRs), also known as neural fields or coordinate-based neural representations [9], have gained attention in image processing mainly induced by their ability to represent complex and high-frequency functions. These networks learn continuous representations of functions that can be evaluated at arbitrary coordinates, making them well suited to high-detail tasks such as 3D shape representation, in the context of biomedical images, for registration, segmentation [10] and reconstruction [9].

In addition to their continuous nature, INRs are inherently differentiable, allowing them to be seamlessly integrated into end-to-end reconstruction algorithms to optimize both image quality and accuracy. For instance, Sitzmann et al. [11] introduced sinusoidal representation networks (SIREN), using a periodic activation function to represent high-frequency contents, and its efficiency has been proven in the application to medical imaging by CT reconstruction [12].

In this work, we explore the application of implicit neural networks in the context of PET image reconstruction by proposing suitable adaptations. Moreover, based on realistic data sets, the effectiveness of the resulting reconstruction approach is proven by comparing it to conventional methods and to a DIP-based strategy, in terms of estimated image content and quality.

# 2. METHODS

## 2.1. End-to-end PET reconstruction

The proposed 2D PET image reconstruction framework is illustrated in Fig. 1. The reconstructed image, $\lambda \in \mathbb{R}^{n \times n}$ with $n \times n$ the number of voxels, is computed using a deep neural network, i.e. a SIREN or DIP network, which parameterizes $\lambda$ as follows:

$$ \lambda = f_N(\theta | v). \quad (1) $$

In (1), $f_N$ represents the function associated to the neural network, $\theta$ refers to the model's weights and $v$ corresponds to the neural network input, that will be defined later. The reconstructed image $\lambda$ is obtained by solving the following constrained optimization problem:

$$ \min_{\lambda} \{-\log p(y | \mathcal{P}(\lambda))\} \quad \text{s.t.} \quad \lambda = f_N(\theta | v), \quad (2) $$

where $p(y | \mathcal{P}(\lambda))$ represents the likelihood of the measured data $y$ given the forward projection $\mathcal{P}(\lambda)$ of the reconstructed image $\lambda$. Specifically, the negative log-likelihood used as the loss function is:

$$ \mathcal{L}_{\text{Poisson}}(y | \mathcal{P}(\lambda)) = \sum_{i} (\mathcal{P}(\lambda)_{i} - y_{i} \log(\mathcal{P}(\lambda)_{i})), \quad (3) $$

which better suits the Poisson nature of the sinogram data than mean square error (MSE). Here, the forward projection

operator is defined as $\mathcal{P}(\lambda) = A\lambda + r$, where $A$ is the system matrix affected by attenuation and sensitivity, and $r$ represents background events, including scatter and random coincidences [13].

Each iteration $k$ of the optimization involves one forward projection and one backpropagation step: in the forward pass, the image $\lambda_k$ is generated through $f_N(\theta_k)$, then projected using $\mathcal{P}$ to obtain a sinogram $y_k$. The loss is computed based on the likelihood between the generated and measured sinogram $y$ as in (3), and the updated neural network weights $\theta_{k+1}$ are obtained accordingly using an optimization algorithm (Stochastic gradient descent, Adam, L-BFGS, etc.). This iterative process is repeated until convergence.

## 2.2. SIREN for PET reconstruction

INRs model the image intensities using a continuous differentiable function, in contrast to traditional representations that map images to a discrete pixel grid. More precisely, an INR, in 2D PET reconstruction, learns a mapping from a 2D coordinate $v := (x, y) \in \mathbb{R}^2$ to the corresponding pixel intensity value in $\mathbb{R}$:

$$ f_N: \mathbb{R}^2 \to \mathbb{R}, \quad (x, y) \to \lambda(x, y) = f_N(\theta | (x, y)) \quad (4) $$

Recently, a neural network architecture for INRs that uses sinusoidal activation functions was proposed in [11] and referred to as SIREN. Each layer of SIREN applies an affine transformation followed by a sine function, allowing the network to model continuous and smooth signals. Given 2D coordinates $(x, y)$, SIREN learns to map these inputs to their corresponding intensities in the image.

Our proposal to apply SIREN as $f_N$ in (2) for PET image reconstruction is motivated by its ability to naturally capture both low and high frequency information, which is critical for accurately reconstructing fine details in PET images. To better adapt the SIREN model to the PET reconstruction framework, we incorporate a SoftPlus activation function in the final layer, which ensures smooth, differentiable, and strictly positive outputs, as negative values are not feasible in this context.

## 2.3. Deep image prior

An alternative neural network model that can be employed within the end-to-end approach is based on a DIP architecture. In this case, $f_N$ in (2) corresponds to a 3D U-Net-inspired design as proposed by Hashimoto et al. [8], with $v$ in (2) initialized as a random image input. This random image serves to guide the reconstruction process gradually, acting as the starting point for the network's training. Adaptations are also introduced to handle 2D sinograms, with modifications like replacing trilinear upsampling layers with bilinear interpolation, as suggested by Gong et al. [14], to avoid checkerboard artifacts. Additionally, no skip connections are used to avoid
<figure><img src="image_2.png" /><figcaption>Fig. 1. Overview of the proposed pipeline for PET reconstruction using SIREN. The model maps the image pixel by pixel to construct the full image. The loss is then computed in the measurement domain between the estimated sinogram obtained by projecting the estimated image and the measured sinogram.</figcaption></figure>

transferring noise from the initial random image too quickly, ensuring a more gradual refinement [15]. Finally, a SoftPlus activation function is applied after the last layer to ensure non-negative outputs, providing a smooth, differentiable solution that avoids the "dead neurons" problem [16].

# 3. EXPERIMENTS

## 3.1. Data and PET projection

To assess the performance of the PET reconstruction, a 2D brain phantom was generated using the BrainWeb database [17]. The reference image from BrainWeb was used to construct a piecewise uniform PET ground truth (GT) that replicates a typical Fluorodeoxyglucose (FDG) brain distribution with standardized uptake values (SUVs) assigned as follows: 6 for gray matter, 2 for white matter, 0.5 for cerebrospinal fluid (CSF), 0.2 for bone, and 1 for other tissues. Additionally, a uniform circular tumor, with SUV = 10 was manually inserted into the white matter in the PET images. The phantom was sampled into 160 x 160 pixels of 2 mm x 2 mm. Synthetic sinograms are obtained by 2D PET analytical simulations conducted using the software described in [13]. These simulations incorporate the geometry of the Siemens Biograph mMR scanner and include attenuation, efficiency, 35% random events, 30% scattered events, and a total of 3.5 million prompts.

## 3.2. Image Quality Metrics

The peak signal-to-noise ratio (PSNR) and the mean structural similarity index measure (MSSIM) are used to assess the reconstructed image quality within the brain. Additionally,

the activity recovery (AR) of a hot tumor region is defined as $AR_h = \hat{\lambda}_h/\bar{\lambda}_h$, where $\hat{\lambda}_h$ and $\bar{\lambda}_h$ represent the mean activity of the hot region in the reconstructed and ground-truth images, respectively. The relative bias (RB) assesses cold region recovery, defined by ventricles filled with cerebrospinal fluid as $RB_c = (\hat{\lambda}_c - \bar{\lambda}_c)/\bar{\lambda}_c$ where $\hat{\lambda}_c$ and $\bar{\lambda}_c$ represent the mean activity of the cold region in the reconstructed and ground-truth images, respectively. Finally, image roughness (IR), computed in the white matter of the brain, measures the standard deviation of voxel values in the phantom background from their mean value:

$$IR = \frac{\sqrt{\frac{1}{L-1} \sum_{i \in ROI} (\hat{\lambda}_{r,i} - \hat{\lambda}_{r,ROI})^2}}{\hat{\lambda}_r} \quad (5)$$

where $\hat{\lambda}_{r,i}$ is the activity of white matter, $\hat{\lambda}_r$ is its mean value, and $L$ is the number of voxels in the region of interest.

## 3.3. Experimental Settings

For all experiments, the optimization is performed using the a quasi-Newton method (the L-BFGS optimizer), with a learning rate of 1 for SIREN and 1.5 for DIP. This approach showed more stable convergence and superior performance in terms of computational time compared to first-order gradient descent algorithms. The SIREN model consists of 5 neural layers, with 256 features per layer, resulting in a total of 329 985 parameters. The model uses sine activations parameterized by $\omega_0 = 25$. The weights are initialized to ensure that the variance of the activations remains consistent across layers, helping to stabilize the training process. The DIP model contains 382 473 parameters, initialized from random Gaussian distribution, with zero mean and unit standard deviation.
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Image Roughness (IR) versus Activity Recovery and Relative Bias, plotted for each iteration for SIREN and DIP, and for different regularization strength in the case of BSREM with quadratic penalty.</figcaption></figure>

The BSREM algorithm is applied using the CASTOR framework [18] with a quadratic penalty, and a manually chosen regularization parameter $\beta = 0.355$ to achieve optimal PSNR and SSIM between the reconstructed and ground truth image.

## 4. RESULTS & DISCUSSION

Fig. 2 shows the performance of SIREN in terms of $AR_h$ and $RB_c$ using ReLU and SoftPlus activation functions, compared to DIP and BSREM with quadratic penalty. The plot shows results for each iteration in the case of DIP and SIREN, and for a set of regularization parameters in the case of BSREM. We see that SIREN with SoftPlus was able to reach an optimal $AR_h$ value of 100% in very few iterations, with significantly less roughness than DIP and BSREM. Similarly, SIREN achieved the lowest relative bias of 23%, with low image roughness compared to both DIP and BSREM.

Fig. 3 shows reconstructed images using the compared algorithms. The BSREM reconstruction is shown with the best regularization weight, giving the highest PSNR. The SIREN and DIP reconstructions are shown at the iterations that give the best PSNR. The figure shows that BSREM, with its optimal values, preserves more detail than the other methods, although some blurring is still present. DIP suffers from increased blurring and loss of finer details. SIREN with ReLU, while providing slightly better contrast, struggles to recover the hot region. In contrast, SIREN with SoftPlus strikes a balance between DIP and BSREM, preserving more detail than DIP and offering better contrast than the other methods, as seen in the zoomed-in region of the phantom. It is important to note that SIREN with SoftPlus performs better in terms of PET-specific metrics such as $AR_h$, $RB_c$, and IR.

In terms of computing time, SIREN achieved the best re-

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Reconstructed images using BSREM with quadratic penalty, DIP and SIREN.</figcaption></figure>

construction in 20 minutes, compared to 67 minutes for DIP. Each iteration for SIREN took approximately 2 minutes, and only 10 iterations were required to reach the best image quality. In contrast, DIP took approximately 8.5 seconds per iteration but required 475 iterations to yield optimal results. For comparison, one BSREM reconstruction required 7.6 seconds per iteration, but the regularization parameters must be carefully chosen—a process that is not straightforward in practice. These results highlight SIREN's ability to achieve high-quality reconstructions faster and with fewer iterations than DIP.

The proposed PET reconstruction method builds on the approach of Hashimoto et al., but improves upon it by replacing the DIP architecture with a SIREN model. This substitution aims to further reduce biases, including artifacts introduced by the data, and focuses on optimizing reconstruction directly from the measured projection data, scatter and random corrections, and attenuation correction. Normalization is also integrated directly into the reconstruction process to improve consistency.

## 5. CONCLUSION

In this study, we introduced an unsupervised method for PET image reconstruction by taking advantage of the SIREN implicit neural network architecture. Inspired by Hashimoto et al, we used a complete pipeline for end-to-end reconstruction, including a forward projection model and a loss function suitable for PET reconstruction directly from sinograms. Comparative evaluations using brain phantom data and realistically simulated sinograms show that the INR-based method outperforms traditional penalized likelihood techniques and DIP-based reconstruction in PET-specific metrics. The results show that our approach improves image contrast and effec-
tively captures fine details. This study highlights the potential
of INR to be applied to end-to-end PET reconstruction and
to improve overall reconstruction performance. Future work
includes extension to 3D PET reconstruction on real data and
the automatic fine tuning of the algorithms.

6. ACKNOWLEDGMENTS

This work has been partly funded by the French "Programme d'Investissement d'Avenir" (ANR-16-IDEX-0007) and region "Pays de la Loire" through their support to I-Site NExT, as well as by Siemens Healthineers, the industrial partner of the NExT research industrial chair IMRAM.

7. REFERENCES

[1] Lawrence A Shepp and Yehuda Vardi, "Maximum likelihood reconstruction for emission tomography," *IEEE transactions on medical imaging*, vol. 1, no. 2, pp. 113-122, 1982.

[2] Joyita Dutta, Richard M Leahy, and Quanzheng Li, "Non-local means denoising of dynamic pet images," *PloS one*, vol. 8, no. 12, pp. e81390, 2013.

[3] Sangtae Ahn and Jeffrey A Fessler, "Globally convergent image reconstruction for emission tomography using relaxed ordered subsets algorithms," *IEEE transactions on medical imaging*, vol. 22, no. 5, pp. 613–626, 2003.

[4] Bo Zhu, Jeremiah Z Liu, Stephen F Cauley, Bruce R Rosen, and Matthew S Rosen, "Image reconstruction by domain-transform manifold learning," *Nature*, vol. 555, no. 7697, pp. 487–492, 2018.

[5] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempit-
sky, "Deep image prior," in *Proceedings of the IEEE
conference on computer vision and pattern recognition*,
2018, pp. 9446–9454.

[6] Jianan Cui, Kuang Gong, Ning Guo, Chenxi Wu, Xi-
axia Meng, Kyungsang Kim, Kun Zheng, Zhifang Wu,
Liping Fu, Baixuan Xu, et al., "Pet image denoising us-
ing unsupervised deep learning," *European journal of
nuclear medicine and molecular imaging*, vol. 46, pp.
2780–2789, 2019.

[7] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville, "On the spectral bias of neural networks," in *International conference on machine learning*. PMLR, 2019, pp. 5301–5310.

[8] Fumio Hashimoto, Kibo Ote, and Yuya Onishi, "Pet image reconstruction incorporating deep image prior and a

forward projection model,” *IEEE Transactions on Radiation and Plasma Medical Sciences*, vol. 6, no. 8, pp. 841–846, 2022.

[9] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng, "Nerf: Representing scenes as neural radiance fields for view synthesis," *Communications of the ACM*, vol. 65, no. 1, pp. 99–106, 2021.

[10] Muhammad Osama Khan and Yi Fang, “Implicit neural representations for medical imaging segmentation,” in *International Conference on Medical Image Computing and Computer-Assisted Intervention*. Springer, 2022, pp. 433–443.

[11] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein, “Implicit neural representations with periodic activation functions,” *Advances in neural information processing systems*, vol. 33, pp. 7462–7473, 2020.

[12] Daniel LeJeune, Lorenzo Luzi, Ali Siahkoohi, Sina Ale-
mohammad, Vishwanath Saragadam, Hossein Babaei,
Naiming Liu, Zichao Wang, and Richard G Baraniuk,
“Titan: Bringing the deep image prior to implicit repre-
sentations,” *arXiv preprint arXiv:2211.00219*, 2022.

[13] Simon Stute, Clovis Tauber, Claire Leroy, Michel Bott-
laender, Vincent Brulon, and Claude Comtat, “Analyti-
cal simulations of dynamic pet scans with realistic count
rates properties,” in *2015 IEEE Nuclear Science Sym-
posium and Medical Imaging Conference (NSS/MIC)*.
IEEE, 2015, pp. 1–3.

[14] Kuang Gong, Ciprian Catana, Jinyi Qi, and Quanzheng Li, "Pet image reconstruction using deep image prior," *IEEE transactions on medical imaging*, vol. 38, no. 7, pp. 1655-1665, 2018.

[15] Alexandre Merasli, Thomas Carlier, Diana Mateus, Saïd Moussaoui, and Simon Stute, “The influence of input and skip connections in pet reconstruction with deep image prior,” in *2023 IEEE Nuclear Science Symposium, Medical Imaging Conference and International Symposium on Room-Temperature Semiconductor Detectors (NSS MIC RTSD)*. IEEE, 2023, pp. 1–1.

[16] Sayan Nag, Mayukh Bhattacharyya, Anuraag Mukherjee, and Rohit Kundu, “Serf: Towards better training of deep neural networks using log-softplus error activation function,” in *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*, 2023, pp. 5324–5333.

[17] Chris A Cocosco, “Brainweb: Online interface to a 3d mri simulated brain database.,” (No Title), 1997.
[18] Thibaut Merlin, Simon Stute, Didier Benoit, Julien Bert, Thomas Carlier, Claude Comtat, Marina Filipovic, Frédéric Lamare, and Dimitris Visvikis, “Caster: a generic data organization and processing code framework for multi-modal and multi-dimensional tomographic reconstruction,” *Physics in Medicine & Biology*, vol. 63, no. 18, pp. 185005, 2018.