## 3 Joint Sparse Sampling and Learned Acquisition for MR-PET Imaging
Here,  
* \(M_{\text{MRI}}\) is the binary MRI sampling mask, selecting which k-space frequencies were acquired. * \(F(\cdot)\) is the Fourier transform operator, mapping the reconstructed image \(x^{\text{MRI}}\) to the k-space domain. * \(y^{\text{MRI}}\) is the actual acquired MRI k-space data (with partial Fourier or compressed sensing sampling). The second term, \(\|M_{\text{PET}}\cdot A_{\text{PET}}(x^{\text{PET}})-y^{\text{PET}}\|_{2}^{2}\), similarly measures the squared \(\ell_{2}\) distance between the predicted PET sinogram and the acquired PET sinogram data. Here,
* \(M_{\text{PET}}\) is the PET sampling mask, indicating the subset of projection angles and detector bins used. * \(A_{\text{PET}}(\cdot)\) is the PET forward projection operator, mapping the reconstructed image \(x^{\text{PET}}\) to the sinogram domain. * \(y^{\text{PET}}\) is the acquired PET sinogram data (often undersampled or noisy). Minimizing \(\mathcal{L}_{\text{data}}\) ensures that the reconstructed MRI and PET images faithfully reproduce the acquired measurements in both domains. This acts as a grounding constraint within the overall optimization, preventing overfitting to sparsity priors or adversarial losses, and guaranteeing that the recovered images remain physically plausible and data-consistent. To further enhance performance, the concept of learned sensing matrices has been incorporated into the reconstruction pipeline [27; 33]. Recent studies in compressed sensing and deep image reconstruction have shown that learned sampling patterns can outperform fixed designs, especially in multimodal contexts. In this work, the sampling masks \(M_{\text{PET}}\) and \(M_{\text{MRI}}\) have been augmented through a learned mask refinement step, in which initial Poisson and PROPELLER patterns are fine-tuned via gradient-based optimization guided by reconstruction loss metrics. This adaptive refinement enables the sensing matrices to better capture salient features specific to the dataset and task, improving reconstruction fidelity under highly sparse sampling conditions. Specifically, the learned masks are optimized by minimizing the reconstruction loss with respect to the mask parameters \(\theta\):  
\[\min_{\theta}\ \mathbb{E}_{x}\Big{[}\mathcal{L}_{\text{GAN}}^{\text{joint}} \Big{(}\mathcal{A}_{\text{joint},\theta}(x^{\text{MRI}},x^{\text{PET}}),\ (y^{\text{MRI}},y^{\text{PET}})\Big{)}\Big{]}+\beta\cdot\|M_{\text{PET}, \theta}\|_{1}+\gamma\cdot\|M_{\text{MRI},\theta}\|_{1},\]  
where \(\mathcal{A}_{\text{joint},\theta}\) is the joint acquisition operator parameterized by \(\theta\), and the \(\ell_{1}\) penalties encourage sparsity in the learned masks. The hyperparameters \(\beta\) and \(\gamma\) control the tradeoff between sampling sparsity and reconstruction fidelity. In this formulation, the entire optimization process is designed to jointly learn both the sampling masks and the image reconstruction pipeline in an end-to-end fashion. The objective minimizes the expected value of the joint adversarial reconstruction loss \(\mathcal{L}_{\text{GAN}}^{\text{joint}}\), evaluated on the current learned acquisition process \(\mathcal{A}_{\text{joint},\theta}\) and the corresponding ground truth raw data \((y^{\text{MRI}},y^{\text{PET}})\). Here,  
* \(\theta\) are the learnable parameters of the joint acquisition process, which include the parameters of the sampling masks \(M_{\text{MRI},\theta}\) and \(M_{\text{PET},\theta}\). * \(\mathcal{A}_{\text{joint},\theta}\) is the joint acquisition operator under the current sampling masks, defined as follows: \[\mathcal{A}_{\text{joint},\theta}(x^{\text{MRI}},x^{\text{PET}})=\Big{(}M_{ \text{MRI},\theta}\cdot F(x^{\text{MRI}}),\ M_{\text{PET},\theta}\cdot A_{ \text{PET}}(x^{\text{PET}})\Big{)}.\]
* \(\mathcal{L}_{\text{GAN}}^{\text{joint}}\) is the joint loss used to train the generator network, incorporating adversarial loss, perceptual loss, and structured joint sparsity priors.