Cohort-Based Active Modality Acquisition

## Abstract

Abstract Real-world machine learning applications often involve data from multiple modalities that must be integrated effectively to make robust predictions. However, in many practical settings, not all modalities are available for every sample, and acquiring additional modalities can be costly.
This raises the question: which samples should be prioritized for additional modality acquisition when resources are limited?
While prior work has explored individual-level acquisition strategies and training-time active learning paradigms, test-time and cohort-based acquisition remain underexplored.
We introduce Cohort-based Active Modality Acquisition (CAMA) , a novel test-time setting to formalize the challenge of selecting which samples should receive additional modalities.
We derive acquisition strategies that leverage a combination of generative imputation and discriminative modeling to estimate the expected benefit of acquiring missing modalities based on common evaluation metrics.
We also introduce upper-bound heuristics that provide performance ceilings to benchmark acquisition strategies. Experiments on multimodal datasets with up to 15 modalities demonstrate that our proposed imputation-based strategies can more effectively guide the acquisition of additional modalities for selected samples compared with methods relying solely on unimodal information, entropy-based guidance, or random selection. We showcase the real-world relevance and scalability of our method by demonstrating its ability to effectively guide the costly acquisition of proteomics data for disease prediction in a large prospective cohort, the UK Biobank (UKBB) . Our work provides an effective approach for optimizing modality acquisition at the cohort level, enabling more effective use of resources in constrained settings. 1 1 1 Code will be published on GitHub .

## 1 Introduction

Consider a clinical healthcare setting where all patients in a cohort undergo a standard, inexpensive set of initial examinations, such as basic blood tests and anamnesis. However, a more advanced, expensive, or invasive procedure, like genomic sequencing or specialized imaging, could offer crucial diagnostic or prognostic information for a subset of these patients (Huang et al., 2021). Given a limited budget or capacity for the more advanced procedure, the central question becomes: which patients should receive these additional resources to maximize the overall diagnostic yield or improve treatment outcomes across the entire cohort?

Figure: Figure 1: Motivational example for CAMA determining the added value of obtaining the MRI modality. (A) A heterogeneous cohort for which each sample has $P$ distinct modalities. (B) Instead of using the initial subset logit scores ${\textnormal{s}}_{i}^{\textnormal{avail}}$, a generative model $f_{\textnormal{imp}}$ imputes the target missing modality for every patient in the cohort. This yields imputed, augmented-modality logit scores $\{{\textnormal{s}}_{i,k}^{\textnormal{imp}}\}_{k=1}^{K}$ that approximate the logits as if that modality were available. When exactly one modality is missing, these scores approximate ${\textnormal{s}}_{i}^{\textnormal{full}}$; otherwise they estimate the counterfactual with only the imputed modality added. (C) An AF utilizes these scores to rank samples by acquisition priority. The graph demonstrates how the global performance metric improves from the initial baseline towards the performance of a model with access to complete multimodal data, as an increasing fraction of the cohort receives the additional modality. This acquisition process is guided by the proposed strategies operating under the acquisition budget constraint $\beta$.
Refer to caption: figures/fig_illustrative_example_3.png

Balancing potential gains from data modalities against the costs and complexities of acquisition is not unique to healthcare. In remote sensing, for instance, decisions must be made regarding which geographical areas warrant costly high-resolution satellite imagery to supplement widely available, lower-resolution data, aiming to optimize regional environmental monitoring under budget constraints. Likewise, in industrial quality assurance, manufacturers could decide which components from a production batch should undergo detailed, time-consuming testing in addition to rapid, standard visual inspections to effectively identify defects at a batch level.
The topic of efficient data acquisition has led to several established paradigms in machine learning, such as Active Learning (AL) (Holzmüller et al., 2023), Active Feature Acquisition (AFA) (Shim et al., 2018), Active Modality Acquisition (AMA) (Kossen et al., 2023), and multimodal learning with missing data (Wu et al., 2024). However, previous research predominantly centers on optimizing acquisition for individual samples and often does not directly address test-time budget constraints for an entire cohort.
Consequently, the strategic, test-time acquisition of entire modalities from a cohort perspective remains a significant, largely unaddressed gap. This setting involves deciding, for a given batch of new samples where different subsets of modalities are available, which specific samples should receive an additional, costly modality to best achieve a global objective, e.g., maximizing overall predictive performance or diagnostic accuracy for the cohort, subject to budget constraints. We hypothesize that imputation-based acquisition functions (AFs) can effectively guide resource allocation under cohort-level constraints. The main contributions of this work are as follows:

- •
The CAMA setting We introduce and formalize CAMA, a previously unexplored setting that addresses the challenge of prioritizing which samples within a test-time cohort should undergo additional modality acquisition based on an available subset of modalities.
- •
Development of AFs for CAMA We propose a theoretical framework, derived from established evaluation metrics, e.g., Area Under the Receiver Operating Characteristic (AUROC) and Area Under the Precision-Recall Curve (AUPRC), that provides a foundation for developing AFs within the CAMA setting.
- •
Architectures for CAMA We develop novel architectures for approaching CAMA, including a) derivations of AFs by combining generative and discriminative deep learning and b) the definition of corresponding upper bounds to serve as performance benchmarks.
- •
Comprehensive evaluation We present a comprehensive empirical evaluation of our proposed methods across several multimodal datasets, which vary in their number of modalities and application domains, with up to 100,000 samples and 15 modalities. This includes an analysis of key assumptions, upper bounds and oracle strategies, performance challenges, and robustness.

## 2 Related Work

In the following, we contextualize our work on CAMA by reviewing the key concepts and contributions from several relevant research domains summarized briefly in [Table˜1](https://arxiv.org/html/2505.16791v2#S2.T1).

**Table 1: Comparison of active data acquisition paradigms. Our proposed CAMA setting is unique in its focus on cohort-level, test-time modality acquisition.**
| Paradigm | Acquisition | Decision Level | Time | Primary Objective |
| --- | --- | --- | --- | --- |
| AL | Labels | Individual | Training | Maximize model performance |
| AFA | Features | Individual | Test | Optimize sample-level prediction |
| AMA | Modalities | Individual | Test | Optimize sample-level prediction |
| CAMA (Ours) | Modalities | Cohort | Test | Maximize global cohort metric |

##### Active Learning (AL)

AL seeks to enhance model training by selecting unlabeled data points for annotation by an oracle (Settles, 2012; Ren et al., 2022; Li et al., 2025). Our methodology draws significantly from AL principles, particularly in the development of an AF to guide the selection process. Consequently, established AL strategies and concepts, such as those rooted in measuring uncertainty (Settles, 2012; Han & Kang, 2021; Hoarau et al., 2025; Raj & Bach, 2022; Ma et al., 2019) or using generative models (Tran et al., 2019; Zhu & Bento, 2017; Zhang et al., 2024; Ma et al., 2019; Peis et al., 2022), are central to our work. Existing work on multimodal acquisition (Rudovic et al., 2019; Das et al., 2022), batch-level selection (Ash et al., 2020; Kirsch et al., 2019; Holzmüller et al., 2023), and balanced AL (Aggarwal et al., 2020; Shen et al., 2023; Zhang et al., 2023; Hoarau et al., 2025) is especially relevant. Our approach, however, diverges from the conventional goals of directly optimizing model training or seeking labels for specific data points: We aim to identify those samples for which the acquisition of an additional data modality would be most beneficial.

##### Active Feature Acquisition (AFA)

AFA builds upon AL by focusing on selecting the most informative individual features for a given sample, often considering their acquisition costs (Rahbar et al., 2025). Similar to AL approaches, methods for AFA encompass a diverse range of techniques, including strategies based on measuring uncertainty (Hoarau et al., 2025; Astorga et al., 2024), the use of generative models (Li & Oliva, 2021; 2024; Gong et al., 2019; Zannone et al., 2019), and Reinforcement Learning (RL) (Valancius et al., 2024; Janisch et al., 2020; Kleist et al., 2025; Shim et al., 2018; Baja et al., 2025). Other common methodologies involve batch-level perspectives (Asgaonkar et al., 2024), leveraging information bottlenecks (Norcliffe et al., 2025), or employing the Kullback-Leibler Divergence (KL-Divergence) (Natarajan et al., 2018). Some AFA techniques rely on gradient calculations (Ghosh & Lan, 2023), while distinct approaches are formulated as individual, sequential recommender systems (Freyberg et al., 2024; Vivar et al., 2020). At an application level, even Large Language Models (LLMs), such as Med-PaLM 2 (Singhal et al., 2025), could be employed for AFA, although such deployments remain unexplored in this context. While our setting shares the core idea of AFA, it differs significantly: We are not concerned with the selection of individual features, but rather with identifying which entire data modalities to acquire. Furthermore, this decision-making process is applied at the cohort level, rather than optimizing for individual samples.

##### Active Modality Acquisition (AMA)

AMA can be conceptualized as an extension of AFA, distinguished by its focus on selecting entire data modalities rather than individual features or labels. Prominent related research includes approaches employing RL for multimodal temporal data (Kossen et al., 2023) and methods utilizing submodular optimization in conjunction with Shapley values (Shapley, 1953; He et al., 2024). The approach by Kossen et al. (2023) differs from ours through its reliance on RL, whereas He et al. (2024) primarily investigate how modalities affect optimal learning performance. Further studies have explored the use of Gaussian mixtures within Bayesian optimal experimental design to enhance data acquisition efficiency for model training (Long, 2022). This objective differs from ours, as our focus is not on improving the model training process itself, but rather on optimizing performance for a downstream task at test time. The relative sparsity of existing work for AMA underscores the significance of the research gap that our proposed setting, i.e., CAMA  aims to address.

##### Multimodal Learning with Missing Data Modalities

Research in multimodal learning with missing data modalities offers techniques for robustly handling incomplete datasets. These methods are broadly classified into strategy design aspects, i.e., architecture-focused designs and model combinations, and data processing aspects, i.e., representation learning and modality imputation (Wu et al., 2024). Acknowledging the utility of these approaches, our work emphasizes imputation-based strategies, and thus this paragraph highlights those methods. Imputation of missing features is commonly performed using Auto Encoders (AEs) (Hinton & Zemel, 1993), Variational Auto Encoders (VAEs) (Kingma & Welling, 2014), Generative Adversarial Networks (GANs) (Goodfellow et al., 2014), or Denoising Diffusion Probabilistic Models (DDPMs) (Ho et al., 2020; Rombach et al., 2022). These methods naturally extend to multiple modalities, for example, with VAE-based (Wesego & Rooshenas, 2024; Sutter et al., 2021; Lewis et al., 2021) and DDPM-based (Wang et al., 2023) approaches. Notably, the latter, i.e., IMDer (Wang et al., 2023), a multimodal deep learning architecture that imputes missing values with DDPMs in latent spaces, is adapted in our work ([Section˜5](https://arxiv.org/html/2505.16791v2#S5)). However, this research area focuses on handling absent modalities rather than deciding which ones to acquire.

## 3 Problem Formulation

Let ${\mathcal{D}}=\{({\bm{x}}_{i},{\textnormal{y}}_{i})\}_{i=1}^{N}$ be a dataset of $N$ samples. For each sample $i$, the full feature set ${\bm{x}}_{i}$ is composed of $P$ distinct data modalities, ${\bm{x}}_{i}=\{{\bm{x}}_{i}^{(1)},\dots,{\bm{x}}_{i}^{(P)}\}$, and ${\textnormal{y}}_{i}\in\{0,1\}$ is the corresponding binary label. In practice, only a subset of these modalities may be available. We denote the set of indices of available modalities for sample $i$ as ${\mathcal{P}}_{i}^{\textnormal{avail}}\subseteq\{1,\dots,P\}$. Our goal is to decide for which samples to acquire costly missing data to maximize a cohort-level performance metric. This decision is guided by predictive scores (logits), and we consider three key predictive scores for each sample $i$:

- •
${\textnormal{s}}_{i}^{\textnormal{avail}}$: The available score, computed using the subset of data modalities that are already observed for the sample.
- •
${\textnormal{s}}_{i}^{\textnormal{full}}$: The full score, a theoretical oracle score that would be computed if all modalities were available. This is unknown for samples with missing data modalities.
- •
$\{{\textnormal{s}}_{i,k}^{\textnormal{imp}}\}_{k=1}^{K}$: A set of $K$ imputed scores that estimate the unknown ${\textnormal{s}}_{i}^{\textnormal{full}}$ using only the available data modalities.

For instance, given the example from [Figure˜1](https://arxiv.org/html/2505.16791v2#S1.F1), in a simple clinical setting with a cheap, universally available base modality, e.g., cardiac biomarkers such as troponin or B-type natriuretic peptide (BNP), and an expensive additional modality, e.g., cardiac MRI, ${\textnormal{s}}_{i}^{\textnormal{avail}}$ would be the score from the blood tests alone, while ${\textnormal{s}}_{i}^{\textnormal{full}}$ would be the score using both tests and MRI. To compute these scores, we assume a single model $f$ parameterized by ${\bm{\theta}}$ that can process any subset of modalities. The available and full scores are thus:

$$ $\displaystyle{\textnormal{s}}_{i}^{\textnormal{avail}}$ $\displaystyle=f({\bm{x}}_{i}^{\textnormal{avail}},{\bm{\theta}})$ (1) $\displaystyle{\textnormal{s}}_{i}^{\textnormal{full}}$ $\displaystyle=f({\bm{x}}_{i}^{\textnormal{full}},{\bm{\theta}})$ (2) $$

where ${\bm{x}}_{i}^{\textnormal{avail}}$ and ${\bm{x}}_{i}^{\textnormal{full}}$ represent the feature sets for the available and full modalities, respectively. To estimate the full score without costly acquisition, we use a generative imputation model $f_{\textnormal{imp}}$. This model generates a set of $K$ plausible embeddings that enable the classifier $f$ to predict the scores $\{{\textnormal{s}}_{i,k}^{\textnormal{imp}}\}_{k=1}^{K}$. These imputation-based scores form the basis of our acquisition functions.

The goal of the optimization is to select a subset of samples ${\mathcal{S}}$ from the cohort of $N$ total samples for which an additional modality should be acquired. This subset ${\mathcal{S}}\subseteq\{1,\dots,N\}$ has a predetermined size $|{\mathcal{S}}|=\beta$, where $\beta$ is the acquisition budget, i.e., the number of samples for which additional modalities will be acquired. The final score ${\textnormal{s}}_{i}({\mathcal{S}})$ used for the evaluation of a sample $i$ is then determined by the selection:

$$ $\displaystyle{\textnormal{s}}_{i}({\mathcal{S}})=\begin{cases}{\textnormal{s}}_{i}^{\textnormal{full}}&\text{if }i\in{\mathcal{S}}\\ {\textnormal{s}}_{i}^{\textnormal{avail}}&\text{if }i\notin{\mathcal{S}}\end{cases}.$ (3) $$

The optimization problem is to find the set ${\mathcal{S}}^{*}$ that maximizes the chosen performance metric:

$$ $\displaystyle{\mathcal{S}}^{*}=\operatorname*{arg\,max}_{{\mathcal{S}}\subseteq\{1,\dots,N\}:|{\mathcal{S}}|=\beta}\text{Metric}({\mathbf{y}},{\mathbf{s}}({\mathcal{S}}))$ (4) $$

where ${\mathbf{y}}=\{{\textnormal{y}}_{i}\}_{i=1}^{N}$ is the vector of true labels, and ${\mathbf{s}}({\mathcal{S}})=\{{\textnormal{s}}_{i}({\mathcal{S}})\}_{i=1}^{N}$ is the vector of resulting scores for all samples in the cohort. Consequently, the task is to identify an optimal, constrained subset for which to acquire additional modalities, while maximizing a performance metric across the entire cohort.

## 4 Acquisition Function Strategies

Directly solving the cohort-level optimization problem to identify the optimal sample set ${\mathcal{S}}^{*}$ is computationally intractable due to its combinatorial nature. Therefore, we employ several heuristic acquisition functions (AFs) that approximate the optimal selection by ranking samples for modality acquisition. These strategies, detailed further in [Appendix˜C](https://arxiv.org/html/2505.16791v2#A3), are derived from standard discriminative metrics ([Section˜C.1](https://arxiv.org/html/2505.16791v2#A3.SS1)) and can be categorized as follows ([Table˜2](https://arxiv.org/html/2505.16791v2#S4.T2)):

- •
Oracle Strategies: As upper-bound benchmarks, they assume perfect knowledge of outcomes and true labels to greedily select samples yielding the largest immediate gain in the target metric.
- •
Upper-Bound Heuristic Strategies: These heuristics assume knowledge of scores under modality completion but are label-agnostic, relying on metrics like the true uncertainty reduction, rank change, or KL-Divergence.
- •
Imputation-Based Strategies: Grounded in counterfactual reasoning, these strategies use a generative model to predict how a sample’s score might change if a missing modality were acquired.
- •
Baseline Information Strategies: These strategies make decisions using only information from the initially available modality, i.e., without any imputation, such as its predicted uncertainty or probability.
- •
Random Strategy: This serves as a fundamental baseline by selecting samples randomly, without regard to any model scores.

**Table 2: Summary of AF Strategies.**
| Category | Strategies | Input Variables | Ranking Criteria |
| --- | --- | --- | --- |
| Oracle | AUROC, AUPRC | True labels & full scores | Greedy selection for maximum gain. |
| Upper-Bound | KL-Divergence, Rank, Uncertainty | True full scores | True change in prediction, cohort rank, or uncertainty. |
| Imputation-Based | KL-Divergence, Rank, Uncertainty & Probability | Imputed full scores | Expected change in prediction, rank, or uncertainty. |
| Baselines | Uncertainty, Probability | Unimodal scores | Uncertainty or probability using the available modality. |
| Random | Random | None | Random selection. |

For evaluation, we introduce a metric that describes the cumulative performance of an AF, normalized by the total possible gain achievable by transitioning all samples to full multimodal performance (see [Figure˜1](https://arxiv.org/html/2505.16791v2#S1.F1) C, for an illustrative curve). Let $M_{\text{AF}}(b)$ denote the performance curve of an AF strategy for a primary metric $M$ as a function of the budget fraction $b$ of the acquisition budget $\beta$, $M_{\text{uni}}$ the performance of the unimodal baseline, and $M_{\text{multi}}$ the performance of the full multimodal model.
The normalized area of gain for an acquisition function AF, which measures the portion of achievable performance gain captured across different budgets, is defined as the area under the performance gain curve, normalized by the maximum possible gain ([Equation˜5](https://arxiv.org/html/2505.16791v2#S4.E5)). Intuitively, a value of 0 indicates no improvement over the unimodal baseline across budgets. A value of $1$ indicates matching the full multimodal performance on average across budgets. Values greater than $1$ occur when the cohort’s performance at intermediate budgets temporarily exceeds the multimodal cohort.

$$ $\displaystyle G_{\text{full}}^{M}(\text{AF})=\frac{\int_{0}^{1}(M_{\text{AF}}(b)-M_{\text{uni}})\,db}{M_{\text{multi}}-M_{\text{uni}}}$ (5) $$

## 5 Evaluation

Figure: Figure 2: End-to-end architectures to determine the scores for different AFs in our proposed CAMA setting. (A) Vanilla late fusion (LF) architecture of a model $f$ that can handle missing data modalities by masking. If all modalities are available, the model creates scores $s_{i}^{\text{full}}$, otherwise scores $s_{i}^{\text{avail}}$ are created. (B) Possible architecture for training (left) and inference (right) with a LF model $f$ and a generative model $f_{\text{imp}}$ to create scores $s_{i}^{\text{imp}}$ for the imputation-based AFs.
Refer to caption: x1.png

To evaluate these strategies in practice, we require architectures that produce the necessary scores. The oracle, upper-bound, baseline, and random AFs can be evaluated using a vanilla discriminative late fusion (LF) model ([Figure˜2](https://arxiv.org/html/2505.16791v2#S5.F2) A), as they operate on true labels ${\textnormal{y}}_{i}$ and true scores ${\textnormal{s}}_{i}^{\textnormal{full}}$ and ${\textnormal{s}}_{i}^{\textnormal{avail}}$ ([Section˜3](https://arxiv.org/html/2505.16791v2#S3)). In contrast, our proposed imputation-based AFs are grounded in counterfactual reasoning: They require the model to predict how its output would change if a missing modality were present. This necessitates a more sophisticated architecture that combines the discriminative classifier with a generative component capable of imputing the missing modality ([Figure˜2](https://arxiv.org/html/2505.16791v2#S5.F2) B).

##### Model Architecture

We implement a multimodal, discriminative, end-to-end LF architecture $f$ combined with generative models $f_{\text{imp}}$ for imputing the missing embeddings ([Figure˜2](https://arxiv.org/html/2505.16791v2#S5.F2) B).
Here, $f$ is composed of modality-specific encoders, , e.g., Vision Transformers (ViTs) (Dosovitskiy et al., 2021) for images and BERT (Devlin et al., 2019) for text, and a classifier for fusing the modality embeddings, i.e., a Transformer encoder (Vaswani et al., 2017).
For the generative component $f_{\text{imp}}$, we primarily employ DDPMs that model conditional mappings between latent spaces, i.e., $p({\bm{z}}_{\text{missing}}\,|\,{\bm{z}}_{\text{available}})$, to sample embeddings for the absent modality (Wang et al., 2023) ([Figure˜4](https://arxiv.org/html/2505.16791v2#A4.F4) in [Appendix˜D](https://arxiv.org/html/2505.16791v2#A4)). Other models are also suitable, e.g., we implement Beta-Conditional-VAEs (BC-VAEs) (Higgins et al., 2017) ([Appendix˜F](https://arxiv.org/html/2505.16791v2#A6)) instead of DDPMs to illustrate architectural trade-offs.
During training, the imputed modality embeddings (latents) produced by $f_{\text{imp}}$ are not passed to the classification head, even when modalities are missing ([Figure˜2](https://arxiv.org/html/2505.16791v2#S5.F2) B, left). Instead, the discriminative components (encoders and head) are trained only on actually available modalities; imputed latents are masked out via the attention mask in the fusion module. The generative and discriminative parts are trained jointly, but the generative outputs do not directly influence the classifier during training beyond the shared encoders being updated by the classification loss.
During inference, the reconstructed predictions from $f_{\text{imp}}$ are also sent to the classifier ([Figure˜2](https://arxiv.org/html/2505.16791v2#S5.F2) B, right), because the imputation-based AFs expect counterfactuals to predict the imputed scores $s_{i}^{\text{imp}}$.
The final loss function for the architecture is defined as

$$ $\displaystyle\mathcal{L}=\lambda_{1}\cdot\mathcal{L}_{\text{CE}}+\sum_{i=1}^{P}\lambda_{2}\cdot\mathcal{L}_{\text{D}_{i}}$ (6) $$

with loss weightings $\lambda$, the classification loss $\mathcal{L}_{\text{CE}}$, the (generative) DDPM losses $\mathcal{L}_{\text{D}_{i}}$, and the total number of modalities $P$. We find that setting $\lambda_{1}=1.0$ is important for downstream performance. For model training, we use the ScheduleFree optimizer (Defazio et al., 2024) with hyperparameters determined through sweeps.
As a detail, we find three architectural decisions essential: (a) applying Layer Normalization (Ba et al., 2016) at the end of each modality’s encoder to stabilize the DDPMs operating between latent spaces, (b) calibrating the model with Label Smoothing (Szegedy et al., 2016) to produce less overconfident and better-calibrated probability distributions, and (c) implementing the score models of the DDPMs as Diffusion Transformers (DiTs) (Peebles & Xie, 2023) for improved resource efficiency.
Regarding missing modalities, we do not pre-train on all available data modalities, in contrast to Wang et al. (2023). We use a predefined, seed-dependent missing-modality mask to control data modality leakage during training unlike batch-dependent masks, which eventually reveal all modalities for every sample across numerous epochs. Further details in [Appendix˜D](https://arxiv.org/html/2505.16791v2#A4).

##### Datasets

We evaluate the setting of CAMA on four real-world multimodal datasets: UKBB (Sudlow et al., 2015), MIMIC Symile (Saporta et al., 2024), MIMIC HAIM (Soenksen et al., 2022a; b), and MOSEI (Zadeh et al., 2018), which cover diverse domains such as healthcare and emotion recognition.
For the publicly available datasets, missing modalities are synthetically created, whereas for UKBB they are an inherent characteristic. We design the datasets for binary classification, resulting in ten binary targets for the MIMIC datasets and one binary target for MOSEI and UKBB. While MOSEI is already class-balanced (Zadeh et al., 2018), HAIM and Symile exhibit significant class imbalance (Soenksen et al., 2022a; Zadeh et al., 2018). To address this, we employ random oversampling during training, which we find essential for the effective operation of AFs. Importantly, during testing we retain the original imbalanced distributions, and no class-balancing steps are applied to UKBB.
We highlight UKBB as the most challenging dataset to demonstrate that CAMA scales to a broad multimodal range and large-scale cohorts with approximately 100,000 samples and 15 modalities. In this setting, we focus on acquiring the exceptionally costly proteomics data for predicting the onset of systemic lupus erythematosus (SLE), which has been shown to benefit from proteomics combined with other clinical data (Yang et al., 2025). Additional details are provided in [Appendix˜E](https://arxiv.org/html/2505.16791v2#A5).

Figure: (a)
Refer to caption: figures/fig_auroc_curves.png

**Table 3: Acquisition performance on Symile, with $G_{\text{full}}$ shown for AUROC/AUPRC as an example for the class with the best and worst performance and the mean value of all ten classes. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold for each column.**
|  | Acquisitions by AUROC, $G_{\text{full}}\uparrow\pm\text{ \lx@glossaries@gls@link{acronym}{sem}{{{}}standard error of the mean (SEM)}}$ | Acquisitions by AUPRC, $G_{\text{full}}\uparrow\pm\text{ \lx@glossaries@gls@link{acronym}{sem}{{{}}SEM}}$ |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Strategy | Cardiomegaly | Pneumothorax | Mean | Lung Lesion | Pneumothorax | Mean |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $2.787\pm 0.139$ | $9.461\pm 1.049$ | $4.580$ | $2.520\pm 0.250$ | $10.623\pm 0.708$ | $4.231$ |
| True KL-Div. | $0.885\pm 0.011$ | $0.910\pm 0.054$ | $0.883$ | $0.828\pm 0.073$ | $0.827\pm 0.043$ | $0.871$ |
| True Rank | $0.878\pm 0.019$ | $0.605\pm 0.053$ | $0.811$ | $0.676\pm 0.088$ | $0.483\pm 0.075$ | $0.776$ |
| True Uncert. | $0.524\pm 0.025$ | $-0.136\pm 0.065$ | $0.481$ | $0.181\pm 0.067$ | $0.293\pm 0.052$ | $0.450$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Divergence | $\mathbf{0.747\pm 0.039}$ | $0.773\pm 0.134$ | $\mathbf{0.833}$ | $\mathbf{0.896\pm 0.146}$ | $0.581\pm 0.084$ | $\mathbf{0.777}$ |
| Probability | $0.350\pm 0.053$ | $\mathbf{0.898\pm 0.061}$ | $0.426$ | $0.320\pm 0.104$ | $\mathbf{0.965\pm 0.027}$ | $0.449$ |
| Rank | $0.378\pm 0.016$ | $0.115\pm 0.082$ | $0.378$ | $0.564\pm 0.086$ | $0.396\pm 0.054$ | $0.407$ |
| Uncertainty | $0.450\pm 0.041$ | $0.055\pm 0.060$ | $0.440$ | $0.130\pm 0.053$ | $0.513\pm 0.066$ | $0.444$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.480\pm 0.013$ | $0.536\pm 0.040$ | $0.480$ | $0.215\pm 0.033$ | $0.811\pm 0.041$ | $0.443$ |
| Probability | $0.431\pm 0.015$ | $0.536\pm 0.040$ | $0.458$ | $0.756\pm 0.136$ | $0.811\pm 0.041$ | $0.550$ |
| Random | $0.385\pm 0.015$ | $0.327\pm 0.061$ | $0.376$ | $0.503\pm 0.103$ | $0.527\pm 0.053$ | $0.388$ |

##### Model and AF Evaluation

For datasets with at least three modalities, we apply five-fold cross-validation. Due to initially noisy results for MIMIC HAIM, we increase the number of folds to ten.
For each sample in the test set, the initial score ${\textnormal{s}}_{i}^{\textnormal{avail}}$ is established by randomly assigning a subset of available modalities ${\mathcal{P}}_{i}^{\textnormal{avail}}$. This procedure is repeated over several runs for robustness. In each run, every sample is stochastically assigned a new subset ${\mathcal{P}}_{i}^{\textnormal{avail}}$. Performance metrics are averaged across these independent runs to ensure our evaluation is robust to any single random assignment of patient data. Acquisition is simulated by incrementally increasing the budget $\beta$. We focus on tasks where the multimodal model demonstrates a performance improvement over the unimodal baseline. For certain prediction tasks, a simpler unimodal model can outperform a more complex multimodal one, potentially due to the introduction of noisy or conflicting signals. In such cases, the final multimodal performance falls below the unimodal baseline, resulting in a negative normalized area of gain, indicating that acquisition was detrimental. To ensure a meaningful evaluation, we exclude any tasks exhibiting this negative gain from the analysis at the split level.
For each budget, the top-ranked samples in ${\mathcal{S}}$ are considered acquired, and their logits are updated from ${\textnormal{s}}_{i}^{\textnormal{avail}}$ to ${\textnormal{s}}_{i}^{\textnormal{full}}$. Final reported results are aggregated across all cross-validation splits, combinations of missing and available modalities, and random runs to ensure robustness of the evaluation.

## 6 Results

**Table 4: Acquisition performance on UKBB, showing $G_{\text{full}}$ for AUROC/AUPRC. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold.**
|  | AUROC | AUPRC |
| --- | --- | --- |
| Strategy | $G_{\text{full}}\pm\text{SEM}\uparrow$ | $G_{\text{full}}\pm\text{SEM}\uparrow$ |
| Upper Bounds (for reference) |  |  |
| Oracle | $1.141\pm 0.051$ | $1.721\pm 0.315$ |
| True KL-Div. | $0.978\pm 0.007$ | $0.986\pm 0.005$ |
| True Rank | $0.887\pm 0.022$ | $0.466\pm 0.110$ |
| True Uncert. | $0.436\pm 0.088$ | $0.507\pm 0.074$ |
| Imputation-based (proposed) |  |  |
| KL-Divergence | $\mathbf{0.641\pm 0.029}$ | $0.658\pm 0.045$ |
| Probability | $0.535\pm 0.026$ | $\mathbf{0.713\pm 0.029}$ |
| Rank | $0.437\pm 0.028$ | $0.340\pm 0.114$ |
| Uncertainty | $0.373\pm 0.053$ | $0.332\pm 0.058$ |
| Baselines (no imputation) |  |  |
| Uncertainty | $0.365\pm 0.042$ | $0.556\pm 0.073$ |
| Probability | $0.365\pm 0.042$ | $0.556\pm 0.073$ |
| Random | $0.528\pm 0.018$ | $0.485\pm 0.052$ |

Our empirical evaluation confirms the effectiveness of CAMA. We benchmark our imputation-based strategies against oracles, upper-bound heuristics, and baselines across multiple datasets. Full results are aggregated in [Tables˜3](https://arxiv.org/html/2505.16791v2#S5.T3), [5](https://arxiv.org/html/2505.16791v2#S6.T5) and [5](https://arxiv.org/html/2505.16791v2#S6.T5) by averaging over permutations of missing input modalities and multiple random instantiations for each missingness configuration. As expected, oracle strategies serve as an upper bound and consistently achieve the highest performance.
Surprisingly, oracle gains can exceed the value of one, as a strategic mix of unimodal and multimodal samples can outperform a purely multimodal cohort. To benchmark the acquisition logic itself, we use label-agnostic upper-bound heuristics that access full scores ${\textnormal{s}}_{i}^{\textnormal{full}}$. Among these, strategies based on KL-Divergence and rank change perform well, indicating that prioritizing large predictive shifts or cohort reordering is an effective heuristic in this setting.
Our main approach for handling the CAMA setting comprises imputation-based strategies that leverage a generative model $f_{\text{imp}}$ to predict counterfactual outcomes. The imputation-based KL-Divergence strategy consistently and significantly outperforms all other non-oracle methods. This AF effectively identifies samples predicted to have the largest shift in their class probability distribution ([Figure˜3](https://arxiv.org/html/2505.16791v2#S5.F3)). In contrast, imputation-based strategies relying on rank change, final uncertainty, or final probability are considerably weaker, suggesting that quantifying the change in prediction is more effective than estimating the final state. While our primary results with respect to imputation-based AFs use DDPMs, a BC-VAE variant offers significantly faster inference for a minor trade-off in performance ([Tables˜6](https://arxiv.org/html/2505.16791v2#S7.T6) and [F](https://arxiv.org/html/2505.16791v2#A6)).
The relative performance ranking of these strategies is largely consistent across all datasets, including the large-scale UKBB cohort with approximately 100,000 samples and 15 modalities. This confirms the robustness and scalability of our framework in a challenging setting. In summary, our results affirm the superiority of the imputation-based KL-Divergence strategy, which achieved substantial and reliable gains over all baselines and heuristic methods. Additional results in [Appendices˜G](https://arxiv.org/html/2505.16791v2#A7), [H](https://arxiv.org/html/2505.16791v2#A8) and [I](https://arxiv.org/html/2505.16791v2#A9).

## 7 Discussion

**Table 6: Efficiency analysis for different architectures.**
| Architecture | Train (sec) $\downarrow$ | Validation (sec) $\downarrow$ | Parameters (M) $\downarrow$ |
| --- | --- | --- | --- |
| Late fusion | 0.02 | 0.015 | 86.5 |
| Late fusion w/ DDPMs | 0.17 | 0.16 | 125 |
| Late fusion w/ BC-VAEs | 0.08 | 0.08 | 313 |

We introduce CAMA to address the challenge of strategic data acquisition under budget constraints. Our experiments consistently demonstrate that imputation-based AFs provide a robust and effective solution. In the following, we discuss the key implications.
The ability of oracles to yield gains exceeding that of a model using complete multimodal data for all samples ([Figure˜3](https://arxiv.org/html/2505.16791v2#S5.F3) (b)), suggests that an underlying predictive model can achieve better global performance with a strategic curation of samples, rather than applying all modalities across the cohort. This likely occurs because additional modalities may introduce variance, redundancy, or conflicting information that imperfect models cannot optimally reconcile. The oracles circumvent this by selecting only additional modalities beneficial to the global metric.
To our surprise, the imputation-based KL-Divergence AF can slightly outperform the corresponding upper-bound heuristic ([Table˜5](https://arxiv.org/html/2505.16791v2#S6.T5)). Conversely, the substantial performance gap between the rank-change heuristic and its imputation-based counterpart suggests that global, rank-based metrics may be particularly vulnerable to imputation noise. While the KL-Divergence AF demonstrated strong performance, not all imputation-based AFs consistently outperformed simpler strategies across all datasets or endpoints ([Appendices˜H](https://arxiv.org/html/2505.16791v2#A8) and [I](https://arxiv.org/html/2505.16791v2#A9)). This indicates that optimal CAMA AFs can be context-dependent and that effectiveness hinges on how imputations are leveraged rather than on imputation quality alone.
We show CAMA s robustness to imputation errors since $f_{\text{imp}}$ models a distribution of plausible outcomes rather than aiming for a single reconstruction. By averaging the expected impact across this distribution, the acquisition decision becomes less sensitive to uncertainty. Additionally, the primary KL-Divergence AF is resilient to noise, as it prioritizes samples expected to cause a large predictive shift, effectively ignoring minor imputation errors.
Taken together, CAMA is not only practical for constrained settings, but also reveals insights into multimodal behavior. The successful KL-Divergence strategy and the surprising oracle performance underscore that the value of an additional modality is not absolute but highly contextual. The most effective AFs are not those that simply predict an outcome, but estimate the magnitude of the predictive shift.

## 8 Conclusion and Future Work

We introduce CAMA, a novel setting addressing the real-world challenge of optimizing global discriminative performance through strategic test-time acquisition of additional modalities under resource constraints. Our evaluation across multiple multimodal datasets shows that imputation-based AFs can effectively guide resource allocation under cohort-level constraints. The generally consistent relative ordering of AFs across diverse datasets and the low variance in overall results lend confidence to the robustness of our core findings. In settings such as healthcare, strategic allocation of costly or invasive diagnostic procedures is essential, and our approach offers a promising direction for these applications.
Future work includes extending CAMA to multi-class problems or regression tasks, exploring additional imputation techniques, directly optimizing cohort-level metrics, and dynamically selecting which modality to acquire.

#### Acknowledgement

We would like to thank Stefan Hegselmann and Lucas Arnoldt for the supportive and insightful discussions throughout the research and development process. The authors acknowledge the Scientific Computing of the IT Division at the Charité - Universitätsmedizin Berlin for providing computational resources that have contributed to the research results reported in this paper. This research has been conducted using the UK Biobank Resource under application number 49966.

## References

- Aggarwal et al. (2020)
Umang Aggarwal, Adrian Popescu, and Céline Hudelot.
Active Learning for Imbalanced Datasets.
In *IEEE Winter Conference on Applications of Computer Vision, WACV 2020, Snowmass Village, CO, USA, March 1-5, 2020*, pp. 1417–1426. IEEE, 2020.
doi: 10.1109/WACV45572.2020.9093475.
- Asgaonkar et al. (2024)
Vedang Asgaonkar, Aditya Jain, and Abir De.
Generator Assisted Mixture of Experts for Feature Acquisition in Batch.
In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (eds.), *Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada*, pp. 10927–10934. AAAI Press, 2024.
doi: 10.1609/AAAI.V38I10.28967.
- Ash et al. (2020)
Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal.
Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds.
In *8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020*. OpenReview.net, 2020.
- Astorga et al. (2024)
Nicolás Astorga, Tennison Liu, Nabeel Seedat, and Mihaela van der Schaar.
Active Learning with LLMs for Partially Observed and Cost-Aware Scenarios.
In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), *Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024*, 2024.
- Ba et al. (2016)
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton.
Layer Normalization, July 2016.
arXiv:1607.06450 [stat].
- Baja et al. (2025)
Hilmy Baja, Michiel Kallenberg, and Ioannis N. Athanasiadis.
To Measure or Not: A Cost-Sensitive, Selective Measuring Environment for Agricultural Management Decisions with Reinforcement Learning, January 2025.
arXiv:2501.12823 [cs].
- Das et al. (2022)
Gyanendra Das, Xavier Thomas, Anant Raj, and Vikram Gupta.
MAViC: Multimodal Active Learning for Video Captioning, December 2022.
arXiv:2212.11109 [cs].
- Defazio et al. (2024)
Aaron Defazio, Xingyu Yang, Ahmed Khaled, Konstantin Mishchenko, Harsh Mehta, and Ashok Cutkosky.
The Road Less Scheduled.
In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), *Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024*, 2024.
- Devlin et al. (2019)
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.
In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), *Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)*, pp. 4171–4186. Association for Computational Linguistics, 2019.
doi: 10.18653/V1/N19-1423.
- Dosovitskiy et al. (2021)
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.
*9th International Conference on Learning Representations, ICLR 2021*, 2021.
- Freyberg et al. (2024)
Jan Freyberg, Abhijit Guha Roy, Terry Spitz, Beverly Freeman, Mike Schaekermann, Patricia Strachan, Eva Schnider, Renee Wong, Dale R. Webster, Alan Karthikesalingam, Yun Liu, Krishnamurthy Dvijotham, and Umesh Telang.
MINT: A wrapper to make multi-modal and multi-image AI models interactive, January 2024.
arXiv:2401.12032 [cs].
- Ghosh & Lan (2023)
Aritra Ghosh and Andrew S. Lan.
DiFA: Differentiable Feature Acquisition.
In Brian Williams, Yiling Chen, and Jennifer Neville (eds.), *Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023*, pp. 7705–7713. AAAI Press, 2023.
doi: 10.1609/AAAI.V37I6.25934.
- Gong et al. (2019)
Wenbo Gong, Sebastian Tschiatschek, Sebastian Nowozin, Richard E Turner, José Miguel Hernández-Lobato, and Cheng Zhang.
Icebreaker: Element-wise Efficient Information Acquisition with a Bayesian Deep Latent Gaussian Model.
In H. Wallach, H. Larochelle, A. Beygelzimer, F. d’ Alché-Buc, E. Fox, and R. Garnett (eds.), *Advances in Neural Information Processing Systems*, volume 32. Curran Associates, Inc., 2019.
- Goodfellow et al. (2014)
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio.
Generative Adversarial Nets.
In Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q. Weinberger (eds.), *Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada*, pp. 2672–2680, 2014.
- Han & Kang (2021)
Jongmin Han and Seokho Kang.
Active learning with missing values considering imputation uncertainty.
*Knowledge-Based Systems*, 224:107079, July 2021.
ISSN 09507051.
doi: 10.1016/j.knosys.2021.107079.
- He et al. (2015)
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification.
In *2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015*, pp. 1026–1034. IEEE Computer Society, 2015.
doi: 10.1109/ICCV.2015.123.
- He et al. (2024)
Yifei He, Runxiang Cheng, Gargi Balasubramaniam, Yao-Hung Hubert Tsai, and Han Zhao.
Efficient Modality Selection in Multimodal Learning.
*Journal of Machine Learning Research*, 25(47):1–39, 2024.
- Higgins et al. (2017)
Irina Higgins, Loïc Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner.
beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework.
In *5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings*. OpenReview.net, 2017.
- Hinton & Zemel (1993)
Geoffrey E. Hinton and Richard S. Zemel.
Autoencoders, Minimum Description Length and Helmholtz Free Energy.
In Jack D. Cowan, Gerald Tesauro, and Joshua Alspector (eds.), *Advances in Neural Information Processing Systems 6, [7th NIPS Conference, Denver, Colorado, USA, 1993]*, pp. 3–10. Morgan Kaufmann, 1993.
- Ho et al. (2020)
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising Diffusion Probabilistic Models.
In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), *Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual*, 2020.
- Hoarau et al. (2025)
Arthur Hoarau, Benjamin Quost, Sébastien Destercke, and Willem Waegeman.
Reducing Aleatoric and Epistemic Uncertainty through Multi-modal Data Acquisition, January 2025.
arXiv:2501.18268 [cs].
- Holzmüller et al. (2023)
David Holzmüller, Viktor Zaverkin, Johannes Kästner, and Ingo Steinwart.
A Framework and Benchmark for Deep Batch Active Learning for Regression.
*J. Mach. Learn. Res.*, 24:164:1–164:81, 2023.
- Huang et al. (2021)
Yu Huang, Chenzhuang Du, Zihui Xue, Xuanyao Chen, Hang Zhao, and Longbo Huang.
What Makes Multi-Modal Learning Better than Single (Provably).
In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), *Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual*, pp. 10944–10956, 2021.
- Janisch et al. (2020)
Jaromír Janisch, Tomáš Pevný, and Viliam Lisý.
Classification with costly features as a sequential decision-making problem.
*Machine Learning*, 109(8):1587–1615, August 2020.
ISSN 0885-6125, 1573-0565.
doi: 10.1007/s10994-020-05874-8.
- Kingma & Welling (2014)
Diederik P. Kingma and Max Welling.
Auto-Encoding Variational Bayes.
In Yoshua Bengio and Yann LeCun (eds.), *2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings*, 2014.
- Kirsch et al. (2019)
Andreas Kirsch, Joost van Amersfoort, and Yarin Gal.
BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning.
In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché Buc, Emily B. Fox, and Roman Garnett (eds.), *Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada*, pp. 7024–7035, 2019.
- Kleist et al. (2025)
Henrik von Kleist, Alireza Zamanian, Ilya Shpitser, and Narges Ahmidi.
Evaluation of Active Feature Acquisition Methods for Time-varying Feature Settings.
*Journal of Machine Learning Research*, 26(60):1–84, 2025.
- Kossen et al. (2023)
Jannik Kossen, Cătălina Cangea, Eszter Vértes, Andrew Jaegle, Viorica Patraucean, Ira Ktena, Nenad Tomasev, and Danielle Belgrave.
Active Acquisition for Multimodal Temporal Data: A Challenging Decision-Making Task.
*Transactions on Machine Learning Research*, 2023.
- Lewis et al. (2021)
Sarah Lewis, Tatiana Matejovicova, Yingzhen Li, Angus Lamb, Yordan Zaykov, Miltiadis Allamanis, and Cheng Zhang.
Accurate Imputation and Efficient Data Acquisition with Transformer-based VAEs.
In *Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021*, 2021.
- Li et al. (2025)
Dongyuan Li, Zhen Wang, Yankai Chen, Renhe Jiang, Weiping Ding, and Manabu Okumura.
A Survey on Deep Active Learning: Recent Advances and New Frontiers.
*IEEE Trans. Neural Networks Learn. Syst.*, 36(4):5879–5899, 2025.
doi: 10.1109/TNNLS.2024.3396463.
- Li & Oliva (2021)
Yang Li and Junier Oliva.
Active Feature Acquisition with Generative Surrogate Models.
In Marina Meila and Tong Zhang (eds.), *Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event*, volume 139 of *Proceedings of Machine Learning Research*, pp. 6450–6459. PMLR, 2021.
- Li & Oliva (2024)
Yang Li and Junier Oliva.
Distribution Guided Active Feature Acquisition, October 2024.
arXiv:2410.03915 [cs].
- Long (2022)
Quan Long.
Multimodal information gain in Bayesian design of experiments.
*Comput. Stat.*, 37(2):865–885, 2022.
doi: 10.1007/S00180-021-01145-9.
- Ma et al. (2019)
Chao Ma, Sebastian Tschiatschek, Konstantina Palla, José Miguel Hernández-Lobato, Sebastian Nowozin, and Cheng Zhang.
EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE.
In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), *Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA*, volume 97 of *Proceedings of Machine Learning Research*, pp. 4234–4243. PMLR, 2019.
- Natarajan et al. (2018)
Sriraam Natarajan, Srijita Das, Nandini Ramanan, Gautam Kunapuli, and Predrag Radivojac.
On Whom Should I Perform this Lab Test Next? An Active Feature Elicitation Approach.
In *Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence*, pp. 3498–3505, Stockholm, Sweden, July 2018. International Joint Conferences on Artificial Intelligence Organization.
ISBN 978-0-9992411-2-7.
doi: 10.24963/ijcai.2018/486.
- Norcliffe et al. (2025)
Alexander Luke Ian Norcliffe, Changhee Lee, Fergus Imrie, Mihaela van der Schaar, and Pietro Lio.
Information Bottleneck for Active Feature Acquisition, 2025.
- Peebles & Xie (2023)
William Peebles and Saining Xie.
Scalable Diffusion Models with Transformers.
In *IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023*, pp. 4172–4182. IEEE, 2023.
- Peis et al. (2022)
Ignacio Peis, Chao Ma, and José Miguel Hernández-Lobato.
Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo.
In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), *Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022*, 2022.
- Rahbar et al. (2025)
Arman Rahbar, Linus Aronsson, and Morteza Haghir Chehreghani.
A Survey on Active Feature Acquisition Strategies, February 2025.
arXiv:2502.11067 [cs].
- Raj & Bach (2022)
Anant Raj and Francis R. Bach.
Convergence of Uncertainty Sampling for Active Learning.
In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), *International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA*, volume 162 of *Proceedings of Machine Learning Research*, pp. 18310–18331. PMLR, 2022.
- Ren et al. (2022)
Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Brij B. Gupta, Xiaojiang Chen, and Xin Wang.
A Survey of Deep Active Learning.
*ACM Computing Surveys*, 54(9):1–40, December 2022.
ISSN 0360-0300, 1557-7341.
doi: 10.1145/3472291.
- Rombach et al. (2022)
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-Resolution Image Synthesis with Latent Diffusion Models.
In *IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022*, pp. 10674–10685. IEEE, 2022.
doi: 10.1109/CVPR52688.2022.01042.
- Rudovic et al. (2019)
Ognjen Rudovic, Meiru Zhang, Björn W. Schuller, and Rosalind W. Picard.
Multi-modal Active Learning From Human Data: A Deep Reinforcement Learning Approach.
In Wen Gao, Helen Mei-Ling Meng, Matthew Turk, Susan R. Fussell, Björn W. Schuller, Yale Song, and Kai Yu (eds.), *International Conference on Multimodal Interaction, ICMI 2019, Suzhou, China, October 14-18, 2019*, pp. 6–15. ACM, 2019.
doi: 10.1145/3340555.3353742.
- Saporta et al. (2024)
Adriel Saporta, Aahlad Puli, Mark Goldstein, and Rajesh Ranganath.
Contrasting with Symile: Simple Model-Agnostic Representation Learning for Unlimited Modalities.
In *Advances in Neural Information Processing Systems*, 2024.
- Settles (2012)
Burr Settles.
*Active Learning*.
Synthesis Lectures on Artificial Intelligence and Machine Learning. Springer International Publishing, Cham, 2012.
ISBN 978-3-031-00432-2 978-3-031-01560-1.
doi: 10.1007/978-3-031-01560-1.
- Shapley (1953)
L. S. Shapley.
A Value for n-Person Games.
In Harold William Kuhn and Albert William Tucker (eds.), *Contributions to the Theory of Games (AM-28), Volume II*, pp. 307–318. Princeton University Press, December 1953.
ISBN 978-1-4008-8197-0.
doi: 10.1515/9781400881970-018.
- Shen et al. (2023)
Meng Shen, Yizheng Huang, Jianxiong Yin, Heqing Zou, Deepu Rajan, and Simon See.
Towards Balanced Active Learning for Multimodal Classification.
In *Proceedings of the 31st ACM International Conference on Multimedia*, pp. 3434–3445, October 2023.
doi: 10.1145/3581783.3612463.
- Shim et al. (2018)
Hajin Shim, Sung Ju Hwang, and Eunho Yang.
Joint Active Feature Acquisition and Classification with Variable-Size Set Encoding.
In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), *Advances in Neural Information Processing Systems*, volume 31. Curran Associates, Inc., 2018.
- Singhal et al. (2025)
Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R. Pfohl, Heather Cole-Lewis, Darlene Neal, Qazi Mamunur Rashid, Mike Schaekermann, Amy Wang, Dev Dash, Jonathan H. Chen, Nigam H. Shah, Sami Lachgar, Philip Andrew Mansfield, Sushant Prakash, Bradley Green, Ewa Dominowska, Blaise Agüera Y Arcas, Nenad Tomašev, Yun Liu, Renee Wong, Christopher Semturs, S. Sara Mahdavi, Joelle K. Barral, Dale R. Webster, Greg S. Corrado, Yossi Matias, Shekoofeh Azizi, Alan Karthikesalingam, and Vivek Natarajan.
Toward expert-level medical question answering with large language models.
*Nature Medicine*, January 2025.
ISSN 1078-8956, 1546-170X.
doi: 10.1038/s41591-024-03423-7.
- Soenksen et al. (2022a)
Luis R. Soenksen, Yu Ma, Cynthia Zeng, Leonard Boussioux, Kimberly Villalobos Carballo, Liangyuan Na, Holly M. Wiberg, Michael L. Li, Ignacio Fuentes, and Dimitris Bertsimas.
Integrated multimodal artificial intelligence framework for healthcare applications.
*npj Digital Medicine*, 5(1):149, September 2022a.
ISSN 2398-6352.
doi: 10.1038/s41746-022-00689-4.
- Soenksen et al. (2022b)
Luis R Soenksen, Yu Ma, Cynthia Zeng, Leonard David Jean Boussioux, Kimberly Villalobos Carballo, Liangyuan Na, Holly Wiberg, Michael Li, Ignacio Fuentes, and Dimitris Bertsimas.
Code for generating the HAIM multimodal dataset of MIMIC-IV clinical data and x-rays, 2022b.
- Sudlow et al. (2015)
Cathie Sudlow, John Gallacher, Naomi Allen, Valerie Beral, Paul Burton, John Danesh, Paul Downey, Paul Elliott, Jane Green, Martin Landray, Bette Liu, Paul Matthews, Giok Ong, Jill Pell, Alan Silman, Alan Young, Tim Sprosen, Tim Peakman, and Rory Collins.
UK Biobank: An Open Access Resource for Identifying the Causes of a Wide Range of Complex Diseases of Middle and Old Age.
*PLOS Medicine*, 12(3):e1001779, March 2015.
ISSN 1549-1676.
doi: 10.1371/journal.pmed.1001779.
- Sutter et al. (2021)
Thomas M. Sutter, Imant Daunhawer, and Julia E. Vogt.
Generalized Multimodal ELBO.
In *9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021*. OpenReview.net, 2021.
- Szegedy et al. (2016)
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.
Rethinking the Inception Architecture for Computer Vision.
In *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, June 2016.
- Tran et al. (2019)
Toan Tran, Thanh-Toan Do, Ian Reid, and Gustavo Carneiro.
Bayesian Generative Active Deep Learning.
In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), *Proceedings of the 36th International Conference on Machine Learning*, volume 97 of *Proceedings of Machine Learning Research*, pp. 6295–6304. PMLR, June 2019.
- Valancius et al. (2024)
Michael Valancius, Max Lennon, and Junier Oliva.
Acquisition Conditioned Oracle for Nongreedy Active Feature Acquisition.
In *Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024*. OpenReview.net, 2024.
- Vaswani et al. (2017)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.
Attention is All you Need.
In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), *Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA*, pp. 5998–6008, 2017.
- Vivar et al. (2020)
Gerome Vivar, Kamilia Mullakaeva, Andreas Zwergal, Nassir Navab, and Seyed-Ahmad Ahmadi.
Peri-Diagnostic Decision Support Through Cost-Efficient Feature Acquisition at Test-Time.
In Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, and Leo Joskowicz (eds.), *Medical Image Computing and Computer Assisted Intervention - MICCAI 2020*, volume 12262, pp. 572–581. Springer International Publishing, Cham, 2020.
ISBN 978-3-030-59712-2 978-3-030-59713-9.
doi: 10.1007/978-3-030-59713-9_55.
Series Title: Lecture Notes in Computer Science.
- Wang et al. (2023)
Yuanzhi Wang, Yong Li, and Zhen Cui.
Incomplete Multimodality-Diffused Emotion Recognition.
In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), *Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023*, 2023.
- Wesego & Rooshenas (2024)
Daniel Wesego and Pedram Rooshenas.
Score-Based Multimodal Autoencoder.
*Transactions on Machine Learning Research*, 2024.
ISSN 2835-8856.
- Wu et al. (2024)
Renjie Wu, Hu Wang, Hsiang-Ting Chen, and Gustavo Carneiro.
Deep Multimodal Learning with Missing Modality: A Survey, October 2024.
arXiv:2409.07825 [cs].
- Yang et al. (2025)
Sisi Yang, Yuanyuan Zhang, Ziliang Ye, Yanjun Zhang, Xiaoqin Gan, Yu Huang, Hao Xiang, Yiting Wu, Yiwei Zhang, and Xianhui Qin.
Plasma proteomics for risk prediction and identification of novel drug targets in systemic lupus erythematosus.
*Rheumatology*, 64(6):4032–4040, June 2025.
ISSN 1462-0324, 1462-0332.
doi: 10.1093/rheumatology/keaf055.
- Zadeh et al. (2018)
Amir Zadeh, Paul Pu Liang, Soujanya Poria, E. Cambria, and Louis-Philippe Morency.
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph.
In *Annual Meeting of the Association for Computational Linguistics*, 2018.
- Zannone et al. (2019)
Sara Zannone, Jose Miguel Hernandez Lobato, Cheng Zhang, and Konstantina Palla.
ODIN: Optimal Discovery of High-value INformation Using Model-based Deep Reinforcement Learning.
In *Real-world Sequential Decision Making Workshop, ICML*, June 2019.
- Zhang et al. (2023)
Jifan Zhang, Shuai Shao, Saurabh Verma, and Robert D. Nowak.
Algorithm Selection for Deep Active Learning with Imbalanced Datasets.
In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), *Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023*, 2023.
- Zhang et al. (2024)
Xulu Zhang, Wengyu Zhang, Xiaoyong Wei, Jinlin Wu, Zhaoxiang Zhang, Zhen Lei, and Qing Li.
Generative active learning for image synthesis personalization.
In *ACM Multimedia 2024*, 2024.
- Zhu & Bento (2017)
Jia-Jie Zhu and José Bento.
Generative Adversarial Active Learning, November 2017.
arXiv:1702.07956 [cs].

## Appendix A Broader Impact and Ethics

The CAMA setting introduced in this paper offers potential for positive broader impacts, primarily by enabling more efficient use of resources in multimodal machine learning. In resource-constrained fields like healthcare, this could facilitate access to more robust and comprehensive model performance by strategically guiding the acquisition of costly or limited additional data modalities. This could translate to improved diagnostic accuracy where such data is critical but not uniformly available for all samples in a cohort.
However, the deployment of CAMA, particularly its core function of ranking and prioritizing samples for modality acquisition, necessitates careful ethical consideration. This raises concerns about equity and fairness, especially if the downstream application impacts critical decisions. A significant risk is the potential to introduce biases, including racial, socioeconomic, or other demographic biases. Therefore, the development and application of CAMA must be approached with a strong commitment to ethical principles.

## Appendix B Reproducibility

To ensure the reproducibility of our results, we provide the following details:

##### Code

The complete source code used for all experiments will be made publicly available on GitHub upon publication. The repository will include scripts for model training and evaluation.

##### Hyperparameters

All hyperparameters, including learning rates, batch sizes, and model-specific parameters, are explicitly listed in [Appendix˜D](https://arxiv.org/html/2505.16791v2#A4). Additionally, we provide the complete sweep configurations used for hyperparameter tuning to allow for full replication of our optimization process.

##### Datasets

Three of the four datasets used in our evaluation are publicly available. For more details see [Section˜5](https://arxiv.org/html/2505.16791v2#S5) and [Appendix˜E](https://arxiv.org/html/2505.16791v2#A5).

##### Implementation Details

We provide a full section in [Appendix˜D](https://arxiv.org/html/2505.16791v2#A4) and a dedicated paragraph in [Section˜5](https://arxiv.org/html/2505.16791v2#S5) describing implementation details that we found to be crucial.

## Appendix C Details about Acquisition Function Strategies

### C.1 AUROC and AUPRC

To derive the proposed acquisition strategies, we briefly explain the metrics used in the following paragraphs.

##### AUROC

The Area Under the Receiver Operating Characteristic (AUROC) measures the model’s ability to discriminate between positive and negative classes and is defined as

$$ $\text{AUROC}({\mathbf{y}},{\mathbf{s}})=\frac{1}{N_{+}N_{-}}\sum_{i:{\textnormal{y}}_{i}=1}\sum_{j:{\textnormal{y}}_{j}=0}\left(\mathbb{I}({\textnormal{s}}_{i}>{\textnormal{s}}_{j})+\frac{1}{2}\mathbb{I}({\textnormal{s}}_{i}={\textnormal{s}}_{j})\right)$ (7) $$

where $N_{+}=|\{i\mid{\textnormal{y}}_{i}=1\}|$ and $N_{-}=|\{j\mid{\textnormal{y}}_{j}=0\}|$.

##### AUPRC

The Area Under the Precision-Recall Curve (AUPRC) summarizes the trade-off between precision ($P_{t}$) and recall ($R_{t}$) across different decision thresholds $t$ and is defined as

$$ $\displaystyle\text{AUPRC}({\mathbf{y}},{\mathbf{p}})=\sum_{k=1}^{N^{\prime}}(R_{k}-R_{k-1})P_{k}$ (8) $$

where points $(R_{k},P_{k})$ are ordered by threshold from the PR curve, $N^{\prime}$ is the number of unique thresholds, and ${\mathbf{p}}=\sigma({\mathbf{s}})$.

### C.2 Oracle Acquisition Strategies: Exact Gain Calculation

Oracle acquisition strategies serve as theoretical upper limits for the performance of greedy acquisition approaches. They operate under the ideal assumption that the true labels ${\textnormal{y}}_{i}$ and the outcome scores ${\textnormal{s}}_{i}^{\textnormal{full}}$ are known for all samples $i\in\{1,\dots,N\}$. While not implementable in practice, these oracle strategies provide benchmarks by selecting samples based on their exact marginal contribution to the global evaluation metric. The general principle is to iteratively select $\beta$ samples. At each step, among the samples for which the additional modality has not yet been acquired, the oracle picks the one that provides the largest true immediate gain to the chosen global metric.

##### AUROC Oracle

The AUROC oracle strategy aims to maximize the cohort’s AUROC by identifying, at each step, the sample $i$ that yields the largest immediate increase in this metric if its additional modality were acquired (changing its score from ${\textnormal{s}}_{i}^{\textnormal{avail}}$ to ${\textnormal{s}}_{i}^{\textnormal{full}}$), i.e., a greedy selection. This prospective increase is quantified by the marginal gain $g_{i}^{\text{\lx@glossaries@gls@link{acronym}{auroc}{{{}}AUROC}}}$. The components of this gain, $g_{i}^{\text{\lx@glossaries@gls@link{acronym}{auroc}{{{}}AUROC}}}({\textnormal{y}}_{i}=1)$ (for positive samples) and $g_{i}^{\text{\lx@glossaries@gls@link{acronym}{auroc}{{{}}AUROC}}}({\textnormal{y}}_{i}=0)$ (for negative samples), reflect the net change in favorable pairwise score comparisons relative to samples of the other class. Recall the definition of AUROC from [Equation˜7](https://arxiv.org/html/2505.16791v2#A3.E7):

$$ $\text{AUROC}({\mathbf{y}},{\mathbf{s}})=\frac{1}{N_{+}N_{-}}\sum_{i:{\textnormal{y}}_{i}=1}\sum_{j:{\textnormal{y}}_{j}=0}\left(\mathbb{I}({\textnormal{s}}_{i}>{\textnormal{s}}_{j})+\frac{1}{2}\mathbb{I}({\textnormal{s}}_{i}={\textnormal{s}}_{j})\right).$ $$

The total marginal gain for sample $i$, representing the exact change in the cohort’s AUROC value, is then, by considering positive and negative samples and neglecting the normalization factor:

$$ $\begin{split}g_{i}^{\text{\lx@glossaries@gls@link{acronym}{auroc}{{{}}AUROC}}}({\textnormal{y}}_{i}=1)&=\sum_{j:{\textnormal{y}}_{j}=0}\bigg(\mathbb{I}({\textnormal{s}}_{i}^{\textnormal{full}}>{\textnormal{s}}_{j}^{\textnormal{avail}})-\mathbb{I}({\textnormal{s}}_{i}^{\textnormal{avail}}>{\textnormal{s}}_{j}^{\textnormal{avail}})\\ &\quad+\frac{1}{2}\Big[\mathbb{I}({\textnormal{s}}_{i}^{\textnormal{full}}={\textnormal{s}}_{j}^{\textnormal{avail}})-\mathbb{I}({\textnormal{s}}_{i}^{\textnormal{avail}}={\textnormal{s}}_{j}^{\textnormal{avail}})\Big]\bigg)\end{split}$ (9) $$

$$ $\begin{split}g_{i}^{\text{\lx@glossaries@gls@link{acronym}{auroc}{{{}}AUROC}}}({\textnormal{y}}_{i}=0)&=\sum_{j:{\textnormal{y}}_{j}=1}\bigg(\mathbb{I}({\textnormal{s}}_{j}^{\textnormal{avail}}>{\textnormal{s}}_{i}^{\textnormal{full}})-\mathbb{I}({\textnormal{s}}_{j}^{\textnormal{avail}}>{\textnormal{s}}_{i}^{\textnormal{avail}})\\ &\quad+\frac{1}{2}\Big[\mathbb{I}({\textnormal{s}}_{j}^{\textnormal{avail}}={\textnormal{s}}_{i}^{\textnormal{full}})-\mathbb{I}({\textnormal{s}}_{j}^{\textnormal{avail}}={\textnormal{s}}_{i}^{\textnormal{avail}})\Big]\bigg)\end{split}$ (10) $$

$$ $g_{i}^{\text{\lx@glossaries@gls@link{acronym}{auroc}{{{}}AUROC}}}=\frac{1}{N_{+}N_{-}}\left(g_{i}^{\text{\lx@glossaries@gls@link{acronym}{auroc}{{{}}AUROC}}}({\textnormal{y}}_{i}=1)\cdot\mathbb{I}({\textnormal{y}}_{i}=1)+g_{i}^{\text{\lx@glossaries@gls@link{acronym}{auroc}{{{}}AUROC}}}({\textnormal{y}}_{i}=0)\cdot\mathbb{I}({\textnormal{y}}_{i}=0)\right)$ (11) $$

##### AUPRC Oracle

The AUPRC oracle strategy seeks to maximize the cohort’s AUPRC. It operates by identifying, at each step, the sample $i$ which, if its additional modality were acquired (changing its score from ${\textnormal{s}}_{i}^{\textnormal{avail}}$ to ${\textnormal{s}}_{i}^{\textnormal{full}}$), would yield the largest immediate increase in the global AUPRC value, i.e., a greedy selection. This marginal gain, $g_{i}^{\text{\lx@glossaries@gls@link{acronym}{auprc}{{{}}AUPRC}}}$, represents the exact change in the cohort’s AUPRC.
To calculate the marginal gain for a sample $i$, we compute the change in the cohort’s AUPRC. Let ${\mathbf{s}}^{\textnormal{current}}$ be the vector of scores for the whole cohort. We define a new vector, ${\mathbf{s}}^{\textnormal{updated}}$, which is identical to ${\mathbf{s}}^{\textnormal{current}}$ except that for sample i, the score is changed from ${\textnormal{s}}_{i}^{\textnormal{avail}}$ to ${\textnormal{s}}_{i}^{\textnormal{full}}$. The marginal gain is then:

$$ $g_{i}^{\text{\lx@glossaries@gls@link{acronym}{auprc}{{{}}AUPRC}}}=\text{\lx@glossaries@gls@link{acronym}{auprc}{{{}}AUPRC}}({\mathbf{y}},{\mathbf{p}}^{\textnormal{updated}})-\text{\lx@glossaries@gls@link{acronym}{auprc}{{{}}AUPRC}}({\mathbf{y}},{\mathbf{p}}^{\textnormal{current}})$ (12) $$

where ${\mathbf{p}}^{\textnormal{current}}=\sigma({\mathbf{s}}^{\textnormal{current}})$ and ${\mathbf{p}}^{\textnormal{updated}}=\sigma({\mathbf{s}}^{\textnormal{updated}})$.

### C.3 Upper-Bound Heuristic Strategies

The preceding oracle strategies make the assumption of perfect foresight into both the true labels ${\textnormal{y}}_{i}$ and the exact outcome scores ${\textnormal{s}}_{i}^{\textnormal{full}}$. We now introduce a distinct class of upper-bound heuristic strategies. These strategies still presume access to the true future scores ${\textnormal{s}}_{i}^{\textnormal{full}}$ for any sample $i$ if its additional modality were acquired. However, the following upper-bound heuristics are label-agnostic, i.e., the true label ${\textnormal{y}}_{i}$ of a candidate sample is not used when determining its priority for acquisition. Consequently, the selection principle for these strategies must rely on how the known change from an initial score ${\textnormal{s}}_{i}^{\textnormal{avail}}$ to the future score ${\textnormal{s}}_{i}^{\textnormal{full}}$ is expected to influence the global evaluation metric, without direct reference to the sample’s ground-truth label.

##### Maximum True Uncertainty Reduction

The uncertainty reduction strategy prioritizes acquiring the additional modality for samples where doing so is expected to yield the largest decrease in predictive uncertainty. For each sample $i$, uncertainty is quantified using the binary entropy $\mathcal{H}(p_{i})$ of its predicted probability $p_{i}$ for the positive class, defined as:

$$ $\mathcal{H}(p_{i})=-p_{i}\log_{2}p_{i}-(1-p_{i})\log_{2}(1-p_{i}),$ (13) $$

The acquisition strategy operates with knowledge of the initial probability $p_{i}^{\textnormal{avail}}=\sigma({\textnormal{s}}_{i}^{\textnormal{avail}})$ derived from the available modalities, and crucially, the true future probability $p_{i}^{\textnormal{full}}=\sigma({\textnormal{s}}_{i}^{\textnormal{full}})$ that would be obtained if the additional modality were acquired (where ${\textnormal{s}}_{i}^{\textnormal{full}}$ is the oracle score). The acquisition score $g_{i}^{\text{UR}}$ for sample $i$ is then the exact reduction in entropy:

$$ $g_{i}^{\text{UR}}=\mathcal{H}(p_{i}^{\textnormal{avail}})-\mathcal{H}(p_{i}^{\textnormal{full}}).$ (14) $$

Samples with higher $g_{i}^{\text{UR}}$ values, indicating a greater expected reduction in uncertainty, are prioritized for modality acquisition.

##### Maximum True Rank Change

This rank change strategy prioritizes samples whose relative standing within the cohort, based on predicted probability of belonging to the positive class, would change most significantly if the additional modality were acquired. For each sample $i$, we consider its rank $R(p_{i})$ when all $N$ samples in the cohort are ordered by their respective probabilities $p_{i}$. The acquisition score $g_{i}^{\text{RC}}$ for sample $i$ is defined as the absolute magnitude of this change in rank:

$$ $g_{i}^{\text{RC}}=|R(p_{i}^{\textnormal{full}})-R(p_{i}^{\textnormal{avail}})|.$ (15) $$

Samples exhibiting a higher $g_{i}^{\text{RC}}$ are prioritized for modality acquisition, since they are expected to cause the largest shift in the sample’s rank-ordered position relative to its peers.

##### KL-Divergence

The KL-Divergence acquisition strategy aims to identify samples for which acquiring the additional modality would lead to the largest change in the predicted probability distribution. Specifically, it quantifies the divergence from the predicted probability distribution based on the true future score, $P_{i}^{\textnormal{full}}\sim\text{Bernoulli}(p_{i}^{\textnormal{full}})$, back to the initial distribution based on baseline data, $P_{i}^{\textnormal{avail}}\sim\text{Bernoulli}(p_{i}^{\textnormal{avail}})$. This is measured by the KL-Divergence $D_{\mathrm{KL}}(P_{i}^{\textnormal{avail}}\|P_{i}^{\textnormal{full}})$ and can be defined as follows for an acquisition function:

$$ $\displaystyle g_{i}^{\text{KLD}}$ $\displaystyle=D_{\mathrm{KL}}\left(P_{i}^{\textnormal{avail}}\middle\|P_{i}^{\textnormal{full}}\right)$ (16) $\displaystyle=p_{i}^{\textnormal{avail}}\log_{2}\frac{p_{i}^{\textnormal{avail}}}{p_{i}^{\textnormal{full}}}+(1-p_{i}^{\textnormal{avail}})\log_{2}\frac{1-p_{i}^{\textnormal{avail}}}{1-p_{i}^{\textnormal{full}}}$ (17) $$

Samples with a higher $g_{i}^{\text{KLD}}$ are prioritized, as this indicates a greater discrepancy between the prediction based on available data and the prediction that would be made with the additional modality.

### C.4 Baseline Information Strategies

Shifting from approaches that leverage oracle knowledge of future scores (${\textnormal{s}}_{i}^{\textnormal{full}}$), the present section details methods serving as practical, label-agnostic baselines. They make acquisition decisions based exclusively on information derived from the initially available modality (${\textnormal{s}}_{i}^{\textnormal{avail}}$). A random acquisition strategy serves as a fundamental baseline.

##### Maximum Baseline Uncertainty

The Maximum Baseline Uncertainty strategy is a baseline that prioritizes samples for which the prediction based on the initially available modality is most uncertain. The acquisition score for sample $i$ is directly the binary entropy $\mathcal{H}(p_{i}^{\textnormal{avail}})$, as defined in [Equation˜13](https://arxiv.org/html/2505.16791v2#A3.E13):

$$ $g_{i}^{\text{UU}}=\mathcal{H}(p_{i}^{\textnormal{avail}}).$ (18) $$

Samples with a higher $g_{i}^{\text{UU}}$, i.e., $p_{i}^{\textnormal{avail}}$ closer to $0.5$, since the entropy $H(p_{i}^{\textnormal{avail}})$ is symmetric around $p_{i}^{\textnormal{avail}}=0.5$, are selected first.

##### Maximum Baseline Probability

This approach prioritizes acquiring the additional modality for samples that the baseline model already predicts as belonging to the positive class with high confidence. The acquisition score $g_{i}^{\text{UP}}$ for sample $i$ is simply its initial probability $p_{i}^{\textnormal{avail}}$ based on the available modality:

$$ $g_{i}^{\text{UP}}=p_{i}^{\textnormal{avail}},$ (19) $$

Samples with a higher $g_{i}^{\text{UP}}$ are prioritized for acquisition.

### C.5 Imputation-Based Strategies

Having explored strategies that assume perfect knowledge of the true labels ${\textnormal{y}}_{i}$ and/or future scores ${\textnormal{s}}_{i}^{\textnormal{full}}$, and simpler baselines relying only on current information ${\textnormal{s}}_{i}^{\textnormal{avail}}$, we now introduce methods aiming to bridge the gap by offering a practical and label-agnostic pathway to modality acquisition. They operate by utilizing an imputation model, $f_{\text{imp}}$, to generate a set of $K$ plausible future scores, denoted $\{{\textnormal{s}}_{i,k}^{\textnormal{imp}}\}_{k=1}^{K}$, conditioned on the initially available data ${\textnormal{s}}_{i}^{\textnormal{avail}}$. The core principle of these strategies is to then derive acquisition scores from statistics of this imputed score distribution, with the goal of emulating the decision-making process, but without requiring true future knowledge at test time.

##### Maximum Expected Probability

The Maximum Expected Probability strategy prioritizes samples which have the highest average probability of belonging to the positive class after modality acquisition. It relies on the set of $K$ imputed future probabilities $\{{\textnormal{p}}_{i,k}^{\textnormal{imp}}\}_{k=1}^{K}$, where each ${\textnormal{p}}_{i,k}^{\textnormal{imp}}=\sigma({\textnormal{s}}_{i,k}^{\textnormal{imp}})$ is derived from an imputed future score ${\textnormal{s}}_{i,k}^{\textnormal{imp}}$. The acquisition score $g_{i}^{\text{eP}}$ for sample $i$ is the mean of these imputed probabilities:

$$ $g_{i}^{\text{eP}}=\frac{1}{K}\sum_{k=1}^{K}{\textnormal{p}}_{i,k}^{\textnormal{imp}}.$ (20) $$

Samples with a higher $g_{i}^{\text{eP}}$ are selected, representing instances where the imputation model, on average, predicts a high likelihood of being positive if the additional modality were acquired.

##### Maximum Expected Uncertainty Reduction

The Maximum Expected Uncertainty Reduction strategy aims to select samples for which the acquisition of the additional modality is anticipated to yield the largest average decrease in predictive uncertainty ([Equation˜13](https://arxiv.org/html/2505.16791v2#A3.E13)). This strategy considers the initial entropy $\mathcal{H}(p_{i}^{\textnormal{avail}})$, and the distribution of entropies $\{\mathcal{H}({\textnormal{p}}_{i,k}^{\textnormal{imp}})\}_{k=1}^{K}$. The acquisition score $g_{i}^{\text{eUR}}$ is the difference between the initial entropy and the mean of the imputed future entropies:

$$ $g_{i}^{\text{eUR}}=\mathcal{H}(p_{i}^{\textnormal{avail}})-\frac{1}{K}\sum_{k=1}^{K}\mathcal{H}({\textnormal{p}}_{i,k}^{\textnormal{imp}}).$ (21) $$

Samples with higher $g_{i}^{\text{eUR}}$ are prioritized, indicating a greater expected clarification of the prediction upon acquiring the new modality.

##### Expected Rank Change

The Maximum Expected Rank Change strategy prioritizes samples for which the acquisition of the additional modality is anticipated to cause the largest change in their rank, relative to the initial ranking based on $p_{i}^{\textnormal{avail}}$. It aims to mirror the "Maximum True Rank Change" strategy by using imputed future probabilities.
Let $R(p_{i}^{\textnormal{avail}})$ denote the rank of sample $i$ when all $N$ samples in the cohort are ordered by their initial probabilities $p_{j}^{\textnormal{avail}}$ (for $j=1,\dots,N$). For each of the $K$ imputed future probabilities ${\textnormal{p}}_{i,k}^{\textnormal{imp}}$ for sample $i$, let $R({\textnormal{p}}_{i,k}^{\textnormal{imp}})$ denote the rank of sample $i$ if its probability were ${\textnormal{p}}_{i,k}^{\textnormal{imp}}$ while all other samples $j\neq i$ retain their initial probabilities $p_{j}^{\textnormal{avail}}$. The acquisition score $g_{i}^{\text{eRC}}$ is then the mean of the absolute differences between these imputed future ranks and the initial rank:

$$ $g_{i}^{\text{eRC}}=\frac{1}{K}\sum_{k=1}^{K}|R({\textnormal{p}}_{i,k}^{\textnormal{imp}})-R(p_{i}^{\textnormal{avail}})|.$ (22) $$

Samples with a higher $g_{i}^{\text{eRC}}$ are selected, as they are expected to experience the largest shift in their rank-ordered position relative to other samples in the cohort upon modality acquisition.

##### Expected KL-Divergence

The Expected KL-Divergence strategy selects samples where the initial probability distribution is expected to diverge most significantly from the future probability distributions derived from the $K$ imputed scores. The acquisition score $g_{i}^{\text{eKLD}}$ is the average KL-Divergence $D_{\mathrm{KL}}(P_{i}^{\textnormal{avail}}\|P_{i}^{(\textnormal{imp},k)})$ over the $K$ imputations:

$$ $g_{i}^{\text{eKLD}}=\frac{1}{K}\sum_{k=1}^{K}D_{\mathrm{KL}}\left(P_{i}^{\textnormal{avail}}\middle\|P_{i}^{(\textnormal{imp},k)}\right).$ (23) $$

A higher $g_{i}^{\text{eKLD}}$ indicates that, on average, the imputed future predictions substantially differ from the initial baseline prediction, suggesting a significant informational update from acquiring the additional modality.

## Appendix D Hyperparameters, Model Details and Compute Environment

We employ domain-specific encoders to process the respective modalities: for language inputs, we use a pre-trained BERT model (Devlin et al., 2019), for vision, a Vision Transformer (ViT) (Dosovitskiy et al., 2021). Other data types, e.g., temporal sequences, tabular data, or pre-extracted embeddings, are handled by Transformer encoders (Vaswani et al., 2017).
We use well-established hyperparameters from the literature for the modality-specific encoders and only optimize the remaining parameters. Notably, our experiments compared three approaches for normalizing the encoder output: No Normalization, Batch Normalization, and Layer Normalization. We found Layer Normalization to be particularly advantageous, as it both stabilized training convergence and significantly enhanced the performance of the DDPMs. We also evaluated the impact of using only the CLS token representation from the encoder versus leveraging the full output sequence. This comparison revealed no substantial effect on performance, suggesting the sufficiency of the CLS token representation for our task. Layers in the network are initialized using He initialization (He et al., 2015) if they were not pre-initialized by the specific encoder architecture. We find this particularly important for stabilizing the DDPMs during the early epochs of end-to-end model training.

We perform hyperparameter sweeps for the remaining parts of the designed model in the following ranges:

- •
Transformer Head
- -
Embedding dimension: [32, 64, 128, 256, 512, 1024]
- -
Feed-Forward network: [128, 256, 512, 1024, 2048]
- -
Dropout: [0, 0.1, 0.2]
- -
Number of heads: [4, 8, 16]
- -
Number of layers: [2, 4, 6, 8]
- •
DDPMs
- -
Embedding dimension: analogous to Transformer head
- -
Hidden dimension: [32, 64, 128, 256, 512, 1024]
- -
Dropout: [0, 0.1, 0.2]
- -
Number of heads: [4, 8, 16]
- -
Number of layers: [2, 4, 6, 8]
- -
Number of steps: [10, 25, 50, 100, 250, 500]
- •
ScheduleFree Optimizer
- -
Learning rate: [1e-1, 1e-2, 1e-3, 3e-4, 1e-4, 1e-5]
- -
Warmup steps: [0, 100, 200]
- -
Weight decay: [0, 0.01, 0.001]

The models are trained with early stopping but without any maximum number of epochs. For the imputation-based acquisition functions, $100$ DDPM samples are used during inference of the model.

Figure: Figure 4: The latent DDPM with its (de)noising functions. Coloring represents less noise in the latent space, starting with pure noise in $X_{i,T}=X_{1,T}$ with $T$ steps and modality $i$. The DDPM is conditioned with two non-missing latent spaces, each from one remaining modality respectively.
Refer to caption: x2.png

Our experiments are conducted on a High-Performance Cluster (HPC) with the following environment:

- 1.
21 Dell PowerEdge R7525 compute nodes, each with:
•
64 AMD Epyc cores (Rome)
•
512GB RAM
•
1 NVIDIA A100 40G GPU
- 2.
2 Dell PowerEdge XE8545 compute nodes, each with:
•
128 AMD Epyc cores (Milan)
•
512GB RAM
•
4 NVIDIA A100 40G GPUs (NVLink-connected)

## Appendix E Dataset Details

We evaluate CAMA on four diverse, real-world multimodal datasets, spanning domains from healthcare to emotion recognition.

##### MIMIC Symile

This clinical dataset is derived from the MIMIC database and is designed for predicting the diagnosis of ten classes (Fracture, Enlarged Cardiomediastinum, Consolidation, Atelectasis, Edema, Cardiomegaly, Lung Lesion, Lung Opacity, Pneumonia, Pneumothorax). It contains 10,345 samples from patients in intensive care units. For our experiments, we utilize three distinct modalities: laboratory values, chest X-ray images, and electrocardiograms (ECGs).

##### MIMIC HAIM

This healthcare benchmark also focuses on the diagnostic prediction of ten classes (Fracture, Enlarged Cardiomediastinum, Consolidation, Atelectasis, Edema, Cardiomegaly, Lung Lesion, Lung Opacity, Pneumonia, Pneumothorax). The bimodal dataset consists of 45,050 samples. The two modalities used in our study are laboratory values and chest X-ray images.

##### CMU-MOSEI

This large-scale benchmark targets multimodal sentiment analysis and emotion recognition with seven classes covering different emotions. It contains 22,856 video samples of speakers expressing opinions. The dataset comprises three modalities: vision, acoustics, and language. Notably, unlike the other datasets, we utilize the pre-computed embeddings provided by the authors rather than the raw data.

##### UK Biobank (UKBB)

The UK Biobank is a large-scale, prospective biomedical database from half a million UK participants. In our experiments, the costly modality targeted for acquisition is proteomics, which is available for only a fraction of the full cohort. We constructed a subset of 100,000 samples in which approximately half include proteomics data, accurately simulating a resource-constrained acquisition scenario. The 15 modalities utilized include electronic health records (EHRs), NMR metabolomics, proteomics, physical activity measurements, diet and alcohol consumption questionnaires, baseline characteristics, smoking status, physiological measurements, anthropometry, hand grip strength, cognitive function tests, ECGs, polygenic risk scores (PRS), and arterial stiffness measurements.

## Appendix F Results for Symile with BC-VAEs

**Table 7: Acquisition performance on Symile (AUROC) with Beta-Conditional Variational Auto Encoders. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $2.807\pm 0.326$ | $4.224\pm 0.501$ | $2.716\pm 0.147$ | $5.901\pm 0.861$ | $2.096\pm 0.078$ | $4.423$ |
| True KL-Div. | $1.009\pm 0.115$ | $0.752\pm 0.095$ | $0.855\pm 0.020$ | $0.689\pm 0.092$ | $0.900\pm 0.007$ | $0.800$ |
| True Rank | $0.714\pm 0.094$ | $0.728\pm 0.070$ | $0.853\pm 0.017$ | $0.777\pm 0.104$ | $0.890\pm 0.007$ | $0.719$ |
| True Uncert. | $0.939\pm 0.088$ | $0.735\pm 0.113$ | $0.571\pm 0.024$ | $0.106\pm 0.079$ | $0.719\pm 0.009$ | $0.555$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div. | $0.834\pm 0.060$ | $0.420\pm 0.109$ | $\mathbf{0.684\pm 0.017}$ | $\mathbf{0.527\pm 0.077}$ | $\mathbf{0.744\pm 0.013}$ | $\mathbf{0.584}$ |
| Prob. | $0.643\pm 0.073$ | $\mathbf{0.559\pm 0.037}$ | $0.489\pm 0.022$ | $-0.304\pm 0.170$ | $0.603\pm 0.009$ | $0.395$ |
| Rank | $0.252\pm 0.109$ | $0.281\pm 0.063$ | $0.526\pm 0.019$ | $0.327\pm 0.087$ | $0.444\pm 0.009$ | $0.366$ |
| Uncert. | $\mathbf{0.911\pm 0.073}$ | $0.557\pm 0.042$ | $0.593\pm 0.023$ | $0.162\pm 0.075$ | $0.637\pm 0.012$ | $0.519$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncert. | $0.862\pm 0.092$ | $0.397\pm 0.052$ | $0.510\pm 0.017$ | $0.423\pm 0.046$ | $0.592\pm 0.008$ | $0.477$ |
| Prob. | $0.127\pm 0.066$ | $0.508\pm 0.026$ | $0.526\pm 0.016$ | $0.054\pm 0.133$ | $0.462\pm 0.006$ | $0.388$ |
| Random | $0.429\pm 0.085$ | $0.290\pm 0.060$ | $0.497\pm 0.019$ | $0.102\pm 0.097$ | $0.480\pm 0.006$ | $0.350$ |

**Table 8: Acquisition performance on Symile (AUPRC) with Beta-Conditional Variational Auto Encoders. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $1.785\pm 0.110$ | $3.468\pm 0.317$ | $2.780\pm 0.146$ | $2.632\pm 0.146$ | $2.460\pm 0.099$ | $4.116$ |
| True KL-Div. | $0.798\pm 0.050$ | $0.700\pm 0.101$ | $0.828\pm 0.020$ | $0.686\pm 0.026$ | $0.888\pm 0.007$ | $0.766$ |
| True Rank | $0.736\pm 0.061$ | $0.625\pm 0.096$ | $0.822\pm 0.022$ | $0.733\pm 0.046$ | $0.843\pm 0.009$ | $0.689$ |
| True Uncert. | $0.778\pm 0.050$ | $0.623\pm 0.044$ | $0.606\pm 0.034$ | $0.206\pm 0.033$ | $0.731\pm 0.011$ | $0.513$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div. | $0.725\pm 0.062$ | $0.574\pm 0.095$ | $\mathbf{0.636\pm 0.023}$ | $\mathbf{0.599\pm 0.034}$ | $\mathbf{0.733\pm 0.013}$ | $\mathbf{0.624}$ |
| Prob. | $0.642\pm 0.033$ | $\mathbf{0.610\pm 0.037}$ | $0.539\pm 0.039$ | $0.072\pm 0.059$ | $0.696\pm 0.009$ | $0.433$ |
| Rank | $0.355\pm 0.050$ | $0.334\pm 0.101$ | $0.409\pm 0.029$ | $0.405\pm 0.032$ | $0.427\pm 0.010$ | $0.409$ |
| Uncert. | $\mathbf{0.757\pm 0.056}$ | $0.584\pm 0.048$ | $0.592\pm 0.032$ | $0.228\pm 0.030$ | $0.637\pm 0.013$ | $0.480$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncert. | $0.713\pm 0.054$ | $0.239\pm 0.127$ | $0.429\pm 0.024$ | $0.335\pm 0.022$ | $0.579\pm 0.008$ | $0.438$ |
| Prob. | $0.265\pm 0.039$ | $0.594\pm 0.023$ | $0.567\pm 0.025$ | $0.405\pm 0.035$ | $0.565\pm 0.009$ | $0.523$ |
| Random | $0.455\pm 0.056$ | $0.248\pm 0.076$ | $0.444\pm 0.027$ | $0.336\pm 0.044$ | $0.485\pm 0.010$ | $0.380$ |

## Appendix G Detailed Results for MOSEI

**Table 9: Acquisition performance on MOSEI (Image imputed by Text), showing $G_{\text{full}}$ for AUROC/AUPRC. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold.**
|  | AUROC | AUPRC |
| --- | --- | --- |
| Strategy | $G_{\text{full}}\pm\text{SEM}\uparrow$ | $G_{\text{full}}\pm\text{SEM}\uparrow$ |
| Upper Bounds (for reference) |  |  |
| Oracle | $0.995\pm 0.006$ | $0.995\pm 0.012$ |
| True KL-Div. | $0.777\pm 0.005$ | $0.790\pm 0.015$ |
| True Rank | $0.763\pm 0.012$ | $0.781\pm 0.014$ |
| True Uncert. | $0.599\pm 0.015$ | $0.673\pm 0.019$ |
| Imputation-based (proposed) |  |  |
| KL-Div | $\mathbf{0.551\pm 0.012}$ | $0.590\pm 0.009$ |
| Probability | $0.473\pm 0.010$ | $\mathbf{0.598\pm 0.006}$ |
| Rank | $0.524\pm 0.012$ | $0.560\pm 0.012$ |
| Uncertainty | $0.500\pm 0.021$ | $0.567\pm 0.009$ |
| Baselines (no imputation) |  |  |
| Uncertainty | $0.507\pm 0.015$ | $0.565\pm 0.018$ |
| Probability | $0.451\pm 0.013$ | $0.578\pm 0.009$ |
| Random | $0.521\pm 0.006$ | $0.575\pm 0.008$ |

**Table 10: Acquisition performance on MOSEI (Image imputed by Audio), showing $G_{\text{full}}$ for AUROC/AUPRC. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold.**
|  | AUROC | AUPRC |
| --- | --- | --- |
| Strategy | $G_{\text{full}}\pm\text{SEM}\uparrow$ | $G_{\text{full}}\pm\text{SEM}\uparrow$ |
| Upper Bounds (for reference) |  |  |
| Oracle | $1.052\pm 0.007$ | $1.011\pm 0.010$ |
| True KL-Div. | $0.785\pm 0.009$ | $0.803\pm 0.005$ |
| True Rank | $0.783\pm 0.010$ | $0.802\pm 0.012$ |
| True Uncert. | $0.672\pm 0.007$ | $0.742\pm 0.006$ |
| Imputation-based (proposed) |  |  |
| KL-Div | $0.566\pm 0.011$ | $0.601\pm 0.009$ |
| Probability | $0.547\pm 0.002$ | $\mathbf{0.629\pm 0.004}$ |
| Rank | $\mathbf{0.576\pm 0.013}$ | $0.614\pm 0.007$ |
| Uncertainty | $0.545\pm 0.009$ | $0.586\pm 0.007$ |
| Baselines (no imputation) |  |  |
| Uncertainty | $0.545\pm 0.009$ | $0.601\pm 0.014$ |
| Probability | $0.526\pm 0.014$ | $0.616\pm 0.008$ |
| Random | $0.553\pm 0.008$ | $0.603\pm 0.005$ |

**Table 11: Acquisition performance on MOSEI (Image imputed by Text and Audio), showing $G_{\text{full}}$ for AUROC/AUPRC. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold.**
|  | AUROC | AUPRC |
| --- | --- | --- |
| Strategy | $G_{\text{full}}\pm\text{SEM}\uparrow$ | $G_{\text{full}}\pm\text{SEM}\uparrow$ |
| Upper Bounds (for reference) |  |  |
| Oracle | $1.321\pm 0.009$ | $1.315\pm 0.012$ |
| True KL-Div. | $0.979\pm 0.006$ | $0.862\pm 0.006$ |
| True Rank | $0.960\pm 0.005$ | $0.811\pm 0.006$ |
| True Uncert. | $0.716\pm 0.004$ | $0.750\pm 0.004$ |
| Imputation-based (proposed) |  |  |
| KL-Div | $\mathbf{0.603\pm 0.007}$ | $0.627\pm 0.009$ |
| Probability | $0.513\pm 0.002$ | $0.610\pm 0.003$ |
| Rank | $0.463\pm 0.005$ | $0.491\pm 0.005$ |
| Uncertainty | $0.499\pm 0.004$ | $0.452\pm 0.004$ |
| Baselines (no imputation) |  |  |
| Uncertainty | $0.513\pm 0.004$ | $0.480\pm 0.005$ |
| Probability | $0.534\pm 0.008$ | $\mathbf{0.676\pm 0.007}$ |
| Random | $0.489\pm 0.002$ | $0.527\pm 0.002$ |

**Table 12: Acquisition performance on MOSEI (Text imputed by Image), showing $G_{\text{full}}$ for AUROC/AUPRC. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold.**
|  | AUROC | AUPRC |
| --- | --- | --- |
| Strategy | $G_{\text{full}}\pm\text{SEM}\uparrow$ | $G_{\text{full}}\pm\text{SEM}\uparrow$ |
| Upper Bounds (for reference) |  |  |
| Oracle | $1.613\pm 0.269$ | $1.493\pm 0.166$ |
| True KL-Div. | $0.845\pm 0.018$ | $0.843\pm 0.010$ |
| True Rank | $0.772\pm 0.024$ | $0.784\pm 0.025$ |
| True Uncert. | $0.633\pm 0.060$ | $0.697\pm 0.030$ |
| Imputation-based (proposed) |  |  |
| KL-Div | $\mathbf{0.900\pm 0.023}$ | $\mathbf{0.896\pm 0.022}$ |
| Probability | $0.806\pm 0.022$ | $0.861\pm 0.023$ |
| Rank | $0.309\pm 0.089$ | $0.420\pm 0.065$ |
| Uncertainty | $0.651\pm 0.049$ | $0.747\pm 0.030$ |
| Baselines (no imputation) |  |  |
| Uncertainty | $0.466\pm 0.045$ | $0.537\pm 0.014$ |
| Probability | $0.418\pm 0.055$ | $0.532\pm 0.045$ |
| Random | $0.417\pm 0.061$ | $0.489\pm 0.051$ |

**Table 13: Acquisition performance on MOSEI (Text imputed by Audio), showing $G_{\text{full}}$ for AUROC/AUPRC. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold.**
|  | AUROC | AUPRC |
| --- | --- | --- |
| Strategy | $G_{\text{full}}\pm\text{SEM}\uparrow$ | $G_{\text{full}}\pm\text{SEM}\uparrow$ |
| Upper Bounds (for reference) |  |  |
| Oracle | $8.867\pm 1.712$ | $16.066\pm 2.592$ |
| True KL-Div. | $0.645\pm 0.051$ | $0.387\pm 0.136$ |
| True Rank | $0.494\pm 0.147$ | $0.015\pm 0.262$ |
| True Uncert. | $0.913\pm 0.121$ | $0.893\pm 0.141$ |
| Imputation-based (proposed) |  |  |
| KL-Div | $2.962\pm 0.925$ | $4.582\pm 1.938$ |
| Probability | $\mathbf{3.157\pm 1.229}$ | $\mathbf{6.861\pm 1.806}$ |
| Rank | $-0.413\pm 0.421$ | $-1.134\pm 0.463$ |
| Uncertainty | $1.592\pm 0.350$ | $3.844\pm 0.806$ |
| Baselines (no imputation) |  |  |
| Uncertainty | $0.591\pm 0.147$ | $0.473\pm 0.204$ |
| Probability | $0.662\pm 0.071$ | $0.971\pm 0.014$ |
| Random | $0.316\pm 0.059$ | $0.376\pm 0.086$ |

**Table 14: Acquisition performance on MOSEI (Text imputed by Image and Audio), showing $G_{\text{full}}$ for AUROC/AUPRC. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold.**
|  | AUROC | AUPRC |
| --- | --- | --- |
| Strategy | $G_{\text{full}}\pm\text{SEM}\uparrow$ | $G_{\text{full}}\pm\text{SEM}\uparrow$ |
| Upper Bounds (for reference) |  |  |
| Oracle | $1.207\pm 0.012$ | $1.280\pm 0.013$ |
| True KL-Div. | $0.851\pm 0.002$ | $0.842\pm 0.004$ |
| True Rank | $0.836\pm 0.004$ | $0.840\pm 0.005$ |
| True Uncert. | $0.649\pm 0.009$ | $0.691\pm 0.009$ |
| Imputation-based (proposed) |  |  |
| KL-Div | $\mathbf{0.892\pm 0.002}$ | $\mathbf{0.894\pm 0.004}$ |
| Probability | $0.665\pm 0.004$ | $0.727\pm 0.003$ |
| Rank | $0.489\pm 0.003$ | $0.520\pm 0.003$ |
| Uncertainty | $0.662\pm 0.008$ | $0.720\pm 0.007$ |
| Baselines (no imputation) |  |  |
| Uncertainty | $0.538\pm 0.009$ | $0.567\pm 0.009$ |
| Probability | $0.379\pm 0.003$ | $0.462\pm 0.004$ |
| Random | $0.512\pm 0.002$ | $0.531\pm 0.003$ |

**Table 15: Acquisition performance on MOSEI (Audio imputed by Image), showing $G_{\text{full}}$ for AUROC/AUPRC. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold.**
|  | AUROC | AUPRC |
| --- | --- | --- |
| Strategy | $G_{\text{full}}\pm\text{SEM}\uparrow$ | $G_{\text{full}}\pm\text{SEM}\uparrow$ |
| Upper Bounds (for reference) |  |  |
| Oracle | $1.238\pm 0.124$ | $1.207\pm 0.093$ |
| True KL-Div. | $0.826\pm 0.014$ | $0.821\pm 0.019$ |
| True Rank | $0.752\pm 0.030$ | $0.780\pm 0.025$ |
| True Uncert. | $0.544\pm 0.034$ | $0.627\pm 0.023$ |
| Imputation-based (proposed) |  |  |
| KL-Div | $\mathbf{0.800\pm 0.012}$ | $\mathbf{0.803\pm 0.017}$ |
| Probability | $0.684\pm 0.020$ | $0.737\pm 0.024$ |
| Rank | $0.326\pm 0.072$ | $0.445\pm 0.052$ |
| Uncertainty | $0.552\pm 0.035$ | $0.625\pm 0.023$ |
| Baselines (no imputation) |  |  |
| Uncertainty | $0.436\pm 0.043$ | $0.502\pm 0.016$ |
| Probability | $0.374\pm 0.034$ | $0.510\pm 0.034$ |
| Random | $0.397\pm 0.052$ | $0.478\pm 0.039$ |

**Table 16: Acquisition performance on MOSEI (Audio imputed by Text), showing $G_{\text{full}}$ for AUROC/AUPRC. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold.**
|  | AUROC | AUPRC |
| --- | --- | --- |
| Strategy | $G_{\text{full}}\pm\text{SEM}\uparrow$ | $G_{\text{full}}\pm\text{SEM}\uparrow$ |
| Upper Bounds (for reference) |  |  |
| Oracle | $4.955\pm 0.417$ | $6.275\pm 0.948$ |
| True KL-Div. | $0.815\pm 0.043$ | $0.817\pm 0.057$ |
| True Rank | $0.563\pm 0.073$ | $0.645\pm 0.098$ |
| True Uncert. | $0.520\pm 0.026$ | $0.604\pm 0.059$ |
| Imputation-based (proposed) |  |  |
| KL-Div | $2.305\pm 0.360$ | $2.335\pm 0.499$ |
| Probability | $\mathbf{2.306\pm 0.259}$ | $\mathbf{2.571\pm 0.483}$ |
| Rank | $-0.210\pm 0.123$ | $-0.093\pm 0.135$ |
| Uncertainty | $1.154\pm 0.141$ | $1.626\pm 0.303$ |
| Baselines (no imputation) |  |  |
| Uncertainty | $0.519\pm 0.023$ | $0.604\pm 0.044$ |
| Probability | $0.416\pm 0.007$ | $0.511\pm 0.018$ |
| Random | $0.326\pm 0.024$ | $0.439\pm 0.023$ |

**Table 17: Acquisition performance on MOSEI (Audio imputed by Image and Text), showing $G_{\text{full}}$ for AUROC/AUPRC. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold.**
|  | AUROC | AUPRC |
| --- | --- | --- |
| Strategy | $G_{\text{full}}\pm\text{SEM}\uparrow$ | $G_{\text{full}}\pm\text{SEM}\uparrow$ |
| Upper Bounds (for reference) |  |  |
| Oracle | $1.215\pm 0.010$ | $1.275\pm 0.011$ |
| True KL-Div. | $0.865\pm 0.002$ | $0.850\pm 0.004$ |
| True Rank | $0.833\pm 0.004$ | $0.837\pm 0.005$ |
| True Uncert. | $0.645\pm 0.007$ | $0.693\pm 0.006$ |
| Imputation-based (proposed) |  |  |
| KL-Div | $\mathbf{0.857\pm 0.002}$ | $\mathbf{0.846\pm 0.004}$ |
| Probability | $0.667\pm 0.003$ | $0.725\pm 0.003$ |
| Rank | $0.459\pm 0.004$ | $0.489\pm 0.004$ |
| Uncertainty | $0.646\pm 0.007$ | $0.694\pm 0.007$ |
| Baselines (no imputation) |  |  |
| Uncertainty | $0.536\pm 0.008$ | $0.566\pm 0.008$ |
| Probability | $0.368\pm 0.005$ | $0.462\pm 0.005$ |
| Random | $0.503\pm 0.003$ | $0.530\pm 0.003$ |

## Appendix H Detailed Results for MIMIC Symile

**Table 18: Acquisition performance on MIMIC Symile for AUROC, showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $2.876\pm 0.368$ | $3.722\pm 0.370$ | $2.856\pm 0.147$ | $4.862\pm 0.356$ | $2.793\pm 0.324$ | $4.580$ |
| True KL-Div. | $1.029\pm 0.165$ | $1.019\pm 0.051$ | $0.885\pm 0.025$ | $0.841\pm 0.045$ | $0.920\pm 0.011$ | $0.883$ |
| True Rank | $0.963\pm 0.153$ | $0.924\pm 0.058$ | $0.946\pm 0.025$ | $0.802\pm 0.068$ | $0.887\pm 0.022$ | $0.811$ |
| True Uncert. | $0.915\pm 0.138$ | $0.763\pm 0.052$ | $0.749\pm 0.031$ | $0.152\pm 0.052$ | $0.648\pm 0.012$ | $0.481$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $0.838\pm 0.164$ | $\mathbf{0.882\pm 0.064}$ | $\mathbf{0.706\pm 0.021}$ | $\mathbf{0.779\pm 0.114}$ | $\mathbf{0.893\pm 0.080}$ | $\mathbf{0.833}$ |
| Probability | $\mathbf{0.861\pm 0.118}$ | $0.638\pm 0.046$ | $0.610\pm 0.021$ | $0.099\pm 0.107$ | $0.514\pm 0.099$ | $0.426$ |
| Rank | $0.123\pm 0.155$ | $0.331\pm 0.043$ | $0.434\pm 0.026$ | $0.371\pm 0.083$ | $0.352\pm 0.030$ | $0.378$ |
| Uncertainty | $0.851\pm 0.138$ | $0.686\pm 0.051$ | $0.701\pm 0.029$ | $0.188\pm 0.094$ | $0.588\pm 0.018$ | $0.440$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.616\pm 0.100$ | $0.482\pm 0.033$ | $0.543\pm 0.024$ | $0.380\pm 0.031$ | $0.539\pm 0.010$ | $0.480$ |
| Probability | $0.153\pm 0.086$ | $0.519\pm 0.035$ | $0.464\pm 0.020$ | $0.446\pm 0.068$ | $0.421\pm 0.010$ | $0.458$ |
| Random | $0.241\pm 0.121$ | $0.399\pm 0.044$ | $0.479\pm 0.020$ | $0.250\pm 0.080$ | $0.425\pm 0.017$ | $0.376$ |

**Table 19: Acquisition performance on MIMIC Symile for AUPRC, showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $2.579\pm 0.343$ | $2.964\pm 0.201$ | $4.784\pm 1.130$ | $3.093\pm 0.201$ | $3.015\pm 0.141$ | $4.231$ |
| True KL-Div. | $0.883\pm 0.094$ | $0.970\pm 0.038$ | $0.933\pm 0.057$ | $0.781\pm 0.038$ | $0.939\pm 0.013$ | $0.871$ |
| True Rank | $0.659\pm 0.176$ | $0.858\pm 0.033$ | $0.903\pm 0.090$ | $0.720\pm 0.064$ | $0.897\pm 0.022$ | $0.776$ |
| True Uncert. | $0.604\pm 0.073$ | $0.812\pm 0.044$ | $0.709\pm 0.036$ | $0.050\pm 0.039$ | $0.645\pm 0.015$ | $0.450$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $\mathbf{0.770\pm 0.210}$ | $\mathbf{0.899\pm 0.044}$ | $\mathbf{0.895\pm 0.159}$ | $\mathbf{0.684\pm 0.063}$ | $\mathbf{0.832\pm 0.039}$ | $\mathbf{0.777}$ |
| Probability | $0.631\pm 0.073$ | $0.673\pm 0.036$ | $0.503\pm 0.106$ | $0.028\pm 0.069$ | $0.624\pm 0.049$ | $0.449$ |
| Rank | $0.461\pm 0.188$ | $0.388\pm 0.034$ | $0.421\pm 0.064$ | $0.318\pm 0.053$ | $0.351\pm 0.022$ | $0.407$ |
| Uncertainty | $0.663\pm 0.108$ | $0.744\pm 0.046$ | $0.611\pm 0.073$ | $0.118\pm 0.034$ | $0.569\pm 0.017$ | $0.444$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.428\pm 0.155$ | $0.524\pm 0.031$ | $0.509\pm 0.036$ | $0.246\pm 0.018$ | $0.519\pm 0.011$ | $0.443$ |
| Probability | $0.242\pm 0.068$ | $0.523\pm 0.026$ | $0.546\pm 0.030$ | $0.512\pm 0.064$ | $0.533\pm 0.013$ | $0.550$ |
| Random | $0.149\pm 0.103$ | $0.423\pm 0.032$ | $0.429\pm 0.097$ | $0.222\pm 0.094$ | $0.448\pm 0.012$ | $0.388$ |

**Table 20: Acquisition performance on MIMIC Symile for AUROC (Image imputed by Lab), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $4.144\pm 2.172$ | $2.120\pm 0.465$ | $2.803\pm 0.306$ | $2.381\pm 0.215$ | $1.684\pm 0.070$ | $4.104$ |
| True KL-Div. | $0.728\pm 0.137$ | $0.816\pm 0.011$ | $0.611\pm 0.084$ | $0.837\pm 0.028$ | $0.813\pm 0.009$ | $0.825$ |
| True Rank | $1.062\pm 0.643$ | $0.614\pm 0.111$ | $0.591\pm 0.157$ | $0.816\pm 0.065$ | $0.765\pm 0.024$ | $0.707$ |
| True Uncert. | $0.686\pm 0.374$ | $0.606\pm 0.068$ | $0.637\pm 0.140$ | $0.496\pm 0.148$ | $0.619\pm 0.024$ | $0.601$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $-1.410\pm 0.037$ | $0.477\pm 0.108$ | $0.282\pm 0.146$ | $\mathbf{0.626\pm 0.137}$ | $0.503\pm 0.022$ | $0.247$ |
| Probability | $\mathbf{0.231\pm 0.288}$ | $0.498\pm 0.036$ | $0.514\pm 0.060$ | $0.503\pm 0.056$ | $0.454\pm 0.020$ | $0.379$ |
| Rank | $-0.725\pm 0.858$ | $0.190\pm 0.201$ | $0.520\pm 0.018$ | $0.403\pm 0.085$ | $\mathbf{0.527\pm 0.032}$ | $0.225$ |
| Uncertainty | $0.168\pm 0.317$ | $0.400\pm 0.025$ | $\mathbf{0.620\pm 0.179}$ | $0.422\pm 0.130$ | $0.493\pm 0.019$ | $\mathbf{0.448}$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.129\pm 0.339$ | $0.332\pm 0.069$ | $0.423\pm 0.201$ | $0.456\pm 0.106$ | $0.503\pm 0.020$ | $0.433$ |
| Probability | $0.129\pm 0.339$ | $\mathbf{0.515\pm 0.057}$ | $0.530\pm 0.049$ | $0.601\pm 0.101$ | $0.477\pm 0.018$ | $0.444$ |
| Random | $-0.509\pm 0.041$ | $0.356\pm 0.070$ | $0.503\pm 0.163$ | $0.609\pm 0.091$ | $0.492\pm 0.023$ | $0.307$ |

**Table 21: Acquisition performance on MIMIC Symile for AUPRC (Image imputed by Lab), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $3.844\pm 2.595$ | $1.792\pm 0.288$ | $2.826\pm 0.677$ | $1.930\pm 0.221$ | $1.778\pm 0.101$ | $4.862$ |
| True KL-Div. | $0.869\pm 0.023$ | $0.827\pm 0.021$ | $0.620\pm 0.156$ | $0.812\pm 0.024$ | $0.775\pm 0.013$ | $0.770$ |
| True Rank | $1.398\pm 0.874$ | $0.566\pm 0.140$ | $0.492\pm 0.345$ | $0.827\pm 0.034$ | $0.759\pm 0.024$ | $0.575$ |
| True Uncert. | $0.712\pm 0.210$ | $0.650\pm 0.111$ | $0.655\pm 0.118$ | $0.538\pm 0.165$ | $0.579\pm 0.041$ | $0.498$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $-1.450\pm 1.121$ | $0.464\pm 0.132$ | $0.149\pm 0.370$ | $\mathbf{0.652\pm 0.149}$ | $0.610\pm 0.010$ | $0.131$ |
| Probability | $\mathbf{0.210\pm 0.451}$ | $0.575\pm 0.018$ | $0.735\pm 0.084$ | $0.591\pm 0.035$ | $0.598\pm 0.016$ | $0.568$ |
| Rank | $-0.777\pm 1.263$ | $0.261\pm 0.185$ | $0.548\pm 0.051$ | $0.438\pm 0.134$ | $0.480\pm 0.042$ | $0.290$ |
| Uncertainty | $0.157\pm 0.477$ | $0.456\pm 0.075$ | $0.516\pm 0.245$ | $0.477\pm 0.170$ | $0.358\pm 0.021$ | $0.441$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.132\pm 0.484$ | $0.369\pm 0.094$ | $0.199\pm 0.387$ | $0.466\pm 0.152$ | $0.368\pm 0.022$ | $0.401$ |
| Probability | $0.132\pm 0.484$ | $\mathbf{0.592\pm 0.029}$ | $\mathbf{0.754\pm 0.084}$ | $0.632\pm 0.071$ | $\mathbf{0.612\pm 0.012}$ | $\mathbf{0.621}$ |
| Random | $-0.110\pm 0.296$ | $0.408\pm 0.060$ | $0.420\pm 0.208$ | $0.599\pm 0.156$ | $0.489\pm 0.037$ | $0.440$ |

**Table 22: Acquisition performance on MIMIC Symile for AUROC (Image imputed by ECG), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $4.028\pm 2.600$ | $1.959\pm 0.358$ | $1.960\pm 0.164$ | $3.526\pm 0.947$ | $1.639\pm 0.050$ | $3.892$ |
| True KL-Div. | $0.963\pm 0.224$ | $0.852\pm 0.051$ | $0.725\pm 0.030$ | $0.614\pm 0.152$ | $0.777\pm 0.003$ | $0.874$ |
| True Rank | $0.192\pm 0.075$ | $0.559\pm 0.101$ | $0.753\pm 0.013$ | $-0.784\pm 1.091$ | $0.719\pm 0.043$ | $0.327$ |
| True Uncert. | $0.934\pm 0.189$ | $0.700\pm 0.131$ | $0.717\pm 0.039$ | $-0.853\pm 0.584$ | $0.658\pm 0.050$ | $0.024$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $0.425\pm 0.323$ | $\mathbf{0.561\pm 0.079}$ | $\mathbf{0.769\pm 0.096}$ | $-0.353\pm 0.827$ | $\mathbf{0.621\pm 0.030}$ | $\mathbf{0.669}$ |
| Probability | $0.650\pm 0.024$ | $0.538\pm 0.107$ | $0.709\pm 0.095$ | $0.162\pm 0.130$ | $0.591\pm 0.024$ | $0.443$ |
| Rank | $0.411\pm 0.193$ | $0.337\pm 0.122$ | $0.538\pm 0.050$ | $-0.165\pm 0.412$ | $0.445\pm 0.038$ | $0.380$ |
| Uncertainty | $\mathbf{0.750\pm 0.011}$ | $0.527\pm 0.112$ | $0.720\pm 0.083$ | $0.466\pm 0.229$ | $0.431\pm 0.026$ | $0.167$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $-0.962\pm 1.403$ | $0.380\pm 0.050$ | $0.567\pm 0.064$ | $\mathbf{0.591\pm 0.180}$ | $0.427\pm 0.022$ | $-0.055$ |
| Probability | $0.133\pm 0.291$ | $0.442\pm 0.050$ | $0.500\pm 0.028$ | $-0.684\pm 0.967$ | $0.427\pm 0.022$ | $0.650$ |
| Random | $-0.803\pm 0.529$ | $0.482\pm 0.038$ | $0.647\pm 0.019$ | $0.227\pm 0.307$ | $0.513\pm 0.018$ | $0.263$ |

**Table 23: Acquisition performance on MIMIC Symile for AUPRC (Image imputed by ECG), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $6.343\pm 5.165$ | $1.593\pm 0.113$ | $1.950\pm 0.162$ | $2.785\pm 0.530$ | $1.832\pm 0.041$ | $4.085$ |
| True KL-Div. | $1.358\pm 0.553$ | $0.818\pm 0.060$ | $0.776\pm 0.039$ | $0.526\pm 0.206$ | $0.855\pm 0.004$ | $0.814$ |
| True Rank | $-0.157\pm 0.576$ | $0.643\pm 0.064$ | $0.808\pm 0.008$ | $-0.477\pm 0.824$ | $0.741\pm 0.031$ | $0.420$ |
| True Uncert. | $1.331\pm 0.520$ | $0.699\pm 0.124$ | $0.790\pm 0.026$ | $-0.640\pm 0.325$ | $0.745\pm 0.040$ | $0.489$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $-0.784\pm 1.571$ | $\mathbf{0.613\pm 0.082}$ | $0.870\pm 0.127$ | $-0.203\pm 0.570$ | $\mathbf{0.737\pm 0.035}$ | $0.434$ |
| Probability | $0.498\pm 0.210$ | $0.552\pm 0.123$ | $0.868\pm 0.103$ | $0.136\pm 0.118$ | $0.722\pm 0.027$ | $\mathbf{0.590}$ |
| Rank | $0.116\pm 0.576$ | $0.468\pm 0.033$ | $0.652\pm 0.045$ | $-0.068\pm 0.285$ | $0.524\pm 0.027$ | $0.419$ |
| Uncertainty | $\mathbf{0.642\pm 0.147}$ | $0.553\pm 0.129$ | $\mathbf{0.874\pm 0.099}$ | $0.297\pm 0.183$ | $0.553\pm 0.015$ | $0.462$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $-4.523\pm 5.002$ | $0.451\pm 0.068$ | $0.668\pm 0.070$ | $\mathbf{0.360\pm 0.105}$ | $0.534\pm 0.013$ | $-0.049$ |
| Probability | $-0.190\pm 0.747$ | $0.496\pm 0.049$ | $0.578\pm 0.027$ | $-0.626\pm 0.854$ | $0.534\pm 0.013$ | $0.382$ |
| Random | $-3.009\pm 2.890$ | $0.530\pm 0.060$ | $0.715\pm 0.037$ | $-0.038\pm 0.461$ | $0.591\pm 0.012$ | $0.098$ |

**Table 24: Acquisition performance on MIMIC Symile for AUROC (Image imputed by Lab and ECG), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $2.679\pm 0.496$ | $3.081\pm 0.257$ | $2.539\pm 0.150$ | $5.098\pm 0.624$ | $2.119\pm 0.086$ | $3.911$ |
| True KL-Div. | $1.006\pm 0.111$ | $1.105\pm 0.082$ | $0.880\pm 0.033$ | $0.703\pm 0.095$ | $1.017\pm 0.023$ | $0.922$ |
| True Rank | $0.760\pm 0.142$ | $1.076\pm 0.073$ | $0.993\pm 0.052$ | $0.930\pm 0.136$ | $1.036\pm 0.023$ | $0.884$ |
| True Uncert. | $0.975\pm 0.140$ | $0.877\pm 0.083$ | $0.886\pm 0.065$ | $0.300\pm 0.043$ | $0.634\pm 0.018$ | $0.494$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $0.770\pm 0.122$ | $\mathbf{0.736\pm 0.077}$ | $0.711\pm 0.031$ | $0.049\pm 0.191$ | $0.535\pm 0.015$ | $0.512$ |
| Probability | $\mathbf{0.883\pm 0.111}$ | $0.563\pm 0.062$ | $0.620\pm 0.034$ | $\mathbf{0.706\pm 0.096}$ | $\mathbf{0.593\pm 0.010}$ | $\mathbf{0.628}$ |
| Rank | $0.594\pm 0.082$ | $0.335\pm 0.079$ | $0.423\pm 0.070$ | $0.130\pm 0.226$ | $0.466\pm 0.012$ | $0.392$ |
| Uncertainty | $0.785\pm 0.085$ | $0.535\pm 0.064$ | $\mathbf{0.728\pm 0.050}$ | $0.545\pm 0.088$ | $0.556\pm 0.015$ | $0.505$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.514\pm 0.151$ | $0.495\pm 0.038$ | $0.529\pm 0.045$ | $0.466\pm 0.056$ | $0.545\pm 0.014$ | $0.474$ |
| Probability | $0.265\pm 0.046$ | $0.463\pm 0.035$ | $0.461\pm 0.037$ | $0.268\pm 0.136$ | $0.401\pm 0.009$ | $0.421$ |
| Random | $0.005\pm 0.204$ | $0.386\pm 0.047$ | $0.467\pm 0.031$ | $0.373\pm 0.145$ | $0.472\pm 0.014$ | $0.425$ |

**Table 25: Acquisition performance on MIMIC Symile for AUPRC (Image imputed by Lab and ECG), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $3.244\pm 0.899$ | $2.660\pm 0.365$ | $3.290\pm 0.320$ | $3.756\pm 0.538$ | $2.484\pm 0.159$ | $4.063$ |
| True KL-Div. | $1.111\pm 0.223$ | $0.983\pm 1.014$ | $0.998\pm 0.079$ | $0.674\pm 0.088$ | $1.014\pm 0.034$ | $0.939$ |
| True Rank | $0.503\pm 0.538$ | $0.727\pm 0.084$ | $1.063\pm 0.108$ | $0.762\pm 0.088$ | $1.014\pm 0.036$ | $0.822$ |
| True Uncert. | $0.826\pm 0.166$ | $0.950\pm 0.103$ | $0.923\pm 0.060$ | $0.143\pm 0.041$ | $0.636\pm 0.029$ | $0.493$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $\mathbf{1.238\pm 0.634}$ | $\mathbf{0.636\pm 0.075}$ | $0.754\pm 0.075$ | $0.330\pm 0.146$ | $0.467\pm 0.020$ | $0.585$ |
| Probability | $0.821\pm 0.184$ | $0.581\pm 0.048$ | $0.714\pm 0.055$ | $\mathbf{0.478\pm 0.083}$ | $\mathbf{0.648\pm 0.012}$ | $\mathbf{0.610}$ |
| Rank | $1.133\pm 0.543$ | $0.375\pm 0.081$ | $0.582\pm 0.103$ | $0.139\pm 0.174$ | $0.469\pm 0.018$ | $0.491$ |
| Uncertainty | $1.018\pm 0.316$ | $0.516\pm 0.058$ | $\mathbf{0.783\pm 0.081}$ | $0.285\pm 0.039$ | $0.505\pm 0.015$ | $0.500$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.843\pm 0.289$ | $0.410\pm 0.050$ | $0.562\pm 0.061$ | $0.275\pm 0.038$ | $0.521\pm 0.019$ | $0.467$ |
| Probability | $0.196\pm 0.143$ | $0.567\pm 0.044$ | $0.520\pm 0.042$ | $0.442\pm 0.111$ | $0.572\pm 0.014$ | $0.515$ |
| Random | $-0.073\pm 0.199$ | $0.397\pm 0.084$ | $0.556\pm 0.053$ | $0.411\pm 0.100$ | $0.470\pm 0.013$ | $0.406$ |

**Table 26: Acquisition performance on MIMIC Symile for AUROC (Lab imputed by Image), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $2.404\pm 0.448$ | $5.403\pm 1.547$ | $2.446\pm 0.350$ | $4.261$ | $4.715\pm 0.988$ | $6.306$ |
| True KL-Div. | $0.901\pm 0.104$ | $0.727\pm 0.113$ | $0.874\pm 0.045$ | $0.807$ | $0.670\pm 0.093$ | $0.634$ |
| True Rank | $0.490\pm 0.208$ | $0.872\pm 0.019$ | $0.873\pm 0.081$ | $1.691$ | $0.627\pm 0.148$ | $0.604$ |
| True Uncert. | $-0.204\pm 0.122$ | $0.690\pm 0.193$ | $0.319\pm 0.084$ | $-0.511$ | $0.507\pm 0.100$ | $0.154$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $0.443\pm 0.028$ | $0.503\pm 0.316$ | $0.523\pm 0.078$ | $\mathbf{0.383}$ | $0.193\pm 0.090$ | $\mathbf{0.684}$ |
| Probability | $0.087\pm 0.217$ | $0.428\pm 0.155$ | $\mathbf{0.574\pm 0.117}$ | $0.221$ | $0.457\pm 0.056$ | $0.380$ |
| Rank | $\mathbf{0.848\pm 0.131}$ | $-0.103\pm 0.100$ | $0.309\pm 0.073$ | $-0.390$ | $0.463\pm 0.125$ | $-0.139$ |
| Uncertainty | $0.074\pm 0.187$ | $0.593\pm 0.168$ | $0.415\pm 0.107$ | $0.278$ | $0.624\pm 0.029$ | $0.311$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.515\pm 0.062$ | $\mathbf{0.781\pm 0.101}$ | $0.394\pm 0.106$ | $0.310$ | $\mathbf{0.636\pm 0.041}$ | $0.487$ |
| Probability | $0.105\pm 0.251$ | $0.444\pm 0.144$ | $0.567\pm 0.112$ | $0.267$ | $0.449\pm 0.055$ | $0.390$ |
| Random | $0.366\pm 0.519$ | $0.662\pm 0.085$ | $0.504\pm 0.111$ | $0.087$ | $0.229\pm 0.142$ | $0.419$ |

**Table 27: Acquisition performance on MIMIC Symile for AUPRC (Lab imputed by Image), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $2.142\pm 0.360$ | $3.772\pm 0.624$ | $4.042\pm 1.747$ | – | $5.560\pm 1.093$ | $4.751$ |
| True KL-Div. | $0.916\pm 0.056$ | $0.648\pm 0.079$ | $0.997\pm 0.160$ | – | $0.568\pm 0.132$ | $0.699$ |
| True Rank | $0.400\pm 0.206$ | $0.686\pm 0.095$ | $1.106\pm 0.342$ | – | $0.603\pm 0.139$ | $0.590$ |
| True Uncert. | $-1.106\pm 0.446$ | $0.506\pm 0.100$ | $0.258\pm 0.136$ | – | $0.477\pm 0.270$ | $0.153$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $0.499\pm 0.117$ | $0.481\pm 0.203$ | $0.413\pm 0.187$ | – | $0.269\pm 0.189$ | $\mathbf{0.680}$ |
| Probability | $-0.381\pm 0.441$ | $0.618\pm 0.008$ | $0.339\pm 0.318$ | – | $\mathbf{0.681\pm 0.114}$ | $0.442$ |
| Rank | $\mathbf{0.824\pm 0.103}$ | $-0.157\pm 0.006$ | $-0.044\pm 0.404$ | – | $0.246\pm 0.115$ | $0.186$ |
| Uncertainty | $-0.429\pm 0.473$ | $0.415\pm 0.161$ | $\mathbf{0.541\pm 0.220}$ | – | $0.570\pm 0.164$ | $0.124$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.455\pm 0.060$ | $0.484\pm 0.099$ | $0.537\pm 0.231$ | – | $0.585\pm 0.160$ | $0.440$ |
| Probability | $-0.362\pm 0.445$ | $0.629\pm 0.007$ | $0.325\pm 0.326$ | – | $0.672\pm 0.112$ | $0.572$ |
| Random | $0.283\pm 0.513$ | $\mathbf{0.635\pm 0.001}$ | $0.303\pm 0.231$ | – | $0.253\pm 0.115$ | $0.356$ |

**Table 28: Acquisition performance on MIMIC Symile for AUROC (Lab imputed by ECG), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $2.830\pm 1.006$ | $6.507\pm 2.813$ | $1.993\pm 0.252$ | $13.751\pm 3.349$ | $5.706\pm 1.964$ | $7.637$ |
| True KL-Div. | $0.807\pm 0.072$ | $1.043\pm 0.176$ | $0.855\pm 0.056$ | $0.933\pm 0.345$ | $0.841\pm 0.028$ | $0.727$ |
| True Rank | $0.454\pm 0.258$ | $0.880\pm 0.255$ | $0.756\pm 0.027$ | $0.462\pm 0.641$ | $0.398\pm 0.151$ | $0.290$ |
| True Uncert. | $0.665\pm 0.104$ | $0.273\pm 0.305$ | $0.688\pm 0.056$ | $-0.218\pm 1.149$ | $0.568\pm 0.041$ | $0.437$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $\mathbf{0.735\pm 0.130}$ | $\mathbf{1.221\pm 0.208}$ | $0.670\pm 0.136$ | $\mathbf{4.739\pm 1.683}$ | $2.234\pm 0.684$ | $\mathbf{2.096}$ |
| Probability | $0.569\pm 0.115$ | $0.126\pm 0.370$ | $\mathbf{0.775\pm 0.075}$ | $-4.498\pm 1.913$ | $\mathbf{2.255\pm 0.646}$ | $-0.333$ |
| Rank | $0.395\pm 0.117$ | $0.160\pm 0.439$ | $0.298\pm 0.039$ | $1.303\pm 1.170$ | $-1.119\pm 0.726$ | $0.619$ |
| Uncertainty | $0.612\pm 0.125$ | $0.499\pm 0.346$ | $0.773\pm 0.078$ | $-4.102\pm 1.614$ | $0.071\pm 0.175$ | $-0.744$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.421\pm 0.290$ | $0.544\pm 0.303$ | $0.456\pm 0.081$ | $0.431\pm 0.170$ | $0.324\pm 0.024$ | $0.574$ |
| Probability | $0.069\pm 0.066$ | $0.372\pm 0.420$ | $0.505\pm 0.102$ | $0.161\pm 0.787$ | $0.324\pm 0.024$ | $0.521$ |
| Random | $0.117\pm 0.418$ | $0.737\pm 0.338$ | $0.531\pm 0.041$ | $-0.400\pm 1.001$ | $0.107\pm 0.142$ | $0.189$ |

**Table 29: Acquisition performance on MIMIC Symile for AUPRC (Lab imputed by ECG), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $4.231\pm 2.318$ | $4.018\pm 1.224$ | $2.093\pm 0.267$ | $6.269\pm 0.383$ | $5.741\pm 1.374$ | $6.317$ |
| True KL-Div. | $0.924\pm 0.069$ | $0.825\pm 0.017$ | $0.883\pm 0.030$ | $0.717\pm 0.570$ | $0.897\pm 0.019$ | $0.972$ |
| True Rank | $0.030\pm 0.698$ | $0.643\pm 0.103$ | $0.846\pm 0.041$ | $0.619\pm 1.260$ | $0.351\pm 0.208$ | $0.900$ |
| True Uncert. | $0.730\pm 0.109$ | $0.793\pm 0.209$ | $0.784\pm 0.034$ | $0.030\pm 0.615$ | $0.740\pm 0.024$ | $0.550$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $\mathbf{0.721\pm 0.166}$ | $\mathbf{1.292\pm 0.411}$ | $0.780\pm 0.157$ | $\mathbf{1.533\pm 0.215}$ | $2.267\pm 0.487$ | $\mathbf{1.779}$ |
| Probability | $0.532\pm 0.179$ | $1.116\pm 0.598$ | $0.920\pm 0.086$ | $-0.332\pm 0.041$ | $\mathbf{2.300\pm 0.464}$ | $0.751$ |
| Rank | $0.092\pm 0.485$ | $0.353\pm 0.268$ | $0.372\pm 0.058$ | $-0.032\pm 0.027$ | $-0.641\pm 0.441$ | $0.458$ |
| Uncertainty | $0.580\pm 0.189$ | $1.080\pm 0.651$ | $\mathbf{0.930\pm 0.088}$ | $-0.549\pm 0.088$ | $0.351\pm 0.131$ | $0.145$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $-1.088\pm 1.658$ | $0.451\pm 0.118$ | $0.605\pm 0.085$ | $0.152\pm 0.036$ | $0.645\pm 0.093$ | $0.260$ |
| Probability | $-0.282\pm 0.309$ | $0.672\pm 0.126$ | $0.594\pm 0.102$ | $0.769\pm 0.144$ | $0.645\pm 0.093$ | $1.112$ |
| Random | $-0.458\pm 0.636$ | $0.631\pm 0.184$ | $0.659\pm 0.051$ | $0.925\pm 0.146$ | $0.264\pm 0.154$ | $0.556$ |

**Table 30: Acquisition performance on MIMIC Symile for AUROC (Lab imputed by Image and ECG), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $2.416\pm 0.727$ | $2.670\pm 0.218$ | $3.090\pm 0.355$ | $3.712\pm 0.446$ | $2.174\pm 0.050$ | $4.233$ |
| True KL-Div. | $1.165\pm 0.267$ | $0.883\pm 0.024$ | $1.050\pm 0.058$ | $1.092\pm 0.080$ | $0.925\pm 0.006$ | $0.981$ |
| True Rank | $1.373\pm 0.427$ | $0.811\pm 0.024$ | $1.076\pm 0.050$ | $0.904\pm 0.066$ | $0.888\pm 0.008$ | $0.883$ |
| True Uncert. | $1.008\pm 0.213$ | $0.720\pm 0.056$ | $0.781\pm 0.030$ | $-0.092\pm 0.044$ | $0.675\pm 0.027$ | $0.494$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $\mathbf{1.088\pm 0.224}$ | $\mathbf{0.875\pm 0.022}$ | $0.667\pm 0.039$ | $\mathbf{1.193\pm 0.091}$ | $\mathbf{0.885\pm 0.008}$ | $\mathbf{1.035}$ |
| Probability | $1.084\pm 0.241$ | $0.647\pm 0.048$ | $0.657\pm 0.039$ | $-0.270\pm 0.085$ | $0.767\pm 0.008$ | $0.390$ |
| Rank | $-0.190\pm 0.547$ | $0.273\pm 0.051$ | $0.415\pm 0.044$ | $0.448\pm 0.088$ | $0.192\pm 0.017$ | $0.350$ |
| Uncertainty | $0.981\pm 0.208$ | $0.718\pm 0.054$ | $\mathbf{0.736\pm 0.028}$ | $-0.143\pm 0.044$ | $0.636\pm 0.027$ | $0.448$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.772\pm 0.126$ | $0.492\pm 0.034$ | $0.627\pm 0.033$ | $0.183\pm 0.030$ | $0.506\pm 0.019$ | $0.469$ |
| Probability | $0.088\pm 0.147$ | $0.529\pm 0.030$ | $0.376\pm 0.025$ | $0.804\pm 0.069$ | $0.324\pm 0.007$ | $0.521$ |
| Random | $0.212\pm 0.278$ | $0.404\pm 0.030$ | $0.503\pm 0.030$ | $0.247\pm 0.113$ | $0.439\pm 0.014$ | $0.268$ |

**Table 31: Acquisition performance on MIMIC Symile for AUPRC (Lab imputed by Image and ECG), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $1.965\pm 0.497$ | $2.573\pm 0.236$ | $3.714\pm 0.473$ | $2.444\pm 0.174$ | $2.962\pm 0.166$ | $3.748$ |
| True KL-Div. | $0.961\pm 0.228$ | $0.918\pm 0.033$ | $1.042\pm 0.082$ | $0.985\pm 0.053$ | $0.954\pm 0.013$ | $0.878$ |
| True Rank | $0.899\pm 0.233$ | $0.896\pm 0.037$ | $1.138\pm 0.112$ | $0.886\pm 0.051$ | $0.922\pm 0.019$ | $0.820$ |
| True Uncert. | $0.610\pm 0.076$ | $0.741\pm 0.066$ | $0.736\pm 0.075$ | $-0.096\pm 0.042$ | $0.718\pm 0.030$ | $0.482$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $\mathbf{0.914\pm 0.236}$ | $\mathbf{0.921\pm 0.039}$ | $0.587\pm 0.056$ | $\mathbf{1.038\pm 0.046}$ | $\mathbf{0.925\pm 0.011}$ | $\mathbf{0.835}$ |
| Probability | $0.811\pm 0.100$ | $0.619\pm 0.048$ | $0.677\pm 0.026$ | $-0.376\pm 0.097$ | $0.848\pm 0.014$ | $0.469$ |
| Rank | $0.516\pm 0.085$ | $0.343\pm 0.047$ | $0.412\pm 0.046$ | $0.355\pm 0.050$ | $0.202\pm 0.019$ | $0.370$ |
| Uncertainty | $0.662\pm 0.086$ | $0.759\pm 0.070$ | $\mathbf{0.723\pm 0.084}$ | $-0.131\pm 0.030$ | $0.691\pm 0.028$ | $0.482$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.573\pm 0.099$ | $0.564\pm 0.041$ | $0.601\pm 0.074$ | $0.152\pm 0.016$ | $0.527\pm 0.023$ | $0.485$ |
| Probability | $0.265\pm 0.173$ | $0.475\pm 0.038$ | $0.424\pm 0.060$ | $0.712\pm 0.038$ | $0.344\pm 0.012$ | $0.518$ |
| Random | $0.504\pm 0.107$ | $0.449\pm 0.031$ | $0.612\pm 0.063$ | $0.242\pm 0.096$ | $0.471\pm 0.024$ | $0.431$ |

**Table 32: Acquisition performance on MIMIC Symile for AUROC (ECG imputed by Image), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $3.489\pm 1.656$ | $21.015\pm 5.247$ | – | – | $37.701$ | $28.964$ |
| True KL-Div. | $0.546\pm 0.073$ | $0.036\pm 0.994$ | – | – | $0.028$ | $0.032$ |
| True Rank | $0.550\pm 0.379$ | $-2.342\pm 1.928$ | – | – | $-1.276$ | $0.122$ |
| True Uncert. | $0.648\pm 0.497$ | $-0.099\pm 0.724$ | – | – | $0.379$ | $0.135$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $0.084\pm 0.465$ | $-1.318\pm 0.868$ | – | – | $\mathbf{6.736}$ | $\mathbf{3.317}$ |
| Probability | $0.542\pm 0.247$ | $0.096\pm 0.156$ | – | – | $-0.629$ | $0.507$ |
| Rank | $-0.355\pm 0.092$ | $-1.669\pm 0.386$ | – | – | $1.017$ | $0.062$ |
| Uncertainty | $\mathbf{0.672\pm 0.578}$ | $0.131\pm 0.351$ | – | – | $-1.407$ | $-0.817$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.637\pm 0.113$ | $-0.592\pm 0.877$ | – | – | $0.868$ | $0.515$ |
| Probability | $0.507\pm 0.072$ | $0.236\pm 0.361$ | – | – | $-0.431$ | $-0.192$ |
| Random | $0.440\pm 0.455$ | $\mathbf{0.389\pm 1.478}$ | – | – | $-1.424$ | $0.484$ |

**Table 33: Acquisition performance on MIMIC Symile for AUPRC (ECG imputed by Image), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $1.631$ | – | $100.646\pm 71.555$ | – | – | $33.428$ |
| True KL-Div. | $0.635$ | – | $2.975\pm 3.762$ | – | – | $1.163$ |
| True Rank | $0.943$ | – | $-7.003\pm 1.999$ | – | – | $-1.214$ |
| True Uncert. | $-0.027$ | – | $0.199\pm 0.327$ | – | – | $0.542$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $\mathbf{0.600}$ | – | $\mathbf{15.994\pm 7.400}$ | – | – | $\mathbf{4.721}$ |
| Probability | $0.075$ | – | $-10.442\pm 1.040$ | – | – | $-2.263$ |
| Rank | $-1.052$ | – | $-0.692\pm 4.909$ | – | – | $-0.467$ |
| Uncertainty | $-0.225$ | – | $-5.787\pm 1.743$ | – | – | $-0.999$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.456$ | – | $0.652\pm 0.902$ | – | – | $0.692$ |
| Probability | $0.408$ | – | $0.503\pm 0.861$ | – | – | $0.567$ |
| Random | $-0.336$ | – | $-7.531\pm 5.766$ | – | – | $-1.297$ |

**Table 34: Acquisition performance on MIMIC Symile for AUROC (ECG imputed by Lab), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $9.961\pm 1.805$ | $7.528\pm 1.743$ | $7.425$ | $9.429\pm 3.404$ | $42.327$ | $14.888$ |
| True KL-Div. | $-0.137\pm 1.838$ | $0.978\pm 0.072$ | $0.178$ | $0.356\pm 0.377$ | $0.288$ | $0.405$ |
| True Rank | $-0.183\pm 0.795$ | $0.976\pm 0.600$ | $0.012$ | $0.962\pm 0.193$ | $-1.285$ | $0.308$ |
| True Uncert. | $0.271\pm 0.133$ | $0.892\pm 0.394$ | $1.249$ | $0.377\pm 0.354$ | $-0.143$ | $0.399$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $-0.756\pm 0.967$ | $0.491\pm 1.880$ | $1.805$ | $\mathbf{1.782\pm 0.998}$ | $\mathbf{12.001}$ | $\mathbf{2.649}$ |
| Probability | $0.203\pm 0.341$ | $\mathbf{2.386\pm 0.784}$ | $1.834$ | $1.222\pm 0.693$ | $-15.313$ | $-1.990$ |
| Rank | $-2.489\pm 1.558$ | $0.182\pm 0.173$ | $-0.177$ | $0.630\pm 0.303$ | $0.104$ | $-0.312$ |
| Uncertainty | $0.078\pm 0.296$ | $1.509\pm 0.221$ | $\mathbf{1.945}$ | $1.538\pm 1.059$ | $1.277$ | $0.335$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.201\pm 0.098$ | $0.537\pm 0.099$ | $0.628$ | $0.023\pm 0.305$ | $-0.118$ | $0.384$ |
| Probability | $0.201\pm 0.098$ | $0.474\pm 0.027$ | $0.760$ | $0.650\pm 0.152$ | $0.394$ | $0.074$ |
| Random | $\mathbf{1.112\pm 1.486}$ | $0.492\pm 0.457$ | $-0.650$ | $0.656\pm 0.102$ | $-1.051$ | $0.387$ |

**Table 35: Acquisition performance on MIMIC Symile for AUPRC (ECG imputed by Lab), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $7.894\pm 1.207$ | $7.935\pm 0.606$ | $7.801\pm 1.620$ | $3.681$ | $13.663\pm 0.667$ | $9.827$ |
| True KL-Div. | $-0.038\pm 1.091$ | $0.843\pm 0.368$ | $-0.530\pm 0.679$ | $0.175$ | $0.815\pm 0.157$ | $0.501$ |
| True Rank | $0.172\pm 0.009$ | $0.532\pm 0.379$ | $1.309\pm 0.247$ | $0.635$ | $-0.719\pm 0.197$ | $0.354$ |
| True Uncert. | $0.480\pm 0.166$ | $0.645\pm 0.089$ | $0.762\pm 0.322$ | $0.205$ | $0.039\pm 0.013$ | $0.240$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $-1.014\pm 0.482$ | $1.979\pm 1.044$ | $0.596\pm 0.626$ | $0.331$ | $\mathbf{3.890\pm 0.009}$ | $\mathbf{1.408}$ |
| Probability | $-0.291\pm 0.867$ | $\mathbf{2.492\pm 0.270}$ | $1.173\pm 0.256$ | $\mathbf{1.229}$ | $-3.963\pm 0.175$ | $-0.348$ |
| Rank | $-3.557\pm 1.533$ | $0.231\pm 0.154$ | $-0.653\pm 0.806$ | $0.206$ | $0.003\pm 0.267$ | $-0.331$ |
| Uncertainty | $0.079\pm 0.427$ | $2.413\pm 0.318$ | $\mathbf{1.197\pm 0.340}$ | $0.817$ | $-0.565\pm 0.214$ | $0.324$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.398\pm 0.120$ | $0.641\pm 0.487$ | $0.114\pm 0.434$ | $-0.013$ | $0.097\pm 0.015$ | $0.265$ |
| Probability | $0.398\pm 0.120$ | $0.796\pm 0.109$ | $0.747\pm 0.126$ | $0.861$ | $0.424\pm 0.202$ | $0.581$ |
| Random | $\mathbf{0.667\pm 1.260}$ | $0.750\pm 0.531$ | $0.520\pm 0.275$ | $0.946$ | $0.091\pm 0.283$ | $0.585$ |

**Table 36: Acquisition performance on MIMIC Symile for AUROC (ECG imputed by Image and Lab), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $2.634\pm 0.819$ | $4.500\pm 1.001$ | $3.062\pm 0.288$ | $4.806\pm 0.645$ | $2.415\pm 0.090$ | $4.486$ |
| True KL-Div. | $1.145\pm 0.450$ | $1.172\pm 0.141$ | $0.774\pm 0.037$ | $0.821\pm 0.061$ | $0.897\pm 0.006$ | $0.839$ |
| True Rank | $1.120\pm 0.318$ | $1.107\pm 0.131$ | $0.858\pm 0.021$ | $0.776\pm 0.057$ | $0.916\pm 0.007$ | $0.836$ |
| True Uncert. | $1.047\pm 0.381$ | $0.787\pm 0.136$ | $0.632\pm 0.069$ | $0.338\pm 0.071$ | $0.675\pm 0.013$ | $0.537$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $\mathbf{1.120\pm 0.452}$ | $\mathbf{1.200\pm 0.150}$ | $\mathbf{0.771\pm 0.037}$ | $\mathbf{0.812\pm 0.065}$ | $\mathbf{0.894\pm 0.006}$ | $\mathbf{0.838}$ |
| Probability | $0.933\pm 0.302$ | $0.676\pm 0.106$ | $0.508\pm 0.038$ | $0.111\pm 0.120$ | $0.349\pm 0.014$ | $0.438$ |
| Rank | $0.126\pm 0.141$ | $0.517\pm 0.086$ | $0.489\pm 0.027$ | $0.507\pm 0.059$ | $0.504\pm 0.009$ | $0.473$ |
| Uncertainty | $1.063\pm 0.392$ | $0.820\pm 0.140$ | $0.638\pm 0.072$ | $0.358\pm 0.070$ | $0.673\pm 0.013$ | $0.551$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.785\pm 0.249$ | $0.509\pm 0.088$ | $0.499\pm 0.058$ | $0.468\pm 0.062$ | $0.604\pm 0.009$ | $0.544$ |
| Probability | $0.097\pm 0.243$ | $0.594\pm 0.105$ | $0.528\pm 0.047$ | $0.435\pm 0.077$ | $0.556\pm 0.012$ | $0.466$ |
| Random | $0.487\pm 0.212$ | $0.363\pm 0.126$ | $0.463\pm 0.043$ | $0.118\pm 0.177$ | $0.458\pm 0.013$ | $0.427$ |

**Table 37: Acquisition performance on MIMIC Symile for AUPRC (ECG imputed by Image and Lab), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $1.578\pm 0.119$ | $3.454\pm 0.465$ | $3.604\pm 0.479$ | $3.295\pm 0.414$ | $2.989\pm 0.181$ | $3.703$ |
| True KL-Div. | $0.634\pm 0.094$ | $1.070\pm 0.078$ | $0.807\pm 0.038$ | $0.702\pm 0.052$ | $0.904\pm 0.017$ | $0.814$ |
| True Rank | $0.782\pm 0.060$ | $1.014\pm 0.057$ | $0.873\pm 0.054$ | $0.707\pm 0.076$ | $0.923\pm 0.019$ | $0.800$ |
| True Uncert. | $0.565\pm 0.081$ | $0.826\pm 0.085$ | $0.531\pm 0.060$ | $0.175\pm 0.056$ | $0.601\pm 0.018$ | $0.431$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $\mathbf{0.621\pm 0.092}$ | $\mathbf{1.087\pm 0.082}$ | $\mathbf{0.815\pm 0.041}$ | $\mathbf{0.702\pm 0.061}$ | $\mathbf{0.903\pm 0.017}$ | $\mathbf{0.812}$ |
| Probability | $0.577\pm 0.055$ | $0.687\pm 0.059$ | $0.484\pm 0.052$ | $-0.037\pm 0.152$ | $0.382\pm 0.026$ | $0.336$ |
| Rank | $0.182\pm 0.091$ | $0.481\pm 0.066$ | $0.422\pm 0.074$ | $0.514\pm 0.052$ | $0.469\pm 0.015$ | $0.430$ |
| Uncertainty | $0.572\pm 0.081$ | $0.851\pm 0.087$ | $0.533\pm 0.061$ | $0.188\pm 0.056$ | $0.602\pm 0.018$ | $0.441$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.456\pm 0.053$ | $0.596\pm 0.072$ | $0.377\pm 0.054$ | $0.287\pm 0.026$ | $0.521\pm 0.017$ | $0.444$ |
| Probability | $0.435\pm 0.052$ | $0.502\pm 0.063$ | $0.684\pm 0.050$ | $0.499\pm 0.081$ | $0.660\pm 0.015$ | $0.585$ |
| Random | $0.374\pm 0.078$ | $0.367\pm 0.062$ | $0.432\pm 0.083$ | $-0.072\pm 0.281$ | $0.430\pm 0.021$ | $0.347$ |

## Appendix I Results for MIMIC HAIM

**Table 38: Acquisition performance on MIMIC HAIM for AUROC, showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $5.121\pm 1.873$ | $6.802\pm 2.953$ | $4.228\pm 1.374$ | $5.862\pm 0.933$ | $3.435\pm 0.792$ | $4.602$ |
| True KL-Div. | $0.842\pm 0.154$ | $0.558\pm 0.187$ | $0.673\pm 0.054$ | $0.518\pm 0.093$ | $0.708\pm 0.025$ | $0.608$ |
| True Rank | $0.853\pm 0.107$ | $0.684\pm 0.056$ | $0.706\pm 0.034$ | $0.676\pm 0.087$ | $0.714\pm 0.041$ | $0.723$ |
| True Uncert. | $0.429\pm 0.264$ | $0.647\pm 0.107$ | $0.729\pm 0.099$ | $0.332\pm 0.116$ | $0.555\pm 0.057$ | $0.538$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $0.827\pm 0.213$ | $0.561\pm 0.142$ | $0.488\pm 0.053$ | $0.505\pm 0.109$ | $0.459\pm 0.074$ | $0.465$ |
| Probability | $0.318\pm 0.280$ | $\mathbf{0.657\pm 0.105}$ | $0.761\pm 0.133$ | $0.620\pm 0.082$ | $0.578\pm 0.051$ | $0.494$ |
| Rank | $0.530\pm 0.685$ | $-0.048\pm 0.305$ | $0.311\pm 0.079$ | $0.429\pm 0.150$ | $0.485\pm 0.032$ | $0.391$ |
| Uncertainty | $0.022\pm 0.463$ | $0.647\pm 0.108$ | $\mathbf{0.880\pm 0.241}$ | $0.503\pm 0.123$ | $0.493\pm 0.056$ | $\mathbf{0.554}$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.492\pm 0.209$ | $0.598\pm 0.038$ | $0.486\pm 0.061$ | $0.244\pm 0.163$ | $0.525\pm 0.032$ | $0.452$ |
| Probability | $0.244\pm 0.302$ | $0.615\pm 0.116$ | $0.550\pm 0.045$ | $\mathbf{0.641\pm 0.065}$ | $\mathbf{0.580\pm 0.042}$ | $0.457$ |
| Random | $\mathbf{1.005\pm 0.367}$ | $0.143\pm 0.289$ | $0.470\pm 0.045$ | $0.546\pm 0.096$ | $0.510\pm 0.025$ | $0.526$ |

**Table 39: Acquisition performance on MIMIC HAIM for AUPRC, showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed ones and baselines in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $1.490\pm 0.249$ | $3.181\pm 0.283$ | $3.084\pm 0.331$ | $2.738\pm 0.363$ | $2.782\pm 0.349$ | $3.087$ |
| True KL-Div. | $0.967\pm 0.098$ | $0.652\pm 0.076$ | $0.435\pm 0.167$ | $0.847\pm 0.065$ | $0.717\pm 0.030$ | $0.662$ |
| True Rank | $1.014\pm 0.085$ | $0.721\pm 0.068$ | $0.541\pm 0.112$ | $0.958\pm 0.097$ | $0.769\pm 0.031$ | $0.754$ |
| True Uncert. | $0.842\pm 0.068$ | $0.592\pm 0.081$ | $0.605\pm 0.047$ | $0.425\pm 0.084$ | $0.611\pm 0.057$ | $0.587$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $0.889\pm 0.097$ | $0.578\pm 0.054$ | $0.301\pm 0.163$ | $0.655\pm 0.110$ | $0.518\pm 0.030$ | $0.516$ |
| Probability | $0.584\pm 0.111$ | $0.672\pm 0.041$ | $\mathbf{0.807\pm 0.075}$ | $0.480\pm 0.018$ | $0.754\pm 0.017$ | $0.686$ |
| Rank | $0.890\pm 0.120$ | $0.337\pm 0.081$ | $0.247\pm 0.115$ | $\mathbf{0.706\pm 0.052}$ | $\mathbf{0.474\pm 0.023}$ | $0.456$ |
| Uncertainty | $0.799\pm 0.056$ | $0.540\pm 0.082$ | $0.680\pm 0.071$ | $0.353\pm 0.047$ | $0.584\pm 0.030$ | $0.570$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.920\pm 0.131$ | $0.556\pm 0.082$ | $0.409\pm 0.085$ | $0.365\pm 0.011$ | $0.487\pm 0.053$ | $0.479$ |
| Probability | $0.480\pm 0.112$ | $\mathbf{0.713\pm 0.046}$ | $0.755\pm 0.054$ | $0.589\pm 0.053$ | $0.752\pm 0.017$ | $\mathbf{0.701}$ |
| Random | $\mathbf{0.941\pm 0.107}$ | $0.404\pm 0.060$ | $0.455\pm 0.057$ | $0.452\pm 0.180$ | $0.546\pm 0.024$ | $0.561$ |

**Table 40: Acquisition performance on MIMIC HAIM for AUROC (Image imputed by Lab), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $4.447\pm 3.050$ | $3.381\pm 1.055$ | $2.055\pm 0.185$ | $6.068\pm 1.280$ | $1.710\pm 0.036$ | $3.810$ |
| True KL-Div. | $1.002\pm 0.142$ | $0.425\pm 0.184$ | $0.623\pm 0.061$ | $0.299\pm 0.113$ | $0.687\pm 0.022$ | $0.476$ |
| True Rank | $0.902\pm 0.059$ | $0.704\pm 0.044$ | $0.667\pm 0.035$ | $0.575\pm 0.139$ | $0.698\pm 0.010$ | $0.671$ |
| True Uncert. | $0.908\pm 0.185$ | $0.768\pm 0.035$ | $0.641\pm 0.022$ | $0.301\pm 0.207$ | $0.709\pm 0.008$ | $0.610$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $0.686\pm 0.060$ | $0.579\pm 0.031$ | $0.463\pm 0.054$ | $0.412\pm 0.085$ | $0.539\pm 0.022$ | $0.416$ |
| Probability | $0.514\pm 0.051$ | $\mathbf{0.773\pm 0.153}$ | $\mathbf{0.560\pm 0.026}$ | $\mathbf{0.768\pm 0.113}$ | $0.590\pm 0.010$ | $0.445$ |
| Rank | $\mathbf{1.530\pm 0.782}$ | $0.045\pm 0.280$ | $0.418\pm 0.037$ | $0.203\pm 0.164$ | $0.450\pm 0.021$ | $0.497$ |
| Uncertainty | $0.710\pm 0.087$ | $0.730\pm 0.091$ | $0.548\pm 0.029$ | $0.597\pm 0.215$ | $0.587\pm 0.012$ | $\mathbf{0.625}$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.512\pm 0.247$ | $0.635\pm 0.040$ | $0.491\pm 0.032$ | $0.086\pm 0.287$ | $0.587\pm 0.015$ | $0.438$ |
| Probability | $0.109\pm 0.371$ | $0.759\pm 0.146$ | $0.533\pm 0.026$ | $0.739\pm 0.095$ | $\mathbf{0.592\pm 0.010}$ | $0.392$ |
| Random | $1.112\pm 0.371$ | $0.483\pm 0.041$ | $0.509\pm 0.015$ | $0.556\pm 0.138$ | $0.542\pm 0.008$ | $0.582$ |

**Table 41: Acquisition performance on MIMIC HAIM for AUPRC (Image imputed by Lab), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $1.490\pm 0.249$ | $3.177\pm 0.420$ | $3.038\pm 0.399$ | $2.586$ | $1.812\pm 0.037$ | $2.790$ |
| True KL-Div. | $0.967\pm 0.098$ | $0.523\pm 0.091$ | $0.372\pm 0.195$ | $0.717$ | $0.664\pm 0.033$ | $0.579$ |
| True Rank | $1.014\pm 0.085$ | $0.632\pm 0.084$ | $0.498\pm 0.128$ | $0.773$ | $0.685\pm 0.019$ | $0.706$ |
| True Uncert. | $0.842\pm 0.068$ | $0.790\pm 0.040$ | $0.624\pm 0.054$ | $0.575$ | $0.776\pm 0.014$ | $0.660$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $0.889\pm 0.097$ | $0.540\pm 0.066$ | $0.260\pm 0.194$ | $0.456$ | $0.586\pm 0.031$ | $0.478$ |
| Probability | $0.584\pm 0.111$ | $0.674\pm 0.058$ | $\mathbf{0.796\pm 0.087}$ | $0.455$ | $0.710\pm 0.011$ | $0.662$ |
| Rank | $0.890\pm 0.120$ | $0.288\pm 0.118$ | $0.229\pm 0.139$ | $\mathbf{0.650}$ | $0.459\pm 0.026$ | $0.416$ |
| Uncertainty | $0.799\pm 0.056$ | $0.683\pm 0.085$ | $0.700\pm 0.081$ | $0.263$ | $0.665\pm 0.022$ | $0.615$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.920\pm 0.131$ | $0.700\pm 0.096$ | $0.448\pm 0.096$ | $0.387$ | $0.651\pm 0.025$ | $0.553$ |
| Probability | $0.480\pm 0.112$ | $\mathbf{0.714\pm 0.071}$ | $0.749\pm 0.063$ | $0.483$ | $\mathbf{0.715\pm 0.011}$ | $\mathbf{0.686}$ |
| Random | $\mathbf{0.941\pm 0.107}$ | $0.429\pm 0.088$ | $0.464\pm 0.067$ | $0.590$ | $0.595\pm 0.012$ | $0.615$ |

**Table 42: Acquisition performance on MIMIC HAIM for AUROC (Lab imputed by Image), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $6.244\pm 0.851$ | $12.503\pm 7.484$ | $11.469\pm 3.810$ | $5.604\pm 1.451$ | $5.899\pm 1.530$ | $6.851$ |
| True KL-Div. | $0.575\pm 0.311$ | $0.780\pm 0.402$ | $0.842\pm 0.049$ | $0.792\pm 0.088$ | $0.737\pm 0.054$ | $0.770$ |
| True Rank | $0.773\pm 0.299$ | $0.649\pm 0.136$ | $0.836\pm 0.030$ | $0.802\pm 0.076$ | $0.736\pm 0.102$ | $0.796$ |
| True Uncert. | $-0.370\pm 0.158$ | $0.445\pm 0.274$ | $1.020\pm 0.440$ | $0.371\pm 0.065$ | $0.334\pm 0.084$ | $0.416$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $\mathbf{1.063\pm 0.606}$ | $0.531\pm 0.397$ | $0.572\pm 0.156$ | $0.622\pm 0.223$ | $0.345\pm 0.175$ | $0.519$ |
| Probability | $-0.010\pm 0.798$ | $0.463\pm 0.079$ | $1.433\pm 0.393$ | $0.435\pm 0.086$ | $0.560\pm 0.129$ | $\mathbf{0.551}$ |
| Rank | $-1.136\pm 0.350$ | $-0.202\pm 0.707$ | $-0.044\pm 0.241$ | $\mathbf{0.712\pm 0.244}$ | $0.535\pm 0.072$ | $0.180$ |
| Uncertainty | $-1.124\pm 0.961$ | $0.508\pm 0.248$ | $\mathbf{1.986\pm 0.840}$ | $0.385\pm 0.063$ | $0.359\pm 0.123$ | $0.492$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $0.460\pm 0.449$ | $\mathbf{0.535\pm 0.076}$ | $0.467\pm 0.283$ | $0.441\pm 0.053$ | $0.438\pm 0.063$ | $0.458$ |
| Probability | $0.470\pm 0.594$ | $0.375\pm 0.158$ | $0.608\pm 0.200$ | $0.519\pm 0.069$ | $\mathbf{0.563\pm 0.105}$ | $0.511$ |
| Random | $0.826\pm 0.879$ | $-0.425\pm 0.748$ | $0.337\pm 0.193$ | $0.534\pm 0.143$ | $0.464\pm 0.056$ | $0.435$ |

**Table 43: Acquisition performance on MIMIC HAIM for AUPRC (Lab imputed by Image), showing $G_{\text{full}}$. Strategies are grouped by category. Best strategy among proposed and baseline methods in bold for each column.**
| Strategy | Fracture | Enl. Card. | Consolidation | Atelectasis | Edema | Mean |
| --- | --- | --- | --- | --- | --- | --- |
| Upper Bounds (for reference) |  |  |  |  |  |  |
| Oracle | $-$ | $3.188\pm 0.307$ | $3.312\pm 0.007$ | $2.814\pm 0.615$ | $4.169\pm 0.490$ | $4.011$ |
| True KL-Div. | $-$ | $0.884\pm 0.047$ | $0.749\pm 0.053$ | $0.912\pm 0.012$ | $0.792\pm 0.043$ | $0.776$ |
| True Rank | $-$ | $0.881\pm 0.081$ | $0.758\pm 0.195$ | $1.051\pm 0.050$ | $0.889\pm 0.035$ | $0.811$ |
| True Uncert. | $-$ | $0.237\pm 0.075$ | $0.508\pm 0.005$ | $0.350\pm 0.067$ | $0.376\pm 0.071$ | $0.467$ |
| Imputation-based (proposed) |  |  |  |  |  |  |
| KL-Div | $-$ | $0.647\pm 0.098$ | $0.508\pm 0.078$ | $\mathbf{0.755\pm 0.079}$ | $0.419\pm 0.032$ | $0.489$ |
| Probability | $-$ | $0.668\pm 0.054$ | $\mathbf{0.858\pm 0.163}$ | $0.492\pm 0.022$ | $\mathbf{0.816\pm 0.022}$ | $0.723$ |
| Rank | $-$ | $0.424\pm 0.082$ | $0.335\pm 0.044$ | $0.734\pm 0.076$ | $0.495\pm 0.042$ | $0.485$ |
| Uncertainty | $-$ | $0.282\pm 0.094$ | $0.580\pm 0.160$ | $0.398\pm 0.022$ | $0.469\pm 0.034$ | $0.461$ |
| Baselines (no imputation) |  |  |  |  |  |  |
| Uncertainty | $-$ | $0.298\pm 0.039$ | $0.214\pm 0.113$ | $0.354\pm 0.001$ | $0.254\pm 0.039$ | $0.294$ |
| Probability | $-$ | $\mathbf{0.712\pm 0.031}$ | $0.782\pm 0.104$ | $0.642\pm 0.002$ | $0.806\pm 0.028$ | $\mathbf{0.726}$ |
| Random | $-$ | $0.361\pm 0.069$ | $0.406\pm 0.089$ | $0.383\pm 0.287$ | $0.478\pm 0.045$ | $0.472$ |

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: Please see "Weakness"
soundness: 3
strengths: 1. The paper is well written and easy to follow. The motivation behind CAMA and the ideas underlying different acquisition strategies are clearly presented.
2. The proposed evaluation metric for the CAMA setting is sound.
3. Experiments are conducted on multiple multimodal datasets, providing comprehensive comparisons across acquisition strategies.
summary: The paper proposes Cohort-based Active Modality Acquisition (CAMA), a novel test-time setting for acquiring missing modalities. This setting focuses on ranking samples to determine which should receive additional, costly modalities in order to best achieve a global objective under a budget constraint. The authors further develop a family of imputation-based methods that combine generative imputation and discriminative modeling to estimate the expected benefit of acquiring missing modalities. Experiments on several multimodal datasets demonstrate that these imputation-based methods generally outperform baseline approaches that do not use imputation.
weaknesses: 1. According to Eqs. (3) and (4), the cost of acquiring missing modalities is assumed to be identical for all patients. This simplification may limit the practical applicability of the proposed setting and methods.
2. Building on the previous point, trivial strategies, such as prioritizing patients with a greater number of missing modalities, might achieve competitive performance.
3. Although the imputation-based methods generally outperform the baselines, their performance appears inconsistent across different evaluation criteria (e.g., KL-Divergence vs. probability metrics), particularly in Table 3. This variability raises uncertainty about which criterion should be preferred in practice.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: Mentioned along with Weaknesses.
soundness: 2
strengths: - The setting of test-time modality selection is quite important and needed, especially in clinical setting
- The paper views the problem from the cohort point of view instead of individual patient point of view (since $\beta$ is constraint on number of patients)
summary: The paper proposes Cohort-based Active Modality Acquisition (CAMA), a test-time setting to formalize the challenge of selecting patients which require additional testing. The proposal uses a generative imputation model $f_{imp}$ to estimate counterfactual scores after acquiring the modality and then ranks samples via heuristics (e.g., KL divergence between current and imputed predictive distributions). The goal is to maximize a global test-time metric (e.g., AUROC) with the budget constraint.
weaknesses: Weaknesses/Questions:
- The setting, as they described is not novel. The paper doesn't cite [1] or [2] where the proposed test-time setting is identical to the setting in [1, section 3] and [2], and Eq (4) of CAMA is related to Lemma 3 in [1], where each modality selected is to maximize the accuracy. As far as I understand, [1] also uses foundation models as encoders, followed by a classification head to predict logit scores. While [1] approaches the issue from an RL perspective, the test-time modality acquisition seems identical since both are maximizing group metrics. The imputation approach mimics both [1] and [2] -- [1] uses the foundation model as generative, and [2] uses spline networks for imputation and uses that to score, however as a sequential RL control problem instead of cohort-level greedy ranking.
- What doesn't make sense to me is eq (3), where if a sample is chosen, all modalities are acquired -- this basically doesn't give relevance to the setting under cost-constraints. Rather, it makes sense to have cost-constraints on the modalities itself, with a heterogenous cost vector $\mathbf{c}$  and have it choose the modalities under constraint $\beta$. This setting with only 1 modality not acquired boils down to the case of the paper. Moreover, the text of figure 1B says that the imputed score *estimates the counterfactual with only imputed modality added* - so is the logit score with all modalities or available + target?
- DDPMs/VAEs have been used previously for missing modality imputation, and the paper seems to compare standard heuristics. Although it claims KL-div performs the best, tables in the appendix don't seem consistent. 
- What is the quality of the imputations? What modalities are acquired? What cost-profiles do we get with performance? Just the curves/tables under a single budget for the $G_\bullet$ metric seems to lack a lot of important details. Cost-profiles for modalities and samples is much more important than inference time since the tasks at hand are not time-bounded. Are there correlations in patients requiring further screening? There are lot of lacking questions and ablations.

[1] Jain, E., Wenckstern, J., von Querfurth, B., & Bunne, C. (2025, March). Test-time view selection for multi-modal decision making. In ICLR 2025 Workshop on Machine Learning for Genomics Explorations.

[2] Li, Y., & Oliva, J. (2024). Towards Cost Sensitive Decision Making. arXiv preprint arXiv:2410.03892.


I believe the paper lacks novelty, and is confusing to read. While the test-time setting is important, sample-wise all-modality acquisition seems less feasible in practice to just modality acquisition. In the current state I lean to reject the paper.

---

confidence: 3
contribution: 2
initial_rating: 2
presentation: 3
questions: 1. During the evaluation in the CAMA setting, how does acquiring the additional modalities for one sample $i$ affect the performance/predictions for a different sample $j$? If there is no interaction between acquiring samples for $i$ and $j$, in what cases is the CAMA setting more interesting/beneficial than active feature acquisition? If there is interaction, how does it differ from active learning?
2. What is a unimodal baseline (l.259)? I assume that this is the baseline with only a single modality for each sample. However, different modalities can have different level of “informativeness” for the outcome of interest, and as such which modality is chosen for the unimodal baseline is a crucial decision that will significantly affect the baseline value. Hence, it seems to me that the unimodal baseline is not well defined.
3. Are the imputation based scores $\{s_{I, k}^{imp}\}$ obtained by taking samples from the generative model? If yes, what value of $K$ was used in the experiments, and how sensitive is the proposed framework to this hyperparameter?
4. What do authors mean by “true scores $s_i^{full}$ and $s_i^{avail}$” (l.291) in the context of their data? I assume this corresponds to the logits (l. 376). If yes, how (if at all) are the “true” logits calculated? Are they used for training?
5. In the evaluation process of their proposed models, I am assuming that the models are trained on the test set (in the cross-validation setting) and then the acquisition process is being run on the test set. Is that correct?
soundness: 2
strengths: - Very clear and extensive related works.
- The authors propose and discuss a comprehensive set of baseline acquisition functions (both oracle and random ones), which allow to put the performance of the proposed metrics in context.
- The experiments are run on several datasets, and over multiple runs, making the results significant.
summary: The paper proposes a new active data acquisition setting called cohort-based active modality acquisition (CAMA), the goal of which is to choose for which samples from some population we should acquire the values of missing modalities in order to maximise some performance metric. The authors propose a set of acquisition functions designed for the CAMA setting that can allow to guide the acquisition. The authors define both oracle metrics, relying on unavailable quantities, and simple baselines such as random acquisition as baselines. To handle the missing modalities in the data during inference, the authors use an architecture based on late fusion models and denoising diffusion probabilities models (DDPMs). With these models in place, the proposed acquisition functions are compared and evaluated on a set of real-world multimodal datasets.
weaknesses: **CAMA setting.**
1. The objective in the CAMA setting is not clear to me. The authors mention that the purpose of CAMA is to `identify those samples for which the acquisition of an additional data modality would be most beneficial`. This is formalised through equation 4. However, it is not clear exactly what “beneficial” means in this context. After the acquisition, the models are not retrained on the newly acquired data (as in active learning). Further, ‘benefits’ are also not measured with respect to improving individual predictions at test time (as in active feature acquisition). Hence, it is not immediate to me what variable CAMA is aiming to gain information about (as it seems to be _neither_ the model parameters, as in AL, _nor_ the outcome variable $Y$, as in AFA). Rather than focusing on gaining information about a specific variable, it seems to me that the “benefits” are measured through improving the target performance metric (e.g. AUROC) on the train/test set $\mathcal{D}$ itself (the distinction between the train and test set is blurry in the paper). While this is indeed a different setting than AL or AFA, I do not see how it translates to measurable gains in real-world settings. It would be great if the authors could clarify why considering such a setting might be of interest to any downstream application.
2. What further makes the setting quite artificial is that the “budget” does not account for the fact that different samples $i$ might require different number of modality acquisitions or that some modalities might be more informative (and also more expensive) than others. Instead, the framework only considers the problem of acquiring _all_ missing modalities or none, making this setting quite simplified.

**Acquisition Functions for CAMA.**
1. The authors claim that their contribution in this work is to propose a “theoretical framework” for developing AFs within the CAMA setting. However, looking at the metrics introduced in Appendix C, it seems to me like the degree of novelty in the derived metrics is limited. In particular, the best performing “Maximum Expected Uncertainty Reduction” and “Expected KL Divergence” seem to be variations of the Expected Information Gain metric (this connection is not discussed in the paper). Meanwhile, some of the other metrics (e.g. Maximum Expected Probability or Expected Rank Change) seem quite arbitrary and not supported theoretically. As such, I do not see theoretical results or insights as a significant contribution of this work.

**Architectures for CAMA.**
1. There is limited formalism offered to help understand the exact design of the proposed architecture, making it more difficult to evaluate it and assess its contribution. In particular, it is unclear to me what are the exact components of the architectures, when each of them are used (training vs inference) and what loss functions are used (in particular, what are the loss functions used for the generative component $f_{imp}$?). Equation 6 does not allow to track how the inputs are processed by the architecture used by the authors. Further, what is meant by the “discriminative components” in l.094 and l. 306? Is “discriminative” used in the GAN sense here?
2. The work seems to be heavily inspired by the architecture proposed in Wang et al. (2023). Compared to Wang et al. (2023), what changes did the authors propose or implement that allowed them to adapt this architecture to the CAMA setting specifically?
3. The contribution sections of the paper mentions that the work establishes upper bounds for CAMA. It is unclear for me what this means, and how it is manifested in the proposed architectures.

---

confidence: 1
contribution: 3
initial_rating: 6
presentation: 2
questions: 1. High Architectural Complexity: The proposed solution requires jointly training a complex discriminative classifier and a sophisticated generative model (e.g., a Diffusion Transformer). 

2. Assumption of a Single Target Modality: The current framework is designed around the scenario of acquiring one specific, pre-determined "target" modality (e.g., proteomics in the UKBB example). 

3. Dependence on Generative Model Fidelity: The success of the imputation-based strategies is fundamentally tied to the quality of the counterfactuals generated by the imputation model (f_imp). If the generative model fails to produce plausible imputations, the acquisition decisions will be misguided.
soundness: 3
strengths: 1. Novel and Practical Problem Formulation: The primary strength is the formalization of the CAMA setting. By shifting the focus from individual-level to cohort-level optimization and from training-time to test-time decisions, the paper addresses a highly practical and underexplored real-world problem. 
2. Comprehensive and Rigorous Evaluation: The experimental setup is exceptionally thorough. The authors evaluate their framework on four diverse, real-world multimodal datasets, including the large and complex UK Biobank, demonstrating scalability and generalizability.
3. Principled and Effective Acquisition Strategy: The use of generative imputation to estimate counterfactuals is a powerful and theoretically grounded approach.
summary: This paper introduces Cohort-based Active Modality Acquisition (CAMA), a novel problem setting that addresses the challenge of strategically acquiring costly additional data modalities for a subset of a cohort at test time, under a fixed budget. Unlike prior work focusing on individual samples or training-time acquisition, CAMA aims to maximize a global performance metric (e.g., AUROC) for the entire cohort. The core of the proposed solution is a family of "imputation-based" acquisition functions that use a generative model (like a DDPM) to predict counterfactual outcomes—simulating how a sample's prediction would change if the missing modality were acquired. By prioritizing samples with the largest expected predictive shift (quantified, for instance, by KL-Divergence), the method intelligently allocates the acquisition budget. Through comprehensive experiments on multiple real-world datasets, including the large-scale UK Biobank with up to 15 modalities, the paper demonstrates that these imputation-based strategies significantly outperform baseline methods like random selection or uncertainty-based sampling.
weaknesses: 1. High Architectural Complexity: The proposed solution requires jointly training a complex discriminative classifier and a sophisticated generative model (e.g., a Diffusion Transformer). 

2. Assumption of a Single Target Modality: The current framework is designed around the scenario of acquiring one specific, pre-determined "target" modality (e.g., proteomics in the UKBB example). 

3. Dependence on Generative Model Fidelity: The success of the imputation-based strategies is fundamentally tied to the quality of the counterfactuals generated by the imputation model (f_imp). If the generative model fails to produce plausible imputations, the acquisition decisions will be misguided.


Decision: Reject