Detecting Data Deviations in Electronic Health Records
Authors: Kaiping Zheng
Horng-Ruey Chua
Beng Chin Ooi
Venue: NeurIPS 2025 poster ()
Keywords: Electronic Health Records
Data Deviations
Primary area: machine_learning_for_sciences
TL;DR: We address the underexplored problem of detecting data deviations in electronic health records and propose a bi-level knowledge distillation approach as the solution.

Abstract: Data deviations in electronic health records (EHR) refer to discrepancies between recorded entries and a patient’s actual physiological state, indicating a decline in EHR data fidelity. Such deviations can result from pre-analytical variability, documentation errors, or unvalidated data sources. Effectively detecting data deviations is clinically valuable for identifying erroneous records, excluding them from downstream clinical workflows, and informing corrective actions. Despite its importance and practical relevance, this problem remains largely underexplored in existing research. To bridge this gap, we propose a bi-level knowledge distillation approach centered on a task-agnostic formulation of EHR data fidelity as an intrinsic measure of data reliability. Our approach performs layered knowledge distillation in two levels: from a computation-intensive, task-specific data Shapley oracle to a neural oracle for individual tasks, and then to a unified EHR data fidelity predictor. This design enables the integration of task-specific insights into a holistic assessment of a patient’s EHR data fidelity from a multi-task perspective. By tracking the outputs of this learned predictor, we detect potential data deviations in EHR data. Experiments on both real-world EHR data from National University Hospital in Singapore and the public MIMIC-III dataset consistently validate the effectiveness of our approach in detecting data deviations in EHR data. Case studies further demonstrate its practical value in identifying clinically meaningful data deviations.

Detecting Data Deviations in Electronic Health Records
Kaiping Zheng', Horng-Ruey Chua Beng Chin Ooi
"School of Computing, National University of Singapore
Division of Nephrology. Department of Medicine, National University Hospital
Department of Medicine, Yong Loo Lin School of Medicine, National University of Singapore Sccooo of Software Technology. Zhejiang University
dcszkai @nus enus.edu. mdcchr@nus.edi .ea.luus.ddu.... sdu ooibc@zju.edu.cm
Abstract
Data deviations in electronic health records (EHR) refer to discrepancies between recorded entries and a patient's actual physiological state, indicating decline in EHR data fidelity. Such deviations can result from pre-analytical variability, documentation errors, unvalidated data sources. Effectively detecting data deviations is clinically valuable for identifying erroneous records, excluding them from downstream clinical workflows, and informing corrective actions. Despite its importance and practical relevance, this problem remains largely underexplored existing research. To bridge this gap, we propose bi-level knowledge distillation approach centered on a task-agnostic formulation of EHR data fidelity as an intrinsic measure of data reliability. Our approach performs layered knowledge distillation in two levels: from a ceapuutation-nntessve task-specific data Shaplcy oracle neural oracle for individual tasks, and then a unified EHR data fidelity predictor. This design enables the integration of task-specific insights into a holistic assessment of patient's EHR data fidelity from multi-task perspective. By tracking the outputs of this learned predictor, we detect potential data deviations in EHR data. Experiments on both real-world EHR data from National University Hospital in Singapore and the public MIMIC-III dataset consistently validate the effectiveness our approach in detecting data deviations in EHR data. Case studies further demonstrate its practical value in identifying clinically meaningful data deviations.
Introduction
In healthcare data analytics, researchers utilize heterogencous data sources to support wide spectrum of applications, including risk prediction, medication recommendation, and discase progression modeling 41). These efforts have yielded tangible benefits for patients, clinicians, and healthcare institutions, contributing to broader societal impact. Among various data types, electronic health records (EHR) have emerged a primary source, with increasing availability in recent years. EHR data capture longitudinal patient information, including laboratory results, diagnoses, and prescriptions during their clinical visits. Leveraging EHR data for analytical purposes can enhance patient management and optimize healthcare resource allocation
Despite the potential of EHR data to reveal valuable insights, the aforementioned success of EHR data analytics critically depends sufficiently high data fidelity, which characterizes how accurately the recorded data capture and reflect the true characteristics of the original EHR data source [89
However, ensuring high fidelity in EHR data is inherently challenging due to the complexity of clinical environments. Hospitals and healthcare institutions serve large, heterogeneous populations
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

of patients, each generating numerous medical observations that must be accurately documented for clinical decision decision-making The scale and variability of EHR data complicate the maintenance of consistent and reliable records. Common sources of errors include pre-analytical variabilityy 46 as well as documentation errors stemming from human mistakes and complex workflows 80 14]. The problem is further exacerbated by the integration of non-validade data sources, particularly behavioral and physiological signals from wearable devices [18 277. which are typically collected in non-clinical, home-based settings and may lack consistency or reliability. These pervasive issues degrade data fidelity and pose substantial barriers to the integrity of EHR-driven analytics and decision support, ultimately constraining the performance of downstream learning models.
during specimen collection and handling [6
Addressing this pressing challenge requires the ability to detect data deviations in EHR is, discrepancies between recorded entries in EHR data and the patient's s actual physiological state. Detecting such deviations offers two benefits. First, it enables the identification and exclusion of erroneous records from clinical workflows, reducing the risk of inappropriate decisions and adverse outcomes. Second, it facilitates targeted data correction strategies, thereby enhancing data quality and supporting more reliable downstream analysis. More specifically, our primary objective is to enable pre-hoc detection of potential data deviations at the point of data entry into the EHR system during clinical practice. The focus is not on post-hoc data cleaning for downstream analytics but on proactive identification of anomalies at the time of data recording. By flagging entries that may exhibit deviations as they are being introduced, we aim to enable real-time quality control and reduce the likelihood of erroneous data entering the system. This proactive mechanism strengthens EHR data quality, which may subsequently enhance the performance of downstream tasks. Nevertheless, this improvement remains secondary effect; the central goal is the reliable assessment of data fidelity in situ, preventing flawed data from affecting clinical decision makinn in real-world settings.
Practically significant as it is, detecting data deviations in EHR data remains a non-trivial and underexplored problem. Data deviations directly impair data fidelity, necessitating a principled way to measure data fidelity. The data Shapley value (26). originally derived from the cooperative game theory [69,, offers a well-established approach to quantify the contribution of individual data samples to learning model. making it a promising candidate for this purpose. However, these contribution- based measures- measures-including the data Shapley value and other data valuation techniques B9] are typically tailored to specific application tasks, and thus may not capture a comprehensive or intrinsic notion of fidelity in a patient's 's EHR data. In contrast, we aim to develop an application-agnost fidelity measure that reflects the inherent reliability of a patient's 's data, independent of downstream tasks, serving as a foundation for more effective detection of data deviations.
Solution. We propose an innovative li-le knowledge distillation approach [33 28 62 for detecting EHR data irviaationsddisinnuune transfer of knowledge. As detailed in Section 3 our approach unfolds in three stages. First, compute task-specific data Shapley values to serve as a data Shapley oracle Oae encoding fine- fine-grained contribution information for each sample (Section 3. Next, to address the high computational cost of Oas. we distill its knowledge into a task-specific neural oracle Oan by amortizing per-sample computation through neural network [3] enabling efficient approximation (Section 3.2). Finally, taking a multi-task perspective on data fidelity, we further distill knowledge from all O.m instances into a unified EHR data fidelity predictor vi (Section 3.3). This hierarchical design uniquely enables the integration of task-specific insights into a general-purpose fidelity estimator, which we use to detect and respond EHR data deviations.
by its task-agnostic formulation of data fidelity and its layered
Novelty. (i) To the best of our knowledge, this work is the first to address the problem of detecting data deviations in EHR data, critical yet underexplored issue with direct implications for clinical reliability and safety. (ii) In contrast to existing data valuation methods, such as the data Shaplcy value, which are inherently task-specifer we propose multi-task perspective derive unified and comprehensive measure of EHR data fidelity. By tracking changes in this task-agnostic fidelity predictor, enable effective detection of potential deviations in EHR data.
Contributions. (i) We address the open problem of measuring EHR data fidelity and detecting data deviations, filling a critical research gap. (ii) We propose a bi-level knowledge distillation approach that transfers information from computationally intensive data Shapley values to task- specific neural oracles, and subsequently to unified EHR data fidelity predictor, enabling effective and efficient fidelity estimation (Section 3). (iii We empirically evaluate our approach on both

---

35.8
(a) Pre-analytical Variability
(b) Documentation Errors (c) Non-validated Data Sources
Figure 1: Three representative sources of data deviations in EHR data.
EHR data from National University Hospital in Singapore and the MIMIC-III dataset (Section 4 The results consistently validate the effectiveness of our approach in detecting data deviations, with representative case studies highlighting its practical utility in identifying specific deviation issues relevant clinical practice.
2 Problem and Our Solution
Necessity of EHR data deviation detection. As discussed, data deviations in EHR data reflect a decline in the fidelity of recorded health data, raising critical concerns about the extent to which such records accurately represent patient's s physiological state at the time of entry. These deviations may arise from various sources, three of which are illustrated in Figure [ The first is pre-analytical vari- ability (Figure where deviations are introduced during specimen collection and handling- -ucch sampling bias, improper storage, unrecognized technical artifacts. The second is documentation errors (Figure [b)), which may occur during data entry, labeling, or integration across systems. These errors, including misclassification, omission, or duplication, typically result from reading mistakes or workflow complexity in interconnected clinical information systems. The third is non-validated data sources (Figure (e)) such patient self-reported measurements, consumer-grade wearable devices, or home-based testing, which lack clinical oversight. These data are prone to inaccuracies due to improper device use, recall bias, or the absence device calibration and standardization.
For instance, patients admitted to the nephrology department may be required to perform urine tests assess kidney function. Patients may collect the urine samples at the wrong hours of the day, may contaminate the urine with other body fluids, or healthcare personnel may delay the urine processing; these lead to distorted laboratory results, which may affect subsequent diagnosis and clinical decision- maaiing. before certain tests for blood glucose, lipids, or parathyroid hormone, for which they may mistakenly consume food prior to testing. In such cases, recall errors or noncompliance can degrade EHR data fidelity, resulting in misleading records and potential data deviations.
addition, patients with kidney disease and diabetes may have to fast
The ability to detect potential data deviations in EHR data holds substantial clinical importance. An effective detector evaluate whether recorded data is trustworthy, thereby improving the accuracy of downstream medical decisions and interventions. Assessing data fidelity allows clinicians to determine whether given measurement accurately reflects the patient's 's condition, enabling more precise clinical management. Additionally, identifying patterns of deviations can further guide improvements in data acquisition, collection, and recording protocols, laying the groundwork for future correction and calibration strategies.
Necessity of distilling knowledge from data Shapley oracle Oas to neural oracle Om To assess EHR data fidelity, the data Shapley value [26]-along with other data valuation methods (39] -offers promising approach by quantifying the contribution of each data sample. Though inherently tailored to particular application, the resulting data Shapley values capture the utility of each sample from the perspective defined by the given task, providing task-specific lens for evaluating data fidelity.
Nevertheless, computing data Shapley values is computationally intensive, with brute-force methods exhibiting exponential complexity. Even with approximation techniques such as Monte Carlo sampling [9], the computational cost remains prohibitive, particularly when applied to large-scale real- world healthcare datasets. This limits the feasibility of applying Shapley-based fidelity assessment in practice, where EHR data fidelity assessment and thus data deviation detection -requires frequent "what-if" analysis. For example, each newly generated medical feature for patient may necessitate
3

---

Data Shapley Oracle das
Neural Oracle on
EHR Data Fidelity
Predictor 4
Task
Task
Task
Data Fidelity
poo
(x.)
k (x,)
000 (x.)
oooo
0000
6000
+ OODO0
Lent
permutation sampling in
Knowledge Distillation
o0000
Knowledge Distillation
Monte Carla
Exim
+
poc o(x))
0000
ooo0
@
Figure 2: Model architecture of our proposed bi-level knowledge distillation approach.
re-evaluating data fidelity to determine potential deviations. To address this, it is advantageous to amortize the computation across samples using a learned model, such as a neural network [1 Motivated by this need for efficiency, we propose distill knowledge from the data Shapley oracle Oas into a task-specific neural oracle Onn thereby preserving task-relevant valuation insights while significantly reducing computational overhead.
Necessity of distilling knowledge from neural oracle On to EHR data fidelity predictor v. We posit that a patient's 's EHR data fidelity should represent an intrinsic, task-agnostic measure of data reliability. In contrast, the approximate data valuation produced by the task-specific neural oracle Om captures only partial, desk-dependent view. To overcome this limitation, perspective to distill the task-speeific knowledge into a unified EHR data fidelity predictor V which provides holistic and integrated assessment of patient's EHR data fidelity by synthesizing insights across multiple clinical tasks, thereby yielding a more comprehensive and robust measure.
take multi-task
Methodology
Given EHR data K00)= = (4/40) for each task (i.e., specific application), where k.()) denotes the i-th sample (i.c., patient in our case), with i E {0..... (O,...,,Y No 1}, t {o...., T 1}. (9 is the number of samples for task f, and T is the total number of tasks. Each sample is represented E,!) (x,, y/ (.), where x; denotes the input EHR data and y; (0) is the task-specific label. We focus on clinical settings where EHR data and associated labels are available for multiple tasks. Our objective is to first quantify data fidelity-the degree which a patient's EHR data can be trusted to be accurate and reliable. Based on this, we then detect potential deviations that indicate anomalies or inconsistencies in the EHR data. We define EHR data deviations follows:
Definition (EHR Data Deviations) Let denote the original EHR data of patient i, and let V (x;) represent its data fidelity. Assume x; is perturbed x; Ax; Given a deviation probability Paev and a predefined threshold u. a data deviation is detected if the following condition holds:
38 such that Paev Pp when A Vx V + Ax) > 8.
(1)
In other words, a substantial decline in data ditafidelity A beyond threshold 8 is associated with high likelihood of deviations in the patient's EHR data.
To detect EHR data deviations, we propose bi-level knowledge distillation approach as depicted in Figure 2 In the first level, we train set of models to approximate the data Shapley values efficiently, one for each prediction task. Concretely, knowledge is distilled from the data Shapley oracle O which provides application-speciff data valuation -ntiiionnniit- a corresponding neural oracle Om for each task. These task-specific neural oracles are trained jointly in multi-task learning setting. In the second level, knowledge from all neural oracles is further distilled into a unified EHR data fidelity predictor V, which serves to identify potential data deviations. This approach consists of three stages: (i) computing data Shapley values per task via Oas. (ii) distilling knowledge from Oa. to O and (iii) distilling knowledge from Om to V. Each stage is detailed below.

---

3.1 Data Shapley Value Computation Per Task in Oas
We begin by quantifying the value of each patient's EHR data for the available tasks. Data valu- ation [39] provides principled framework to assess the contribution of individual data samples to downstream learning performance. Among various approaches, the data Shapley value [26], derived from the Shapley value in cooperative game theory [69], is a widely adopted method with demonstrated effectiveness across multiple domains [65].
Given the task-specific nature of data Shapley values, we compute these values for each sample within each task, resulting in a task-wise data Shapley oracle, denoted Oas For task let f(t) represent the corresponding prediction model, and consider evaluating its performance on subset C K(0), Given an evaluation metric m, the performance of (()) on is denoted by m(S,f())
The data Shapley value of specific sample k:4)) for task is then defined as:
m(S U (K()) (() m(S, N(O - |S|
(4) 13
M
Σ
(2)
where the summation considers all subsets of the samples task t, excluding k/9) and M is a normalization constant. Equation can be equivalently reformulated as follows:
(0) 7/3 E.pro(m(S U (, (0)) mm( (())
where & denotes uniform distribution over all permutations of 1C(1), and t is the set of samples preceding ( in the permutation yp Given the exponential complexity of exact Shapley value com- putation, approximate '/2
(f) using Monte Carlo permutation sampling 9 for enhanced efficiency.
3.2 Knowledge Distillation from Ose to Om
Despite the efficiency gains from approximation methods such Monte Carlo permutation sampling, the computational overhead remains prohibitive, particularly for large-scale real-world healthcare datasets. To address this, adopt amortized modeling approach, leveraging neural networks to approximate per-sample outputs and mitigate the cost individual computations [5].
Concretely, after obtaining task-specific data valuations from the data Shapley oracle Oas. we distill the knowledge into neural oracle On for each task using a neural network gd) of"),, which is
trained to approximate the data Shapley values v1; by minimizing the following loss function, where I(.,-) denotes the mean squared error:
Σ ( (()))
(4)
The neural oracles Om across all tasks are trained jointly in a multi-task learning fashion. To balance task-specific contributions, we assign weight to each loss d-s-tm for task f, defined as the ratio between its value at the current iteration and that at the previous itcration s - 1:
(())
( c -m (8) 4
(5)
This weighting strategy encourages loss terms across tasks to decrease at comparable rates, mitigating scale discrepancies and promoting balanced multi-task optimization 30 [5]] We further denote the
output of the last hidden layer in Om for task t as (9) (x:), representing the task-specific learned representation of input x,,
3.3 Knowledge Distillation from Onn to V for EHR Data Deviation Detection
In this stage, we aim learn the final EHR data fidelity predictor (x 0) by distilling knowledge from the task-specific neural oracles O. trained in the previous stage. We construct (x 0) as neural network that aggregates and transfers information from all Om across tasks, We denote the hidden representation of learned by (x 0) as o(x).
5

---

Knowledge Distillation Loss. To ensure that the learned EHR data fidelity predictor & effectively aggregates information from all task specific neural oracles On we introduce knowledge distillation loss Lıd- This loss minimizes the -aaasau by the mean squared error- error-between the output of (xx, 0) and a weighted aggregation the outputs from the individual On oracles:
Lud
(vx,,, X.
Σ " (x:)g (xi, (xx, (x,, ))
(6)
The weight (x) for task & is computed using attention subnetwork that integrates both ((0)) the task-specific representation of x from Om. and o(x), the hidden representation of o(x) derived by (xx, The attention subnetwork first concatenates these representations and applies affine transformation, followed by a ReLU activation:
p(() (x) ReLLU(Wc(o(x);
b.)
The attention weight a (1) (x) then computed after normalization across tasks:
a (x) w (+))())) + bao a (x) exp(ā (x))/ Σ exp(á (x)))
(x))
(8)
Relative Entropy Constraint. In addition to the primary Lid. introduce relative entropy-based regularization term to encourage the EHR data fidelity predictor to leverage information from a broader range of neural oracles, rather than over-relying the relative entropy (i.e., Kullback-Leibler divergence) between the learned task-specific attention weights (x and a uniform prior (() 1/"T, thereby encouraging a more balanced utilization of all available neural oracles 176, 60 59]. This regularization promotes diversity in knowledge aggregation by favoring higher entropy in the attention distribution. The relative entropy constraint is defined
small subset. Specifically, we minimize
Len (aii(a' (x)ll((1) = log T - =+(a((
where log T is constant, and (x))) denotes the entropy of a learned from Equation &
Similarity Constraint. Beyond Lui and Lem above, redundancy among neural oracles. When two task-specific neural oracles O.. produce highly similar outputs across samples, assigning high weights to both is unnecessary. To mitigate this, we introduce similarity constraint to discourage simultaneous high attention weights for such similar oracle pairs:
also crucial to account for potential
Leim
ΣΣ X IEE
(x:)) (x:)a (x:)
(10)
where Pr, (x;) quantifies the output similarity between the neural oracles for tasks and if as:
(x,e(),)) = exp( exp(-E((or) (x ((t))
(x) ((x,,o(())) ))///)
(11)
133 A larger Pt.t' (x,) indicates higher similarity, thus leading to a larger penalty if both tasks are assigned high attention weights.
with J as a temperature parameter controlling sensitivity
Overall Objective. We integrate the three loss terms- terms--nowlledg distillation, entropy regularization, and similarity constraint-ing into a unified loss function:
Cea + Dent + Asim Csim
(12)
where the weights ARa Aent and Asim are dynamically adjusted based on the ratio of the loss values across successive training iterations:
Aud= = Lxa()) Lsals 1), Aent = Lem(s)/)eemm Le 1), Asim = Leim (:)m(s)/ Lxim 1)
(13)
We have thus far described the bi-level knowledge distillation process: first, distilling tpec-ipe data valuation from the data Shapley oracle Oaz into neural oracles Omn and subsequently distilling this information across tasks into the final EHR data fidelity predictor V. The predictor trained using the overall loss L and the parameters 0 are optimized iteratively until convergence. Once trained, the EHR data fidelity predictor is used assess EHR data fidelity and identify potential data deviations, enabling EHR systems issue alerts when anomalies are detected.

---

Experimental Evaluation
We evaluate the effectiveness of proposed EHR data fidelity predictor V/ in detecting data deviations using real-world EHR data from National University Hospital in Singapore. To demonstrate its broader applicability, we further assess V/ on the public MIMIC-III benchmark dataset [40. with results provided in Appendix
4.1 Experimental Set-up
The evaluation cohort includes patients diagnosed with acute kidney injury (AKI) between November 2015 and October 2016, which in total comprises 2.237 patients from the EHR data of National University Hospital in Singapore. The anonymized dataset contains 43 distinct laboratory tests, result- ing in 130,755 recorded test entries. AKI, characterized by a sudden decline in kidney function 4. raises substantial clinical concern regarding long-term prognosis. These patients were subsequently followed up for five years to track clinically significant outcomes. We focus on post-AKI progression by using data from the 90-day period following AKI diagnosis (the "observation window") to predict the occurrence of four major adverse kidney events below during the subsequent 5-ar follow-up period (the "prediction window").
New Progressive Chronic Kidney Disease (CKD) Prediction: Predict whether the patient will experience new-onset or progressive CKD, defined as a decline of more than 30% in baseline estimated glomerular filtration rate (eGFR) after the 90-day observation window.
Stage 5 CKD Onset Prediction: Predict whether the patient's cGFR will decline below 15mL//min mtm/mm/mmmm111m 71m after 90 days, indicating progression to near end-stage kidney disease.
PO Renal Replacement Therapy (RRT) Dependence Prediction: Predict whether the patient will require RRT during the follow-up period, indicating persistent loss of kidney function requiring long-term intervention.
Mortality Prediction: Predict whether the patient will pass away during the prediction window following the 90-day observation window.
We partition the cohort into 85% for model development and 15% as a held-out set for computing data Shapley values. Within the 85%, further split the data into 80% for training. 10% for validation, and 10% for testing. Hyperparameters are selected based on the best validation performance, measured by the minimum loss L (Equation 12p, across three independent runs. The final model is evaluated on the test set using the selected hyperparameter configuration.
4.2 Validation of T Detection Effectiveness under Controlled Deviation Injection
To evaluate the effectiveness of our proposed EHR data fidelity predictor V/ in quantifying data fidelity and detecting deviations in EHR data, we design controlled deviation injection experiment. For each sample in the 10% test set, we identify its most prominent feature, defined as the dimension the value of which is closest the 99th percentile of the corresponding distribution in the training set, and apply a controlled perturbation. The deviation magnitude is scaled by a multiple of the feature's 's standard deviation o in the training set. This procedure produces paired samples: the original instance (label 0) and its perturbed counterpart (label = 1), forming labeled benchmark for fidelity evaluation under controlled deviation conditions.
More specifically, this can be conceptually viewed as a "real versus corrupted" classification experi- ment aimed at distinguishing genuine EHR samples from their perturbed counterparts. Under the controlled deviation injection setup, we (i) perturb each sample by scaling its most prominent feature with its standard deviation a to simulate physiologically plausible deviations, (ii) generate paired (label 0) and perturbed (label = 1), and (iii) construct a labeled benchmark dataset for fidelity evaluation. We then compute the expected EHR data fidelity decline, A V,, as defined in Equation []] and use its sign to determine whether the proposed approach successfully detects the introduced deviations.
We compare against several widely adopted unsupervised anomaly detection baselines:
One-Class SVM: Constructs decision boundary in a high-dimensional feature space using a radial basis function (RBF) kernel. During training, up to 5% of the samples are allowed to be
7

---

1.
x16
Mr Max Rangs
09
Methods
5o after Deviiaiio Injection
Mod
05
V/orr propceet)
Local Outler
2-
Gauetan Weum Mode!
PCA Reconemuction Time
0.5
Deviation Magritude (times
Deviation Magnitute (times ol
Figure 3: Performance comparison between V and baselines for EHR data deviation detection.
Figure 4: Impact of deviation magnitudes on A
(data fidelity decline) after deviation injection.
treated as outliers. Anomaly scores are derived based on each sample's distance to the learned decision boundary.
Local Outlier Factor: Estimates the local density of each sample based on its 20 nearest neighbors. Samples exhibiting substantially lower local reachability density are identified as outliers. The detection threshold is set the 5thh percentile of the Local Outlier Factor scores computed on training data.
generated from mixture of Gaussian distributions. Anomaly scores are computed based on the log-likelihood of each sample under the fitted model. Samples with log- regliiklihoods below the 3th percentile of the training distribution flagged as outliers.
Gaussian Mixture Model: Assumes that the samples
k-Means Distance: Computes the Euclidean distance between each sample and its assigned cluster centroid obtained via k-Means clustering. Samples with distances exceeding the 90th percentile of the training distances are classified as outliers.
PCA Reconstruction Error: Applies principal component analysis (PCA) to reduce dimensionality while retaining 60% of the total variance. Anomaly scores are computed as the reconstruction error for each sample. The 90th percentile of reconstruction errors the training set is used the detection threshold.
The comparative results in terms of the area under the ROC curve (AUC) are shown in Figure 3 Overall, all evaluated methods exhibit a monotonic increase AUC as the magnitude of injected deviation grows, which is consistent with intuition: larger perturbations our proposed approach, based on the sign the fidelity decline (AV), demonstrates significantly higher sensitivity small deviations. Remarkably, even with a perturbation as small 0.1o, V detects deviations in the majority of test samples, achieving an AUC of 0.93. As the deviation magnitude increases, the detection performance improves further, reaching an AUC of 0.95 at 5a.
easier detect. However,
In contrast, baseline methods show little to no response at low perturbation levels. At 0.1or, their AUC scores remain close to 0.5. indicating effective discrimination between perturbed and unperturbed samples. Most basclines only begin to exhibit meaningful detection performance when the deviation exceeds la, and none approach the accuracy of Y until the deviation reaches 5
These results highlight the distinct advantage of our approach detecting subtle yet clinically significant deviations. The fidelity predictor V derived through bi-level knowledge distillation from task-specific data Shapley oracles, provides a powerful signal for identifying deviations EHR data. In practical clinical settings, deviations of several standard deviations are often already identifiable through conventional rule-based validation. However, often overlooked yet potentially impactful- that excels detecting. This capability enables timely intervention and supports more robust data assurance in real-world healthcare applications.
precisely the small-magnitude deviations-
4.3 Validation of i Output Sensitivity to Varying Deviation Magnitudes
Next, we conduct the output sensitivity experiment using the same paired samples described in Section 4.2 while varying the magnitudes of the injected deviations. Specifically, we examine the fidelity decline, denoted as AV, across the perturbed test samples as the deviation magnitude increases from 0.1o to 5a. Figure presents the distribution of A W at each deviation level, reporting

---

200
32
175
Volue (0ther Lab Trxsss)
40-
100
Valure totner Lab Tests)
150
Volue (Creatinincl
30-
150
23
125
20-
103
100
Since AKI
Slyceiincea AX
Creakin no (umott) Uiza (mmelii
Carbon Diox (mume(i) Albuin (95)
Creakinne (umoli))
Albumin (gS,)
Bicidbonate, POCT (nsmlt))
hoomoglazin (g)U)
(mmoet))
Hoosphiti (mmoot)
+ Haumag POCT (g9a))
Figure 5: Case study
Figure Case study 2.
the median, interquartile range (IQR), and full range. As expected, the absolute magnitude of AV increases monotonically with larger deviations, in line with our formulation in Definition [ Importantly, even when A / is close zero absolute terms, its sign remains a reliable signal for identifying deviations. This robustness near the decision boundary is critical, as it ensures V/'s effectiveness detecting subtle data shifts without being confounded by noise. Together with the results in Figure 3] this analysis further confirms our approach's 's strong performance in identifying early-stage deviations that conventional approaches often miss, reinforcing utility in real-world clinical applications.
4.4
Case Study: Detecting Potential Data Deviations in Real-world Scenarios
To further investigate how our proposal could detect potential data deviation issues in real-world scenarios, we zoom in on two example patients from the investigated cohort with low data fidelity values, with their data in the observation window shown.
The patient's creatinine level showed a steady decline from 231 ymol/L to 100 umol/L within short period, indicating either marked improvement in renal function the effect of dialysis to augment renal function [36]. However, the urea level remained elevated mossly above 10 mmol/L-showing a slight inconsistency with the downward trend of creatinine (35] Further, in the context of coexisting acidosis (with carbon dioxide persistently below 19 mmol/L (55)) and moderate to severe anemia (with haemoglobin as low 6.0 g/dL (75).. the serum albumin level paradoxically increased to as high as 48 g/L. This contradicts the fundamental physiological principle that albumin synthesis typically suppressed under inflammatory and nutritionally imbalanced states [6I. In addition, discrepant duplicate haemoglobin records (e.g., 6.6 and 8.8 g/dl. on the same day, highlighted yellow) were observed, which may suggest the effect of blood transfusion. In nutshell, the lack of intrinsic physiological consistency among multiple key laboratory tests reflects a lack of pathophysiologccal coupling between data points, hence suggesting the presence of data deviations in this patient's EHR data. Such records may fail to accurately reflect the patient's 's actual physiological state the time and pose risk of misleading subsequent clinical decisions.
Case 1 (Figure 5):
Case 2 (Figure ;) The patient's creatinine levels remained consistently within the range of 30-70 nearly normal only mildly reduced glomerular filtration rate. However, the patient's urea levels remained persistently elevated, with a peak exceeding 20 mmol/L. Such a combination is uncommon (35 in the absence of high protein intake 42 or gastrointestinal bleed- ing (19, indicating a discordance between the two laboratory test results. Moreover, in the context of hypoalbuminemia (low Albumin (61), phosphate levels generally below 1.0 mmol/L (with mini- mum of 0.6 mmol/L. [58], and development of metabolic acidosis (Bicarbonate, POCT levels (55) repeatedly exhibited sharp drops highlighted in pink), no corresponding metabolic disturbances were observed [4]. Additionally. the patterns of phosphate levels showed no clear relationship with renal function parameters, suggesting the possibility of potential errors or fluctuating nutritional intake. Furthermore, the point-of-care haemoglobin (Haemoglobin, POCT) [57] readings fluctuated markedly within the same day (e.g., from 6.9 to 9.9 g/dl highlighted in yellow). Overall, the biochemical indicators in this patient tend to lack internal physiological consistency, and the dynamic changes in multiple laboratory tests are odds with the expected pathophysiological characteristics of renal discase, suggesting potential deviations in the recorded data.
jmol/L, corresponding

---

5 Related Work
Data valuation provides a principled framework to quantify the contribution of individual data samples to the performance of downstream analytic models [39 65 73). Various strategies have been proposed for this purpose. The leave-one-out approach evaluates sample importance by measuring the change in model performance when sample is excluded from training. Influence functions [43] assess importance based on the model's s sensitivity to infinitesimal upweighting of a sample. More recently, the data Shapley value [26), inspired by the Shapley value from cooperative game theory [69]. has been introduced as an equitable and theoretically grounded data valuation method. Subsequent work
has extended its theoretical foundations [25] 45 82], improved its computational efficiency [38] [5].
and evaluated its practical utility across diverse applications 886 777.
Amortized computation (or optimization) leverages learning- based models, such neural net- works, to capture shared structure across similar problem instances, enabling efficient prediction of solutions and reducing per-instance computational cost. Compared to -onotiott methods, amortized approaches can offer speedups of several orders of magnitude [. This paradigm has been adopted across various domains improve efficiency, including meta learning [63] 21), explainable machine learning [88] 37], and reinforcement learning [47] 32). In the context of feature attribution and data valuation, amortized computation is advocated in [15] to address scenarios where exact labels are unavailable or prohibitively costly to obtain. To this end, a stochastic amortization technique proposed, which trains neural networks using noisy labels while maintaining strong performance with theoretical guarantees.
is commonly employed for model compression and acceleration, aiming to transfer knowledge from a large, cumbersome model ("teacher" model) to a smaller, more efficient model ("student" model). The goals of knowledge distillation
Knowledge distillation [33 62 28
multifaceted [36]. with
two particularly relevant objectives: (i) knowledge compression, where the student model is trained to retain performance comparable the teacher while being significantly more compact [33 66 64];
and (ii) knowledge adaptation, where the student learns to generalize to new or unseen target domains by leveraging knowledge transferred from teacher models trained on related source domains 33 54
6 Conclusion
This paper addresses the underexplored problem of data deviations in EHR data, which undermines data fidelity in real-world healthcare settings. To detect such deviations, we formulate EHR data fidelity as an intrinsic, task-agnostic property of the data. We then propose bi-level knowledge distillation approach that transfers knowledge from a task-specific data Shapley oracle (Oaz) ) to a neural oracle (O..) for each individual task, and subsequently (V) that integrates information across tasks. By monitoring the outputs of V our approach enables effective detection of EHR data deviations and, more specifically, supports pre-hoc identification of potential data quality issues at the point of data entry, allowing clinicians to recognize and address erroneous records before they contaminate downstream clinical workflows. Experimental results on the EHR dataset from National University Hospital in Singapore for post-AKI analysis, and on the public MIMIC-II benchmark confirm the effectiveness of the proposed approach. Additionally, representative case studies from the National University Hospital data demonstrate the proposal's s ability to pinpoint deviation issues, supporting the identification of erroneous records and guiding correction strategies with practical utility for healthcare practice. Inspired by our case studies, when data fidelity is low or further declines, the detected data deviations may be closely related to complex physiological dynamics in clinical settings. This suggests that additional, uncollected EHR data (even in other data modalities) may need to be gathered and analyzed to fully understand the data deviations and hence the patient's 's physiological state. This remains open problem and warrants further in-depth investigation.
unified EHR data fidelity predictor
Acknowledgments and Disclosure of Funding
We would like to thank the anonymous reviewers for their constructive comments. NUS' research is partially supported by The Lee Foundation in the form of its NUS Lee Kong Chian Centennial Professorship grant. The electronic AKI data from National University Hospital were supported by the National Kidney Foundation of Singapore Research Grant (NK/)C2014001/144
10

---

References
[1] Brandon Amos. Tutorial on amortized optimization for learning to optimize over continuous domains, arXiv preprint arXiv:2202. 00665, 2(3):9, 2022.
[2] Tian Bai, Shanshan Zhang, Brian L. Egleston, and Slobodan Vucetic. Interpretable represen- tation learning for healthcare via capturing disease progression through time. In KDD, pages 43-51. ACM, 2018.
[3] Sigall K Bell, Tom Delbanco, Joann G Elmore, Patricia S Fitzgerald, Alan Fossa, Kendall Har- court, Suzanne G Leveille, Thomas H Payne. Rebecca Stametz, Jan Walker, et al. Frequency and types of patient-reported errors in electronic health record ambulatory care notes. JAMA network open, 3(0)<2258877<205867, 2020.
[4] Rinaldo Bellomo, John Kellum, and Claudio Ronco. Acute kidney injury. The Lancet 380(9843):756-766, 2012.
[5] Qingpeng Cai, Kaiping Zheng, H. V. Jagadish, Beng Chin Ooi, and James Wei Luen Yip. Cohortnet: Empowering cohort discovery for interpretable healthcare analytics. Proc. VLDB Endow, 17(10)22477-2500 2024.
[6] Qingpeng Cai, Kaiping Zheng, Beng Chin Ooi, Wei Wang, and Chang Yao. ELDA: learning explicit dual-interactions for healthcare analytics. In ICDE, pages 393-406. IEEE, 2022.
[7] Shaofeng Cai, Kaiping Zheng, Gang Chen, H. V. Jagadish, Beng Chin Ooi, and Meihui Zhang. Arm-net: Adaptive relation modeling network for structured data. In SIGMOD Conference, pages 207-220. ACM, 2021.
[8] Wei Cao, Dong Wang. Jian Li, Hao Zhou, Lei Li, and Yitan Li. BRITS: bidirectional recurrent imputation for time series. In NeurIPS, pages 6776-6786, 2018
(9] Javier Castro, Daniel Gómez, and Juan Tejada. Polynomial calculation of the shapley value based sampling. Comput. Oper. Res., Res., 36():: 2009.
[10] Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):6085, 2018.
[11] Chacha Chen, Junjie Liang, Fenglong Ma, Lucas Glass, Jimeng Sun, and Cao Xiao. UNITE: uncertainty-based health risk prediction leveraging multi-sourced data. In WWW, pages 217-226. ACM/IW3CC 2021.
[12] Edward Choi, Mohammad Taha Bahadori, Le Song. Walter F. Stewart, and Jimeng Sun. GRAM: graph-based attention model for healthcare representation learning. In KDD, pages 787-795. ACM, 2017.
[13] Horng-Ruey Chua, Kaiping Zheng, Anantharaman Vathsala, Kee- Yuan Ngiam, Hui-Kim Yap, Liangjian Lu, Ho-Yes Tiong, Amartya Mukhopadhyay, Graeme MacLaren, Shir-Lynn Lim, et al. Health care analytics with time-invariant and time- variant feature importance to predict acute kidney injury: observational longitudinal study. Journal of Medical Internet Research, 23(12)=330805 2021.
[14] Genna R Cohen, Charles P Friedman, Andrew M Ryan, Caroline R Richardson, and Julia Adler-Milstein. Variation in physicians' electronic health record documentation and potential patient harm from that variation. Journal of general internal medicine, 34:2355-222 2367, 2019.
[15] Ian Covert, Chanwoo Kim, Su-In Lee, James Y. Zou, and Tatsunori B. Hashimoto. Stochastic amortization: A unified approach to accelerate feature and data attribution. In NewrIPS, 2024.
[16] Bridgit O Crews, Julia C Drees, and Dina N Greene. Data-driven quality assurance to prevent erroneous test results. Critical Reviews in Clinical Laboratory Sciences, 57(3):146-16M) 2020.
[17] Xiaotie Deng and Christos Papadimitriou, On the complexity of cooperative solution concepts. Math. Oper. Res., 19(2):257-26,, 1994.

---

[18] Catherine Dinh-Le, Rachel Chuang, Sara Chokshi, and Devin Mann. Wearable health technology and electronic health record integration: scoping review and future directions. JMIR mHealth and uHealth, 7(9)= 2861, 2019.
[19] Amy A Ernst, Mary Lou Haynes, Todd G Nick, and Steven J Weiss. Usefulness of the blood urea nitrogen/creatinine ratio in gastrointestinal bleeding. The American journal of emergency medicine, 17(1):70-72, 1999.
[20] Andre Esteva, Alexandre Robicquet, Bharath Ramsundar, Volodymyr Kuleshov, Mark DePristo, Katherine Chou, Claire Cui, Greg Corrado, Sebastian Thrun, and Jeff Dean. A guide to deep learning in healthcare. Nature medicine, 25(1): 2019.
[21] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. In ICML, volume 70 of Proceedings of Machine Learning Research, pages 1126-112 1135. PMLR, 2017.
[22] Junyi Gao, Cao Xiao, Lucas M. Glass, and Jimeng Sun. COMPOSE: cross-modal pseudo- siamese network for patient trial matching. In KDD, pages 803-812. ACM, 2020.
[23] Junyi Gao, Cao Xiao, Yasha Wang, Wen Tang, Lucas M. Glass, and Jimeng Sun. Stagenet: Stage-aware neural networks for health risk prediction. In WWW, pages 530-540. ACM/ IW3C2, 2020.
[24] Junyi Gao, Yinghao Zhu, Wenqing Wang. Zixiang Wang, Guiying Dong, Wen Tang, Hao Wang, Yasha Wang, Ewen M. Harrison, and Liantao Ma. A comprehensive benchmark for COVID-19 predictive modeling using electronic health records in intensive care. Patterns, 5(4):100951, 2024.
[25] Amirata Ghorbani, Michael P. Kim, and James Zou. A distributional framework for data valuation. In ICML, volume 119 of Proceedings of Machine Learning Research, pages 3535- 3544. PMLR, 2020.
[26] Amirata Ghorbani and James Y. Zou, Data shapley: Equitable valuation of data for machine learning. In ICML, volume 97 of Proceedings of Machine Learning Research, pages 2242-2251. PMLR, 2019.
[27] Geoffrey S Ginsburg, Rosalind W Picard, and Stephen H Friend. Key issues as wearable digital health technologies enter clinical care. New England Journal of Medicine, 390(12):1118 -1277 2024.
[28] Jianping Gou, Baosheng Yu, Stephen J. Maybank, and Dacheng Tao. Knowledge distillation: A survey. Int. J. Comput. Vis., 17996111789-11 1819, 2021.
[29] David A Grimes and Kenneth F Schulz. Cohort studies: marching towards outcomes. The Lancet, 359(9303):341 -345, 2002.
[30] Rick Groenendijk, Sezer Karaoglu, Theo Gevers, and Thomas Mensink. Multi-loss weighting with coefffcient of variations. In WACV, pages 1468-1477. IEEE, 2021.
[31] Hrayr Harutyunyan, Hrant Khachatrian, David C Kale, Greg Ver Steeg, and Aram Galstyan. Multitask learning and benchmarking with clinical time series data. Scientific data, 6(1):96, 2019.
[32] Nicolas Heess, Gregory Wayne. David Silver, Timothy P. Lillierap. Tom Erez, and Yuval Tassa. Learning continuous control policies by stochastic value gradients. In NIPS. pages 2952-2952. 2015.
[33] Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in neural network. CoRR, Abs/1503,.0531, 2015.
[34] Judy Hoffman, Eric Tzeng, Taesung Park, Jun Yan Zhu, Phillip Isola, Kate Saenko, Alexei A. Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In ICML., volume 80 of Proceedings of Machine Learning Research, pages 1994-2003. PMLR, 2018.
12

---

[35] Adrian O Hosten. Bun and creatinine. Clinical Methods: The History. Physical, and Laboratory Examinations. 3rd edition, 1990.
[36] Chengming Hu, Xuan Li, Dan Liu, Haolun Wu, Xi Chen, Ju Wang, and Xue Liu. Teacher-student architecture for knowledge distillation: A survey. CoRR, abs/2308.04200 2023.
[37] Neil Jethani, Mukund Sudarshan, Ian Connick Covert, Su-In Lee, and Rajesh Ranganath. Fastshap: Real-time shapley value estimation. In ICLR. OpenReview.net, 2022.
[38] Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas J. Spanos. Towards efficient data valuation based on the shapley value. In AISTATS, volume 89 of Proceedings of Machine Leurning Research, pages 1167-1176. PMLR, 2019.
[39] Kevin Fu Jiang, Weixin Liang, James Y. Zou, and Yongchan Kwon. Opendataval: unified benchmark for data valuation. In NeurlPS, 2023.
[40] Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1-9, 2016.
[41] Sulaiman Khan, Habib Ullah Khan, and Shah Nazir. Systematic analysis of healthcare big data analytics for efficient care and disease diagnosing. Scientific Reports, 12(1):22377, 2022.
[42] Gang-Jee Ko, Connie M Rhee, Kamyar Kalantar-Zadeh, and Shivam Joshi. The effects of high- protein diets on kidney health and longevity. Journal of the American Society of Nephrology, 3168):1667-167, 2020.
[43] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In ICML, volume 70 of Proceedings of Machine Learning Research, pages 1885-1894, PMLR, 2017.
[44] Jeffrey A Kraut and Nicolaos E Madias. Metabolic acidosis: pathophysiology. diagnosis and management. Nature Reviews Nephrology, 655):274-285, 2010.
[45] Yongchan Kwon and James Zou. Beta shapley: a unified and noise-reduced data valuation framework for machine learning. AISTATS, volume 151 of Proceedings of Machine Learning Research, pages 8780-8802. PMLR, 2022.
[46] Rachel Le, Stacy EF Melanson, Athena K Petrides, Ellen M Goonan, Ida Bixho, Adam B Landman, Anne Marie Brogan, David W Bates, and Milenko J Tanasijevic. Significant reduction preanalytical errors for nonphlebotomy blood draws after implementation of a novel integrated specimen collection module. American Journal of Clinical Pathology, 146(4):456 2016.
[47] Sergey Levine and Vladlen Koltun. Guided policy search. In ICML (3), volume 28 of JMLR Workshop and Conference Proceedings, pages 1-9. JMLR.org. 2013.
[48] Zachary Chase Lipton, David C. Kale, Charles Elkan, and Randall C. Wetzel. Learning to diagnose with LSTM recurrent neural networks. In ICLR (Poster), 2016
[49] Changshuo Liu, Lingze Zeng, Kaiping Zheng, Shaofeng Cai, Beng Chin Ooi, and James Wei Luen Yip. Neuralcohort: Cohort-aware neural representation learning for healthcare analytics. In ICML. OpenReview.nee 2025.
[50] Shikun Liu, Edward Johns, and Andrew J. Davison. End-to-end multi-task learning with attention. In CVPR, pages 1871- Computer Vision Foundation / IEEE, 2019.
[51] Zhaojing Luo. Shaofeng Cai, Can Cui, Beng Chin Ooi, and Yang Yang. Adaptive knowledge driven regularization for deep neural networks. In AAAI, pages 8810-8818. AAAI Press, 2021.
[52] Zhaojing Luo, Shaofeng Cai, Yatong Wang, and Beng Chin Ooi. Regularized pairwise re- lationship based analytics for structured data. Proc. ACM Manag. Data, 1(1):82:1-822 1-2:277, 2023.
13

---

[53] Fenglong Ma, Jing Gao, Qiuling Suo, Quanzeng You, Jing Zhou, and Aidong Zhang. Risk prediction on electronic health records with prior medical knowledge. KDD, pages 1910- 1919. ACM, 2018.
[54] Tambet Matüisnn, Avital Oliver, Taco Cohen, and John Schulman. Teacher-student curriculum learning. IEEE Trans. Neural Networks Learn. Syst, 31(9):3732-3722 3740, 2020.
[55] Mayo Clinic. Carbon Dioxide Test, 2025. Accessed: 2025-05-10. https://www
[56] Mayo Clinic. Creatinine Test, 2025. Accessed: 2025-05-10.
[57] Mayo Clinic. Hemoglobin Test, 2025.
Accessed:
2025-05-10.
/
https://wwe httper//www.mmyyllnnil...... www.clliiclllas.....m.......a.a. htt.a//a....oooessccaaaaaaoo///e o
-linical-and-Internn Interrrrettteeeeeiiiiv
[58]
Mayo Clinic. Phosphate Test, 2025. Accessed: 2025-05-10,
1ttp:o/00.tttss///..aaaalmml.... deest-caataloo/Deeviieeete http:///000001111ll11ae1eo -nw--and--nnneeettiii-e
com com/tes http:///w.....aooooooooo/o...oaa
[59] Clara Meister, Elizabeth Salesky, and Ryan Cotterell. Generalized entropy regularization or: There's nothing special about label smoothing. In ACL, pages 6870-6886. Association for Computational Linguistics, 2020.
[60] Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey E. Hinton. Regularizing neural networks by penalizing confident output distributions. In ICLR (Workshop). OpenReview.net, 2017.
[61] Theodore Peters Jr. All about albumin: biochemistry, genetics, and medical applications. Academic press, 1995.
[62] Mary Phuong and Christoph Lampert. Towards understanding knowledge distillation. In ICML, volume 97 of Proceedings of Machine Learning Research, pages 3142-5151. PMLR, 2019.
[63] Aravind Rajeswaran, Chelsea Finn, Sham M. Kakade, and Sergey Levine. Meta-learning with implicit gradients. NeurlPS, pages 113-124, 2019.
[64] Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. In ICLR (Poster), 2015.
[65] Benedek Rozemberezki, Lauren Watson, Péter Bayer, Hao-Tsung Yang. Oliver Kiss, Sebastian Nilsson, and Rik Sarkar. The shapley value in machine learning. In IJCAI, pages 5572-5579. ijcai.org, 2022.
[66] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. CORR, abs/1910.01 108, 2019.
[67] Tabinda Sarwar, Sattar Seifollahi, Jeffrey Chan, Xiuzhen Zhang, Vural Aksakalli, Irene Lena Hudson, Karin Verspoor, and Lawrence Cavedon. The secondary use of electronic health records for data mining: Data characteristics and challenges. ACM Comput. Surv., (512)333:1333:44 2023.
[68] Emily J See, Nigel D Toussaint, Michael Bailey, David W Johnson, Kevan R Polkinghorne, Raymond Robbins, and Rinaldo Bellomo. Risk factors for major adverse kidney events the first year after acute kidney injury. Clinical Kidney Journal, 14(2):556-563, 2021.
[69] Lloyd S Shapley et al. A value for n-person games. 1953.
[70] Jiyun Shi, Yuqiao Wang, Chi Zhang, Zhaojing Luo, Chengliang Chai, and Meihui Zhang. Dmrnet: Effective network for accurate discharge medication recommendation. In ICDE, pages 3393-3406. IEEE, 2024.
[71] Xi Shi, Charlotte Prins, Gijs Van Pottelbergh, Pavlos Mamouris, Bert Vaes, and Bart De Moor. automated data cleaning method for electronic health records by incorporating clinical knowledge. BMC Medical Informatics Decis. Mak., 21(1):267, 2021.
14

---

[72] Michelle Si and Jian Pei. Counterfactual explanation of the shapley value in data coalitions. Proc. VLDB Endow., 17(11):3332-3355 2024.
[73] Rachael Hwee Ling Sim, Xinyi Xu, and Bryan Kian Hsiang Low. Data valuation in machine learning: "ingredients", strategies, and open challenges. In IJCAI, pages 5607-5614 ijcai.org, 2022.
[74] Ge Su, Kaiping Zheng, Tiancheng Zhao, and Jianwei Yin. CLEAR: addressing representation contamination in multimodal healthcare analytics. In KDD (1), pages 1289-1300. ACM, 2025,
[75] Kevin M Sullivan, Zuguo Mei, Laurence Grummer-Strawn, and Ibrahim Parvanta. Haemoglobin adjustments to define anaemia. Tropical Medicine & International Health, 13(10):1267-1271, 2008
[76] Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In CVPR, pages Computer Society, 2016.
IEEE
[77] Siyi Tang, Amirata Ghorbani, Rikiya Yamashita, Sameer Rehman, Jared A Dunnmon, James Zou, and Daniel L Rubin. Data valuation for medical imaging using shapley value and applica- tion to a large-scale chest x-ray dataset. Scientific reports, 1 11(1): (11):8366, 2021.
[78] Seth Ting, Horng-Ruey Chua, and Matthew Edward Cove. uous kidney replacement therapy with glucose-free solution: report of cases. American Journal of Kidney Diseases, 78(2):305-308, 2021.
Euglycemic ketosis during contin-
[79] Allan Tucker, Zhenchen Wang, Ylenia Rotalinti, and Puja Myles. Generating high-fidelity synthetic patient data for assessing machine learning healthcare software. NPJ digital medicine, 3(1):147, 2020.
[80] Michael M Wagner and William R Hogan. The accuracy of medication data in an outpatient electronic medical record. Journal of the American Medical Informatics Association, 3(3):234- 244, 1996.
[81] Hao Wang, Jiyun Shi, Yuhao Chen, Haochen Xu, Chi Zhang, Zhaojing Luo, and Meihui Zhang. PFCA: efficient path filtering with causal analysis for healthcare risk prediction. In ICDE, pages 2323-2336. IEEE, 2025.
[82] Jiachen T. Wang, Tianji Yang, James Zou, Yongchan Kwon, and Ruoxi Jia. Rethinking data shapley for data selection tasks: Misleads and merits. In ICML. OpenReview.net, 2024.
[83] Shirly Wang, Matthew B. A. McDermott, Geeticka Chauhan, Marzyeh Ghassemi, Michael C. Hughes, and Tristan Naumann. Mimic-extract: data extraction, preprocessing, and representa- tion pipeline for MIMIC-III. In CHIL, pages 222-235. ACM, 2020.
[84] Xiaofeng Wang and Michael W Kattan. Cohort studies: design, analysis, and reporting. Chest, 158(1):872-S78 2020.
[85] Zhan Wang, Serhan Dagtas, John R. Talburt, Ahmad Baghal, and Meredith N. Zozus. Rule- hased data quality assessment and monitoring system in healthcare facilities. In ITCH volume 257 of Studies in Health Technology and Informatics, pages 460-467. IOS Press, 2019.
[86] Xinyi Xu, Zhaoxuan Wu, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Validation free and replication robust volume-based data valuation. In NewrlPS, pages 10837-10848, 2021.
[87] Pranjul Yadav, Michael S. Steinbach, Vipin Kumar, and György J. Simon. Mining electronic health records (ehrs): A survey. ACM Comput. Surv., 5006):85:1-85::2 2018.
[88] Jinsung Yoon, James Jordon, and Mihaela van der Schaar. INVASE: instance-wise variable selection using neural networks. In ICLR (Poster). OpenReview.net, 2019.
[89] Jinsung Yoon, Michel Mizrahi, Nahid Farhady Ghalaty, Thomas Jarvinen, Ashwin S Ravi, Peter Brune, Fanyu Kong, Dave Anderson, George Lee, Arie Meir, et al. Ehr-safe: generating high-fidelity and privacy-preservinn synthetic electronic health records. NPJ digital medicine, 641):141, 2023.
15

---

[90] Angela Zhang. Lei Xing, James Zou, and Joseph C Wu. Shifting machine learning for healthcare from development to deployment and from models to data. Nature biomedical engineering. 6(12):1330-1344 1345, 2022.
[91] Chaohe Zhang, Xin Gao, Liantao Ma. Yasha Wang, Jiangtao Wang, and Wen Tang. GRASP: generic framework for health status representation learning based on incorporating knowledge from similar patients. In AAAI, pages 715-723. AAAI Press, 2021.
[92] Muhan Zhang, Christopher Ryan King, Michael Avidan, and Yixin Chen. Hierarchical attention propagation for healthcare representation learning. In KDD, pages 249-256. ACM, 2020.
[93] Xinlu Zhang, Shiyang Li, Zhiyu Chen, Xifeng Yan, and Linda Ruth Petzold. Improving medical predictions by irregular multimodal electronic health records modeling. In ICML, volume 202 of Proceedings of Machine Learning Research, pages 41300-41313 41313. PMLR, 2023.
[94] Kaiping Zheng, Shaofeng Cai, Horng Ruey Chua, Melanie Herschel, Meihui Zhang, and Beng Chin Ooi. Dyhealth: Making neural networks dynamic for effective healthcare analytics. Proc. VLDB Endow., 15(12)3445-344 2022.
[95] Kaiping Zheng, Shaofeng Cai, Horng Ruey Chua, Wei Wang. Kee Yuan Ngiam, and Beng Chin Ooi. TRACER: A framework for facilitating accurate and interpretable analytics for high stakes applications. In SIGMOD Conference, pages 1747-1763. ACM, 2020.
[96] Kaiping Zheng, Gang Chen, Melanie Herschel, Kee Yuan Ngiam, Beng Chin Ooi, and Jinyang Gao. PACE: learning effective task decomposition for human-in-the-lopy healthcare delivery. In SIGMOD Conference, pages 2156-2168. ACM, 2021.
[97] Kaiping Zheng, Horng Ruey Chua, Melanie Herschel, H. V. Jagadish, Beng Chin Ooi, and James Wei Luen Yip. Exploiting negative samples: A catalyst for cohort discovery in healthcare analytics. In ICML. OpenReview.net, 2024.
[98] Kaiping Zheng, Jinyang Gao, Kee Yuan Ngiam, Beng Chin Ooi, and James Wei Luen Yip. Resolving the bias in electronic medical records. In KDD, pages 2171-2188 ACM, 2017.
16

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We address the underexplored problem of detecting data deviations in EHR data by proposing a bi-level knowledge distillation approach.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
We discuss the potential limitations and directions for further investigation in Section
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
17

---

Answer:
[NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We present the necessary details in Section 4.1 with additional information provided in Appendix E We also include the code in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
18

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: provide the code the supplementary materials. Although the EHR dataset from National University Hospital in Singapore is proprietary, we also evaluate the proposed approach on the publicly accessible MIMIC-III benchmark, with results reported in Appendix F
F.5 and F.6
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
Sublic/guides http://uuiddee/eeeuuubbeeeeeee
CCodeSubmissiooooll11
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
httpa///ggiidee/e/eeuuubbbibsee htt.:/ddee/coeeeSuumaassiioolloo wwtpimisssiiooooiiioooo cilips.cc/publle httpe//uwlccggguddeeeeeeeeeeeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide implementation details, including data splits and hyperparameter settings, in Section 4.1 and Appendix E
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
We report detailed statistics on the performance metrics of our proposed approach.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
19

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide details on the computational resources used in Appendix B
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https://neurrps hwtp://wwb..ctiiicccsiieeeeeeeee
Answer: [Yes]
Justification: We check the NeurIPS Code of Ethics and ensure that this paper complies with its guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss the potential positive societal impacts and negative societal impacts of our proposed approach in Appendix
Guidelines:
The answer NA means that there is no societal impact of the work performed.
20

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper does not involve the release of data or models that pose high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We follow the standard procedure for accessing the publicly available MIMIC- III benchmark dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
21

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswipphaeeeooccaaappp thcode. com/ com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This paper does not involve the release of any new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: We obtain an IRB approval to conduct research using the EHR dataset from National University Hospital in Singapore.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
22

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
This paper does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurd) htto::///eerrrppcccooooferrrre/ /c/////feeeeeees//222////.. for what should or should not be described.
23

---

APPENDIX
Appendix
Notation Table.
Appendix
Extended Related Work.
Appendix
B.
EHR Data Analytics.
Appendix
Data Valuation and Data Shapley Value.
Appendix
B
Amortized Computation.
Appendix
B 4
Knowledge Distillation.
Appendix
Pseudocode for Core Stages of the Methodology.
Appendix
Algorithm 1: Data Shapley Value Computation Per Task in Oae
Appendix
Algorithm 2: Knowledge Distillation from Oas to Om.
Appendix
Algorithm 3: Knowledge Distillation from O. to V.
Appendix
Computational Complexity Analysis.
Appendix
D.1
Computational Complexity Analysis of Algorithm 1.
Appendix
D.2
Computational Complexity Analysis of Algorithm 2.
Appendix
D.3
Computational Complexity Analysis of Algorithm 3.
Appendix
Extended Experimental Set-up.
Appendix E.
Experimental Set-up on the AKI Dataset.
Appendix E. 2
Experimental Set-up on the MIMIC-III Dataset.
Appendix
Supplementary Experimental Results.
Appendix
Hyperparameter Sensitivity Study on the AKI Dataset.
Appendix
F2
Evaluation of Osa 's Approximation the AKI Dataset.
Appendix
E3
Ablation Study on the AKI Dataset.
Appendix
F4
Evaluation of Controlled Deviation Injection on the MIMIC-III Dataset.
Appendix
F.5
Evaluation of Y Output Sensitivity on the MIMIC-III Dataset.
Appendix
E.
Comparison with Rule-based Methods the MIMIC-III Dataset.
Appendix
G
Broader Impact.
24

---

A Notation Table
We adopt the standard convention of using italic symbols (e.g.. 2) for scalars, bold lowercase (e.g.. x) a summary of the notations
for vectors, and bold uppercase (e.g., X) for matrices. Table used throughout the paper. The table is organized into four sections: (i) general notations, (ii) notations for Data Shapley Value Computation Per Task in Oas (Section 3.1), (iii) notations for
Knowledge Distillation from Oas to On (Section 3.2). and (iv) Knowledge Distillation from O to for EHR Data Deviation Detection (Section
Table 1: Summary of notations used.
Notation
Description
Oss
Data Shapley oracle
Onn
Task-specific neural oracle
V
EHR data fidelity predictor
Index of task or specific application Total number of tasks Number of samples in task t EHR data for task
T
No
KCO)
The i-th EHR sample in task t Input features of k:(")
X,
(0)
Task-specific label of kg for task t Data fidelity of x;
((x))
AQ
Decline in data fidelity
(()
Prediction model used in Oss A subset of K(()
S
m
Performance evaluation metric
(1)
Data Shapley value of k(() for task Uniform distribution over all permutations of XC())
7
A permutation of ((())
9p
Samples preceding k.()) in yp
g(9) (x,
Neural oracle O.n for task t
Knowledge distillation loss from Oas to Om for task t
dm wolt)
Weight assigned to c
h(t) (xx)
Learned representation of x, in task
(xx 0)
Final EHR data fidelity predictor as a neural network
oxx)
Hidden representation of learned by 4 0)
Exd
Knowledge distillation loss from O. to
(0 (x)
Attention weight for task
Lent
Relative entropy constraint from Omn to Vp
Esim
Similarity constraint from Onn to
Pr (x:)
Output similarity between neural oracles for tasks and t' Temperature parameter in similarity computation
T
L
Overall loss integrating Lxa- Lenss and Lsim
Aed
Weight of Lua in
Aunt
Weight of Len in L
Asim
Weight of Lam in C
25

---

B Extended Related Work
B.1 EHR Data Analytics
In EHR data analyties, heterogeneous EHR data are extensively utilized to support a broad spectrum of applications, including risk prediction, medication recommendation, clinical trial matching, and disease progression modeling [20] 22 90 24]. To fully exploit the potential these valuable
data sources, prior studies have investigated multiple methodological directions, such as enhanced
representation learning to improve downstream analytic performance [48
98
g
94 93 74),
cohort modeling to uncover shared patterns across patient subgroups
29 23
91
5 97 49)
improved explainability and reliability to strengthen clinicians' trust in model outputs 12 99 11
1 96
6
52 and knowledge-gund guideed decision support to integrate domain expertise into predictive
modeling 53
92
51 70 81.
Collectively, these methodological advances enhance patient management and optimize healthcare resource allocation [67 (7], yielding tangible benefits for patients, clinicians, and healthcare institu- tions. Nevertheless, such progress fundamentally depends on often implicit assumption that the underlying EHR data are of high fidelity. This assumption, though critical, is frequently overlooked in real-world clinical environments, posing major challenge for reliable EHR-base analytics.
B.2 Data Valuation and Data Shapley Value
Data valuation provides principled framework measure the contribution of individual data samples to the performance of downstream analytic models (39]633] 73). Several strategies have been developed for this purpose. The leave-one-ous approach measures sample importance by evaluating the change in model performance upon excluding each sample. Influence functions importance by analyzing the model's sensitivity to infinitesimal upweighting of a sample.
[43] estimate
The data Shapley value [26), inspired by the Shapley value in cooperative game theory [69]. has emerged as theoretically grounded and equitable method for data valuation. Building on this foundation, subsequent works have sought to enhance its theoretical and practical properties. For instance, the distributional Shapley framework (25) generalizes the original formulation by defining the value of each data point over an underlying data distribution. Beta Shapley [45] introduces a relaxation of the efficiency axiom of the Shapley value, which is not essential in machine learning contexts, to achieve desirable statistical properties for efficiency. More recently, a hypothesis testing framework [82) has been presented to examine the data Shapley value under different utility function constraints, motivating a class of utility functions that ensure optimal data selection in such scenarios.
Other researchers seek to reduce the computational overhead associated with the data Shapley value. For example, (38] proposes a suite of techniques to accelerate its computation by introducing specific assumptions on the utility function, enabling practical estimation algorithms for machine learning tasks. In contrast, a unified framework called stochastic amortization [15] is introduced to speed up both feature attribution and data valuation by leveraging amortized computation, which will be discussed in detail in Appendix B.3
In addition, the practical utility of the data Shapley value has been evaluated in diverse worldwordd ap- plications. For instance, [86) investigates scenarios where a validation set is unavailable and proposes using the diversity of data samples as an intrinsic property of the dataset, therefore independent of validation. In another study [77), the data Shapley value is employed to quantify the contribution of each training sample to the model's performance in pneumonia detection, using large chest X-ray medical imaging dataset.
B.3 Amortized Computation
Amortized computation (or optimization) I uses learning- models, such as neural networks, to exploit shared structure across similar problem instances, thereby enabling efficient solution prediction and significantly reducing per-instance computational cost. Compared to non-amortized methods, amortized approaches can achieve several orders of magnitude in speedup [1] and have been widely adopted to improve efficiency across various domains.
In meta learning, the Model-Agnostic Meta-Learning (MAML) algorithm (2 is designed to be compatible with any model trained via gradient descent, enabling the learning of model parameters that can be casily adaptable. A variant that incorporates implicit differentiation 63) further separates
26

---

the computation of the meta- aata-gradient from the specific choice of the inner-loop optimizer, allowing the proposal tackle greater number of gradient steps without suffering from vanishing gradients or excessive memory usage.
In explainable machine learning, INVASE [88] performs instance-wise feature selection using selector-predice preddictorbbaeeline architecture trained jointly to identify informative subsets of features, where the baseline network is devised train the selector network. FastSHAP [37] further amortizes the estimation of Shapley values by training an explainer model that approximates them in a single forward pass, using a stochastic gradient descent objective based on weighted least squares.
In reinforcement learning, guided policy search 47 integrates trajectory optimization to direct policy learning, mitigating the risk of poor local optima encountered in direct policy search. Similarly, Stochastic Value Gradients continuous control policies more effectively.
32 provide a unified framework that leverages backpropagation to learn
Finally, in the context of feature attribution and data valuation, amortized computation is employed in [15 to address settings where exact labels unavailable expensive to obtain. A stochastic amortization technique is therefore proposed to train neural networks using noisy labels, while still maintaining strong performance backed by theoretical guarantees.
B.4 Knowledge Distillation
Knowledge distillation [33 62 28 is widely adopted technique for model compression and accelcration. It facilitates the transfer of knowledge from large, cumbersome model ("teacher" model) to a smaller, more efficient model ("student" model). The objectives of knowledge distillation multifaceted (36), with two particularly relevant objectives: (i) knowledge compression and (ii) knowledge adaptation, as described below.
In knowledge compression, the goal is to preserve the predictive performance of the teacher model in significantly more compact student model. 3 for example, the authors distill the knowledge of an ensemble of models into single model, achieving competitive performance. In natural language processing, DistilBERT [66] compresses the BERT model via distillation, resulting in improved inference efficiency while preserving core language understanding capabilities. FitNets [64] extend the basic distillation paradigm by training thin and deep student networks using both output predictions and intermediate representations from wide, shallower teacher networks additional supervision.
In knowledge adaptation, the student model is trained to generalize to new or unseen target domains by leveraging knowledge from teacher models trained on related source domains. For instance, Cycle-Consistee Adversarial Domain Adaptation (CyCADA) 34 enhances domain adaptation by facilitating the alignment in both the generative image space and that in the latent representation space while preserving task-relevant semantics. Another example the Teacher-Student Curriculum Learning framework (54), where the teacher automatically selects subtasks for the student, enabling progressive learning through a curriculum- based approach.
C Pseudocode for Core Stages of the Methodology
This section outlines the pseudocode for the three core stages of the proposed bi-level knowledge dis- tillation approach for detecting data deviations in EHR data. The stages task-specific data Shapley values using the data Shapley oracle Oaa (ii) distilling knowledge from Oan the corresponding task-specific neural oracle O.. and (iii) distilling knowledge from O. to the unified EHR data fidelity predictor V.
follows: (i) computing
C.1 Algorithm 1: Data Shapley Value Computation Per Task in Oas
Algorithm D
conceptually describes the computation of data Shapley values, which serve as the ground truth supervision for the first level of knowledge distillation.
C.2 Algorithm 2: Knowledge Distillation from Oas to Omn
Algorithm 2 details the training procedure for task-specific neural networks g (x, 0("),, which serve neural oracles to approximate the data Shapley values produced by Oas-
27

---

(600 Algorithm Data Shapley Value Computation Per Task in Oas
Input:
K(0) {(
()) Ji= N Dataset for task & {O..... 1}
f(t); Prediction model for task t
m(-, f()): Performance evaluation metric for task t
Output:
( -1, Data Shapley values for each task / {n fi=0
1: for each task L {0, ...,,
do
2:
for each sample k.") (x.,Mt E KC()) do
shi S. be the data samples in IC(1) preceding ke in a permutation yp
Let
4: T
(t) Bipr m S4 (Kk ,()
m Sp
(()
f(())
5: end for
6: end for
(11 (1 -11 1i=0 =1=0
7: return
Algorithm Knowledge Distillation from Oas to Onn
Input:
Duain Training dataloader yielding batches (Xtatch { Umatch )
D Validation dataloader
( ) Set of task lask-specific neural oracle models
optg: Optimizer for parameters {e)²))
CMSE: Mean squared error loss function
Ej: Max epochs. Pi: Early stopping patience. ej Stability constant for weighting
Output:
Trained models (g
1: Initialize (0 (0($)177 T-11
Jt=
2: Initialize (C
1.0]
1==
T-1
(for dynamic per-task loss weighting)
3: for epoch E - 1 to E; do
4: for all models g( do
5: g'",.rain|)
6: end for
7: for each batch (Xeatch, Moonchh (t TT-1
in Derain do
8: opty. (etggzero_gra(()) grad()
9:
Liotal w f-
10:
for all tasks E (0....,T-1)) do
Touch t g 0641)
(1)
11:
12:
batch +- LMSE (nsuuch
(t)
(Phunch Thanch))
13:
Compute dynamic task weight (t);
wlll) (e 1)?1.0 hatch// (Ctt), + E1))
15:
LiaaL + LiotaL + (1) batch
end for
17:
Licaa w. a..bacwward)) step))
19:
end for
20: Update (C(2))]T 17-1 with current epoch's 's computed average task losses
21: Perform validation Deai if improvement, save (0(() ); check early stopping (P:)
22: end for
23: Load best saved (0(4) T-1 Jt=0
24: return (,, (1) 1
28

---

C.3
Algorithm 3: Knowledge Distillation from On to y
describes the training of the final predictor v by aggregating knowledge from the task-specific neural oracles gl) (x, ) trained in Algorithm
Algorithm B
Algorithm 3 Knowledge Distillation from Om to y
Input:
Daiin Training dataloader yielding batches Xbatch
Dvat Validation dataloader
(glt) (1,0g ( (=0 TT-1 Trained task-specific neural oracles from Algorithm (frozen)
(1, ):) Unified EHR data fidelity predictor model
A(,, (.)): Attention subnetwork
opte: Optimizer for 0y, fAA.
Temperature for Lsim- T: Number of tasks
Eg Max epochs. P2: Early stopping patience. 62: Stability constant for weighting
Output:
Trained (..00)) and 0()
1: Initialize 8g, A
2: Initialize Lpeekdd Eprey,ent, Eprevisim <- 1.0 (for dynamic loss term weighting)
3: for epoch e to E2 do
4: V V.train(); A.train()
5: for each batch Xbarch in D train do
6:
opty zero ( Perform model forward propagation:
7:
(Thatch < (0 (Xhatch, )TO
8:
(h (ggt).get_hhheee (ee.geeetddeexxxxee (bacch)
9:
10:
baach < V/ (Xtauchh pv Obatch + V @ot_hiddennXXe
11:
abatch A(Obaach | 4x)
(2)
12:
Compute loss terms:
13:
Lad LMSE V batch, Ee batch detach detach(Uhaaahh (ianae))
(e)
Lent f- mean (DrI. (Chatch Uniform( l/T)))
15:
Lsim meansa ss batch Coster
(1)) a exp( exp(-MMS(())" mn )/T))
16:
Compute dynamic weights for loss terms:
17:
Akd t (e = 1)?110 (Cea/ (E + 62)): similar for Aent Asim
18:
Liotal Aka + AeatCeun + Asim
19:
Liotal Bank-bbckkardf/ otty-)tep()
20:
21:
end for
22:
Update Eprevied Epreveent Eprevsim with current epoch's computed averages
23:
Perform validation on Dvati if improvement, save Ov, 0,4: check early stopping (P2)
24:
end for
25:
Load best saved Og, 0
return (e 0.)
D Computational Complexity Analysis
The computational complexity of the three proposed algorithms is analyzed below. The following notations are adopted:
T: Total number of tasks.
(t); Number of data samples for task t.
M: Number of Monte Carlo permutations for data Shapley value approximation.
Ni and Ng: Total number of training samples in Algorithm[aan aand Algorithm
3 respectively.
E1, Eg: Number of training epochs for Algorithm 2 and Algorithm 3 respectively.

---

B1. Bg: Batch sizes for Algorithm and Algorithm respectively.
Computational cost single inference (prediction) using the task-specific model (Algorithm
inl
(
1.
mc Performance evaluation metric. Evaluating m subset S using a fixed model
f()) incurs cost of ((S) C
For a generic neural network model Net with parameters Net:
Crua batch sorch_size): Cost of forward pass.
-eela batch_size), Cost of a backward pass (gradient computation), typically approximated as Cred R B Clwa for some constant / 2 1.
-
Copim (Net): Cost of updating parameters via optimizer, typically O((xxee))-
stop (N batch size): Total cost of single training step, computed as:
Crein_step (Net, batch_sizeg Crwa batch_ + (Net, batch batch_size) Coptim (Net)
g)); Task-specific neural oracle.
V: Unified EHR data fidelity predictor.
A: Attention subnetwork.
D.1
Computational Complexity Analysis Algorithm
The exact computation of data Shapley values is known to be #P-hard 72 17
To address this, the proposed algorithm adopts a Monte Carlo approximation. For each task t and each of its ()) samples ( the data Shapley value is estimated by averaging the marginal contributions across M random permutations of (()))
For given permutation sp, the marginal contribution of k.9)) requires two evaluations of the perfor- mance metric m: namely, m(S U U f()) and m(S ! f()), where s denotes the set of data points preceding (:(4) in the permutation, with an expected size of O(N(O). Hence, computing one marginal contribution has time complexity of ((N(). cl
Aggregating across all M permutations and (1) samples for each task t yields total complexity of:
Σ M. (NWW))) (nt) ) =
(00mm) M (
Assuming uniform dataset size and inference cost across tasks, i... (+) Navg and ciO Ont this simplifies to:
O(T - - N2 M- Cin
Moreover, under the high-level asymptotic assumption that the per-sample inference cost constant, i.e. Cinf_nng _nn__wg O(1) + Ollog Nmvg), where the first term arises from the fixed model architecture and input dimensionality. and the second term accounts for AUC computation, the overall computational complexity can be expressed follows:
O(T M N2 2 loog Navg)
D.2 Computational Complexity Analysis Algorithm
In this stage, task-specific neural oracle models (g 2-11 are jointly trained for E₁ epochs. Each epoch processes [, batches, where N1 is the total number of training samples and B, is the batch size. During each batch, a joint training step performed using a shared loss function Lioal ws involving all I' models.
30

---

Let the per- batch training cost for model g(() be denoted by Cuain stop (, B1). The total training cost across all tasks given by:
N
T-1
O
E
Σ
Cuain Serin_ste((9 (0 .B))
B1
mm0
Assuming that all task-specific models g() have similar computational complexity, define representative per-batch training cost as Cg train (B1),, yielding:
NI B
O(Ei
T Cg "raiirinnsai step (1))
Further assuming that the per-batch training cost scales linearly with the batch size (i.e., Cg snaii_sseepp (Be = OB1), with a constant factor determined by fixed model architecture), the overall time complexity simplifies to:
O .T)
D.3
Computational Complexity Analysis Algorithm
In this algorithm, the unified predictor and attention subnetwork A are trained for Eg epochs using [ Ba] batches. The computational cost per batch consists of the following components:
1. Inference from frozen models: Forward passes through the T frozen task-specific models (t)) to obtain hidden representations h(0 with total cost D (g) B2).
2. V forward pass: Computes the prediction and hidden representation o, with cost Ciwa( V B2).
3. A forward pass: Computes attention scores with cost Crwa(A, Bz).
4. Loss computation:
Knowledge distillation loss Lia: O(B2T).
Relative entropy constraint Lem: O(BTT).
Similarity constraint Leim: O(B2T²) (due pairwise comparisons).
The dominant cost among these is: Closs_cale )
5. Backward pass for V A: Oowa (V, B2) + Chwal (A, B2).
6. Optimizer update for 0g, 0A: Copum ( VV) + Coptim
Aggregating these components, the total time is:
N₂ B2
0
F2
(t)) B2) + Cirain (V, B2) + Cirin shiin_steppe (A, B2) + Class cale
Assuming average forward pass cost Cw_dwdaavg (B2) for each of the T frozen (t) models, and recalling that Closs,cale- O(BI?², the total time complexity simplifies to:
N2 B2
C2 Fudd ovg (B Cirain step (V, B2) + Ctrnin_step( (A, + BBTz)
O
E2
In the high-level asymptotic case where all forward and training step costs scale linearly with batch size (i.e., Cg_twd_ wog (Bz) = (B2),, Curain (V, B2) O(B2), and Cirin step (A, B2) O(B2))
the expression becomes:
31

---

Table Prevalence per task the original AKI dataset.
Task
# Positive
%
New Progressive CKD
941
32.58
Stage 5 CKD Onset
295
10.21
Post-AKI RRT Dependence
118
4.09
Mortality
670
23.20
o
0
N2 E2
(T Bg + B2 + B2 + B2T3) = -0(E (2(++13))
B2
Therefore, the most simplified dominant time complexity is:
O N -
E Extended Experimental Set-up
This section provides additional details of the experimental set-up for both the AKI (acute kidney injury) dataset from National University Hospital in Singapore and the publicly available MIMIC-III benchmark dataset [40]
For both datasets, we partition the extracted samples into 85% for model development and 15% as a held-out set for computing data Shapley values. Within the development set, we further divide the data into 80% for training, 10% for validation, and 10% for testing. Model training is performed using the Adam optimizer. Hyperparameters are selected based on the best validation performance, measured by the minimum loss L (Fquation 12]. averaged over three independent runs. The final model is then evaluated on the test set using the selected hyperparameter configuration. Additional setup details specific to each dataset are provided in Appendices E.I and E.2
The experiments are conducted on a server equipped with two Intel Xeon Gold 6248R CPUs, 768 GB of memory, and eight NVIDIA V100 GPUs connected via NVLINK. All models implemented using Py PyTcrc version 1.12.1.
To evaluate the effectiveness of our proposed bi-level knowledge distillation approach, compute the expected EHR data fidelity decline, A&, as defined Equation and use its sign to indicate the presence or absence of detected deviation.
E.1 Experimental Set-up on the AKI Dataset
The original cohort from National University Hospital in Singapore comprises 2,888 patients diag- nosed with AKI (acute kidney injury) 4 between November 2015 and October 2016, with follow-up data collected over five-year period to monitor their post-AKI outcomes. We focus on four major adverse kidney events (MAKE) [68]. which serve as four prediction tasks in setting and reflect long-term deterioration in kidney function. These events include: (i) the development of new or progressive CKD (chronic kidney disease), defined (ii) the onset of Stage 5 CKD, indicated by eGFR falling below 15ml./ming mouin/1..33mm 1.mm2 (iii) dependence on RRT (renal replacement therapy); and (iv) mortality. The prevalence of each task (i.e., proportion of positive samples) in the original AKI dataset is summarized in Table
decline of more than 30% in baseline eGFR;
We define 90-day observation window following the initial AKI diagnosis and use the patients' laboratory test results within this period as model input. Applying this criterion results in the exclusion of 651 patients (without laboratory tests in the observation window), yielding final cohort of 2.237 patients. The objective is to predict the occurrence of the four target events within a subsequent prediction window. The temporal relationship between the observation and prediction windows is illustrated Figure 7 Specifically, we extract 43 distinct types of laboratory tests recorded during the observation window, comprising total of 130,755 test entries.
Regarding the hyperparameter settings, the task-specific neural oracle Omn implemented as multilayer perceptron (MLP) with three hidden layers of sizes 32, 16, and 8, respectively. The unified
32

---

AKI Diagnosis
90 days
years- 90 days
Time
Observation
Prediction
Window
Window
Figure 7: Relationship between observation window and prediction window in the AKI dataset.
Table Prevalence per task in the MIMIC-III dataset.
Task
# Positive
%
Essential hypertension
17573
41.94
Coronary atherosclerosis and other heart disease
13540
32.31
Cardiac dysrhythmiaa
13458
32.12
Disorders of lipid metabolism
12162
29.02
Fluid and electrolyte disorders
11254
26.86
Congestive heart failure; nonhypertensive
11220
26.78
Acute and unspecified renal failure
8964
21.39
Complications of surgical procedures medical care
8695
20.75
Diabetes mellitus without complication
8074
19.27
Respiratory failure; insufficiency; arrest (adult)
7566
18.06
Septicemia (except in labor)
5975
14.26
Pneumonia (except that caused by tuberculosis sexually transmitted discase) Chronic kidney discase
5815
13.88
5607
13.38
Hypertension with complications and secondary hypertension
5547
13.24
Chronic obstructive pulmonary disease and bronchiectasis Acute myocardial infarction
5455
13.02
4337
10.35
Diabetes mellitus with complications
3988
9.52
Other liver diseases
3723
8.89
Pleurisy; pneumothorax; pulmonary collapse
3658
8.73
Shock
3291
7.85
Acute cerebrovascular disease
3079
7.35
Gastrointestinal hemorrhage
3067
7.32
Conduction disorders
3011
7.19
Other lower respiratory disease
2168
5.17
Other upper respiratory disease
1702
4.06
EHR data fidelity predictor also an MLP, with hidden layers of sizes 64, 32, and The representation dimension (x)()x) in the attention subnetwork (Equation is learning rate of 0.01 for training O.n and 0,0001 for V. The temperature parameter 7 in Equation IT] set to 0.5. Training conducted for a maximum of 1000 epochs with batch size of 128. Early stopping is employed if the validation performance does not improve for 50 consecutive epochs.
32. We use
E.2 Experimental Set-up on the MIMIC-III Dataset
(Medical Information Mart Intensive Care) widely used benchmark dataset in EHR data analytics. It comprises EHR data from over forty thousand patients admitted to intensive care units (ICUs) between 2001 and 2012. this study, we adopt the multi-task learning benchmark established in (31] focusing the phenotype classification application. This application involves predicting the presence of 25 distinct acute care conditions (i.c., phenotypes) during a given ICU stay, formulated multilabel classification problem.
MIMIC-III 40
In this dataset, a single patient may have multiple hospital admissions, and each admission can include multiple ICU stays. Following the protocol in [11, we treat each ICU stay as an individual sample, resulting
total of 41,902 samples. The goal of phenotype classification to predict the presence of specific acute care conditions (phenotypes) for each ICU stay. Detailed descriptions of the phenotypes and their corresponding prevalences (i.e., the proportion of positive samples) are provided in Table
33

---

Table 4: Tuning range for hyperparameters on the AKI dataset.
Hyperparameters
Tuning Range
Dimensions of g (x. (
[64, 32, 16), [32, 32, 16], [32, 16, 8]
Dimensions of V (x(x 0)
[128, 64, 32], [64, 32, 16), [32, 16, 8]
Dimension of y(() (x)
{16, 32)
Learning rate of (()) (x, 0
{0.0001, 0.001, 0.01)
Learning rate of / (x, 0)
[0.0001, 0.001, 0.01) {0.5, 1.0, 2.0)
Temperature
Varidation Losses Across hyperparameee Configurations
32-16-6 120-64-00
-hinarr Conrr 2 MIX 9),
82. 10 02 16
79-184 MA12116 32
50-50-15 16, 1280452
19-30-11, 120-00-12
-0.45
52-92-18, 32 194, 15
FOM
D415
16,
32-00-16, PA12316
32-92 16, 92-6, 32
27864-32,
64-32-16, 120-4-32,
34.32-10 10, 32 164, 18
64-32-16,
-.35
64 32 16, 64 3216, 16
C00
64-32-15, 64-22-66
car
0.0001, 0.000.0.000 0.5
0.000 0.0001,
0.0001 0.001
LI0003 (L001,
0.0011, 001,
0.001, 200000110 0.5
0.001, 0.00111
5.0011
o001 0.000,100 0.5
0.001 0.001,
0.001 0.001,
0.01, 000,0 0.5
0.001 005,
0001, 001,
0.11 5.0001
0.1 o.0m11
0.0 0.% 0L001,1
0 0.
6.01.
0.1, 001.03 0.
001,001,1
0.1 700.000.
000011 0.001
0.0011
50001
Iypersorameters learning (" e("),
of Vix. 8).
Figure 8: Hyperparameter sensitivity study results of our proposed bi-level knowledge distillation approach the AKI dataset.
For each ICU stay, extract 17 physiological variables as input features. Each variable is aggregated across seven predefined time span ranges: the first and last 10%, 25%, and 50% of the stay duration, as well as the entire time span. Within each time range, we compute six statistical measures per variable: minimum, maximum, mean, standard deviation, skewness, and the number of recorded measurements. This preprocessing yields a total of 714 features per sample for subsequent analysis.
The task-specific neural oracle Om implemented as an MLP with three hidden layers of dimen- sions 512, 256, and 128, respectively. The unified EHR data fidelity predictor V/ shares the same architecture, also comprising hidden layers of sizes 512, 256, and 128. The representation dimension of rf (x) in the attention subnetwork (Equation 7 set to 128. We use a learning rate of 0.01 for training O and 0.001 for V. The temperature parameter 7 in Equation[i]]a fixed at 2.0. Models are trained for a maximum of 1000 epochs with a batch size 512. Early stopping is employed based on validation performance, with a patience of 50 epochs.
F
Supplementary Experimental Results
F.1
Hyperparameter Sensitivity Study on the AKI Dataset
We perform a comprehensive hyperparameter sensitivity analysis on the AKI dataset collected from National University Hospital in Singapore. The tuned hyperparameters and their corresponding search ranges are summarized in Table 4 The number of training epochs is fixed at 1000, with a batch size of 128. Early stopping is applied if the validation performance does not improve for 50 consecutive epochs. The sensitivity results are presented in Figure
g As illustrated, the optimal
validation performance is achieved when the architecture of g(() (x, 0(0) is configured with layer
34

---

Table 5: Statistical precision data Shapley value estimates across different Monte Carlo sample sizes for four prediction tasks on the AKI dataset. Standard Error (SE) measures estimation uncertainty, while Rolling Variance (RV) indicates convergence stability.
(a) Mortality and RRT dependence prediction tasks.
Sample Size
Mortality Prediction
RRT Dependence Prediction
SE (x 10
RV (x10-
SE (x10-5) 5)
RV (x10-9)
1,000
7.88
3.63
13.57
5,88
2,000
5.56
1.75
10.60
2.85
5,000
3.51
0.69
7.57
1.12
10,000
2.48
0.34
5.79
0,56
20,000
1.76
0.17
4.23
0.28
50,000
1.12
0.07
2.79
0.11
100,000
0.79
0.03
2.01
0.06
(b) New or progressive CKD and Stage 5 CKD onset prediction tasks.
Sample Size
New or Progressive CKD) Prediction
Stage CKD Onset Prediction
SE (x10~9)
RV (x10-9)
SE (x 10-5)
RV (x10-
1,000
16.19
1.89
24.48
6.42
2,000
11.80
0.91
17.78
3.09
5,000
7.71
0.36
11.83
1.21
10,000
5.51
0.18
8.49
0.60
20,000
3.93
0.09
6.04
0.30
50,000
2.51
0.04
3.83
0.12
100,000
1.78
0.02
2.72
0.06
dimensions [32, 16, 8]. and (xx, 0) with [64, 32, 16]. The representation dimension of r(()(x) is set
to 32. The learning rates for (*) (x, o(()) and (x, 0) are set to 0.01 and 0.0001, respectively. The temperature parameter is fixed at 0.5. This hyperparameter configuration is subsequently applied to the test dataset for reporting the final evaluation results.
F.2 Evaluation of Oas 's Approximation the AKI Dataset
Although computing exact data Shapley values would, in principle, yield higher accuracy than permutation-basee approximations, the exact computation known to be #P-hard [72] [7], rendering it infeasible for large-scale, real-world datasets such as those used in experiments. In practice, permutation sampling remains a widely adopted and theoretically grounded approximation strategy
for data valuation [26]]] (65)
To assess the influence of this approximation in Oas. we conduct 100,000 independent Monte Carlo simulations to estimate data Shapley values and evaluate convergence using two complementary statistical metrics across four Aolt-AKI progression tasks. The results are summarized in Table [ Standard Error (SE a vn) is used to construct confidence intervals for the estimated values. following T + 1.96 SE for the 95% confidence level. The empirical decay of S at the rate of O(n -1/2) is consistent with the Central Limit Theorem, and the final SE falls below one percent across all tasks. Rolling Variance (RV) measures temporal stability by computing the variance of cumulative means over sliding windows of 100 samples, serving as a practical indicator of when additional samples yield negligible improvements. The observed monotonic decline in RV demonstrates the statistical stabilization of the data Shapley value estimates.
Collectively, the aforementioned analysis provides both theoretical grounding and empirical evidence supporting the reliability of the permutation-bant based data Shapley value approximation employed by Oss in later stages of this study.
35

---

Table 6: Comparison between our approach and variant teacher fusion strategies.
Fusion Strategy
Loss
Strategy Description
Our approach
0.008135
Attention-based aggregation as defined in Equation 6
Random weights
0.010720
Fixed random weights across teachers
Simple average
0.028935
Equal-weight aggregation across teachers
2 teachers
0.026531
Aggregation of the two best-perforning teachers
Top-1 teacher 0.144956
Use of the single best-performing teacher only
Table 7: Comparison between our approach and variant objective loss functions.
Configuration
Loss
Modification Details
Our approach
0.008135
Full objective loss as defined in Equation 12
No entropy constraint
0.024595
feni term removed from objective
No similarity constraint
0.029499
faim term removed from objective
Static weighting
0.104307
Fixed Ard. Aent Asim in Equation 12
High temperature
0.017665
Increased temperature for softer distributions (r 2.0)
Low temperature
0.109919
Decreased temperature for sharper distributions (T 0.1)
E3
Ablation Study on the AKI Dataset
We first clarify that the knowledge distillation process from Oas to Onn responslie for deriving application-specifii data valuation for downstream inalysis-is an integral component of our approach and cannot be removed. We then compare our full approach against several weakened variants of the subsequent distillation step from Om to V. As summarized in Table 6. our approach, which integrates an attention-basee aggregation strategy, consistently achieves the lowest loss. These results validate the effectiveness of the proposed teacher fusion mechanism and highlight the necessity of employing principled aggregation strategy during knowledge distillation.
We further perform detailed ablation analysis to examine the contribution of each component in the overall objective function (Equation T2). In particular, we evaluate the effects of removing individual loss terms, disabling the dynamic weighting mechanism, and varying the temperature hyperparameter. As reported in Table 7 our full approach achieves the best performance, while any component's s removal or alteration results noticeable degradation. These findings collectively demonstrate that the interplay among loss terms, dynamic weighting, and appropriate temperature calibration is critical to achieving optimal knowledge distillation performance.
F4 Evaluation of Controlled Deviation Injection on the MIMIC-III Dataset
In this section, we evaluate the effectiveness of the proposed Y for detecting deviations in EHR data using the MIMIC-III dataset. We adopt controlled deviation injection experiment similar to that conducted on the AKI dataset (Section 4.2). introducing deviations with magnitudes ranging from 0.10 to 5σ. The comparative results between V and baseline methods are presented in Figure 9
Consistent with the observations on the AKI dataset (Figure 3) both the baselines and V exhibit improved AUC performance as the deviation magnitude increases, which aligns with the expectation that larger perturbations are more easily detectable. However, v demonstrates superior sensitivity to small deviations. Specifically, it achieves an AUC of 0.91 when the deviation is small as 0. and this performance further improves to 0.94 at 5a. In contrast, most baseline methods fail to respond effectively at lower deviation levels and do not reliably distinguish between perturbed and unperturbed samples.
Among the baselines, One-Class SVM, Local Outlier Factor, and M-Means Distance show negligible response until the deviation exceeds 3o, and their performance remains suboptimal even at 5a. PCA Reconstruction Error outperforms these methods but only achieves an AUC of 0.8 at 5o. Gaussian Mixture Model is the strongest baseline on MIMIC-III, reaching a competitive AUC under 5a, yet remains insensitive to deviations smaller than 1o, highlighting its limited robustness relative to V.
36

---

1.0
10
Mn-Max farge
127
Range nOM
09
Methods
de atter Devittenn Injectionn
Modian
V/orr propceat)
7.x100
1.0x10
Local outker Padeer
Gaueban Weun Mode!
0
2.5x1-
5.:10
PCA Reconemuttion Tmme
2.5x10
0.5
-10
Deviation Magritude (times
Deviation Nagnitude (times of
Figure 9: Performance comparison between V Figure 10: Impact of deviation magnitudes on V and baselines for EHR data deviation detection on the MIMIC-III dataset.
(data fidelity decline) after deviation injection the MIMIC-II dataset.
Notably, in contrast to the AKI results, the baseline methods on MIMIC-III consistently fall short of Tt's performance, even higher deviation magnitudes. This suggests that the MIMIC-III dataset, comprising 25 tasks (phenotypes to classify), poses a more complex deviation detection challenge. Nevertheless, V/ maintains high performance across the entire range of deviation magnitudes, demon- strating the effectiveness of bi-level knowledge distillation from task- specific data Shapley oracles in enhancing EHR deviation detection.
F.5 Evaluation of vt' Output Sensitivity on the MIMIC-III Dataset
We further assess the output sensitivity of V on the MIMIC-II dataset by measuring the fidelity decline, denoted A under varying magnitudes of injected deviations from 0.1o to 5c. The results are summarized in Figure TO, which reports the median, interquartile range (IQR), and Hinimum-maximun range of AV.
Consistent with the observations on the AKI dataset (Figure A increases monotonically as the deviation magnitude grows. Notably, even when V is close to zero, V/ remains effective at detecting subtle deviations by leveraging the sign of A as a reliable indicator. This corroborated by the high AUC observed in Figure at small deviation levels, reinforcing the ability of V/ to identify early-stage deviations that are typically overlooked by baseline methods. This property is particularly valuable for early warning applications in EHR data analytics.
Additionally, we observe that the absolute values of A on the MIMIC-III dataset are generally smaller than those on the AKI dataset, suggesting that the output sensitivity of t varies less markedly with increasing deviation magnitude in this setting. Furthermore, A can be negative under small deviations, indicating occasional prediction errors by V in extreme cases. These findings underscore the increased difficulty of the MIMIC-III scenario, consistent with the baseline performance degra- dation reported in Appendix F.4 Nevertheless, Vi continues to deliver robust deviation detection performance, even in this more complex and diverse application context.
F.6 Comparison with Rule-based Methods on the MIMIC-III Dataset
Existing rule-based methods that incorporate domain smowledee-particulle for laboratory test data-mainly focus on detecting values that fall outside normal physiological ranges [71
85 83
In this study, we refer to publicly available list of clinically validated variable ranges associated with the MIMIC-III multi-task learning benchmark. This list, developed in consultation with clinical experts, reflects their domain understanding of physiologically plausible measurement intervals (83). Each variable is defined by upper and lower bounds specifying the physiologically acceptable range, and any observed value that lies outside these thresholds can be flagged as abnormal. This constitutes a straightforward,
informed rule-basee based detection method.
We summarize key features from our dataset and report their empirical means and standard deviations (std a)) alongside the corresponding clinically valid ranges in Table It is noteworthy that clinically defined ranges typically span more than 10o, which is substantially broader than the deviations considered in our experiments (limited to perturbations up to 5 Consequently, such rule-based methods only identify overt abnormalities and are insufficient for detecting more
37

---

Table 8: Statistics and clinically valid value ranges for key features on the MIMIC-III dataset.
Feature
Mean
Std
Valid Low
Valid High
Valid Range/Std
Diastolic blood pressure
59.57
15.26
0.00
375.00
24.57
Fraction inspired oxygen
0,47
0.26
0.21
1.00
3.08
Glascow coma scale total
11.44
3.74
3.00
15.00
3.21
Glucose
138.27
67.38
33,00
2000.00
29.19
Heart rate
101.39
32.85
0.00
350.00
10.66
Mean blood pressure
78.37
18.20
14.00
330.00
17.36
Oxygen saturation
96.82
4.25
0.00
100.00
23.55
Respiratory rate
24.65
14.68
0.00
300.00
20.44
Systolic blood pressure
119.65
25.74
0.00
375.00
14.57
Temperature
36.90
0.85
26.00
45.00
22.43
subtle yet clinically meaningful deviations. In contrast, our proposed is specifically designed to detect small- magnituee deviations often overlooked yet potentially impactful (see Figures [3 and [9). This capability enables earlier clinical intervention and supports more reliable data quality assurance in real-world healthcare settings.
G Broader Impact
The proposed bi-level knowledge distillation approach enables reliable detection of potential data deviations EHR data arising from sources such as pre-analytical variability, documentation errors, unvalidated data sources. By assessing the EHR data fidelity, the approach enhances the reliability and accuracy of downstream clinical decisions and interventions. As such, represents a promising direction for improving data acquisition, collection, and recording protocols and may serve foundation for future error correction and calibration mechanisms.
Beyond these technical contributions, it is essential to involve clinicians and healthcare professionals when applying EHR deviation detection in clinical practice. In particular, decline in data fidelity may not solely stem from artifacts or errors-it may also reflect meaningful underlying physiological dynamics related to iatrogenic reasons, or medical interventions, reflecting the complex interplay of various acute medical conditions that occur concurrently in real-world patients (78). In such cases, additional contextual information multimodal data sources may required to interpret these deviations accurately and to understand the patient's clinical condition. For example, from a nephrological perspective, acute dialysis introduces substantial fluctuations in renal function measurements. These fluctuations do not align with the progressive trajectory of worsening or severe renal failure but instead reflect the treatment-induced modulation of physiological parameters. Consequently, approach for detecting EHR data deviations may not be limited removing erroneous entries or correcting recording mistakes. Rather, it can facilitate disease-specifis analysis by filtering out concurrent medical noise that arises from complex patient care processes, where multiple acute diseases, transfusions, infusions, and medications jointly influence observed trends. Addressing these challenges necessitates collaborative efforts between computational researchers and domain experts, and highlights an important open area for future investigation.
38


Reviews:
[{"review_id": "VPhs8Qel9M", "reviewer": "Reviewer_oTHf", "summary": "The paper addresses the under-explored challenge of data deviation, discrepancies between recorded EHR entries and a patient’s true physiological state, which diminishes data fidelity and jeopardises downstream clinical analytics. It models fidelity as a task-agnostic property and introduces a bi-level knowledge-distillation pipeline: task-specific data-Shapley values (the “Data Shapley Oracle”) are first approximated by neural oracles and then fused into a unified predictor Ψ for patient-level fidelity estimation. Monte-Carlo Shapley sampling combined with neural amortisation slashes the exponential cost of exact valuation, making near-real-time fidelity scoring feasible on large healthcare datasets. An anomaly is flagged whenever Ψ registers a fidelity drop beyond a learned threshold, enabling alerts for documentation errors, specimen-handling issues, or unreliable wearable-device readings.", "questions": "Please refer to the weakness.", "limitations": "Please refer to the weakness.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n- The paper tackles the important problem of detecting anomalies in EHR data.\n- The manuscript is clearly written and generally easy to follow.\n\nWeaknesses\n\n- The “Data Shapley Oracle” is produced via permutation sampling rather than by computing exact Shapley values. This approximation can introduce non-trivial error that may be further amplified when distilled into the proposed Neural Oracle. The paper should report sanity checks that quantify this gap before treating the oracle as ground truth.\n- The amortized Shapley framework may fail under domain shift. Out-of-distribution samples, common in healthcare, where each clinical sub-domain is highly specialized, could yield unreliable value estimates and undermine the proposed approach.\n- Definition 1 specifies a deviation threshold, yet in practice, this threshold behaves like a tunable hyperparameter. In real clinical settings, such cut-offs are hard to set, raising questions about whether the fidelity metric can robustly flag true abnormalities in EHR data.\n- The data fidelity used in this paper for detecting data deviation is highly dependent on the provided value function. However, the paper lacks explanation and analysis of the sanity of the value function. If the value function losts its sanity in providing prediction, then the entire scoring system will be not realiable and meaningless. The paper is recommended to include either empirical assessment or domain experts evaluation for sanity check.", "was_revised": true, "final_justification": "My concerns are still remaining. I personally believe this would be a good paper in the future, but it would need further quality improvement. Thus, I would keep my score.", "rebuttal": "**W1.** While we acknowledge that computing exact data Shapley values would yield higher accuracy than permutation-based sampling, the exact computation incurs exponential complexity and is intractable for the large-scale, real-world datasets used in our experiments. Further, permutation sampling remains a widely adopted and theoretically grounded approximation strategy in this line of research [14, 46].\n\nTo evaluate the potential impact of this approximation, we have performed 100K independent Monte Carlo simulations to estimate data Shapley values, employing two complementary metrics to assess convergence with statistical rigor across four post-AKI progression tasks. **Standard Error** ($SE = \\sigma/\\sqrt{n}$) underpins the construction of confidence intervals: $\\bar{x} \\pm 1.96 \\cdot SE$ provides 95% confidence bounds for the data Shapley values. The observed $O(n^{-1/2})$ decay rate aligns with the Central Limit Theorem, and the final SE reaches sub-percentage levels across all tasks. **Rolling Variance** ($RV$) quantifies temporal stability by measuring the variance of cumulative means over sliding windows of 100 samples. This serves as a practical indicator to assess when additional sampling yields diminishing returns. The observed monotonic decline in $RV$ indicates statistical stabilization of the estimates.\n\nThese analyses provide both theoretical justification and empirical support for the reliability of the permutation-based data Shapley value estimates used in our subsequent analyses.\n\n|Sample Count|Mortality Prediction||RRT Dependence Prediction||CKD Prediction||ESKD Onset Prediction||\n|-|-|-|-|-|-|-|-|-|\n|| $SE (10^{-5})$ | $RV (10^{-9})$ | $SE (10^{-5})$ | $RV (10^{-8})$ | $SE (10^{-5})$ | $RV (10^{-9})$ | $SE (10^{-5})$ | $RV (10^{-8})$ |\n|1k|7.88|3.63|13.57|5.88|16.19|1.89|24.48|6.42|\n|2K|5.56|1.75|10.60|2.85|11.80|0.91|17.78|3.09|\n|5K|3.51|0.69|7.57|1.12|7.71|0.36|11.83|1.21|\n|10K|2.48|0.34|5.79|0.56|5.51|0.18|8.49|0.60|\n|20K|1.76|0.17|4.23|0.28|3.93|0.09|6.04|0.30|\n|50K|1.12|0.07|2.79|0.11|2.51|0.04|3.83|0.12|\n|100K|0.79|0.03|2.01|0.06|1.78|0.02|2.72|0.06|\n\n**W2.** We thank the reviewer for highlighting the potential influence of domain shift on our amortized Shapley framework. We agree that domain shift—characterized by changes in the underlying data distribution—is a critical and challenging problem, particularly in healthcare settings where new diseases (e.g., COVID-19) or highly specialized sub-domains can result in out-of-distribution (OOD) samples. We acknowledge this as a potential limitation of our study, but emphasize that it is unlikely to affect our current setting for the following reasons.\n\nFirst, our work focuses on EHR data within a single hospital. Although federated EHR scenarios involving multiple hospitals are valuable, they are beyond the scope of our current study, as we do not manage inter-hospital domain shifts.\n\nSecond, within our hospital, data heterogeneity across different departments and diseases is already addressed through our multi-task modeling framework. This approach effectively unifies diverse clinical contexts by incorporating task-specific information without requiring explicit domain adaptation.\n\nThird, while domain shift research often investigates models trained on data from one domain and deployed in another, such a setting is not highly relevant to our routine clinical workflows. Except for emerging infectious diseases or other rare events, our clinical decision support system is typically developed and applied within the same institutional and departmental context.\n\nFinally, our approach is primarily designed for **pre-hoc detection of data deviations at the point of data entry**, aiming to alert clinicians and healthcare staff before erroneous data affects downstream processes. From this perspective, the risk posed by domain shift is minimal because the features of interest—such as frequently recorded vital signs and laboratory results—are consistent across common clinical tasks. Domain shift in rarely used or highly specialized tests (e.g., biomarkers for rare diseases) has far less impact on data quality assurance, which is our main focus.\n\nWhile we recognize domain shift as an important area for future extension, we believe it does not undermine the contributions or validity of our current work. We will revise the paper to explicitly discuss this potential limitation and outline possible directions for integrating domain adaptation techniques in broader, cross-institutional settings.\n\n**W3.** Yes, the threshold $\\delta$ in Definition 1 can be adapted to suit specific practical requirements and deployment scenarios. For instance, in our experiment evaluating the sensitivity of the proposed approach $\\Psi$ to varying deviation magnitudes (results shown in Figure 4), we set $\\delta = 0$ and interpret the sign of $\\Delta \\Psi$ as an indicator of whether a deviation is detected. As shown in Figure 4, when the detection objective emphasizes reliable performance within the interquartile range (P25–P75), this zero-threshold setting is sufficient. In particular, the sign of $\\Delta \\Psi$ consistently provides a stable and effective signal for identifying deviations, even for subtle perturbations near the decision boundary.\n\nWe acknowledge that in real-world applications, the threshold $\\delta$ should be customized based on domain-specific considerations. We work closely with our frontline clinicians to calibrate this threshold for practical use, balancing detection accuracy with the need to minimize alert fatigue in clinical workflows.\n\n**W4.** To assess the fidelity of EHR data, we leverage data valuation techniques to quantify the contribution of individual data samples. Among various approaches, the data Shapley value has emerged as a prominent method in the data valuation literature [14, 25, 46], and is increasingly recognized as a standard tool for assessing data quality in high-stakes domains such as healthcare [54, 57, a, b]. Accordingly, we adopt the data Shapley value as our data valuation method to derive EHR data fidelity. While the selection of data valuation methods is not the main focus of our work, our proposed approach is readily applicable to other, potentially more advanced, alternatives that may further enhance detection performance.\n\nGiven the high computational cost of exact data Shapley value calculation, we employ Monte Carlo sampling for approximation. The validity of this approximation has been discussed in detail in our response to **your comment W1**. Empirical results demonstrate that our approach achieves strong performance in detecting EHR data deviations and provides clinically meaningful insights into potential data quality issues of real-world cases, as validated by our clinicians.\n\nImportantly, our approach is designed as a pre-hoc data fidelity assessment procedure that addresses critical gaps in current EHR workflows by (i) identifying and excluding erroneous records from clinical workflows, thereby mitigating the risk of inappropriate interventions and adverse outcomes; and (ii) enabling targeted data correction strategies to improve overall data quality and ensure more reliable downstream analyses. While it is possible that our approach may produce potential false positives or negatives, the deviations it identifies are sufficiently informative to warrant clinical investigation prior to data ingestion into the EHR system. In this regard, the proposed approach functions as a practical and effective data fidelity screening mechanism, reinforcing the robustness and reliability of the EHR data infrastructure.\n\n> [a] Pandl, Konstantin D., et al. \"Trustworthy machine learning for health care: scalable data valuation with the shapley value.\" Proceedings of the Conference on Health, Inference, and Learning. 2021.\n\n> [b] Bloch, Louise, Christoph M. Friedrich, and Alzheimer’s Disease Neuroimaging Initiative. \"Data analysis with Shapley values for automatic subject selection in Alzheimer’s disease data sets using interpretable machine learning.\" Alzheimer's Research & Therapy 13.1 (2021): 155."}, {"review_id": "mE2bUlY7C4", "reviewer": "Reviewer_opND", "summary": "The authors study the novel problem of detecting data deviation in Electronic Health Records, which has implications on the reliability of patient data for downstream tasks. They propose a 2-step knowledge distillation approach that calculator Shapley values for individual tasks, and eventually learns a general neural model for data valuation. They use controlled deviation injection to show the ability for their method to identify anomalies on data from both public and private EHR records.", "questions": "One might ask if the goal is to train a good model on EHR data, then can we just assume that data deviation is random noise that a good model should ignore? Are there any experiments you could run outside of controlled injection studies to show that this actually leads to a performance difference?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The authors study a very interesting and novel problem, and the paper is very well-written. The overall approach considering Shapley scores makes intuitive sense, and the experiments are thorough across several interesting tasks. \n\nSome weaknesses include: \n\nW1. Lack of studying ablations of the proposed method (e.g. removing KD steps) versus just comparing against other baselines. Furthermore, are there any simple rule-based approaches leveraging the domain knowledge that could be used as a baseline? \n\nW2. Clarification whether the primary goal is to remove data for the purpose of downstream prediction, or other uses. Since the method only outputs an anomaly score,  it doesn’t help shed light on why any deviation occurred. The case studies in the paper were really useful, but primarily seemed to highlight an issue with duplication: meanwhile, the paper’s introduction highlights a wider source of data deviation. Could the authors provide more insights from the data on the types of deviation error?\n\nW3. Discuss any limitations around removing data deviation points – similar to arguments around how incorrect data in LMs are actually useful for calibration and misinformation detection (or training on toxic text helps LMs know toxic vs. non-toxic and generate less toxic text), what if a model built on EHR data might actually benefit from being aware of potential data deviations? Are there concerns around robustness?", "was_revised": true, "final_justification": "Thank you for the thorough response, and I appreciate the clarifications surrounding pre-hoc detection. While the authors mention that clinicians have confirmed the practice necessity of pre-hoc methods, it would be great if the paper reflected this as the introduction still largely motivates the work with the potential of EHR analytics.", "rebuttal": "**W1.** **Ablation study results.** We first clarify that the knowledge distillation process from $\\mathcal{O} _ {ds}$ to $\\mathcal{O} _ {nn}$—which enables the derivation of application-specific data valuation for downstream analysis—is an essential component of our approach and cannot be removed. In response to the suggested ablation study, we have compared our proposal against several weakened variants of the subsequent distillation step from $\\mathcal{O} _ {nn}$ to $\\Psi$. As shown in the table below, our approach, which incorporates an attention-based aggregation strategy, consistently achieves the lowest loss. These results underscore the effectiveness of the proposed teacher fusion mechanism and demonstrate the necessity of a principled aggregation strategy in this knowledge distillation process.\n\n|Variant|Loss|Strategy Description|\n|-|-|-|\n|**Our approach**|**0.008135**|Attention-based aggregation as defined in Equation (6)|\n|Random weights|0.010720|Fixed random weights across teachers|\n|Simple average |0.028935|Equal-weight aggregation across teachers|\n|Top-2 teachers|0.026531|Aggregation of the two best-performing teachers|\n|Top-1 teacher|0.144956|Use of the single best-performing teacher only|\n\nWe have further conducted a fine-grained ablation analysis to assess the contribution of each component in the overall objective loss in Equation (12). Specifically, we examine the effects of removing individual loss terms, disabling the dynamic weight adjustment mechanism, and modifying the temperature setting. As shown in the table below, our full approach achieves the best performance. Performance degrades noticeably when any component is removed or altered, confirming the importance of each design choice. These results collectively underscore that the synergy among loss terms, dynamic weighting, and appropriate temperature calibration is critical for optimal knowledge distillation.\n\n|Variant|Loss|Strategy Description|\n|-|-|-|\n|**Our approach**|**0.008135**|Full objective loss as defined in Equation (12)|\n|Without entropy constraint|0.024595|$\\mathcal{L}_{ent}$ removed from objective|\n|Without similarity constraint|0.029499|$\\mathcal{L}_{sim}$ removed from objective|\n|Fixed weights|0.104307|Static $\\lambda_{kd}$, $\\lambda_{ent}$, $\\lambda_{sim}$ in Equation (12)|\n|High temperature|0.017665|Temperature set to $\\tau=2.0$|\n|Low temperature|0.109919|Temperature set to $\\tau=0.1$|\n\n**Rule-based approaches leveraging domain knowledge.** Existing rule-based methods that leverage domain knowledge—particularly for laboratory test results—primarily focus on detecting \"values outside normal physiological ranges\" [c, d, e]. For this study, we have identified a publicly available list of clinically validated variable ranges associated with the MIMIC-III multi-task learning benchmark. This list was developed in consultation with clinical experts and reflects their domain knowledge of physiologically plausible measurement ranges [e]. Each variable in this list is associated with predefined upper and lower bounds that specify a physiologically acceptable range. Observed values falling outside these thresholds can be flagged as abnormal, thus forming a simple domain-informed rule-based detection method.\n\nWe have summarized key features from our dataset and reported their empirical means and standard deviations ($\\sigma$) alongside the corresponding valid ranges in the table below. Notably, the clinically defined ranges typically exceed $10\\sigma$, far beyond the scale of deviations considered in our experiments (which are limited to perturbations up to $5\\sigma$). As a result, such rule-based methods are only capable of identifying overt anomalies and are inadequate for capturing more subtle yet meaningful deviations. In contrast, our proposed $\\Psi$ is specifically designed to detect small-magnitude deviations often overlooked yet potentially impactful (see Figure 3). This capability supports timely clinical intervention and contributes to more robust data quality assurance in real-world healthcare settings.\n\n|Feature|Mean|Std|Valid Low|Valid High|Valid Range/Std|\n|-|-|-|-|-|-|\n|Diastolic blood pressure|59.57|15.26|0.00|375.00|24.57$\\sigma$|\n|Fraction inspired oxygen|0.47|0.26|0.21|1.00|3.08$\\sigma$|\n|Glascow coma scale total|11.44|3.74|3.00|15.00|3.21$\\sigma$|\n|Glucose|138.27|67.38|33.00|2000.00|29.19$\\sigma$|\n|Heart Rate|101.39|32.85|0.00|350.00|10.66$\\sigma$|\n|Mean blood pressure|78.37|18.20|14.00|330.00|17.36$\\sigma$|\n|Oxygen saturation|96.82|4.25|0.00|100.00|23.55$\\sigma$|\n|Respiratory rate|24.65|14.68|0.00|300.00|20.44$\\sigma$|\n|Systolic blood pressure|119.65|25.74|0.00|375.00|14.57$\\sigma$|\n|Temperature|36.90|0.85|26.00|45.00|22.43$\\sigma$|\n\n> [c] Shi, Xi, et al. \"An automated data cleaning method for Electronic Health Records by incorporating clinical knowledge.\" BMC Medical Informatics and Decision Making 21.1 (2021): 267.\n\n> [d] Wang, Zhan, et al. \"Rule-based data quality assessment and monitoring system in healthcare facilities.\" Improving Usability, Safety and Patient Outcomes with Health Information Technology. IOS Press, 2019. 460-467.\n\n> [e] Wang, Shirly, et al. \"Mimic-extract: A data extraction, preprocessing, and representation pipeline for mimic-iii.\" Proceedings of the ACM conference on health, inference, and learning. 2020.\n\n**W2.** We clarify that the primary goal of our proposal is to enable **pre-hoc detection of potential data deviations at the point of data entry** into the EHR system during clinical practice. The aim is not data cleaning or post-hoc removal for downstream analytics, but rather early identification of anomalies at the time of data recording. By highlighting entries with possible deviations as they are introduced, our approach supports real-time quality control and reduces the likelihood of erroneous data entering the system. This leads to improved EHR data quality, which may in turn enhance the performance of downstream analytical tasks. However, this improvement is a secondary benefit; our central goal remains the reliable assessment of data fidelity in situ, to prevent flawed data from influencing clinical decision-making for real-world patients.\n\nWe also clarify that the presented case studies go beyond simple duplication scenarios. For instance, the Point-of-Care Testing (POCT) laboratory tests featured in our case studies are conducted at or near the site of patient care rather than in centralized laboratories, with the aim of enabling rapid clinical decisions. When multiple test results for the same feature are recorded within a short timeframe and exhibit inconsistencies or abnormalities, the underlying causes can be multifactorial. These include misattribution of results to the wrong patient, errors in sample collection, device malfunctions in uncontrolled environments, or testing conducted at inappropriate times relative to clinical interventions or medication schedules.\n\nOur clinicians confirm the practical necessity and clinical value of detecting such potential data deviations in a pre-hoc manner. It is emphasized that alerting healthcare workers to possible issues at the time of testing can help prevent erroneous data from being integrated into subsequent clinical workflows, thereby improving the robustness of patient care and decision-making.\n\n**W3 & Questions.** We would like to clarify that our primary goal is neither post-hoc data removal or cleaning, nor the development of high-performing EHR analytic models. Instead, our focus is on **pre-hoc detection of potential data deviations prior to their integration into the EHR system** in clinical practice. Such early detection enables clinicians and healthcare workers to intervene promptly, facilitating root cause investigation and the refinement of clinical workflows—for example, by improving device stability or updating standard operating procedures.\n\nWhile we acknowledge the possibility of leveraging our deviation detection outputs as auxiliary information in EHR analytics—for example, using them as pseudo-labels to inform model training—we note that the impact of such inherent noise or deviations is nuanced. Depending on the context, these signals could either degrade performance or promote model robustness. Exploring this trade-off, however, goes beyond the primary scope of our current study. We will revise the paper to clarify these points and incorporate further discussion accordingly."}, {"review_id": "pEcCcQ91jm", "reviewer": "Reviewer_8yxT", "summary": "This paper addresses the problem of detecting EHR discrepancies between recorded values and the patient’s true physiological state (ie \"data deviation\") in Electronic Health Records (EHRs). They propose a multi-step framework to estimate whether the data is 'good' based on first training task-specific models (using Shapley values) to measure how much each data point contributes to downstream clinical tasks and then then combining those models into a single, general-purpose predictor. This final model assigns a \"fidelity score\" to new data points. If that score drops sharply after a small change to the input, it signals that the data may be unreliable. The method is tested on both private and public (in the appendix) datasets.", "questions": "- None of the models are directly looking at the time series, though such information would probably be very valuable for the underlying question of whether the collected data is a valid reflection of the patient’s state. Was this considered? If so, why was it not leveraged?\n- Intuitively why were the elements of this model combined? For instance, why did you think to use Shapley values as a first step?\n- Am I correct in my understanding that this approach does not verify whether the EHR data is an accurate measurement of patient physiology but rather it measures whether the data appears useful in downstream models? These are not always the same. Misleading data can sometimes improve task predictions if it correlates spuriously with outcomes (e.g., due to confounding).", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 4, "strengths_and_weaknesses": "Strengths\n- This area (identifying and potentially addressing incorrect input values) is very valuable for every domain, especially including healthcare which has many opportunities for noise to occur.\n- Section 4.4's case studies did a very good job explaining how one could identify data quality issues (eg two labs that should probably move together actually end up diverging in their series).\n- I appreciated seeing baselines to compare against to illustrate the task & alternative solutions.\n\n\nWeaknesses\n- This area, while important, is hard to measure whether particular contributions are meaningful. For instance, the corruptions to the data you chose might not be meaningful or generalizable to other data collection issues.\n- Some of the baselines were weak. Many are designed to identify global outliers, rather than the e.g. multi-lab time series correlations illustrated in the case study. I would’ve liked to see something like “real vs corrupted” classifier trained to predict actual data points (label=0) vs the corrupted data points you create (label=0).\n\nMinor Weaknesses\n- The writing in the paper often used technical jargon (which made it hard to follow on first pass) when simpler explanations could've worked just as well if not better. As an illustration, instead of \"Knowledge is distilled from the data Shapley oracle  𝑂 𝑑 𝑠 O  ds ​  … into a corresponding neural oracle  𝑂 𝑛 𝑛 O  nn ​  … trained jointly in a multi-task learning setting.\" I would have benefitted from seeing early on \"We train a set of models to mimic the Shapley values efficiently, one for each prediction task.\"\n- Until Section 4.4 I was a little confused about the inputs/outputs of the task setup (eg is it predicting the fidelity of a single data element vs a batch of results vs something else). This could have been made more clear earlier on.", "was_revised": true, "final_justification": "I thank the authors for their clarifications and responses.\n\nI had initially misunderstood the controlled deviation injection experiment. On re-reading, I see that it is indeed present, though I would suggest slightly clearer framing in the main text to aid comprehension.\n\nI'm still unconvinced about both:\nW1) how generalizable are these data corruptions?\nQ3) is this method detecting data fidelity or feature impact on performance.\n\nFor Q1, in my experience, perturbed numeric values is one kind of data entry, though not the most egregious. There are non-smooth data entry errors such as using the wrong units or entering an extra digit (eg someone who ways 2122 pounds). That said, this method doesnt need to account for *all* data errors. I merely speculate that the utility of this approach may be more narrow than the authors believe.\n\nOn Q3, I think the authors and I agree on the mechanics: the approach identifies data elements whose perturbation most affects downstream model utility. Where we differ is in interpretation. The authors argue that their goal is not to enhance downstream performance, and thus their approach measures fidelity rather than utility. My view is that (even with a multi-task setup) model-derived utility remains an indirect proxy for fidelity. The method does not verify whether the EHR data reflect accurate physiological measurements (which is understandably infeasible), but instead assesses how useful the data appears to be for model performance. Whether that is an acceptable stand-in for \"fidelity\" remains open to interpretation.\n\nI will maintain my score of borderline accept.", "rebuttal": "**W1.** We respectfully disagree with the concern regarding the relevance and generalizability of our perturbation strategy. In clinical practice, point-of-entry validation for EHR data is essential due to the risk of serious downstream consequences from erroneous inputs. However, existing validation practices rely largely on coarse plausibility checks—such as broad value ranges or thresholds based on standard deviations—which are insufficient for detecting subtle yet clinically significant deviations (see also our response to **Reviewer opND's comment W1**).\n\nOur strategy, which perturbs feature values by their standard deviation $\\sigma$, aligns with established clinical quality control practices and offers a clinically interpretable way to model abnormal laboratory readings [c, d, e]. Rather than an arbitrary choice of corruptions, what our work contributes is a principled, sensitive framework that addresses the limitations of current checks by enabling pre-hoc detection and alerting of meaningful deviations prior to downstream contamination.\n\nThis design reflects realistic data fidelity issues observed in operational EHR workflows. Our clinicians have confirmed that our experimental setup is not only meaningful but also directly aligned with the operational needs and safety requirements of healthcare data entry. We will revise the paper to articulate this clinical grounding and to discuss the broader applicability of our approach beyond the specific perturbations evaluated.\n\n**W2.** We are not entirely certain if we have interpreted this comment correctly and would appreciate any clarification if our current understanding is inaccurate.\n\nWe would like to first clarify a potential misunderstanding on the experimental evaluation. Our paper does include the suggested “real vs corrupted” classification experiment to distinguish actual data samples from perturbed ones. In our controlled deviation injection setup, we (i) perturb data with the feature's standard deviation $\\sigma$, (ii) generate paired samples—original (label = 0) and perturbed (label = 1), and (iii) construct a labeled benchmark dataset for fidelity evaluation. We then compute the expected decline in EHR data fidelity $\\Delta \\Psi$ and use its sign to indicate whether a deviation is detected.\n\nAs shown in Figure 3, our approach outperforms anomaly detection baselines, particularly in detecting small-magnitude deviations. We have also compared against domain-knowledge-based rules and presented a detailed ablation study assessing different constituent components. Please refer to our response to **Reviewer opND’s comment W1** for related discussion. We remain open to additional suggestions on methods relevant to our setting and will incorporate such comparisons where feasible.\n\n**W3.** We will revise the paper to introduce our core idea using clearer and more accessible descriptions before presenting the detailed design and terminology. We believe this restructuring will improve clarity and make our contributions more approachable to a broader audience.\n\n**W4.** Our approach outputs a predicted EHR data fidelity score $\\Psi$ for each data sample. To detect deviations, we compute the expected EHR data fidelity decline $\\Delta \\Psi$ and use its sign as an indicator.\n\nIn the controlled deviation injection experiment (Figure 3), we simulate physiologically plausible deviations by perturbing features with multiples of their standard deviation $\\sigma$, forming paired samples ⟨original (label = 0), perturbed (label = 1)⟩. We apply our approach to compute $\\Delta \\Psi$ for each pair and use the sign to identify deviations. Results demonstrate the strong detection performance of our approach compared to baselines.\n\nIn the output sensitivity experiment (Figure 4), we use the same paired samples as input with varying magnitudes of the injected deviations. The resulting $\\Delta \\Psi$ values demonstrate that our approach reliably captures subtle deviations, underscoring its practical utility in early-stage deviation detection for clinical applications.\n\nWe acknowledge that the rationale and structure of these experiments may not be immediately clear on a first reading. To address this, we will revise the paper to introduce the experimental setups and key design choices earlier, improving clarity and readability.\n\n**Q1.** We follow the established MIMIC-III benchmark for data processing, aggregating 17 physiological variables across seven predefined time ranges: the first and last 10%, 25%, 50% of the ICU stay, and the entire duration. For each variable within each range, six statistics are computed—minimum, maximum, mean, standard deviation, skewness, and count—yielding 714 features per sample. This preprocessing strategy has demonstrated strong performance on benchmark MIMIC-III tasks in prior work and is adopted here to ensure consistency. As our focus is on pre-hoc detection of data deviations at the point of entry rather than optimizing preprocessing, we do not explore alternative preprocessing strategies. \n\nFrom a practical standpoint, incorporating more advanced time series models would exacerbate the already significant computational cost of estimating data Shapley values. Exact data Shapley computation is intractable for large-scale EHR data due to its exponential complexity [14]. Even with Monte Carlo approximation, the process requires repeated model retraining for each sample across multiple permutations. To address this, we adopt an amortized modeling approach, training a neural network to approximate per-sample data Shapley values [1, 6]. While this strategy reduces inference-time cost, it still requires computing ground-truth data Shapley values during training. Given these constraints, more complex time series models are currently infeasible. Improving the efficiency of Shapley-based data valuation continues to be an important and active area of research [46].\n\n**Q2.** Our methodology is designed in a top-down, stepwise manner, guided by the central goal of detecting EHR data deviations. Below, we clarify the rationale behind each component.\n\nTo begin, detecting deviations requires a principled measure of data fidelity. Data valuation methods—particularly data Shapley values—are increasingly used to quantify sample-level contributions to learning models [14, 25, 46, 54, 57, a, b], making them a natural proxy for data fidelity. However, data Shapley values are inherently task-specific and thus insufficient for capturing a more comprehensive notion of fidelity across a patient’s EHR data.\n\nTo overcome this, we adopt a multi-task perspective that distills task-specific data Shapley values into a unified EHR data fidelity predictor. This integration enables a more comprehensive assessment by incorporating insights across multiple clinical tasks.\n\nIn summary, we propose two levels of knowledge distillation:\n(i) from computationally intensive data Shapley values to task-specific neural oracles, to address efficiency concerns (see Sections 3.1 and 3.2), and\n(ii) from the task-specific neural oracles to a unified EHR data fidelity predictor that captures cross-task insights.\n\nThe motivation for this design and the necessity of each knowledge distillation level are detailed in Section 2. We have also included an ablation study evaluating the impact of different components (see our response to **Reviewer opND’s comment W1**), and a sanity check validating the approximation of data Shapley values (see our response to **Reviewer oTHf's comment W1**).\n\nWe will incorporate the above discussion into the paper to further clarify the rationale behind our proposed methodology and facilitate readers' understanding of our technical design choices.\n\n**Q3.** We would like to clarify that our objective is fundamentally different from enhancing the analytic performance of downstream models. Instead, our focus lies in **pre-hoc detection of potential data deviations at the point of entry**, with the explicit goal of preventing such records from contaminating the EHR system. Our approach raises alerts upon detecting deviations, enabling clinicians and healthcare staff to take timely actions, such as investigating root causes or refining data entry workflows, thereby ensuring overall data fidelity.\n\nWe understand that this comment likely arises from our use of data Shapley values as a basis for constructing a fidelity measure. Data Shapley values estimate a sample’s contribution to downstream tasks, which does not always equate to its alignment with true physiological measurements. This distinction is well noted, and addressing this gap forms the core innovation of our work. Specifically, we integrate task-specific data Shapley values derived from multiple clinically relevant tasks into a unified fidelity predictor, which captures the holistic quality of EHR data beyond any single downstream objective. By combining these multi-task contributions, our approach reduces the risk of over-reliance on task-specific correlations and produces a more robust indicator of data fidelity. Experimental results and expert evaluations from clinicians confirm that this integrated approach effectively identifies deviations and highlights clinically meaningful data quality issues observed in real-world cases.\n\nWe also acknowledge the complex relationship between misleading data and downstream performance. Some deviations degrade performance by introducing noise or systematic errors, while others may inadvertently boost performance via spurious correlations or confounding factors. Although exploring such task-dependent effects is beyond the primary scope of this study, we agree that this is an important perspective and will revise the paper to (i) restate our emphasis on data fidelity rather than downstream performance enhancement, and (ii) expand the discussion to reflect the broader implications and impact of misleading data on downstream performance."}]

Author rebuttal:

Meta-review: The paper tackles the important problem of detecting data deviation in electronic health records (EHRs) using a bi-level knowledge distillation approach. The authors define data deviation as discrepancies between recorded entries and a patient's actual physiological state, and propose a system that first approximates computationally expensive data Shapley values with neural oracles for individual clinical tasks, then distills these into a unified EHR data fidelity predictor. The approach aims to support pre-hoc detection of potential data quality issues at the point of data entry, enabling clinicians to identify and address erroneous records before they contaminate downstream clinical workflows.

The reviewers and authors had an active discussion with several substantive concerns. Reviewer oTHf raised critical questions about the reliability of using Monte Carlo approximation instead of exact Shapley values, the potential for domain shift issues, and the practical challenge of setting detection thresholds in clinical settings. The authors responded with extensive Monte Carlo validation studies showing convergence, but oTHf remained concerned about error propagation through the two-stage distillation process. Reviewer opND questioned the scope and practical utility of the approach, asking for better ablation studies and clarification about whether the goal was data cleaning or pre-hoc detection. The authors provided comprehensive ablation results and clarified their focus on pre-hoc screening. Reviewer 8yxT acknowledged the importance of the problem but questioned the generalizability of the perturbation strategy to real-world data entry errors and whether the method truly measures data fidelity versus feature impact on model performance. The authors maintained that their physiologically-scaled perturbations align with clinical quality control practices and that their multi-task approach provides a robust proxy for data fidelity.

In reviewing the paper, reviews, and author responses, I recommend acceptance because the paper addresses a clinically important and under-explored problem with a technically sound approach that demonstrates clear practical value. While legitimate concerns remain about the approximation errors in the Shapley value estimation and the distinction between measuring data utility versus true physiological fidelity, the authors provided substantial additional analyses addressing most reviewer concerns. The multi-task knowledge distillation framework is methodologically novel, the experimental validation is thorough with both controlled experiments and real-world case studies, and the clinical motivation is well-established. The work makes a meaningful contribution to healthcare AI by providing a principled approach to data quality assessment that could improve clinical decision-making, despite some remaining questions about generalizability and threshold calibration in practice.
Decision: Accept (poster)