PERSONALIZED LAB TEST RESPONSE PREDICTION WITH KNOWLEDGE AUGMENTATION
Authors: Suman Bhoi
Mong-Li Lee
Wynne Hsu
Hao Sen Andrew Fang
Ngiap Chuan Tan
Venue: ICLR 2022 Submitted (2022)
Keywords: Lab test responses
Patient Representation
Electronic Health Records

Abstract: Personalized medical systems are rapidly gaining traction as opposed to “one size
fits all” systems. The ability to predict patients’ lab test responses and provide justification for the predictions would serve as an important decision support tool and
help clinicians tailor treatment regimes for patients. This requires one to model
the complex interactions among different medications, diseases, and lab tests. We
also need to learn a strong patient representation, capturing both the sequential
information accumulated over the visits and information from other similar patients. Further, we model the drug-lab interactions and diagnosis-lab interactions
in the form of graphs and design a knowledge-augmented approach to predict patients’ response to a target lab result. We also take into consideration patients'
past lab responses to personalize the prediction. Experiments on the benchmark
MIMIC-III and a real-world outpatient dataset demonstrate the effectiveness of
the proposed solution in reducing prediction errors by a significant margin. Case
studies show that the identified top factors for influencing the predicted lab results
are consistent with the clinicians' understanding.

Under review as a conference paper at ICLR 2022
PERSONALIZED LAB TEST RESPONSE PREDICTION WITH KNOWLEDGE AUGMENTATION
Anonymous authors Paper under double-bling review
ABSTRACT
Personalized medical systems are rapidly gaining traction as opposed to "one size fits all" systems. Predicting patients' lab test responses and providing justifica- tion for the predictions would help clinicians tailor treatment regimes for patients. This requires modeling the complex interactions among different medications, dis- eases, and lab tests. We need to learn strong patient representation that captures both the sequential information accumulated over the visits and information from other similar patients. We model drug-lab interactions and diagnosis-lab interac- tions as graphs and design a knowledge-augmentt response for target lab result. We also take into consideration patients' past lab responses to personalize the prediction. Experiments on real-world datasets demonstrate the effectiveness of the proposed solution in reducing prediction er- rors by significant margin. Case studies show that the identified factors for influencing the predicted results are consistent with elinicians' understanding.
approach to predict patients'
INTRODUCTION
Electronic Health Records (EHR) provide a rich repository of patient related information such as disease diagnosis, lab test results, prescribed medications, etc.
history of visits. Advances in machine learning have led to the large-scale analytics of EHR for disease inference (Ni et al. 2017). mortality prediction (Tan al. 2019), personalized medication recommendation (Wang al. 2019
Shang et al. 2019: Bhoi et al. 2021). At the same time, there is a growing trend towards using pa- tient analyties for decision support (Oei al., [021] to improve patient care and clinical outcomes, particularly in chronic disease management such as hypertension and diabetes. Often, these chronic disease patients are prescribed some standard treatment regime for each chronic condition indepen- dently and their conditions are monitored periodically based on lab test results such HbAlq] However, for patients with co-morbidities, studies have shown that their lab test results are often influenced by other treatments related to their co-morbidities, making it hard to assess the effective- ness of the prescribed treatment (Unnikrishnan et al. 2012). The ability to predict the target lab test result of a patient's condition, taking into consideration medications that are prescribed for his/her other conditions, would enable the clinician to personalize the treatment regime for the patient. This can help eliminate invasive procedures associated with sample collection for the lab test.
Existing research use patient specific information such as demographics and past visit records to predict lab test results (Luo al. 2016 Kang (0018). The work in Luo et al. (2016) predicts Ferritin lab test result by using patient demographics and the results of other lab tests that have been ordered the same time as the Ferritin lab test. Kang (2018) examines the task of predicting HbAlc test results and proposes a recurrent neural network (RNN) based architecture to model the sequential dependencies across the past visits. These works do not consider the impact of medications on the target lab test result. For example, patients with high blood pressure are prescribed medications like Propranolol which is known to increase HbAle (Dornhorst al. 1985).
In practice, lab test results are often influenced by drugs and diagnosis. However, these drug-lab interactions and liagnosis-lab interactions are largely ignored by current works on lab response predictions. We observe that labg interaction can be positive where the drug increases the lab test value, or negative where the lab test value is decreased by the drug, e.g., HbAlc is increased
httg:://nnciiiihhhssse////ssssso alc-hbalc-tsst//

---

Under review as a conference paper at ICLR 2022
by Propranolol and decreased by Metformin (Dornhorst et al. Similarly, diagnosis-lab interaction can be positive or negative, e.g. HbAle is increased by iron
1985
Conzález-Ortiz et al. 2012)
deficiency anemia and decreased by hemolytic anemia (Program 2013
In this work, we use a transformer encoder (Vaswani et al. 2017) to capture the patient specific information, while the information of similar patients is modeled using the modified graph attention network (GATv2) Brody et al., 2021). We combine the outputs from both the transformer encoder and the GATv2 to obtain strong latent patient representation so as to accurately predict patient response to a target lab test. We incorporate fine-grained dosage information to increase the accuracy of the predictions in the presence of medication titrations for chronic disease patient management. This is in contrast to existing works which mainly consider medication type.
We augment the patient representation with the knowledge of drug-lab interactions, diagnosis-lab interactions, and use graph attention networks to model the positive and negative effects of drugs and diagnosis on the target lab result. This enables us to model the complex relationships between drugs, co-morbidities, and lab test results. We also take into consideration patients' past lab responses to personalize the prediction. Finally, we leverage the attention weights in the proposed solution to identify the top factors that may have influenced the predicted lab test results. This allows clinicians to have insight into the underlying cause for any changes in the lab test result. Extensive quantitative and qualitative experiments etary outpatient dataset demonstrate the effectiveness of the proposed system to significantly lower the prediction errors by large margin. Case studies reveal the usefulness of identifying the factors that contributed to the predicted lab test results,
the benchmark MIMIC-III (Johnson et al.. 2016) EHR and a propri-
RELATED WORKS
Lab test prediction has been studied for the purpose of clinical decision support. The work in (Luo et al. 2016) employs multiple machine learning algorithms to predict Ferritin test response from patient demographics and other accompanying lab test responses from EHR. (Kang et al. 2015) uses an ensemble of support vector machines to predict HbAlc test results using patient demographics, prescribed medications, and the last known HbAlc value. (Kang et al.) (017)) proposes to use a heterogeneous ensemble of classifiers and reduces the prediction error by incorporating an option to reject an instance when the confidence of any of the classifiers is low. These methods rely on only the current patient visit information and ignore the sequential dependency of past patient visits. (Kang) 2018) proposes a recurrent neural network based architecture to model the sequential dependencies across the visits for the task of prediction of HbAle Ic test results. Given the black-box nature of RNN, this approach is unable to identify the factors that contribute to the lab test result prediction. In contrast, our solution is able to highlight these factors via attention based mechanism.
Learning a strong latent patient representation is key to the accurate prediction of lab test results. The work in (Choi et al., 2016 Ma et al., 2017) use a recurrent neural network (RNN) and a bi- directional RNN respectively to create patient representation for disease prediction by capturing the sequential dependency in medical codes over time. Similarly, T-LSTM (Baytas et al. (0017) pro- poses time aware long-short term memory for patient subtyping. DMNC (Le et al. 2018) learns the patient representation by infusing memory augmented neural networks (Sukhbaatar et al. 2013) with RNNs handle long-range dependencies in patient visit history. Recent works have demonstrated the superiority of transformer encoder representations over RNN based approaches in capturing the sequential dependency in EHR for disease prediction (Li et al. 2020), mortality prediction (Darabi et al. 2020), medication recommendation (Prakash et al. 2021). HiTANet (Luo et al.) 2020) intro- duces time-aware transformer to obtain a patient representation using local and global attention for risk prediction. BEHRT (Li et al.) 2020) adapts Transformer (Vaswani et al. 2017 Devlin et al.) 2019) based architecture to learn patient representation using patient's 's diagnostic and demographic information to predict future disease occurrence. All these works do not consider information from other similar patients and do not use fine-grained dosage information.
Recently, (Lu et al., diagnosis and heart failure prediction. This work uses the ontology of diseases along with the patient diagnosis information to jointly learn the representation for patients and diseases. Our work differs from this in that our notion of patient similarity includes demographics, prescribed medications information in addition to disease diagnosis to predict lab test responses.
202I) model patient similarity and incorporate clinical notes for the task of
2

---

Under review as a conference paper at ICLR 2022
Lab Test
Knowledge Augmercation
Lab Interaction Information
Past Lab Test Response Information
Agg Aggeegator
Pollent Reprssentation
Transformer Encoder
Transformer
GATv2
G
Embredding Layer
Embedding Layer
41
dy
Why-1
Figure I: Overview of KALP.
[Diubates mullitu, Hyperlipidemls]
[Dlobetes mellitus, Hyperlipidemin)
Hyparipidemia)
(HHAA,, 9.1Il
[[Sitagliptn, 75)
[tHbAIc,
[|Sttaglipcin, 75).
[|bbAle 7.8]]
[(Stagglptin, 50)
(Simuastatin, 20) (Metformin, (Glclaudde, 160)
(Simmaetatin, 20),
[Simwastadin, 20).
(Metformin, (Glicladde, 180).
(Menformin, 850).
(Gliclazide, 150).
(Chorpheniraminc 4),
(Ascorbic Acid, 200
(Ascorbic Adid, 200]]
Figure 2: EHR representation of patient across three visits with HbAle the lab test under consideration. Information sources are color-coded with different colors, green indicates medication with dosage, red indicates diagnosis, and blue indicates lab test response.
METHODOLOGY
Given a sequence of past patient visits, our task is to predict the result of a target lab test for the patient's is current visit. Figure D givess overview of the proposed solution called KALP for Knowledge-Augnet
Lab test Prediction. There are two key components: (a) learning an effec- tive latent patient representation and (b) augmenting it with knowledge of drug-lab and lisease-lab interactions, as well as patient's historical responses to the lab test.
3.1 PATIENT REPRESENTATION
Our proposed patient representation takes into account both the sequential dependency in patient diagnosis and medication information, as well as information from similar patients. Suppose patient has T visits prior his current visit. We represent the 7th visit of a patient as
e; |d m,,
where d; is multi-hot vector depicting the diagnosis, m, is vector of (medication, dosage) pairs, and 1, is continuous variable denoting the results the target lab test, 1 S < T
We capture the dosage information in the form of a continuous vector where each entry contains the dosage information for the corresponding medication type. All the continuous values are normalized using min-max normalization (Han et al. 2011). Figure? 2 illustrates the visit information extracted from a patient's EHR.
Sequential Dependency. We utilize two transformers, one for the diagnosis and the other for med- ication, to model the sequential dependency in the patients' diagnosis and medication information over the visits. Having dual transformers enable to obtain representation for patients who may
3

---

Under review as a conference paper at ICLR 2022
Aggrepator
Ag--1
GAIv2
Encoder
Encoder
Encoder
Encoder
Encoder
Encoder
Transformeri
Concatarmetoo
Encoder
Encoder
u.s knnoder
Encoder
Encoder
kncoder
Embedding Layer
Embedding Layer
Diagnosis
Medicine
Codes
Doses
Patient Similarity Graph
Figure 3: Details of the patient representation component.
have missing diagnosis or medication information. We linearly embed d, and m,, into low dimen- sional space using embedding matrices E' and ET respectively. These linear embeddings then combined with positional embeddings (Vaswani et al. (017)) to obtain the embeddings for diagnosis di and medication m.. The embeddings d, and m passed to their respective transformers as shown in Figure S
Each transformer consists of multiple layers with each layer having T encoders. Each encoder has position-wise fully connected feed-forward network and a multi-head self-attention mecha- nism. The position-wise feed-forward network has two linear transformations with ReLU layer in between. The multi-head attention employs scaled dot-product attention to obtain the weights on the patient visits given the visit history. Since this multi-head attention can attend future time steps, to ensure that the model's 's predictions are only conditioned on past visits, we apply triangular mask the embedding de and m.- This mask is the same as the one used the decoder component of (Vaswani et 2017 The input the 7th encoder in the first layer is the concatenation of d'j. 1 S j S i. Inputs to the ith encoder in the subsequent layers is the concatenation of the outputs from the first to the th encoders of the previous layers.
Similar to Vaswaan et al. 2017p, we employ residual connection around the self-attention mecha- nism and the feed-forward network is followed by layer normalization. We also apply dropout (Sri- vastava et al. 2014) to avoid over-fitting. The outputs of the transformers for diagnosis and medica- tions are given by:
h h .... h4 Transformer (ddidd,,.. di
(1)
ht h hT Transformer (mm m ..., m)
(2)
where he and h are the outputs of ith encoder in the last layer of the respective transformers. Since our goal is to predict the lab test result for the current visit T, we use the outputs h and hT- as the encoded sequential diagnosis and medication information.
Patient Similarity Information. The work in Jia et al. 2020) uses pair-wise patient similarity as a feature and extreme gradient boosting improve the accuracy of diagnosis prediction. (Wirbka al. 2020) constructs cohorts of patients who have similar demographics, diagnosis and prescribed medications for treatment recommendation Here, we observe that patients with similar age, weight, gender, diagnosis, etc. tend to have similar lab test responses. As such, want incorporate information from similar patients to learn a more effective patient representation that improves the prediction accuracy of our task.
We construct a weighted patient similarity graph G = EV, where each node n; E V denotes patient and each labelled edge (ne, (i, Wiy) F. denotes that patient is similar to patient
4

---

Under review as a conference paper at ICLR 2022
with degree of similarity Way. For this work, try two popular similarity metries namely, cosine similarity and Jaccard similarity, and find that cosine similarity has slightly better performance. We cosine similarity to compute Wa based patient information at their first visit, together with their age, weight, and gender information. Note that any similarity measure can be used.
Although GAT (Veličković et al. 2018) is a popular choice obtain node representation from graph, it represents nodes with limited expressivity (Brody et al., 2021). To mitigate these short- comings adapt GATv2 (Brody et al.) 2021) learn the node representation of the weighted patient similarity graph G. GATv2 has two layers with attention heads in the first layer, and attention head the second layer shown in Figure The attention weight between nodes n, and ng for the 6t attention head in the first layer is given by:
ajt = Wikh X GATv2(EE (n, mx])
(3)
EntEs, Wij X GATv2 E
[ng, [n.))
where || denotes concatenation, S, the set of nodes whose similarity with node j is non-zero, GATv20) is the graph attention network operation comprising single layer feed-forward neural network, followed by a LeakyReLU operation, E the embedding matrix of the 5th attention head, and Wikk denotes the edge weight representing the similarity between patients j and k.
The output from the first layer for node n, is given by:
oj = =11=10
Σ
jjk
E.
T8k
(4)
where a is a sigmoid function. With this, the output from the second layer for node ny can be obtained follows:
uj =
Σ ask ok nsES,
(5)
where ayk is calculated by using the average of the attention weight between node j and k from the first layer as new edge weights, i.e., Wykk is updated the average of a for S S 2, E is the embedding matrix for the second layer. Combining this u, with the diagnosis and medication representation of the most recent visit h - and h obtain the patient representation PT-1:
TT-1 h4 -1 + hT-11 Uj
3.2 KNOWLEDDGE AUGMENTATION
Next, we want to augment the patient representation obtained with drug-lab interactions and disease- lab interactions, as well as patients' past responses the lab test.
Lab Interaction Information, We obtain the drug- interactions from the AACC Effects on Clini- cal Laboratory Tests database) SIDER (Kuhn et al. 2016)). MEDI (Wei et al. 2013) and lisease-lab interactions from AACC Effects on Clinical Laboratory Tests database. Incorporating this informa- tion makes the system aware of the impact these interactions on the lab test values. Since the interactions can be positive (increases the lab test value) or negative (decreases the lab test value), we use two graphs to represent them. We have the positive lab interaction graph G = (X, YY+ and negative lab interaction graph G (X, Y "). Each node E X denotes either lab test, medica- tion or diagnosis. Each labelled edge (U; 9,) E Y denotes positive interaction between nodes U; and uy. while each labelled edge (v,, 0;) Y denotes negative interaction between nodes 2ú and Uy. Note that G+ and G are bipartite graphs where the lab tests constitute one type of nodes, and medications, diagnosis together constitute another type of nodes, in other words, there is no edge between nodes of the same type.
Once again, we use GATv2 with two layers and two attention heads in the first layer learn the node representations from the positive and negative lab interaction graphs as we have done for
5

---

Under review as a conference paper at ICLR 2022
Aspregatior
Iab Interaction Information
Response
GAIvz
Lanr2
Xy-a
Patlent
Figure 4: Details of the knowledge augmentation component.
the weighted patient similarity graph (see Figure 4 The node representations of G and G are mapped to low dimensional space by applying the embedding matrices E and E respectively the first layer for the 8th attention head where < b < 2 With this, we can obtain the node representations, and g, of the target lab test from G+ and G respectively, Then the lab interaction vector e is given by:
p Xq
where À regulates the fusion of positive and negative lab interaction information.
Past Lab Test Response Information. Since patients respond to treatments differently, incorporat- ing a patient's 's own past lab test responses will guide KALP to generate personalized lab responses. As such, we have repository of patients' past lab test responses stored in the form of key-value pairs where the key is the patient representation and the value the lab test response. Given the patient representation rT-1, compute the attention the previous representations r;, i stored in the repository as follows:
ôT-1 = softmaa((1 TT-1, T2 -1. TT-2 FT-1)
(8)
With this attention 8 E T-22, obtain the weighted past visit lab response fr-1 follows:
T-2
1--1= Σ 11] 1++1
where -10 depicts the ith entry the attention vector -T- and 4,+1 the lab test result at the (i 1)th visit. By concatenating TT-1, c, and fr-1, we obtain the final output vector which is then passed through a linear layer to predict the result of the target lab test:
fr w (rr-1 || c fr-1)
(10)
where is the gradient vector of the transformation function in the linear layer. Since the prediction of lab test result is regression task, our objective function is to minimize the square of the error defined as follows:
1
Cusse
(v 91)²
(11)
where T is the total number of visits, ii and are the predicted and ground-truth lab test result for the 7th visit respectively.
3.3 INFLUENTIAL FACTORS
The design of system allows to identify the factors that have influenced the predicted value fr the target lab test based on the attention scores of GAT. Let w", wº, and w/ be the sub- vectors of the linear transformation in Equation 10 corresponding to patient representation TT-1,

---

Under review as a conference paper at ICLR 2022
lab interaction c, and past lab test results fr-1 respectively. In other words, we have
w" || w" w
Then the influence of each diagnosis for the past visits given by:
1 = 1 [1] x (ur". E" k:) x (b)
(12)
where -1-1 is the average of attention weights across all the heads and layers of the encoders in the transformer for the diagmosis information for time T'' - 1, - is the jth entry in af- Ed is the embedding matrix for diagnosis, and d. [k] is the 1:th diagnosis in the 7th visit. Similarly, the influence of a patient's 's prescribed medication on the predicted lab test result is:
TM 1TTT [1] x (w" E. ki) x mk]k]
(13)
where a7-1 is the average of attention weights across all the heads and layers of the encoders in the transformer for the medication information for time T 1, -7-111] [t] is the 7th entry in af--1- E" the embedding matrix for medication, m; [k] is the 4 medication the jth visit. We obtain the influence of patient's ss similarity with other patients as follows:
n kIkEs, a5[k] x w" (E"
(14)
where a the average of attention weights across all heads and layers of GATv2 for node j. az [A]
is the bthh entry in a; Ex is the average of embedding matrix E over all heads for the nodes in the
weighted patient similarity graph, and n& is the one-hot vector of the node for the k patient. The
influence of both positive and negative lab interactions can be obtained as follows:
n IINES, of [k] x w (E" Us)
(15)
17" = IKES, A a?(k)] x w (E*... 0%)
(16)
where a" and az are the averages of attention weights across all heads and layers of the GATv2
for the lab interaction graphs G+ and G respectively for node j. af [k] is the w entry in af and
ay [K] is the tt entry in a). E. and E are the average of embedding matrices E++ and E over all heads in G+ and G respectively, and dy is the one-hot vector of the th node in the graphs. Finally, the influence of patient's 's past lab test responses is given by:
n' | w/
81-1(d x 4-11
(17)
We normalize these influences and rank them to obtain the top factors that could have influenced the prediction of the target lab test response. These scores need not necessarily mean explanabil- ity/interpretability in general context.
4 PERFORMANCE STUDY
In this section, we evaluate the effectiveness of our proposed KALP in predicting lab test results. KALI implemented in PyTorch and the models are trained on two NVIDIA Titan RTX GPU. We adopt the widely used metrics for regression namely, root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) in our evaluation (see Appendix A. We use the following datasets in our experiments:
MIMIC-III (Johnson et al. 2016). This is the largest publicly available EHR dataset which con- tains clinical data for 7870 neonates (infants) and 38,597 adults admitted to ICU between 2001 and 2008, and captures attributes such as lab reports, medications, etc.
PRIVATE. This is 10-year outpatient proprietary dataset. Compared to the inpatient MIMIC III, the number of diagnosis per patient is fewer in this dataset.
In our experiments, the target lab test is HbAlc. We filter out the patients who have less than two HbA le results. We divide the datasets into training, validation, and test sets in the ratio of 8:1:1. We report the results in the format μ + where μ and are the mean and standard deviation over the
TThe code will be available in Github.
7

---

Under review as a conference paper at ICLR 2022
Table Results for comparative study.
MIMIC-III
PRIVATE
Methods
RMSE
MAE
MAPE
RVSE
MAE
MAPE
PV
2.39 - 0.21
1,91 0.19
14.47 = 2.51
2.08 = 0.17
1.95 + 0.19
13.22 2.18
LR
2.91 + 0.25
2.52 0.22
18,26 + 2.76
2.38 + 0.21
2.05 0.200
2:.45
NN
2.61 0.20
2.18 0.21
16.58 2.57
2.11 0.19
1.17
14.48 2.33
RNN
2.37
1,88 0.19
14.04 - 2.21
2.21
2.04 0.17
1.72
12.53 + 2.13
DMNC
1.95
1.67 0.15
13.19 + 2.08
1.83 + 0.15
1.55 0.11
10.64 2.01
HiTANet
1,89 0.17
1.51 0.15
12.37 2.11
1.72 0.14
0.11
9.52 + 1.93
BEHRT 1.57 + 0.14
1,38 + 0.13
11.28 + 1.95
1,41 + 0.11
1.24 0.13
8.49 + 1.82
KALP
1,15 + 0.11*
0,80
0.10
6.87
+ 1.63*
0.85 + 0.09
0.69 + 0.07
3,42 + 1.47*
indicates that the result is statistically significant when compared to the second best with p-value < 0.05.
Table 2: Results when there are changes in medication dosage.
MIMIC-II
PRIVATE
Methods
RMSE
MAE
MAPE
RMSE
MAE
MAРE
PV
4.16 + 0.23
3.73 + 0.21
23.11 + 2.61
3.91 + 0.24
3.27 + 0.21
20.52 + 2.32
LR
4.56 1: 0.31
3.98 0.28
24.03 1: 2.89
4.05
3.43 0.26
21.04 2.67
NN
4.12 1.22 0.28
3.69 0.27
22.45 + 2.77
3.87 + 0.24
3.22 + 0.22
20.26 + 2.51
RNN
3.62 0.25
3.26 1 0.26
20.64 2.64
3.01 0.21
19.16 + 2.39
DMNC
3.18 0.22
2.76 0.23
19.01 1: 2.55
2.89 0.21
2.33 0.20
17.02 2.25
HiTANet
3.08 - 0.19
2.75 + 0.15
18.41 + 2.41
2.65 0.18
2.19 + 0.17
16.18 + 2.21
BEHRT
2.65 0.18
2.39 1 0.17
17.70
2.43
2.43
2.26 0.16
2.02 0.15
15.02 1. 2.23
KALP
1,85
0.13*
1.34 0.12
10.28
2.13*
1.27 0.10
1.10 0.11
8.18 + 1.96*
indicates that the result is statistically significant when compared to the second best with p-value < 0.05.
10-folds. Appendix A.2 gives the statisties of the resulting datasets, pre-processing steps, training and optimization details. Appendix A.3 shows the optimal parameters as determined by sensitivity experiments. We compare KALP with the following methods:
Previous Value (PV). The predicted lab response for the current visit is given by the previous visit.
Linear Regression (LR). This is the least square method based linear regression.
Nearest Neighbour (NN). The lab test response of the most similar patient is used.
RNN Kang 2018). This method employs gated recurrent neural networks to predict lab test results.
DMNC (e et al. 2018). The original DMNC only considers diagnosis and procedure. Here we adapt it to predict lab response taking into account demographics, medication, and diagnosis.
HiTANet (Luo et al. 2020). The original HiTANer Net uses only diagnosis to perform risk prediction. We adapt this to predict lab response by using demographics, medication, and diagnosis.
BEHRT (Li et al. 2020). This BERT-based technique originally takes as input diagnosis and demo- graphics, but we have adapted it to provide lab response prediction by using medication in addition to demographics and diagnosis.
Table shows that KALP outperforms all baselines on both MIMIC-III and PRIVATE. The one- way ANOVA (Fisher 1992) test shows that the improvements are statistically significant with p- values < 0.05. Compared
DMNC and BEHRT which model only sequential dependency of patient's visits over time, we see that our knowledge augmented approach dramatically widens the performance gap. This suggests that incorporating similar patients, lab interactions and past lab test responses is efffcttive in lowering the prediction errors. The ablation study in Appendix A.4 shows the effect of different information sources. Using different neural architectures show that KALP continues to give good performance over existing methods (see Appendix A.5).
Table 2 shows the performance of the various methods to predict lab test results after the clinician has prescribed change in the medication dosage. Here, cations have dosage changes between visits. Compared to Table T all the methods show an increase in the prediction errors. This is because the lab test result for patients with changes in medication
subset of the patients whose medi-
8

---

Under review as a conference paper at ICLR 2022
dosage may not follow the general trend in the EHR and would be difficult to predict their lab test results. We observe that KALP has the lowest prediction errors, demonstrating its upplicability in the real world where clinicians often titrate medication dosage for patient management.
CASE STUDY
Finally, we present a case study from PRIVATE demonstrate the performance of KALP and its ability to identify the top influencing factors that led to its lab result predictions. The top influencing factors can be obtained from the attention weights (recall Section 3.3
Figure 5 shows a chronic diabetic patient with HbAlc ranging between 5.4 and 6.5. KALP is able to predict the HbAle to an accuracy of 0.2 while the state-of-the-art BEHRT has a prediction error as large as 0.7 (Visit 4). On visit 3,
that KALP predicted the HbAlc to be 5.5 while BEHRT predicted it to be 5.1. The top influencing factor for this prediction is the past lab response. This highlights that taking into consideration patients' past responses helps in personalizing the prediction. Further for Visit 4, we again see that KALP is able to predict the rise in the HbA lc value within 0.1 of the ground truth value compared BEHRT. closer examination reveals that there is a reduction in the dosage of Glipizide on visit 3 and KALP has attributed the top influencing factor for this prediction be the reduction in dosage of Glipizide.
To the best of our knowledge, KALP is the only system that takes into account dosage information when making lab test predictions. In practice, clinicians often need to adjust medication treatment to manage patient care. With KALP's ability to model medication dosages, we have demonstrated here highly accurate prediction for the target lab test result. Appendix prolide additional case studies to demonstrate the importance of modeling other patient information such medication dosage, patients' past lab responses.
Diognoce
Elggrands Uhportonnton, Hypert pidemia,
Daggzonh
Pogertersien, Htpelipiderii, plabetea mellitua, Respiiatory inflctt
meditts
Dlabebes mellitun lhoedbeche
Olcbetes melliton, Heselachee
Diaberis resiition, Bespirahay Indectin
Medicction
Medictaiin
Medication
Meniicatoon
Mediccation
Metfornin
Glipiáde
Clipbrdde .B
Fenolibrute, Gliclaide, Mettornis Recavsratin
Arrlodipine, Diseprofod, Fenofibrati, dliduidss
Bround
Truth
KALP
5ct IT
Ground Iuihh
KALP
sennY
Ground Truth
KALF
scinT
Grmand
Iruth
KALP
BCHRT
Geound Truth
KALP
BEIKK
54
5.5
5.7
2,5
6.5
6.7
6.5
Infusmarg
inflionongg hactors
i-fuwnang
Influmrc faciors
Iifuuessn Factaxs
[Madiatioon (fiiiadd (58.27) DMedictiinn (lrllommi))
responoe, HBAtd (75, diippaddl (29.8))
[Medication, Ciipirat-8) (e5:26), Jiiquuniis Headarkel (58.75
[Mtudication Mutioomnr (90.24) (medication. Glidaaniee
Figure 5: Predicted and ground truth HbAlc. Influencing factors are obtained from KALP. Blue depicts lb values, red depicts diagnosis, and green depicts medication. I and f depict decrease and increase of dosage respectively.
6 CONCLUSION
In this work, we have described a personalized lab test result prediction approach that learns a strong patient representation incorporating both patient information accumulated over the visits as well as information from similar patients. This representation also captures fine-grained dosage information enabling us to adjust the prediction in response changes in treatment regime, which is often needed in the management of chronic patient care. To increase the prediction accuracy for patients with complex so-morbidities, we have augmented the patient representation with external knowledge on lab interactions and patients' historical responses to the target lab test. Experimental results on two real-world datasets demonstrate the effectiveness of KALP in providing predictions that are close to the actual lab results. Future work includes extending KALP to incorporate information on the severity and frequency of lab interactions. Further, KALP only uses the structured information in the EHR. However, EHR encompasses plethora of information in the form of doctors' notes, physiological signals, and medical images. One promising direction would be to jointly model these multi-modal sources in KAL.P.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Inci M Baytas, Cao Xiao, Xi Zhang, Fei Wang, Anil K Jain, and Jiayu Zhou. Patient subtyping via time-aware Istm networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 65-74, 2017.
Suman Bhoi, Mong Li Lee, Wynne Hsu, Hao Sen Andrew Fang, and Ngiap Chuan Tan. Person- alizing medication recommendation with graph-based approach. Transactions on Information Systems, I(I), 2021. doi: 10. 145/3488668.
Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks? arXiv preprint arXiv:2005.0.. 14491, 2021.
Edward Choi, Mohammad Taha Bahadori, Joshua A Kulas, Andy Schuetz, Walter F Stewart, and Jimeng Sun. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. Advances in Neural Information Processing Systems, pp- 3512-3522 2016.
Sajad Darabi, Mohammad Kachuee, Shayan Fazeli, and Majid Sarrafzadeh. Taper: Time-aware
patient ehr representation. IEEE journal of biomedical and health informatics, 24(1 1):3268- 3275, 2020.
Jacob Devlin, Ming- Ming-Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (1), 2019.
Anne Dornhorst, StephenH Powell, and Jack Pensky. Aggravation by propranolol of hypergly- caemic effect hydrochlorothiazide in type ii diabetics without alteration of insulin secretion. The Lancet, (2568821):12331111 1985.
Ronald Aylmer Fisher. Statistical methods for research workers. In Breakthroughs in statistics, pp- 66-70. Springer, 1992.
Manuel González-Ortiz, Esperanza Martinez-Abundis, José A Cobles-Cervantes, Maria G Ramos- Zavala, Carmelita Barrera-Durán, and Jorge González-Canudas, Effect of metformin glycinate on glycated hemoglobin alc concentration and insulin sensitivity in drug-naive adult patients with type 2 diabetes mellitus. Diabetes technology & therapeutics, 14(12): 140-11444 2012.
Jiawei Han, Micheline Kamber, and Jian Pei. Data mining concepts and techniques third edition. The Morgan Kaufmann Series Data Management Systems, 5(4):83-124, 2011.
Zheng Jia, Xian Zeng, Huilong Duan, Xudong Lu, and Haomin Li. A patient-simillarty--aass model for diagnostic prediction. International journal of medical informatics, 135:104073, 2020.
Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-Wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii, freely accessible critical care database. Scientific data, 3(1):1 9, 2016.
Seokho Kang. Personalized prediction of drug efficacy for diabetes treatment via patient-level quential modeling with neural networks. Artificial intelligence in medicine, 85:1-6, 2018.
Seokho Kang, Pilsung Kang, Tachoon Ko, Sungzoon Cho, Su-jin Rhee, and Kyung-Sang Yu. An ef- ficient and effective ensemble of support vector machines for anti-diabetic drug failure prediction. Expert Systems with Applications, 42(9):4265-42777 2015.
Seokho Kang, Sungzoon Cho, Su-jin Rhee, and Kyung-Sanp Yu. Reliable prediction of kunt--iabette drug failure using a reject option. Pattern Analysis and Applications, 20(3):883 891, 2017.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR, 2015.
Michael Kuhn, Ivica Letunic, Lars Juhl Jensen, and Peer Bork. The sider database of drugs and side effects. Nucleic acids research, 44(D1):
2016.
Hung Le, Truyen Tran, and Svetha Venkatesh. Dual memory neural computer for asynchronous two- view sequential learning. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1637-1645, 2018.
10

---

Under review as a conference paper at ICLR 2022
Yikuan Li, Shishir Rao, Jose Roberto Ayala Solares, Abdelaali Hassaine, Rema Ramakrishnan, Dexter Canoy, Yajie Zhu, Kazem Rahimi, and Gholamreza Shomi-Koorshiii. Behrt: transformer for electronic health records. Scientific reports, 10(1):1-12, 2020.
Chang Lu, Chandan K. Reddy, Prithwish Chakraborty, Samantha Kleinberg, and Yue Ning. Collab- orative graph learning with auxiliary text for temporal event prediction in healthcare. In Zhi-Hua Zhou (ed.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCA/ 2021, Viriual Event / Montreal, Canada, 19-27 August 2021, pp. 4529-3535, ijcai.org, pettps://doi.oeo www.doo..ooooooooooooooooooooo.o 0 24963 jiaii 2001/
2021. doi: 0..24963/jjcaii22221444 U httph:ttss///ttt
486
Junyu Luo, Muchao Ye, Cao Xiao, and Fenglong Ma. Hitanet: Hierarchical time-aware atten- tion networks for risk prediction electronic health records. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. pp. 647-656, 2020.
Yuan Luo, Peter Szolovits, Anand S Dighe, and Jason M Baron. Using machine learning to predict laboratory test results. American journal of clinical pathology, 145(6):778- 788, 2016.
Fenglong Ma. Radha Chitta, Jing Zhou, Quanzeng You, Tong Sun, and Jing Gao. Dipole: Di- agnosis prediction in healthcare via attention- based bidirectional recurrent neural networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 2017
Jingchao Ni, Hongliang Fei, Wei Fan, and Xiang Zhang. Cross-network clustering and cluster ranking for medical diagnosis. In 20/7 IEEE 33rd International Conference on Data Engineering (ICDE), pp. 163-166. IEEE, 2017.
Ronald Wihal Oei. Hao Sen Andrew Fang, Wei- Ying Tan, Wynne Hsu, Mong-Li Lee, and Ngiap-Chuan Tan. Using domain knowledge and data-driven insights for patient similarity analytics. Journal of Personalized Medicine, 11(8))699, Jul 2021. ISSN 2075-4426. doi: 10.3390/jpm 1080699.
PKS Prakash, Srinivas Chilukuri, Nikhil Ranade, and Shankar Viswanathan. Rarebert: Transformer architecture for rare disease patient identification using administrative claims. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 453-460, 2021.
National Glycohemoglobin Standardization Program. Factors that interfere with hbalc test results, 2013. UtL/sttp://www. ngsp wttg/fotooo..ooaaaeo...
Junyuan Shang, Cao Xiao, Tengfei Ma, Hongyan Li, and Jimeng Sun. Gamenet: Graph augmented memory networks for recommending medication combination. In proceedings of the AAAI Con- ference Artificial Intelligence, volume 33, pp. 1126-1133, 2019.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 191))11999-1988 2014.
Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. End-to-end memory networks Advances in Neural Information Processing Systems, 2015:2440-244 2015.
Qingxiong Tan, Andy Jinhua Ma, Mang Ye, Baoyao Yang, Huiqi Deng, Vincent Wai-Sun Wong, Yee-Kit Tse, Terry Cheuk-Fung Yip, Grace Lai- Lai-Hung Wong, Jessica Yuet-Ling Ching, et al. Ua- crnn: Uncertainty- aware convolutional recurrent neural network for mortality risk prediction. In Proceedings of the 28th ACM International Conference on Information and Knowledge Manage- ment, pp. 109-1 18, 2019.
Thimas, CE Cormen, RL. Leiserson, and RC Stein. Introduction to algorithms, 2009.
Ranjit Unnikrishnan, Ranjit Mohan Anjana, and Viswanathan Mohan. Drugs affecting hbalc levels.
Indian journal of endocrinology and metabolism, 16(4):528, 2012.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31st In- fernational Conference on Neural Information Processing Systems, pp. 6000-6010, 2017.

---

Under review as a conference paper at ICLR 2022
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. 6th International Conference on Learning Representations, ICLR, 2018.
Shanshan Wang, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Jun Ma, and Maarten de Rijke. Order- free medicine combination prediction with graph convolutional reinforcement learning. In Pro- ceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1632, 2019.
Wei-Qi Wei, Robert M Cronin, Hua Xu, Thomas Lasko, Lisa Bastarache, and Joshua C Denny. Development and evaluation of ensemble resource linking medications to their indications. Journal of the American Medical Informatics Association, 20(5):954-96) 961, 2013.
Lucas Wirbka, Walter E Haefeli, and Andreas Meid. framework to build similarity- cohorts for personalized treatment advice standardized, but flexible workflow with the package 2020. 2020.
simbaco. PloS one, 15()).e0233686,
12

---

either Under review as a conference paper at ICLR 2022
A
APPENDIX
A.1 EVALUATION METRICS
Suppose T the total number of visits, VI and yi are the predicted and ground-truth lab test results for the hh visit respectively. The formula of the evaluation metrics used are as follows:
T
RMSE
(3
0.)²
(18)
T
E=
T
MAE
91
(19)
T
100
1/1
MAPE
(20)
T
y1
A.2 DATASET STATISTICS, PRE-PROCESSING, TRAINING AND OPTIMIZATION
Table and Table give the statisties and the patient characteristics of the experiment datasets respectively.
Table 3: Characteristics of datasets.
Attribute
MIMIC-II
PRIVATE
Number of patients
6412
6312
Number of diagnosis
1919
139
Numher of medications
2941
45
Average number of visits per patient
2.67
13.06
Average number of diagnosis per visit
13.47
5.90
Average number medications per visit
32
4.07
Maximum number of diagnosis visit
39
27
Maximum number of medications per visit
148
12
For both MIMIC-III and PRIVATE use the ICD-9 coding system for diagnosis and the generic names for medications. Some of the important pre-processing steps are discussed next. Since the dosages reported in different units convert all the dosage values to correspond milligram or milliliter. For this study, we distinguish unique visits based on the unique hospital admission time (given by unique hospital ID in the datasets). Furthermore, in case of multiple dosage prescriptions of the same medicine and multiple lab test results within the same visit we consider their latest entries. We also filter out the patients who have less than two instances of the target lab test result.
The embedding size for our model is fixed 128 and training done using Adam (Kingma & Baj 2015). The learning rate is 0.0002, and the best performing model is chosen on the validation set after 50 epochs.
For MIMIC-III, we apply a dropout of 0.4 on the input embedding layer. The transformer encoder consists of four layers with 4 attention heads each. The graph attention network (GATv2) used to
Table 4: Patient characteristics of both datasets.
Variable
Type
MIMIC-II
PRIVATE
Age
Discrete
65.36 (13.27)
58.45 (15.67)
Gender
Categorical
4136 (M), 2276 (F)
3324 (M), 2988 (F)
Weight
Continuous
66.23 (15.25)
71.45 (14.12)
HbAlc
Continuous
9.2 (3.9)
10.5 (4.2)
13

---

Under review as a conference paper at ICLR 2022
model patient similarity consists of two layers with 2 attention head in the first layer and attention head in the second layer. The parameter value is set to be À = 0.49 based on the validation set. The learning rate is 0.0001, and the best performing model is chosen based the performance validation set after 50 epochs. The prrper-prramterss for all the baselines were chosen on the valida- tion set. For Recurrent Neural Network of 64. The transformer component of HiTANet has 2 layers, 6 attention heads, intermediate layer size of 1024, hidden size of 256 while other parameters remain the same as the original work. The time interval is set as 1 for HiTANet. BEHRT with an architecture of 4 layers, 6 attention heads, intermediate layer size of 512, and hidden size of 288 provided optimal performance. The word and memory size for DMNC model is 64 and 16 used in the original work itself.
the
GRU as the recurrent unit with hidden dimension
For PRIVATE, the dropout 0.6 and the transformer encoder consists of two layers with attention heads each. The graph attention network (GATv2) used to model patient similarity consists of two layers with 4 attention heads in the first layer and attention head in the second layer. The parameter value is A 0.35. The recurrent unit used for Recurrent Neural Network was GRU with a hidden dimension 64. The transformer component of HiTANet has layer, 2 attention heads, intermediate layer size of 1024, hidden size of 256 while other parameters remain the same as the original work. The time interval is set as for HiTANet. BEHRT with an architecture layers, 4 attention heads, intermediate layer size of 512, and hidden size of 288 provided optimal performance. The word and memory size for DMNC model is 64 and 32.
A.3 SENSITIVITY EXPERIMENTS
We examine the effect of embedding dimension for the inputs, number of layers and the number of attention heads in the transformer encoders on the performance of KALP.
Figure aaa) shows the results as vary the input embedding dimension from 8 to 256. We see that the smallest RMSE is achieved when the embedding dimension is 128 for both MIMIC-III and PRIVATE datasets.
Figure shows how KALP performs when we vary the number of transformer layers. We observe that having layers gives the best performance in MIMIC-III whereas layers is sufficient for the PRIVATE dataset. Figure ()) shows that the best results is obtained when the number of attention heads in the transformer for both the datasets.
We also vary the number of attention heads in the first layer of GATv2 used to model the patient similarity graph. Figure d)d) shows that having 2 attention heads gives the smallest RMSE on MIMIC-III dataset while having 4 attention heads leads to the best performance for the PRIVATE dataset.
A.4 ABLATION STUDY
We perform an ablation study to understand the effect of knowledge from different information sources on the performance of KALP. We implement the following variants of KALP:
KALP w/o medication. This variant uses all the information sources except the prescribed medications and their dosages.
KALP w/o similar patients. This variant does not include information from similar pa- tients in the patient representation.
KALP w/o lab interactions. This model does not use any information from the lab inter- actions. This depicts the model variant not using any external information.
KALP w/o positive lab interaction. Here, we do not the positive lab interaction information.
KALP w/o negation lab interaction. This model does not use the negative lab interaction information.
KALP w/o past lab test results. Here, we not use information from the patient's past lab test results.
KALP w/o knowledge augmentation. In this final variation, we do not use information from the patient's past lab test results as well as the lab interaction information.
14

---

Under review as a conference paper at ICLR 2022
MIMICHI
MMIC
1.6
1.4
1.6
12
BMASE
0.8
0.6
0.6
0.4
0.4
0.2
16
128
250
Embedding dimension
Number of layers
Dimension of input emhedding layer
Layers of transformer encoder
MIMIC
MMII-III
PRIVATE
144
E
RMASE
0.8
066
0.4
01
0.2
Aumber heads
Number heads
(c) Attention head of transformer encoder
(d) Attention head of GATv2
Figure Effect of embedding dimension and attention heads on PREMIER.
Table 5: Ablation Study
MIMIC-III
PRIVATE
Methods
RMSE
MAE
MAPE
RMSE
MAE
MAPE
KALP w/o medication
1.55
1.28
11.01
1.21
1.08
8.07
KALP w/o similar patients
1.49
1.22
8.39
1.15
1.04
7.75
KALP w/o lab interactions
1.35
1.13
8.24
1.11
0.91
7.50
KALP w/o positive lab interactions
1.22
7.55
0.91
0.79
6.66
KALP w/o negative lab interactions
1.28
1.02
7.63
0.99
0.87
7.43
KALP w/o past lab results
1.42
1.18
8.42
1.15
1.01
7.59
KALP knowledge augmentation
1.45
1.20
8.40
1.16
1.03
7.62
BEHRT
1.57
1.38
11,28
1.41
1.24
8.49
KALP
1.15
0.80
6.87
0.85
0.69
3.42
Table 5 shows the performance of these variants on MIMIC-III and PRIVATE datasets. We observe that the best performance achieved when all the information sources are utilized. Not using any medication or patient similarity information leads to the highest RMSE, MAE, and MAPE on both datasets. This indicates that having a patient representation that incorporates the patient's prescribed medications and information from similar patients can boost the accuracy of lab test result predic- tions. We observe that the performance of KALP without the knowledge augmentation component still outperforms the state-of-the-ee art technique, BEHRT suggesting that the patient representation learnt by KALP is indeed effective.
A.5 ARCHITECTURAL VARIANTS OF KALP
We also earry out experiments to understand the impact of architectural changes on the performance of KALP. The following variants of KALP are implemented:
KALPGAY- We graph attention network, GAT (Veličković al. 2018 to model the patient similarity and the lab interaction information. The rest of the architecture is the same that of K.I.P.
15

---

Under review as a conference paper at ICLR 2022
sparse. We sparsify the patient similarity graph by retaining edges whose similarity scores are above some threshold. Here we set the threshold to 0.95.
KALP
KALP We sparsify the patient similarity graph by using a minimum spanning tree, MST (Thimas et al., 2009) to minimize the inverse of the edge weights (i.e. similarity scores)
Table shows that KALP has the best performance compared to KALPGAT, KALP sparses and
KALPAN MST.
Table 6: Comparison of Architectural Variants of KALP.
MIMIC-III
PRIVATE
Methods
RMSE
MAE
MAPE
RMSE
MAE
MАPE
KALP GAT
1.35
1.11
7.21
1.09
0.91
5.17
KALP sporse
1.27
1.02
7.09
0.98
0.85
4.56
KALP MST
1.19
0.91
6.95
0.89
0.76
3.72
KALP
1.15
0.80
6.87
0.85
0.69
3.42
A.6 ADDITIONAL CASE STUDIES
Recall that PRIVATE is outpatient dataset with patients having chronic diseases and co- morbiditics. These patients have regular follow-up visits to monitor and manage their conditions. This case study demonstrates the importance of capturing medication dosage and patients' past lab response.
Figure 7 shows Patient A from the PRIVATE dataset who has HbAle test result of 11.4 during his first visit to the clinic and is diagnosed with Acute upper respiratory infection, Hyperlipidemia, Diabetes mellitus. We see that KALP is able to accurately predict the HbAlc values within 0. margin, whereas BEHRT's margin of error tends to be larger, particularly for the prediction at Visit This is because prior to Visit 5, there is downward titration of Metformin and the reduced dosage may lead to an increase in the HbAlc value. KALP is able to infer the rise in HbAlc values as it has captured the medication dosage in its patient representation, whereas BEHRT does not capture such dosage information. This is confirmed by the top influencing factors for Visit 5 where the top factor indicated as the reduced dosage in Metformin.
Vidi?
Vidl
VISE
VAG
Visit
Diagroob
Diasnoso
Dengrooss
Nagrosk
Dingsemi Hyperipptennt, Ulideetss rreiituss
Anate resplastory infection, typarlisieeni,
Diabetos mellitu
Hyporippdennts, Dlabetes mailttus
Ipperippttente Ufuhetts realitur
Hyporipide mia, Blahwite meliine
Ekobetis mellito
Mediaation
Medication
Mechoation
Llsaglipit,, ?
Lmsggtotnn,
Glickelde, Uiurelipttr, Matfrrnnn
diiduide
Sround
newer
Ground
Trui
IXL
no IBT
Ground
Ground
Sround Truth
EALP
Gisound Inch
KALL
BEHRT
8ir
Parton
Infuencing Twires
intorenting Bectors
Influeniing trp Eachnrs
Influmndre Fectors
Inllwecing Swetnrs
[Metikation Meclaomir)
Ieedication ciistrite JRn,7% Reegarsb, Eicbuttec mollinul (2,07)
[Mecccatios, Gaickssto (34.20) (Metllation, [ioopplttis00r.ag
Mettominnji (rideetde116.55
IModication Merfomirth peg, (Medd cation, chanson (22.7))
(71.55). IDlapoodis. piubelea (hiiitoo) (20.1))
Figure 7: Predicted and ground truth HbAle results for Patient A. Influencing factors obtained from KALP.
A.6.1 SAMPLES FROM MIMIC-III.
The patients in MIMIC-III are in ICU with acute conditions and complex co-morbidities. These case studies demonstrate the importance of taking into account knowledge of the lab interactions.
Figure shows a Patient B from the MIMIC-III with a HbAle value of 5.5 at his first visit and is diagnosed with multiple diseases such congestive heart failure, diabetes, hypertension, etc. He is
16

---

Under review as a conference paper at ICLR 2022
Diagnosis
Diagnosis
Disgnosts
Congestive heart failure, Disbetes mullitus, Hypertenaion
Heriiplegis, Diebetes me peeliitu congestive beatt faikere, Hypertension
Malignant neoplasm, Diabetes mellitus, Hypothyroldiom
Medication
Medication
Aspiris, Dillisnem, Vencommccn
Diltiarem, Heparin
Diitiazem Vancomycing
Ground
Truth
KALP
BEHRT
Ground Truth
KALP
BEHRT
Ground
Truth
KALP
DENRT
5.4
6.7
6.2
5.6
5.8
6.0
infuancing Factors
Influencing Factors
Influencing Factors
[Positive lab interaction, Dilliaaee)
[Medication, Aspitin] (61.63), [Positive lab interaction, Hypertension) (14.98)
(57.18).
[Positive lab interaction, Aspirin]
Figure 8: Predicted and ground truth HbAlc for Patient B. Influencing factors are obtained from KALP.
Visi
Visi
Vidt
Visit
Diagnosis
Diegnosis
Diagnosis
Diagnosis
Atrial libillation, Urinary tract infection, Hyper iiiieeaaa
Hypertension, Haematuria, Diabetes Mellitus
Atrial fibillltioo, Hypertension
Abrial libillattoo, Hypertension, Neoplasm cerebral meninges
Medication
Medication
Medication
Medication
teetammeth//mmmehpppee Atorvastatin, .., Werfarin, Diltiazem
Diltiazem_
Aopliin
Vancomyyln, Warfarin Usinopril
Lishoopill, ... Digoxin
Ground
KALP
Ground
BEHRT
6.0
9.8
9.0
6.4
6.6
Influencing Factors
Influencing Factors
Influancing Factors
Influencing Factors
[Positive
[Medication, Insulin] (32.65),
[Negative interaction, Warfaris] (44.64) [Positive leb interaction,
Atorvastatin] (45.33), [Negative leb interaction,
[Past lab response, HbA1:] (30.98)
Sulfemethy Sulfsmethy/rimettune /rimethoprii) (36.76)
[typertension] (20.52)
Figure 9: Predicted and ground truth HbAlc results for Patient C. Influencing factors are obtained from KALP.
prescribed with various medications including Asprin and Diltiazem. Both Diltiazem and Aspirin known increase the HbAle value. With the augmented lab interaction knowledge, KALP is able to accurately predict the HbAlc value within 0.1 of the actual lab result, and has correctly identified the top influencing factors as the positive lab interaction of Diltiazem and Aspirin on HbAlc values in Visit 2.
Figure g shows the details of Patient C and the HbAle predictions by KALP and BEHRT along with the influencing factors identified by KALP. We see that the patient has a HbAle le value of 6.0 Visit and diagnosed with urinary tract infection (UTT), atrial fibrillation, hyperlipidemia, etc. Medications like Trimethoprim, Atorvastatin, etc. prescribed this patient.
At Visit 2, KALP predicted an increase in the HbAle value and identified the top two influencing factors as the positive lab interaction of Atorvastatin and the negative lab interaction of Trimethoprim on HbAle le levels. However, we see that the actual HbAlc lc is 9.8 but KALP gives a a lower predicted value of 9.0. There two possible reasons for this. First. Timethoprim is prescribed to treat bacterial infection and the patient would have stopped taking this medication long before her second visit the ICU, thus removing the negative lab interaction of Timethoprim HIAlc. Second,
17

---

Under review as a conference paper at ICLR 2022
the reported interactions in the AACC, SIDER, MEDI datasets do not include the frequency of occurrence and the severity of the interaction, leading to less accurate prediction by KALP this case.
18


Reviews:
[{"review_id": "Yza4mQFlZyw", "reviewer": "Reviewer_MoEw", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n+ The proposed model outperforms other baselines by incorporating external knowledge including drug-lab and disease-lab interactions.\n+ The paper shows the case study for interpreting its prediction by analyzing attention weights to know influential factors. \n\nWeaknesses:\n- Although achieving better results than baseline methods for future lab test prediction, the contribution of this work is negligible in terms of technical perspective. In particular, Transformer and GAT are well-known architecture coming from previous works. The only clear contribution of this work is to utilize external knowledge such as patient similarity (some previous works did that) and drug-lab and disease-lab interactions.\n- There are some concerns about the experimental settings with respect to the baseline methods. \n-- The baseline methods are not competitive. There're many well-known models for EHR prediction such as Dipole, TLSTM, HiTaNet, etc., authors should adapt these models to their problem and compare them with their proposed models beside conventional models such as linear regression, k-nearest neighbor, and recurrent neural network. \n-- It's trivial that the proposed model achieves better results compared to baseline methods because it uses external information. Authors need to emphasize their claims by comparing their model with the baseline methods using this external information. It's trivial to incorporate this information, for example, using binary vectors to represent the positive and negative impacts of drugs and diseases to lab tests and concatenating these vectors to drug and disease representations.\n- The motivation for using patient similarity is not clear. In particular, the model encodes information from the patient similarity matrix constructed from their demographic information only without using lab test information from these patients. Thus, it is not clear how this information source can help to improve performance.\n- Minor comments:\n-- Some notations are used without any introduction (e.g., ni, nj, nk in equation (3); E in equation (5))\n-- EHR record is temporal data by nature. Authors should consider incorporating time information into their model.\n", "summary_of_the_paper": "This paper proposes a deep learning architecture for predicting future lab tests, in particular HbA1c based on EHR records. The proposed model leverages (1) attention mechanism (i.e., Transformer) to model sequence of diagnosis and medication, (2) graph attention network (GAT) to capture patient similarity, and drug-lab and disease-lab interactions. The experimental results show that this model outperforms other baselines on the two EHR datasets (i.e., MIMIC-III and private dataset) for lab test prediction.", "summary_of_the_review": "A paper with an interesting idea but with some problems", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "HKiZwHiKySn", "reviewer": "Reviewer_QtXC", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Patient data in EHRs is complex - longitudinal, irregularly sampled, and missingness is very often informative.  There have been very many prior works applying the deep neural net architecture of the day, most recently transformer-like encoders, to this sort of data, usually for problems such as predicting mortality at various timeframes.  This work is a bit of a departure from the typical paper in that it focuses on a somewhat unusual target - HbA1c - which is useful as a marker for metabolic disease such as diabetes.  The authors claim that this is useful because such the management of chronic conditions like metabolic disease often involve empirically adjusting doses of medications to achieve target lab test values.  In this review I will leave aside questions of whether this is truly a clinically useful task and focus on the novelty and effectiveness of the methods presented. \n\nTask formulation\nThe problem is set up as a regression problem (HbA1c lab test results are numeric - percentages), and formulated in an autoregressive manner - to predict a lab test result at time T, all patient information from time points 0,...,T-1 can be used.  The space in which this paper operates is: how best can do we distill patient information prior to prediction time T into a feature vector that allows us to estimate the lab test result.  Each patient has at least 2 visits, and two HbA1c lab test results.  Note that there is no prediction on the first visit.  \n\nNovelty\nThis work presents a method called KALP. With respect to the models used to encode patient data, the authors essentially take a sequential prediction approach using transformer-like models as encoders.  For reasons that are not entirely convincing, this work uses separate encoders for diagnoses and medications; the resulting embeddings are added together to get an embedding for each time point for each patient, $\\textbf{r}_1, \\dots, \\textbf{r}_{T-1}$. This is not new ground, and indeed the strongest baseline used in the work (BEHRT) is essentially the same thing (with a couple notable differences discussed shortly).  The paper does, however, add quite a bit of side information that is more interesting: \n\n1. Patient similarity - a fully connected graph whose vertices are patients is constructed; edges are weighted with a similarity, which is calculated as a cosine similarity between patient data on the first visit.  This graph is encoded using a GAT, and the resulting node representations are used to construct an additional embedding per patient that is added to the time-point specific embedding. The resulting embedding is added to $\\textbf{r}_t$.   \n2. Diagnosis/Medication-lab test information - this work also uses a GAT to encode a bipartite graph of diagnosis/medication-lab test effects, with edges having a direction (causes increase or causes decrease).  These edges do not denote quantitative information - just directional.  This results in an additional embedding $\\textbf{c}$. \n3. Past lab test results - previous lab test results are combined as a weighted sum, with weights derived from inner products of the patient representations $\\textbf{r}_t$ for each of the previous time steps.  This results in an additional input $\\textbf{f}_{T-1}$.  \n\nThe final representation is the concatenation $\\[\\textbf{r}_{T-1}, \\textbf{c}, \\textbf{f}_{T-1}\\]$.  This is passed through a linear layer, and the entire thing is trained jointly, with  squared error loss. \n\nThis method is applied to two datasets: MIMIC-3, and ICU dataset, and a proprietary outpatient dataset.  Compared to a range of baselines, this method  achieves the lowest error by a pretty large margin in ten-fold cross validation.  The appendix also presents a quite thorough ablation study in which various components of the model are taken out to assess their contribution to performance. The appendix also contains a lot of information about the tuning of the models and baselines; again, this work seems quite thorough.  Kudos to the authors!  \n\nOverall, I liked this work because it presented some (to me) relatively novel (in the EHR-predictions space) methods to a potentially useful task.  The methods are, for the most part, not new but I appreciate their efforts to incorporate side data to improve predictions a lot, as well as the clarity and thoroughness of the presentation. \n\nHowever, I find the evaluation lacking in a couple ways.  First, I would note that it's always a good idea to present a 'carry last value forward' baseline for this sort of task - it can be surprisingly hard to beat sometimes.  Second, and more seriously, I would note that the characterization of BEHRT, the strongest baseline, as originally presented (and as described in Related Works) uses only demographics and diagnoses.  BEHRT achieves an MSE in the MIMIC dataset of 1.57, higher than the 1.15 for KALP.  But in the ablation study presented in the appendix, KALP without medication information suffers a sharp degradation of performance - 1.55, which is pretty much the same as BEHRT, which as described does not have access to medication information.  Thus, in my mind, in order to convince the reader that the complexity of KALP actually matters very much, I would want to see a BEHRT baseline with medication information.  I can't see any particular reason this is technically infeasible.  The basic question for this paper is - does all that patient similarity, diagnosis/medication-lab test interactions, and past lab test results stuff really matter over a relatively simple baseline with access to some pretty critical information.  For me, this is the key question - without knowing this, I would not feel comfortable concluding that all that juicy side data is really helping very much!  \n\nOne final remark - in the discussion, the authors say, \"This allows clinicians to simulate the effect of different treatment regimes before finalizing the best treatment option for the patient\".  This is a very strong claim, and effectively says that KALP is getting good causal effect estimates from intrinsically observational data.  I strongly recommend backing off this claim because that is not at all supported by the work presented.  \n\n", "summary_of_the_paper": "This work presents a method for predicting HbA1c lab test results from electronic health record (EHR) data.  The method builds on lots of prior work that uses encoders to summarize longitudinal patient data, and adds some interesting elements.  Most notably, it incorporates various bits of domain specific knowledge about drug-lab test interactions (e.g., drug A tends to increase HbA1c lab measurements), patient similarity, and per-patient historical information about lab test results. The overall system is evaluated in MIMIC-3, comprising ICU stays, and a proprietary outpatient dataset, and seems to perform markedly better than baseline methods as measured by MSE, etc.  The authors present a pretty thorough ablation study, which has some interesting results. ", "summary_of_the_review": "This work is does not present any fundamentally new methods, but rather tries to show the value of using largely pre-existing methods to incorporate additional information to the task of predicting HbA1c lab test results.  That is, in my view, a perfectly fine kind of paper for ICLR.  But given the above question about how BEHRT (a baseline) fares with the addition of medication data, I am reluctant to recommend acceptance for this work because that is a key piece of evidence regarding whether the complexity of KALP is really necessary.  I would reconsider if this was added to the paper.  \n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1Okcf_G4dFx", "reviewer": "Reviewer_JWHn", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n- The proposed model KALP incorporates dosage information which usually neglected in the literature.\n- The proposed model outperforms SOTA signficantly on two real-world EHR datasets.\n\nCons: \n- The novelty of the proposed representation is rather limited. It is a combination of well-known models that have been previously applied in similar context. \n- The model capacity/size is not assessed vs. SOTA model sizes. It is unclear whether the choice of the representation or the model size is driving the prediction performance.\n- The choice of hyper-parameters in the model is not well justified: Why the factor $\\lambda$ for adjusting the fusion positive and negative lab interactions is set as a global hyper-parameters ?", "summary_of_the_paper": "The paper proposes a representation learning model from EHR data for the prediction of patients' lab test response.  The proposed model, named KALP (Knowledge-Augment Lab test Prediction), consists of two modules: 1) Patient Representation ($\\bold{r}_{T-1}$) where T is the number of visits): It is an aggregation of two transformers: The first (transformer encoder) for patient-specific data and the second (the modified graph attention GATv2) for modeling similar patients. The output of the patient representation is passed to 2) Knowledge Augmentation ($\\bold{c}$): It consists of two modules: a) Lab Interaction Module: GATv2 is used to extract a representation of drug-lab interactions and disease-lab interactions from two external domain knowledge databases. Separate graphs are defined for positive (increase of lab test value) and negative interactions. b) Past Lab Test Response Module ($\\bold{f}_{T-1}$): The patient representation previously obtained ($\\bold{r}_{T-1}$) is used to compute an attention representation from the different patient visits. The final representation is obtained as a concatenation of the three components $\\bold{r}_{T-1}$, $\\bold{c}$ and $\\bold{f}_{T-1}$. A last linear layer is used to fit the lab test result as a regression task. Attention weights and the last linear layer are used to define what is called influential factors as to make the model interpretable. The model is evaluated on two EHR datasets (MIMIC-III and PRIVATE). The obtained results in terms of regression metrics (RMSE, MAE and MAPE ) are better (statistically significant) than baselines and SOTA such as BEHRT, DMNC, RNN. Case studies are given to show how the model can be used to interpret the model output.", "summary_of_the_review": "While the novelty in terms of individual components of the proposed model rather is limited, the overall method seems to provide good improvement compared to SOTA on real-world datasets. \n\nAdditional comments:\n- Patient information includes  demographics information and the first visit. This is not well justified. The first visit does not necessarily represent a global/static view of patient state. Clarification on this point would be needed.\n-  PRIVATE dataset has a slightly smaller cohort (6321 from Table 3) contrary to what mentioned in the paper while MIMIC-III has 6412 patients.\nI was expecting that the performance for the inpatient dataset (MIMIC-III) would be much better compared to the outpatient dataset (PRIVATE).  Inpatient is a well controlled environment compared to outpatient. For example, mortality prediction in inpatient is highly accurate compared to outptient models. This is not the case in the experimental results of the paper. This needs clarifications\n- last line of page 8: We observe see  that -> we observe that\n-  Figure 5, the arrows (up and down) needs to be explained (negative vs. positive interaction) in the figure legend.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "0_ECNvDiG6t", "reviewer": "Reviewer_3iB5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Some of the stronger aspects of this paper are as follows:\n\n- Predicting individual patients' lab response can be of crucial importance in a clinical decision support system. The authors propose a method that incorporates both external knowledge as well as auto-correction based on patients' past lab response.\n- The presented results, in both the main paper and the appendix section, seems to provide some justification about the efficacy of the method via point estimates\n- The authors conducted several case studies (distributed over the main paper and the appendix) to uncover some of the important aspects of the patients\n\n\nThe paper can be improved upon by addressing the following aspects\n- One of the biggest shortcomings of the paper lies in the experimental settings. The authors fail to report confidence bounds around the point estimates. Without these numbers the significance of the results cannot be established\n- Furthermore, the proposed method seems to learn from a sequential setting. To properly evaluate such methods, one needs to make sure there are no data leakage across folds. Thus more information is needed around how the folds were created e.g. were they split based on the data points or unique patients. Were they validated in a temporal cross validation manner? \n- The authors provide some high level information about the datasets. However, a proper ``Table 1`` [1] for the data is missing - this is crucial to understand the data characteristics especially for the PRIVATE data to understand the importance of the results \n- The paper also seems doesn't analyze why and how the two datasets are useful dataset for this problem. For example, MIMIC-III mainly contains ICU data where data is collected at regular interval and are mostly dense. On the other hand, the authors mention that PRIVATE is an outpatient setting - is the data sparse and irregular in that case? Did the authors analyze this aspect? If so how did they normalize the different data streams that may be arriving at different interval (A setting which DMNC explicitly models)\n- The presented literature survey on this topic is also limited.  For example there has been both classical as well as more recent work on using intra- and inter- patient dependencies for similar class of problems [2] \n- Finally, the authors didn't mention whether they intend to publish their code\n\nSome other comments\n\n- This is an interesting insight from the paper `we observe that patients with similar age, weight, gender, diagnosis, etc. tend to have similar lab test responses` \n- In the introduction section and subsequently the authors claim that the attention values lends itself for easy interpretability. It is crucial to note that recent literature has debated this practice and the debate about \"whether attention is explanation\" and how \"attention can be adversarially affected\" is an active topic. While it is acceptable to use the attention method for this problem, the critique needs to be acknowledged. \n- The authors may consider describing in more details the effort and scale involved in encoding the domain knowledge beyond what was presented. e.g. was any expert involved in this process? The authors describe HBA1c as their target problem - can these methods scale to other populations?\n- The ablation analysis should be in main paper\n\n\n\n[1]: https://www.sciencedirect.com/science/article/abs/pii/S0895435618309867\n[2]: https://www.ijcai.org/proceedings/2021/0486.pdf\n", "summary_of_the_paper": "The paper proposes a novel architecture to predict an individual patients lab response and provide supporting evidence about predicted risk for use in a clinical decision support tool. The authors also claim to model the drug-lab interactions and diagnosis-lab interactions in the form of graphs and used this to create a knowledge augmented individual risk prediction model. The authors experimented on MIMIC-III dataset and another closed real dataset to analyze the efficacy of the solution. They also provided a case study report on the consistency of the top discovered factors with a clinician's understanding. ", "summary_of_the_review": "The paper targets an interesting problem and the provided results seem to support the efficacy of the model. However, the paper's experimental section needs to be improved upon to understand the importance of the results. The paper can be improved upon by providing a more thoughtful analysis of why the model works and where can such models be used.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper uses several types of information to predict one specific
lab test response for patients. The predictions are made by combining
and tailoring mainly existing techniques.

The reviewers raised a number of concerns, and the authors clarified
many of them and provided additional results. In particular the
following issues were discussed: Comparing to state-of-the-art methods
and methods having the same information available, specifics of
empirical evaluations and of methodological novelty, choice of the
particular data sets, and justifiability of conclusions.

The main remaining weakness is the limited novelty, which should not
be interpreted as the contributions of the paper being trivial.

In contrast, the solid engineering work done by the authors in this
paper will be valuable in developing clinical decision support tools,
and the authors are encouraged to incorporate the new results and
feedback in future work and submissions.
Decision: Reject