P ERSONALIZED L AB T EST R ESPONSE P REDICTION WITH K NOWLEDGE A UGMENTATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Personalized medical systems are rapidly gaining traction as opposed to “one size ﬁts all” systems.
Predicting patients’ lab test responses and providing justiﬁcation for the predictions would help clinicians tailor treatment regimes for patients.
This requires modeling the complex interactions among different medications, diseases, and lab tests.
We need to learn a strong patient representation that captures both the sequential information accumulated over the visits and information from other similar patients.
We model drug-lab interactions and diagnosis-lab interactions as graphs and design a knowledge-augmented approach to predict patients’ response for a target lab result.
We also take into consideration patients’ past lab responses to personalize the prediction.
Experiments on real-world datasets demonstrate the effectiveness of the proposed solution in reducing prediction errors by a signiﬁcant margin.
Case studies show that the identiﬁed factors for inﬂuencing the predicted results are consistent with clinicians’ understanding.
I NTRODUCTION
Electronic Health Records (EHR) provide a rich repository of patient related information such as disease diagnosis, lab test results, prescribed medications, etc.
over a history of visits.
Advances in machine learning have led to the large-scale analytics of EHR for disease inference (Ni et al., 2017), mortality prediction (Tan et al., 2019), personalized medication recommendation (Wang et al., 2019;
Shang et al., 2019; Bhoi et al., 2021).
At the same time, there is a growing trend towards using patient analytics for decision support (Oei et al., 2021) to improve patient care and clinical outcomes, particularly in chronic disease management such as hypertension and diabetes.
Often, these chronic disease patients are prescribed some standard treatment regime for each chronic condition independently and their conditions are monitored periodically based on lab test results such as HbA1c 1 .
However, for patients with co-morbidities, studies have shown that their lab test results are often inﬂuenced by other treatments related to their co-morbidities, making it hard to assess the effectiveness of the prescribed treatment (Unnikrishnan et al., 2012).
The ability to predict the target lab test result of a patient’s condition, taking into consideration medications that are prescribed for his/her other conditions, would enable the clinician to personalize the treatment regime for the patient.
This can help eliminate invasive procedures associated with sample collection for the lab test.
Existing research use patient speciﬁc information such as demographics and past visit records to predict lab test results (Luo et al., 2016; Kang, 2018).
The work in Luo et al.
(2016) predicts Ferritin lab test result by using patient demographics and the results of other lab tests that have been ordered at the same time as the Ferritin lab test.
Kang (2018) examines the task of predicting HbA1c test results and proposes a recurrent neural network (RNN) based architecture to model the sequential dependencies across the past visits.
These works do not consider the impact of medications on the target lab test result.
For example, patients with high blood pressure are prescribed medications like
Propranolol which is known to increase HbA1c (Dornhorst et al., 1985).
In practice, lab test results are often inﬂuenced by drugs and diagnosis.
However, these drug-lab interactions and diagnosis-lab interactions are largely ignored by current works on lab response predictions.
We observe that drug-lab interaction can be positive where the drug increases the lab test value, or negative where the lab test value is decreased by the drug, e.g., HbA1c is increased by Propranolol and decreased by Metformin (Dornhorst et al., 1985; Gonz´alez-Ortiz et al., 2012).
Similarly, diagnosis-lab interaction can be positive or negative, e.g., HbA1c is increased by iron deﬁciency anemia and decreased by hemolytic anemia (Program, 2013).
In this work, we use a transformer encoder (Vaswani et al., 2017) to capture the patient speciﬁc information, while the information of similar patients is modeled using the modiﬁed graph attention network (GATv2)(Brody et al., 2021).
We combine the outputs from both the transformer encoder and the GATv2 to obtain a strong latent patient representation so as to accurately predict patient response to a target lab test.
We incorporate ﬁne-grained dosage information to increase the accuracy of the predictions in the presence of medication titrations for chronic disease patient management.
This is in contrast to existing works which mainly consider medication type.
We augment the patient representation with the knowledge of drug-lab interactions, diagnosis-lab interactions, and use graph attention networks to model the positive and negative effects of drugs and diagnosis on the target lab result.
This enables us to model the complex relationships between drugs, co-morbidities, and lab test results.
We also take into consideration patients’ past lab responses to personalize the prediction.
Finally, we leverage the attention weights in the proposed solution to identify the top factors that may have inﬂuenced the predicted lab test results.
This allows clinicians to have insight into the underlying cause for any changes in the lab test result.
Extensive quantitative and qualitative experiments on the benchmark MIMIC-III (Johnson et al., 2016) EHR and a proprietary outpatient dataset demonstrate the effectiveness of the proposed system to signiﬁcantly lower the prediction errors by a large margin.
Case studies reveal the usefulness of identifying the factors that contributed to the predicted lab test results.
R ELATED W ORKS
Lab test prediction has been studied for the purpose of clinical decision support.
The work in (Luo et al., 2016) employs multiple machine learning algorithms to predict Ferritin test response from patient demographics and other accompanying lab test responses from EHR.
(Kang et al., 2015) uses an ensemble of support vector machines to predict HbA1c test results using patient demographics, prescribed medications, and the last known HbA1c value.
(Kang et al., 2017) proposes to use a heterogeneous ensemble of classiﬁers and reduces the prediction error by incorporating an option to reject an instance when the conﬁdence of any of the classiﬁers is low.
These methods rely on only the current patient visit information and ignore the sequential dependency of past patient visits.
(Kang, 2018) proposes a recurrent neural network based architecture to model the sequential dependencies across the visits for the task of prediction of HbA1c test results.
Given the black-box nature of
RNN, this approach is unable to identify the factors that contribute to the lab test result prediction.
In contrast, our solution is able to highlight these factors via attention based mechanism.
Learning a strong latent patient representation is key to the accurate prediction of lab test results.
The work in (Choi et al., 2016; Ma et al., 2017) use a recurrent neural network (RNN) and a bidirectional RNN respectively to create a patient representation for disease prediction by capturing the sequential dependency in medical codes over time.
Similarly, T-LSTM (Baytas et al., 2017) proposes time aware long-short term memory for patient subtyping.
DMNC (Le et al., 2018) learns the patient representation by infusing memory augmented neural networks (Sukhbaatar et al., 2015) with RNNs to handle long-range dependencies in patient visit history.
Recent works have demonstrated the superiority of transformer encoder representations over RNN based approaches in capturing the sequential dependency in EHR for disease prediction (Li et al., 2020), mortality prediction (Darabi et al., 2020), medication recommendation (Prakash et al., 2021).
HiTANet (Luo et al., 2020) introduces a time-aware transformer to obtain a patient representation using local and global attention for risk prediction.
BEHRT (Li et al., 2020) adapts Transformer (Vaswani et al., 2017; Devlin et al., 2019) based architecture to learn patient representation using patient’s diagnostic and demographic information to predict future disease occurrence.
All these works do not consider information from other similar patients and do not use ﬁne-grained dosage information.
Recently, (Lu et al., 2021) model patient similarity and incorporate clinical notes for the task of diagnosis and heart failure prediction.
This work uses the ontology of diseases along with the patient diagnosis information to jointly learn the representation for patients and diseases.
Our work differs from this in that our notion of patient similarity includes demographics, prescribed medications information in addition to disease diagnosis to predict lab test responses.
M ETHODOLOGY
Given a sequence of past patient visits, our task is to predict the result of a target lab test for the patient’s current visit.
Figure 1 gives an overview of the proposed solution called KALP for
Knowledge-Augment Lab test Prediction.
There are two key components: (a) learning an effective latent patient representation and (b) augmenting it with knowledge of drug-lab and disease-lab interactions, as well as patient’s historical responses to the lab test.
3. P ATIENT R EPRESENTATION
Our proposed patient representation takes into account both the sequential dependency in patient diagnosis and medication information, as well as information from similar patients.
Suppose a patient has T − 1 visits prior to his current visit.
We represent the i th visit of a patient as
[Equation]
where d i is a multi-hot vector depicting the diagnosis, m i is a vector of (medication, dosage) pairs, and l i is a continuous variable denoting the results of the target lab test, 1 ≤ i ≤ T − 1 .
We capture the dosage information in the form of a continuous vector where each entry contains the dosage information for the corresponding medication type.
All the continuous values are normalized using min-max normalization (Han et al., 2011).
Figure 2 illustrates the visit information extracted from a patient’s EHR.
Sequential Dependency.
We utilize two transformers, one for the diagnosis and the other for medication, to model the sequential dependency in the patients’ diagnosis and medication information over the visits.
Having dual transformers enable us to obtain representation for patients who may have missing diagnosis or medication information.
We linearly embed d i and m i , into a low dimensional space using embedding matrices E d and E m respectively.
These linear embeddings are then combined with positional embeddings (Vaswani et al., 2017) to obtain the embeddings for diagnosis d ei and medication m ei .
The embeddings d ei and m ei are passed to their respective transformers as shown in Figure 3.
Each transformer consists of multiple layers with each layer having T − 1 encoders.
Each encoder has a position-wise fully connected feed-forward network and a multi-head self-attention mechanism.
The position-wise feed-forward network has two linear transformations with a ReLU layer in between.
The multi-head attention employs scaled dot-product attention to obtain the weights on the patient visits given the visit history.
Since this multi-head attention can attend to future time steps, to ensure that the model’s predictions are only conditioned on past visits, we apply a triangular mask to the embedding d ei and m ei .
This mask is the same as the one used in the decoder component of (Vaswani et al., 2017).
The input to the i th encoder in the ﬁrst layer is the concatenation of d ej , 1 ≤ j ≤ i .
Inputs to the i th encoder in the subsequent layers is the concatenation of the outputs from the ﬁrst to the i th encoders of the previous layers.
Similar to (Vaswani et al., 2017), we employ residual connection around the self-attention mechanism and the feed-forward network is followed by layer normalization.
We also apply dropout (Srivastava et al., 2014) to avoid over-ﬁtting.
The outputs of the transformers for diagnosis and medications are given by:
[Equation]
where h di and h mi are the outputs of i
th encoder in the last layer of the respective transformers.
Since our goal is to predict the lab test result for the current visit T , we use the outputs h dT − 1 and h mT − 1 as the encoded sequential diagnosis and medication information.
Patient Similarity Information.
The work in (Jia et al., 2020) uses pair-wise patient similarity as a feature and extreme gradient boosting to improve the accuracy of diagnosis prediction.
(Wirbka et al., 2020) constructs cohorts of patients who have similar demographics, diagnosis and prescribed medications for treatment recommendation.
Here, we observe that patients with similar age, weight, gender, diagnosis, etc. tend to have similar lab test responses.
As such, we want to incorporate information from similar patients to learn a more effective patient representation that improves the prediction accuracy of our task.
We construct a weighted patient similarity graph G =
( V, E ) where each node n i ∈
V denotes a patient i , and each labelled edge ( n i , n j , w ij ) ∈
E denotes that patient
i is similar to patient j with a degree of similarity w ij .
For this work, we try two popular similarity metrics namely, cosine similarity and Jaccard similarity, and ﬁnd that cosine similarity has slightly better performance.
We use cosine similarity to compute w ij based on patient information at their ﬁrst visit, together with their age, weight, and gender information.
Note that any similarity measure can be used.
Although GAT (Veliˇckovi´c et al., 2018) is a popular choice to obtain node representation from a graph, it represents nodes with limited expressivity (Brody et al., 2021).
To mitigate these shortcomings we adapt GATv2 (Brody et al., 2021) to learn the node representation of the weighted patient similarity graph G .
GATv2 has two layers with z attention heads in the ﬁrst layer, andattention head in the second layer as shown in Figure 3.
The attention weight between nodes n j and n k for the b th attention head in the ﬁrst layer is given by:
[Equation]
where (cid:107) denotes concatenation, S j is the set of nodes whose similarity with node j is non-zero,
GATv2() is the graph attention network operation comprising a single layer feed-forward neural network, followed by a LeakyReLU operation, E b is the embedding matrix of the b th attention head, and w jk denotes the edge weight representing the similarity between patients j and k .
The output from the ﬁrst layer for a node n j is given by:
[Equation]
where σ is a sigmoid function.
With this, the output from the second layer for node n j can be obtained as follows:
[Equation]
(5) where α jk is calculated by using the average of the attention weight between node j and k from the ﬁrst layer as new edge weights, i.e., w jk is updated to the average of α bjk for 1 ≤ b ≤ z , E is the embedding matrix for the second layer.
Combining this u j with the diagnosis and medication representation of the most recent visit h dT − 1 and h mT − 1 , we obtain the patient representation r T − 1 :
[Equation]
3. K NOWLEDGE A UGMENTATION
Next, we want to augment the patient representation obtained with drug-lab interactions and diseaselab interactions, as well as patients’ past responses to the lab test.
Lab Interaction Information.
We obtain the drug-lab interactions from the AACC Effects on Clinical Laboratory Tests database 2 , SIDER (Kuhn et al., 2016), MEDI (Wei et al., 2013) and disease-lab interactions from AACC Effects on Clinical Laboratory Tests database .
Incorporating this information makes the system aware of the impact of these interactions on the lab test values.
Since the interactions can be positive (increases the lab test value) or negative (decreases the lab test value), we use two graphs to represent them.
We have the positive lab interaction graph G + =
( X, Y + ) and negative lab interaction graph G − =
( X, Y − ) .
Each node v ∈ X denotes either a lab test, medication or diagnosis.
Each labelled edge ( v i , v j ) ∈
Y + denotes positive interaction between nodes v i and v j , while each labelled edge ( v i , v
j ) ∈
Y − denotes negative interaction between nodes v i and v j .
Note that G + and G − are bipartite graphs where the lab tests constitute one type of nodes, and medications, diagnosis together constitute another type of nodes, in other words, there is no edge between nodes of the same type.
Once again, we use GATv2 with two layers and two attention heads in the ﬁrst layer to learn the node representations from the positive and negative lab interaction graphs as we have done for 2 https:/ the weighted patient similarity graph (see Figure 4).
The node representations of G + and G − are mapped to a low dimensional space by applying the embedding matrices E b + and E b − respectively in the ﬁrst layer for the b th attention head where 1 ≤ b ≤ 2 .
With this, we can obtain the node representations, p and q , of the target lab test from G + and G − respectively.
Then the lab interaction vector c is given by:
[Equation]
where λ regulates the fusion of positive and negative lab interaction information.
Past Lab Test Response Information.
Since patients respond to treatments differently, incorporating a patient’s own past lab test responses will guide KALP to generate personalized lab responses.
As such, we have a repository of patients’ past lab test responses stored in the form of key-value pairs where the key is the patient representation and the value is the lab test response.
Given the patient representation r T − 1 , we compute the attention on the previous representations r i , 1 ≤ i ≤ T − 2 , stored in the repository as follows:
[Equation]
With this attention δ T − 1 ∈ R T − 2 , we obtain the weighted past visit lab response f T − 1 as follows:
[Equation]
where δ T −
[ i ] depicts the i th entry in the attention vector δ T − 1 , and l i +1 is the lab test result at the ( i + 1) th visit.
By concatenating r T − 1 , c , and f T − 1 , we obtain the ﬁnal output vector which is then passed through a linear layer to predict the result of the target lab test:
[Equation]
where w is the gradient vector of the transformation function in the linear layer.
Since the prediction of lab test result is a regression task, our objective function is to minimize the square of the error deﬁned as follows:
[Equation]
where T is the total number of visits,
ˆ y i and y
i are the predicted and ground-truth lab test result for the i th visit respectively.
3. I NFLUENTIAL F ACTORS
The design of our system allows us to identify the factors that have inﬂuenced the predicted value ˆ y T of the target lab test based on the attention scores of GAT.
Let w r , w c , and w f be the subvectors of the linear transformation in Equation 10 corresponding to patient representation r T − 1 , lab interaction c , and past lab test results f T − 1 respectively.
In other words, we have
[Equation]
Then the inﬂuence of each diagnosis for the past visits is given by:
[Equation]
) where a dT − 1 is the average of attention weights across all the heads and layers of the encoders in the transformer for the diagnosis information for time T − 1 , a dT − 1 [ i ] is the i th entry in a dT − 1 , E d is the embedding matrix for diagnosis, and d i [ k ] is the k th diagnosis in the i th visit.
Similarly, the inﬂuence of a patient’s prescribed medication on the predicted lab test result is:
[Equation]
[Equation] where a mT − 1 is the average of attention weights across all the heads and layers of the encoders in the transformer for the medication information for time T − 1 , a mT − {+
+} [ i ] is the i th entry in a mT − 1 , E m is the embedding matrix for medication, m i
[ k ] is the k th medication in the i th visit.
We obtain the inﬂuence of a patient’s similarity with other patients as follows:
[Equation]
14) where a sj is the average of attention weights across all heads and layers of GATv2 for node j , a sj [ k ] is the k th entry in a sj , E s is the average of embedding matrix E b over all heads for the nodes in the weighted patient similarity graph, and n k is the one-hot vector of the node for the k th patient.
The inﬂuence of both positive and negative lab interactions can be obtained as follows:
[Equation]
[Equation] where a pj and a qj are the averages of attention weights across all heads and layers of the GATv for the lab interaction graphs G + and G − respectively for node j , a pj [ k ] is the k th entry in a pj and a qj [ k ] is the k th entry in a qj , E p and E q are the average of embedding matrices E b + and E b − over all heads in G + and G − respectively, and v k is the one-hot vector of the k th node in the graphs.
Finally, the inﬂuence of a patient’s past lab test responses is given by:
[Equation]
We normalize these inﬂuences and rank them to obtain the top factors that could have inﬂuenced the prediction of the target lab test response.
These scores need not necessarily mean explanability/interpretability in general context.
P ERFORMANCE S TUDY
In this section, we evaluate the effectiveness of our proposed KALP in predicting lab test results.
KALP 3 is implemented in PyTorch and the models are trained on two NVIDIA Titan RTX GPU.
We adopt the widely used metrics for regression namely, root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) in our evaluation (see Appendix A.1).
We use the following datasets in our experiments:
MIMIC-III (Johnson et al., 2016).
This is the largest publicly available EHR dataset which contains clinical data for 7870 neonates (infants) and 38,597 adults admitted to ICU between 2001 and 2008, and captures attributes such as lab reports, medications, etc.
PRIVATE.
This is a 10-year outpatient proprietary dataset. Compared to the inpatient MIMIC III, the number of diagnosis per patient is fewer in this dataset.
In our experiments, the target lab test is HbA1c.
We ﬁlter out the patients who have less than two
HbA1c results.
We divide the datasets into training, validation, and test sets in the ratio of 8:1:1.
We report the results in the format µ ± σ where µ and σ are the mean and standard deviation over the [Table] [Table] 10-folds.
Appendix A.2 gives the statistics of the resulting datasets, pre-processing steps, training and optimization details.
Appendix A.3 shows the optimal parameters as determined by sensitivity experiments.
We compare KALP with the following methods:
Previous Value (PV).
The predicted lab response for the current visit is given by the previous visit.
Linear Regression (LR).
This is the least square method based linear regression.
Nearest Neighbour (NN).
The lab test response of the most similar patient is used.
RNN (Kang, 2018).
This method employs gated recurrent neural networks to predict lab test results.
DMNC (Le et al., 2018).
The original DMNC only considers diagnosis and procedure.
Here we adapt it to predict lab response taking into account demographics, medication, and diagnosis.
HiTANet (Luo et al., 2020).
The original HiTANet uses only diagnosis to perform risk prediction. We adapt this to predict lab response by using demographics, medication, and diagnosis. BEHRT (Li et al., 2020). This BERT-based technique originally takes as input diagnosis and demographics, but we have adapted it to provide lab response prediction by using medication in addition to demographics and diagnosis.
Table 1 shows that KALP outperforms all baselines on both MIMIC-III and PRIVATE.
The oneway ANOVA (Fisher, 1992) test shows that the improvements are statistically signiﬁcant with pvalues < 0.05.
Compared to DMNC and BEHRT which model only sequential dependency of a patient’s visits over time, we see that our knowledge augmented approach dramatically widens the performance gap.
This suggests that incorporating similar patients, lab interactions and past lab test responses is effective in lowering the prediction errors.
The ablation study in Appendix A.4 shows the effect of different information sources.
Using different neural architectures show that KALP continues to give good performance over existing methods (see Appendix A.5).
Table 2 shows the performance of the various methods to predict lab test results after the clinician has prescribed a change in the medication dosage.
Here, we use a subset of the patients whose medications have dosage changes between visits.
Compared to Table 1, all the methods show an increase in the prediction errors.
This is because the lab test result for patients with changes in medication dosage may not follow the general trend in the EHR and would be difﬁcult to predict their lab test results.
We observe that KALP has the lowest prediction errors, demonstrating its applicability in the real world where clinicians often titrate medication dosage for patient management.
C ASE S TUDY
Finally, we present a case study from PRIVATE to demonstrate the performance of KALP and its ability to identify the top inﬂuencing factors that led to its lab result predictions.
The top inﬂuencing factors can be obtained from the attention weights (recall Section 3.3).
Figure 5 shows a chronic diabetic patient with HbA1c ranging between 5.4 and 6.5.
KALP is able to predict the HbA1c to an accuracy of 0.2 while the state-of-the-art BEHRT has a prediction error as large as 0.7 (Visit 4).
On visit 3, we see that KALP predicted the HbA1c to be 5.5 while
BEHRT predicted it to be 5.1.
The top inﬂuencing factor for this prediction is the past lab response.
This highlights that taking into consideration patients’ past responses helps in personalizing the prediction.
Further for Visit 4, we again see that KALP is able to predict the rise in the HbA1c value within 0.1 of the ground truth value compared to BEHRT.
A closer examination reveals that there is a reduction in the dosage of Glipizide on visit 3 and KALP has attributed the top inﬂuencing factor for this prediction to be the reduction in dosage of Glipizide.
To the best of our knowledge, KALP is the only system that takes into account dosage information when making lab test predictions.
In practice, clinicians often need to adjust medication treatment to manage patient care.
With KALP’s ability to model medication dosages, we have demonstrated here a highly accurate prediction for the target lab test result.
Appendix A.6 provides additional case studies to demonstrate the importance of modeling other patient information such as medication dosage, patients’ past lab responses.
C ONCLUSION
In this work, we have described a personalized lab test result prediction approach that learns a strong patient representation incorporating both patient information accumulated over the visits as well as information from similar patients.
This representation also captures ﬁne-grained dosage information enabling us to adjust the prediction in response to changes in treatment regime, which is often needed in the management of chronic patient care.
To increase the prediction accuracy for patients with complex co-morbidities, we have augmented the patient representation with external knowledge on lab interactions and patients’ historical responses to the target lab test.
Experimental results on two real-world datasets demonstrate the effectiveness of KALP in providing predictions that are close to the actual lab results.
Future work includes extending KALP to incorporate information on the severity and frequency of lab interactions.
Further, KALP only uses the structured information in the EHR.
However, EHR encompasses a plethora of information in the form of doctors’ notes, physiological signals, and medical images.
One promising direction would be to jointly model these multi-modal sources in KALP.
A A PPENDIX A. E VALUATION M ETRICS
Suppose T is the total number of visits,
ˆ y i
and y i are the predicted and ground-truth lab test results for the i th visit respectively.
The formula of the evaluation metrics used are as follows:
RMSE = (cid:118)(cid:117)(cid:117)(cid:116)
T −
T (cid:88) i =
( y i − ˆ y i )(18)
MAE =
T −
T (cid:88) i = | y
i − ˆ y i | (19)
MAPE = 100 T −
T (cid:88) i = (cid:12)(cid:12)(cid:12) (cid:12) y
i − ˆ
y i
y
i (cid:12)(cid:12)(cid:12) (cid:12) (20)
A. D ATASET S TATISTICS , P RE - PROCESSING , T RAINING AND O PTIMIZATION
Table 3 and Table 4 give the statistics and the patient characteristics of the experiment datasets respectively.
[Table]
For both MIMIC-III and PRIVATE we use the ICD-9 coding system for diagnosis and the generic names for medications.
Some of the important pre-processing steps are discussed next.
Since the dosages are reported in different units we convert all the dosage values to correspond to either milligram or milliliter.
For this study, we distinguish unique visits based on the unique hospital admission time (given by unique hospital ID in the datasets).
Furthermore, in case of multiple dosage prescriptions of the same medicine and multiple lab test results within the same visit we consider their latest entries.
We also ﬁlter out the patients who have less than two instances of the target lab test result.
The embedding size for our model is ﬁxed at 128 and training is done using Adam (Kingma & Ba, 2015).
The learning rate is 0.0002, and the best performing model is chosen on the validation set after 50 epochs.
For MIMIC-III, we apply a dropout of 0.4 on the input embedding layer.
The transformer encoder consists of four layers with 4 attention heads each.
The graph attention network (GATv2) used to [Table] model patient similarity consists of two layers with 2 attention head in the ﬁrst layer and 1 attention head in the second layer.
The parameter value is set to be λ = 0 .
49 based on the validation set.
The learning rate is 0.0001, and the best performing model is chosen based on the performance on the validation set after 50 epochs.
The hyper-parameters for all the baselines were chosen on the validation set.
For Recurrent Neural Network we use GRU as the recurrent unit with a hidden dimension of 64.
The transformer component of HiTANet has 2 layers, 6 attention heads, intermediate layer size of 1024, hidden size of 256 while other parameters remain the same as the original work.
The time interval is set as 1 for HiTANet.
BEHRT with an architecture of 4 layers, 6 attention heads, intermediate layer size of 512, and hidden size of 288 provided optimal performance.
The word and memory size for DMNC model is 64 and 16 as used in the original work itself.
For PRIVATE, the dropout is 0.6 and the transformer encoder consists of two layers with 4 attention heads each.
The graph attention network (GATv2) used to model patient similarity consists of two layers with 4 attention heads in the ﬁrst layer and 1 attention head in the second layer.
The parameter value is λ = 0 .
The recurrent unit used for Recurrent Neural Network was GRU with a hidden dimension of 64.
The transformer component of HiTANet has 1 layer, 2 attention heads, intermediate layer size of 1024, hidden size of 256 while other parameters remain the same as the original work.
The time interval is set as 1 for HiTANet.
BEHRT with an architecture oflayers, 4 attention heads, intermediate layer size of 512, and hidden size of 288 provided optimal performance.
The word and memory size for DMNC model is 64 and 32.
A. S ENSITIVITY E XPERIMENTS
We examine the effect of embedding dimension for the inputs, number of layers and the number of attention heads in the transformer encoders on the performance of KALP.
Figure 6(a) shows the results as we vary the input embedding dimension from 8 to 256.
We see that the smallest RMSE is achieved when the embedding dimension is 128 for both MIMIC-III and
PRIVATE datasets.
Figure 6(b) shows how KALP performs when we vary the number of transformer layers.
We observe that having 4 layers gives the best performance in MIMIC-III whereas 2 layers is sufﬁcient for the
PRIVATE dataset.
Figure 6(c) shows that the best results is obtained when the number of attention heads in the transformer is 4 for both the datasets.
We also vary the number of attention heads in the ﬁrst layer of GATv2 used to model the patient similarity graph.
Figure 6(d) shows that having 2 attention heads gives the smallest RMSE on
MIMIC-III dataset while having 4 attention heads leads to the best performance for the PRIVATE dataset.
A. A BLATION S TUDY
We perform an ablation study to understand the effect of knowledge from different information sources on the performance of KALP.
We implement the following variants of KALP:
• KALP w/o medication.
This variant uses all the information sources except the prescribed medications and their dosages.
• KALP w/o similar patients.
This variant does not include information from similar patients in the patient representation.
• KALP w/o lab interactions.
This model does not use any information from the lab interactions.
This depicts the model variant not using any external information.
• KALP w/o positive lab interaction.
Here, we do not use the positive lab interaction information.
• KALP w/o negation lab interaction.
This model does not use the negative lab interaction information.
• KALP w/o past lab test results.
Here, we do not use information from the patient’s past lab test results.
• KALP w/o knowledge augmentation.
In this ﬁnal variation, we do not use information from the patient’s past lab test results as well as the lab interaction information. [Figure]
[Table] Table 5 shows the performance of these variants on MIMIC-III and PRIVATE datasets.
We observe that the best performance is achieved when all the information sources are utilized.
Not using any medication or patient similarity information leads to the highest RMSE, MAE [Table] and MAPE on both datasets.
This indicates that having a patient representation that incorporates the patient’s prescribed medications and information from similar patients can boost the accuracy of lab test result predictions.
We observe that the performance of KALP without the knowledge augmentation component still outperforms the state-of-the-art technique, BEHRT suggesting that the patient representation learnt by KALP is indeed effective.
A. A RCHITECTURAL V ARIANTS OF KALP
We also carry out experiments to understand the impact of architectural changes on the performance of KALP.
The following variants of KALP are implemented:
• KALP GAT .
We use graph attention network, GAT (Veliˇckovi´c et al., 2018) to model the patient similarity and the lab interaction information.
The rest of the architecture is the same as that of KALP.
• KALP sparse .
We sparsify the patient similarity graph by retaining edges whose similarity scores are above some threshold.
Here we set the threshold to 0.95.
• KALP MST .
We sparsify the patient similarity graph by using a minimum spanning tree, MST (Thimas et al., 2009) to minimize the inverse of the edge weights (i.e. similarity scores) .
Table 6 shows that KALP has the best performance compared to KALP GAT , KALP sparse , and
KALP MST .
[Table]
A. A DDITIONAL C ASE S TUDIES
Recall that PRIVATE is an outpatient dataset with patients having chronic diseases and comorbidities.
These patients have regular follow-up visits to monitor and manage their conditions.
This case study demonstrates the importance of capturing medication dosage and patients’ past lab response.
Figure 7 shows a Patient A from the PRIVATE dataset who has a HbA1c test result of 11.4 during his ﬁrst visit to the clinic and is diagnosed with Acute upper respiratory infection, Hyperlipidemia,
Diabetes mellitus.
We see that KALP is able to accurately predict the HbA1c values within a 0.
margin, whereas BEHRT’s margin of error tends to be larger, particularly for the prediction at Visit 5.
This is because prior to Visit 5, there is a downward titration of Metformin and the reduced dosage may lead to an increase in the HbA1c value.
KALP is able to infer the rise in HbA1c values as it has captured the medication dosage in its patient representation, whereas BEHRT does not capture such dosage information.
This is conﬁrmed by the top inﬂuencing factors for Visit 5 where the top factor is indicated as the reduced dosage in Metformin.
A.6. S AMPLES FROM MIMIC-III.
The patients in MIMIC-III are in ICU with acute conditions and complex co-morbidities.
These case studies demonstrate the importance of taking into account knowledge of the lab interactions.
Figure 8 shows a Patient B from the MIMIC-III with a HbA1c value of 5.5 at his ﬁrst visit and is diagnosed with multiple diseases such as congestive heart failure, diabetes, hypertension, etc.
He is prescribed with various medications including Asprin and Diltiazem.
Both Diltiazem and Aspirin are known to increase the HbA1c value.
With the augmented lab interaction knowledge, KALP is able to accurately predict the HbA1c value within 0.1 of the actual lab result, and has correctly identiﬁed the top inﬂuencing factors as the positive lab interaction of Diltiazem and Aspirin on
HbA1c values in Visit 2.
Figure 9 shows the details of Patient C and the HbA1c predictions by KALP and BEHRT along with the inﬂuencing factors identiﬁed by KALP.
We see that the patient has a HbA1c value of 6.
at Visit 1 and is diagnosed with urinary tract infection (UTI), atrial ﬁbrillation, hyperlipidemia, etc.
Medications like Trimethoprim, Atorvastatin, etc. are prescribed to this patient.
At Visit 2, KALP predicted an increase in the HbA1c value and identiﬁed the top two inﬂuencing factors as the positive lab interaction of Atorvastatin and the negative lab interaction of Trimethoprim on HbA1c levels.
However, we see that the actual HbA1c is 9.8 but KALP gives a a lower predicted value of 9.0.
There are two possible reasons for this.
First, Timethoprim is prescribed to treat bacterial infection and the patient would have stopped taking this medication long before her second visit to the ICU, thus removing the negative lab interaction of Timethoprim on HbA1c.
Second, the reported interactions in the AACC, SIDER, MEDI datasets do not include the frequency of occurrence and the severity of the interaction, leading to a less accurate prediction by KALP in this case.