Multi-task learning (MTL) [2] is a method to train a model, or multiple models jointly for multiple tasks to obtain improved generalization, by sharing knowledge among them. One of the most critical problems in multi-task learning is the problem known as negative transfer, where unreliable knowledge from other tasks adversely affects the target task. This negative transfer could be critical for safety-critical applications such as clinical risk prediction, where we cannot risk losing performance on any of the tasks. To prevent negative transfer, researchers have sought ways to allow knowledge transfer only among closely related tasks, by either identifying the task groups or learning optimal sharing structures among tasks [6, 22] . However, it is not only the task relatedness that matters, but also the relative reliability of the task-specific knowledge. Recent asymmetric multi-task learning (AMTL) models [17, 18] tackle this challenge by allowing tasks with low loss to transfer more. While the asymmetric knowledge transfer between tasks is useful, it does not fully exploit the asymmetry in the case of time-series analysis, which has an additional dimension of the time axis.
With time-series data, knowledge transfer direction may need to be different depending on the timestep. For instance, suppose that we predict infection and mortality for patients in intensive care units based on their medical records. At earlier timesteps, prediction of Infection may be more reliable than Mortality, where we may want knowledge transfer to happen from task Infection to Mortality; at later timesteps, we may want the opposite situation to happen. Moreover, knowledge transfer may happen across timesteps. For example, a high risk of Infection in early timestep will alert high risk of Mortality at later timesteps. To exploit such temporal relationships between tasks, we need a model that does not perform static knowledge transfer between two tasks ( Figure 1a ), but transfers knowledge across the timesteps in two different tasks, while dynamically changing the knowledge transfer amount and direction at each timestep ( Figure 1b) . To this end, we propose a multi-task learning framework for time-series data, where each task not only learns its own latent features at each timestep but also leverages aggregated latent features from the other tasks at the same or different timesteps via attention allocation ( Figure 2 ).
Yet this brings in another challenge. On what basis should we promote asymmetric knowledge transfer? For asymmetric knowledge transfer between tasks, we could use task loss as a proxy of knowledge reliability [17, 18] . However, loss is not a direct measure of reliability, as loss might not be available at every step for time-series prediction. Also, a model trained with few instances (Task 1 in Figure 1c -1e) may have a small loss and thus transfer more knowledge to other tasks (Figure 1c ), but the knowledge from this model could be highly biased and unreliable as it may have overfitted ( Figure 1d ). Thus, we propose a novel probabilistic Bayesian framework for asymmetric knowledge transfer, which leverages feature-level uncertainty, instead of task loss, to measure the reliability of the knowledge (Figure 1b) . Basically, if a latent feature learned at a certain timestep has large uncertainty, our model will allocate small attention values for the feature; that is, the attention will be attenuated based on the uncertainty, where knowledge transfers from the task with low uncertainty to high uncertainty ( Figure 1e ) (Please see Section 4.1 and Table 1 ).
We experimentally validate our Temporal Probabilistic Asymmetric Multi-Task Learning (TP-AMTL) model on four clinical risk prediction datasets against multiple baselines. The results show that our model obtains significant improvements over strong multi-task learning baselines ( Table 2, 3) : up to 4.56% on average (Table 2 ) over the best-performing baseline [10] , with no negative transfer on any of the tasks. We further show that both the asymmetric knowledge transfer between tasks at two different timesteps and the uncertainty-based attenuation of attention weights are effective in improving generalization. Finally, with the actual knowledge transfer graph plotted with uncertainty obtained for each timestep, we could interpret the model behaviors according to actual clinical events in clinical risk prediction tasks ( Figure 3 , Figure 15 , 16, 17) . This interpretability makes it more suitable for clinical risk prediction in real-world situations.
Our contribution in this work is threefold:
• We propose a novel probabilistic Bayesian formulation for asymmetric knowledge transfer, where the amount of knowledge transfer depends on the uncertainty at the feature level.
• We extend the framework to an asymmetric multi-task learning framework for timeseries analysis, which utilizes feature-level uncertainty to perform knowledge transfer among tasks and across time-steps, thereby exploiting both the task-relatedness and temporal dependencies.
• We validate our model on four clinical risk prediction tasks, on which it significantly outperforms with no negative transfer. With the help of clinicians, we further analyze the learned knowledge transfer graph to discover meaningful relationships between clinical events.
Multi-task Learning While the literature on multi-task learning [2, 1] is vast, we selectively mention the prior works that are closely related to ours. Historically, multi-task learning models have focused on what to share [34, 35, ?] , as the jointly learned models could share instances, parameters, or features [13, 16, 20] . With deep learning, multi-task learning can be implemented rather straightforwardly by making multiple tasks to share the same deep network. However, since solving different tasks will require diversified knowledge, complete sharing of the underlying network may be suboptimal and brings in a problem known as negative transfer, where certain tasks are negatively affected by knowledge sharing. To prevent this, researchers are exploring more effective knowledge sharing structures. Soft parameter sharing method with regularizer [6] can enforce the network parameters for each task to be similar, while a method to learn the optimal combination of shared and task-specific representations is also proposed [22] in computer vision. Losses can be weighed based on the uncertainty of the task in a multi-task framework [15] , reducing negative transfer from uncertain tasks. While finding a good sharing structure can alleviate negative transfer, negative transfer will persist if we perform symmetric knowledge transfer among tasks. To resolve this symmetry issue, the asymmetric MTL model with inter-task knowledge transfer [17] was proposed, which allows task-specific parameters for tasks with smaller loss to transfer more. [18] proposed a model for asymmetric task-to-feature transfer that allows reconstructing features with task-specific features while considering their loss, which is more suitable for deep neural networks and scalable.
Clinical time-series analysis While our method is generic and applicable to any time-series prediction task, we mainly focus on clinical time-series analysis in this paper. Multiple publicly available benchmark datasets [5, 12] have been disclosed for the benchmark tasks. Also, several recent works have proposed clinical prediction benchmarks with publicly available datasets [3, 9, 11, 24, 25] . We construct our datasets and tasks specific to our problem set (Section 4.2), in part referring to previous benchmark tasks. Furthermore, there has been some progress on this topic recently, mostly focusing on the interpretability and reliability of the model. An attention-based model [4] that generates attention for both the timesteps (hospital visits) and features (medical examination results) was proposed to provide interpretations of the predictions. However, attentions are often unreliable since they are learned in a weakly-supervised manner, and a probabilistic attention mechanism [10] was also proposed to obtain reliable interpretation and prediction, that considers uncertainty as to how to trust the input. Our work shares the motivation with these prior works as we target interpretability and reliability. Recently proposed model for multi-task time-series prediction, which is with Transformer architecture [29] , is susceptible to negative transfer as all tasks share a single base network.
In this section, we describe our framework of probabilistic asymmetric multi-task learning (P-AMTL) in a general setting. Suppose that we have D tasks with datasets {X d , Y d } D d=1 , in which the sets X 1 , X 2 , ..., X D can be identical, overlapping or even disjoint. We further suppose that we have D different probabilistic networks {p d (.)} D d=1 , each of which generates high-level latent features of task d (task-specific) via Z d ∼ p d (X d ). In a single-task learning setting, these latent features Z d are in turn used to make predictions for task d. However, in our asymmetric multi-task learning framework, we want to borrow some learned features from other tasks to share knowledge and to improve generalization performance. Specifically, in order to perform prediction for task d, we leverage latent features learned from other tasks, Z j,d ∼ p j (X d ), ∀j = d. Given the source features Z j,d and the target features Z d , the model needs to decide on the following:
1) The amount of knowledge to transfer Existing asymmetric multi-task learning models [17, 18] often use task loss to decide on the amount of knowledge transfer, in a way that tasks with low training loss are allowed to transfer more, while tasks with high loss only receive knowledge from Figure 2 : Temporal probabilistic asymmetric knowledge transfer. This illustrates how we apply the probabilistic asymmetric knowledge transfer between tasks and across time-series tasks, at the same timestep and at different timesteps. (Right) Features of task j at timestep 1 is more reliable than features of task d at timestep 1, so the model will learn to transfer more from task j to task d and transfer less from task d to task j. other tasks. Yet, the task loss may be unreliable as a measure of the knowledge from the task and may not be available in some cases (Figure 1c , cases in Section 1). To overcome these limitations, we propose to learn the amount of knowledge transfer based on the feature-level uncertainty. Our model learns the transfer weight 2) The form of transferred knowledge Since the learned features for different tasks may have completely different representations, directly adding α j,d Z j,d to Z d would be sub-optimal. For this combining process, we train two additional networks G 1 k and G 2 k for each task k where G 1 k is used to convert the learned task-specific features from p k (.) to a shared latent space and G 2 k is used to convert the features from that shared latent space back to the task-specific latent space. Finally, we can compute the combined feature map for task d as (Figure 2 (Right)):
The combined feature map C d can be used for the final prediction of task d: this is also applied to the prediction of other tasks in the same manner.
We now apply our probabilistic asymmetric multi-task learning framework for the task of time-series prediction. Our goal is to jointly train time-series prediction models for multiple tasks at once.
Suppose that we are given training data for D tasks, D = {(X 1 , Y 1 ), . . . , (X D , Y D )}. Further suppose that each data instance x where x ∈ X d for some task d, consists of T timesteps. That is,
, where x (t) ∈ R m denotes the data instance for the timestep t. Here we assume the number of timesteps T is identical across all tasks for simplicity, but there is no such restriction in our model. Additionally, y d is the label for task d; y d ∈ {0, 1} for binary classification tasks, and y d ∈ R for regression tasks. Given time-series data and tasks, we want to learn the task-specific latent features for each task and timestep, and then perform asymmetric knowledge transfer between them. Our framework is comprised of the following components:
Shared Low-Level Layers We allow our model to share low-level layers for all tasks in order to learn a common data representation before learning task-specific features. At the lowest layer, we have a shared linear data embedding layer to embed the data instance for each timestep into a continuous shared feature space. Given a time-series data instance x, we first linearly transform the data point for each timestep t, x (t) ∈ R m , which contains m variables:
where W emb ∈ R m×k and k is the hidden dimension. This embedded input is then fed into shared RNN layer r = (r (1) , r (2) , ..., r (T ) ) = RN N (v (1) , v (2) , ..., v (T ) ) for pre-processing. Figure 2 ) After embedding and pre-processing the input into a continuous space, we further encode them into task-and timestepspecific features. Since hard-sharing layers may result in negative transfer between tasks, we allow separate embedding layers for each task to encode task-specific knowledge in our 'Base Network'. For each task d, the separate network consists of L feed-forward layers, which learn disentangled knowledge for each timestep. These L feed-forward layers for task embedding can be formulated as:
where W i d ∈ R k×k , b i d ∈ R k and σ is a non-linear activation function (e.g. leaky relu).
Modeling feature-level uncertainty While the above embedding can capture knowledge for each task and timestep, we want to further model their uncertainties as well, to measure the reliability of the knowledge captured. Towards this objective, we model the latent variables as probabilistic random variables, with two types of uncertainty [14] : 1) epistemic uncertainty, which comes from the model's unreliability from the lack of training data, and 2) aleatoric uncertainty, that comes from the inherent ambiguity in the data. We capture the former by using dropout variational inference [7] , and the latter by explicitly learning the model variance as a function of the input ( Figure 2 ). Suppose that we have the task-specific latent features:
where ω is the set of all parameters. This can be formulated as
. As mentioned before, we use dropout approximation [7] with parameter M as the variational distribution q M (ω) to approximate p(ω|D).
Asymmetric knowledge transfer across tasks and time steps Now we apply the proposed probabilistic asymmetric knowledge transfer method to perform knowledge transfer across timesteps, both within each task and across tasks, to exploit intra-and inter-task temporal dependencies. In order to transfer knowledge from task j to task d with temporal dependencies, we allow the latent features of task d at time step t (f
) to obtain knowledge from task j at all previous time steps (Figure 2 ), and then combine them into a single feature map C
As mentioned in the previous subsection, the transfer weight α
j (again, we perform MC sampling to get the variance). Here, we choose to constrain the knowledge transfer to happen only from past to future timesteps because of the time complexity at inference time. With our proposed model, for each update at the clinical environment in an online manner, we only need to transfer the knowledge from previous time steps to the current one, making the complexity to be O(T). This is on a par with other models like RETAIN [4] or UA [10] , making it highly scalable. However, if we allow the knowledge to transfer from future timestep to past timestep, we also need to update the knowledge at previous timesteps for a single update. The time complexity of the model in this case is O(T 2 ), which is undesirable. In the ablation study (Section B in Appendix), we show that this constraint also brings in a small performance gain. The total complexity of the whole training or inference is still O(T 2 ) due to the inter-timestep transfer, but this is on par with state-of-the-art models such as Transformer [30] .
Finally, we use the combined features C
d , which contain temporal dependencies among tasks, for prediction for each task d. We use an attention mechanism:
Then the model can perform prediction as follows, 
We first validate the effectiveness of the Uncertainty (UC) -based Knowledge Transfer (KT) for asymmetric multi-task learning, using a non-temporal version of our model. We use a variant of the MNIST dataset which contains images of handwritten digits 0-9 with random rotations and background noise. From this dataset, we construct 5 tasks; each task is a binary classification of determining whether the given image belongs to class 0-4. We sample 5000, 5000, 1000, 1000, and 500 examples for task 0, 1, 2, 3, and 4 respectively, such that asymmetric knowledge transfer becomes essential to obtain good performance on all tasks. As for the base network, we use a multi-layer perceptron, which outputs mean and variance of the task-specific latent features. We use the following baselines for comparison: 3) AMTL-Loss. A model that is similar to 4), with the transfer weight from task j to task d learned by a network F j,d with the average task loss over all instances as the input. 4) P-AMTL. Our probabilistic asymmetric MTL model.
Results (Table 1) shows that MTL outperforms STL, but suffers from negative transfer (Task 4, highlighted in red (Table 1) ). AMTL-Loss underperforms MTL, which shows that the loss is not a good measure of reliability; a model that is overfitted to a task will have a small loss, but its knowledge may be unreliable (Figure1c) . Finally, our model outperforms all baselines without any sign of negative transfer, demonstrating the superiority of uncertainty-based knowledge transfer.
We further validate our full model on multiple clinical risk prediction tasks against relevant baselines.
We experiment on four datasets that we compile for clinical risk prediction from two open-source electronic health records (EHR) datasets.
We compile a dataset out of the MIMIC III dataset [12] , which contains the record of 53,423 distinct hospital admissions between 2001 and 2012 to the intensive care unit (ICU) of a hospital. We use records of patients over the age 15, where we hourly sample to construct 48 timesteps from the first 48 hours of admission. Following clinician's guidelines, we select 12 infection-related variables for the features at each timestep. Tasks considered for this dataset are the 2) PhysioNet [5] . Total of 4,000 ICU admission records were included, each containing 48 hours of records (sampled hourly) and 31 physiological signs, including variables displayed in Table 4 . Task used in the experiment includes four binary classification tasks, namely, 1) Stay<3: whether the patient would stay in ICU for less than three days, 2) Cardiac: whether the patient is recovering from cardiac surgery, 3) Recovery: whether the patient is staying in Surgical ICU to recover from surgery, and 4) Mortality prediction (Mortality). We use a random split of 2800/400/800 for training/validation/test.
Baselines of our model are as follows (1-4) STL baselines, 5-11) MTL baselines). 1) STL-LSTM. Long short-term memory network.
2) STL-Transformer. Similar to 1), but with Transformer [30] as the base network.
3) STL-UA [10] . Uncertainty-Aware probabilistic attention model. 4) STL-RETAIN [4] . The attentional RNN for interpretability of clinical prediction with EHR. 5) MTL-LSTM. The naive hard-sharing multi-task learning method where all tasks share the same network except for the separate output layers for prediction, with LSTM as the base network. 6) MTL-Transformer, 7) MTL-UA, 8) MTL-RETAIN Same as 5), but with Transformer [30] , UA [10] , RETAIN [4] as the base network, respectively. 9) AMTL-LSTM [17] . This learns the knowledge transfer graph between task-specific parameters shared across all timesteps with static KT between tasks based on the task loss (Figre 1a). 10) MTL-RETAIN-Kendall [15] . The UC-based loss-weighing scheme with base MTL-RETAIN. 11) TP-AMTL. Our probabilistic temporal AMTL model that performs both intra-and inter-task KT.
We first evaluate the prediction accuracy of the baseline STL and MTL models and ours on the four clinical time-series datasets, by measuring the Area Under the ROC curve (AUROC) (MIMIC-III Infection ( Table 2 ) and PhysioNet (Table 3) ). We observe that hard-sharing MTL models outperform STL on some tasks, but suffers from performance degeneration on others (highlighted in red in Table 2 and 3), which shows a clear sign of negative transfer. MTL models especially work poorly on MIMIC-III infection, which has clear temporal relationships between tasks. Probabilistic models (e.g., UA) generally outperform their deterministic counterparts (e.g., RETAIN). However, MTL-RETAIN-Kendall, which learns the weight for each task loss based on UC, significantly underperforms even the STL-LSTM, which may be due to the fact that losses in our settings are at almost similar scale unlike with the task losses in [15] that have largely different scales. AMTL-LSTM improves on some tasks, but degenerates the performance on the others, which we attribute to the fact that it does not consider inter-timestep transfer. On the other hand, our model, TP-AMTL, obtains significant improvements over all STL and MTL baselines on both datasets. It also does not show performance degeneration on any of the tasks, suggesting that it has successfully dealt away with negative transfer in multi-task learning with time-series prediction models.
To further analyze the relationships between UC and KT, we visualize KT from multiple sources (Figure 3a ) normalized over the number of targets, and to multiple targets (Figure 3b ) normalized over the number of sources, along with their uncertainties. Specifically, the UC of a task at a certain timestep is represented by the average of the variance of all feature distributions. The normalized amount of KT from task j at time step t to task d is computed as (α (t,t)
Similarly, the normalized amount of KT to task d at time step t from task j is (α (1,t) j,d +α
We observe that source features with low uncertainties transfer knowledge more, while at the target, features with high uncertainties receive more KT. However, note that they are not perfectly correlated, since the amount of KT is also affected by the pairwise similarities between the source and the target features as well.
With the help of a physician, we further analyze how transfer weights and uncertainties are related with the patient's actual medical conditions (see Table 4 and Figure 4 ). We first consider an example record of a patient from the MIMIC-III Infection dataset who was suspected of infection on admission, and initially had fever, which was confirmed to be the symptom of bacterial infection later. Figure 4a shows the amount of KT from task Fever at 3:00 to all later timesteps of task Infection. At this timestep, the patient's condition changes significantly. We observe that the patient had a fever, and the WBC level has increased to the state of leukocytosis, and both the SBP and DBP decrease over time.
Most importantly, the patient is diagnosed to have an infection, as the bacterial culture test result turns out to be positive at 2:57. With the drop of UC of the task Infection around the time window where the event happens (dotted arrow in Figure 4a ), the amount of KT from Fever to Infection drops as well, as the knowledge from the source task becomes less useful.
As for another case study, we consider a record of a patient from PhyisoNet dataset who recovered from cardiac surgery and passed away during admission (Table 4 and Figure 4b ) for mortality Table 4 , we observe that sign of multi-organ failure develops, as features related to respiratory (F iO 2 , HCO − 3 , Lactate), renal (BUN, Creatinine), and cardiac (DBP) function deteriorates. As patient's condition after surgery gets worse, UC of Cardiac starts to decrease at later timesteps (dotted arrow in Figure 4b ) and KT from Cardiac to Mortality increases as the UC of the source task Cardiac starts to drop, since the knowledge from the source task becomes more reliable. As suggested in this interpretation, we can identify timesteps, where the meaningful interactions occur between tasks happens, by analyzing the learned knowledge graph from our model.
We propose a novel probabilistic asymmetric multi-task learning framework that allows asymmetric knowledge transfer between tasks at different timesteps, based on the uncertainty. While existing asymmetric multi-task learning methods consider asymmetric relationships between tasks as fixed, the task relationship may change at different timesteps in time-series data. Moreover, knowledge obtained for a task at a specific timestep could be useful for other tasks in later timesteps. Thus, to model the varying direction of knowledge transfer and across-timestep knowledge transfer, we propose a novel probabilistic multi-task learning framework that performs knowledge transfer based on the uncertainty of the latent representations for each task and timestep. We validate our model on clinical time-series prediction tasks on four datasets, on which our model shows strong performance over the baseline symmetric and asymmetric multi-task learning models, without any sign of negative transfer. Several case studies with learned knowledge graphs show that our model is interpretable, providing useful and reliable information on model predictions.
In this paper, we present Temporal Probabilistic Asymmetric Multi-Task Learning (TPAMTL) model, which is appropriate for various multi-task time-series predictions with strong performance while preventing negative transfer. Our model is especially applicable to safety-critical clinical risk predictions, as it provides a reliable explanation of its final prediction on each task at each timestep. Here in this section, we introduce the clinical and social impact of our model.
With the actual knowledge transfer graph plotted with uncertainty obtained at each timestep, healthcare professionals could interpret the behaviors of model according to actual clinical events in multiple clinical risk prediction tasks ( Figure  4, 3, 15, 16, 17) . By checking the knowledge transfer graph and the amount of uncertainty measured for each task at each timestep, our model can provide medical professionals with a better explanation of the underlying model behavior. Based on this explanation, medical experts can have a clue on the direction of knowledge transfer, or say interaction, between tasks, which might help them decide future action: whether to consult other departments for medical opinion, to run a further test to confirm the predicted event, or to come up with better treatment plan. This interpretability of our model will be useful in building a safe time-series analysis system for large-scale settings where both the number of time-series data instances and timestep are extremely large, such that manual analysis is impractical.
Monitoring clinical events of critical ill patient Due to the unprecedented pandemic outbreak of coronavirus infection this year, hospitals worldwide are suffering from the lack of healthcare professionals and facilities to monitor and care for patients [33] . In this era of difficulty, machine learning models can aid the diagnosis [19, 31] , drug repositioning [27] , predicting patient condition, and epidemiological trends of disease [21] . Our model can be applied in this setting as the healthcare resource is scarce, and the patient population proliferates. Therefore, monitoring multiple comorbidities and events of COVID-19 patients would be one example of the large-scale setting mentioned before. Currently, the number of newly diagnosed patients is increasing daily [23] , and there is a need to follow-up on multiple outcomes (pneumonia [32] , cardiac disease related to myocardial injury [8] , etc.) of this infection, also in relation to underlying disease each patient had, where no pathophysiological and case information is available. In this situation, when the number of patients exceeds the capacity of the healthcare system, a prediction model with a better capacity of handling large-scale data can help reduce the burden of healthcare staff. Not only the acute clinical conditions (e.g., infection) but also the chronic clinical conditions, such as conditions and abnormalities related to heart failure (Task MIMIC-III Heart Failure) or respiratory failure (Task MIMIC-III Respiratory Failure), can be monitored with our model. This is because our model provides long-term prediction without degeneration of performance, still providing reliable information.
The dataset used in this experiment is a subset of the MIMIC III dataset [12] , which contains tasks with clear temporal dependencies between them. We use the records for the first 48 hours after admission for each patient to only consider the patient condition on admission, as infection occurring after 48 hours is more likely to be acquired at the Intensive Care Units (ICUs). However, our method can be further applied to time-series prediction tasks with longer time steps. Following guidelines of clinicians, we select 15 infection-related variables including Heart rate, Systolic Blood Pressure, Diastolic Blood Pressure, Glasgow Coma Scale (GCS) -Verbal, Motor, Eye, invasive procedures: this includes endoscopic procedure, intubation, dialysis, chest tube placement, Lumbar drainage, and biopsy etc., serum albumin and total protein which represents nutritional status of patients, and intravenous steroids, for the features at each timestep (see Table 11 ). Figure 5 ). When a patient in ICU is infected with any of the pathogens, body temperature elevates (Fever (Task 1)) as a sign of infection. Next, Infection (Task 2), is confirmed when the blood culture result turns out to be positive with bacteria, fungus, or virus. Lastly, Mortality (Task 3) can be resulted from infection.
PhysioNet dataset includes three tasks that are temporally correlated to the last task, Mortality ( Figure  6 ). For instance, Stay<3 (Task 1) is reversely related to Mortality (Task 4), as the patient staying for less than three days are less likely to die. The other two tasks, namely Cardiac (Task 2) and Recovery (Task 3) are temporally correlated with task Mortality as patients who have undergone cardiac or any other surgeries are more likely to have critical events that might lead to mortality. We select 29 physiological signs as listed in this section: age, gender, height, weight, Systolic Blood Pressure, Diastolic Blood Pressure, mean arterial pressure, heart rate, respiratory rate, body temperature, glucose, bilirubin, serum electrolytes (sodium, potassium, magnesium, bicarbonate), lactate, pH, Hematocrit, platelets, Partial Pressure of Oxygen (PaO 2 ), Partial Pressure of carbon dioxide (PaCO 2 ), Oxygen Saturation (SaO 2 ), Fraction of Inspired Oxygen (FiO 2 ), Glasgow Coma Scale (GCS), blood urea nitrogen (BUN), Creatinine, Urine, Mechanical Ventilation Status.
To test the generalized performance of our model, we compiled two additional datasets, namely Heart Failure and Respiratory Failure, out of the MIMIC III dataset [12] .
From the MIMIC III dataset, we collected 3, 577 data instances, each of which corresponds to the record of a patient (age between 18 and 100) admitted to the ICU of a hospital. This dataset contains 15 features which are associated to the risk of heart failure occurrence, including Heart rate (HR), Systolic Blood Pressure (SBP), Diastolic Blood Pressure (DBP), Body Temperature (BT), Fraction of inspired oxygen (FiO 2 ), Mixed venous oxygen saturation (M v O 2 ), Oxygen Saturation of arterial blood (S a O 2 ), Brain natriuretic peptide (BNP), Ejection Fraction (EF), Glasgow Coma scale (GCS) -Verbal, Motor, Eye. We considered four tasks that might lead to heart failure. The first task (Ischemic) is the patient condition where a patient is diagnosed with ischemic heart disease. The second task (Valvular) is related to the diagnosis of valvular heart disease, and the third task (Heart Failure) contains the condition where a patient is diagnosed with various types of heart failure. Lastly, Mortality (Task 4) can be a possible outcome of heart failure. We used random split of approximately 1850/925/925 instances for train, valid, test set. ICD code for the diagnosis of each disease is summarized in Table 13 and Table 14 summarizes all feature information used to construct this dataset. 
We collected total of 37, 818 distinct ICU admissions of adult patients, between the age of 18 and 100. To further test the generalization performance of our model to larger dataset, we test the model performance on three partial datasets. First we run the experiment with full admission instances sampled for 48 hours after admission (Table 7) , full 37, 818 instances of data sampled for 48 hours ( Table 8 ). The disease and clinical event label, ICD code and item ID are summarized in Table 15 . This dataset contains 29 features, which are predictive of respiratory failure occurrence, including Fraction of inspired oxygen (FiO 2 ), Oxygen Saturation of arterial blood (S a O 2 ) and venous blood (S v O 2 ), Partial Pressure of oxygen in alveoli (P A O 2 ), Arterial/Venous oxygen content (C a O 2 ), which is summarized in Table 16 . Tasks considered in this task includes four respiratory conditions that are required for the prediction of respiratory failure (Task 5) and mortality (Task 7) (Table 15) : Hypoxemia (Task1, P a O 2 < 60mmHg), Hypercapnia (Task2, P a CO 2 > 50mmHg), VQ Mismatch (Task 3, A a DO 2 > 10), Acidosis (Task 4, pH < 7.35), Respiratory Failure (Task 5), Cyanosis (Task 6, S a O 2 < 60mmHg). The collected dataset was randomly split into 26, 472/5, 672/5, 674 for training/validation/test in full dataset, and 2, 800/600/600 for total 4, 000 instances.
Here we describe the baselines with more details. 
The single-task learning method which uses RNNs to capture the temporal dependencies.
(v
The single-task learning method which uses Transformer architecture [30] to capture the temporal dependencies:
where P OS_EN C is the positional encoding used in Transformer, T RAN S_BLOCK is also the architecture used in the paper, which consists of two sublayers: M U LT I_HEAD (with four heads) and F F W . We also used residual connection and layer norm after each sublayer as in the original paper.
The naive hard-sharing multi-task learning method where all tasks share the same network except for the separate output layers for prediction, whose base network is Long Short-Term Memory Network (LSTM).
(v (1) , v (2) , ..., v (T ) ) = v = xW emb ∈ R T ×k (11) (h (1) , h (2) , ..., h (T ) ) = LST M (v (1) , v (2) , ..., v (T ) ) (12) The same as MTL-LSTM, but with Transformer [30] as the base network.
where P OS_EN C is the positional encoding used in Transformer, T RAN S_BLOCK is also the architechture used in the paper, which consists of 2 sublayers: M U LT I_HEAD (with four heads) and F F W . We also used residual connection and layer norm after each sublayer as the original paper. The same as MTL-LSTM, but with RETAIN [4] as the base network. Specifically, after getting the shared context vector c i , separated output layers will be applied to form the prediction for each task. The same as MTL-LSTM, but with UA [10] as the base network. Specifically, after getting the shared context vector c i , separated output layers will be applied to form the prediction for each task. This can be seen as the probabilistic version of MTL-RETAIN. This is asymmetric multi-task learning [17] adopted for our time-series prediction framework, where we learn the knowledge transfer graph between task-specific parameters, which is learned to perform asymmetric knowledge transfer based on the task loss. The parameters for each task are shared across all timesteps, which will result in static asymmetric transfer between tasks.
This model is similar to MTL-RETAIN. However, we followed the uncertainty-aware loss weighting scheme from [15] to weight the loss for each task by its uncertainty:
A.2.8 TP-AMTL.
Our probabilistic temporal asymmetric multi-task learning model that performs both intra-task and inter-task knowledge transfer.
The probabilistic AMTL model with uncertainty-aware knowledge transfer, but performs knowledge transfer only within the same task at the transfer layer. Note that, however, this model can still share inter-task knowledge in a symmetrical manner since it still has shared lower layers (the embedding and the LSTM layers). Figure 13 : Architecture overview. The amount and form of (d) knowledge transfer is computed by networks F and G, which is described in detail on subsections 3.1 and 3.2.
The probabilistic model with uncertainty-aware knowledge transfer, which performs knowledge transfer only between the features at the same timestep, at the transfer layer. Again, note that this model can still capture the temporal dependencies among the timesteps to certain degree, as it has shared lower layers. 
A.3.4 TP-TAMTL -no constraints.
We trained all the models using Adam optimizer. We set the maximum iteration for Adam optimizer as 100,000, and for other hyper-parameters, we searched for the optimal values by cross-validation, within predefined ranges as follows: To show the effectiveness of the inter-task and inter-timestep knowledge transfer, we further compare our model on the PhysioNet dataset against several variations of our model (please refer to our supplementary file for more details our these variants): 1) AMTL-intratask, 2) AMTL-samestep: The variant of our model that knowledge only transfers 1) within a same task and 2) within a same time-step.
3) TD-AMTL: The deterministic variant of our model. ( Table 5 shows that our model outperforms the "intratask" and "samestep" variants, which demonstrates the effectiveness of inter-task and inter-step knowledge transfer. Moreover, the deterministic counterpart largely underperforms any variants, which may be due to overfitting of the knowledge transfer model, that can be effectively prevented by our Bayesian framework.
B.2 Future-to-past transfer.
We also compare our model against a variation of our method with no temporal constraint on the inter-step knowledge transfer (TP-AMTL (unconstrained)), such that the knowledge transfer can happen from the later timestep to earlier ones. Table 5 shows that the constrained model outperforms the unconstrained model.
Furthermore, we examine the effect of two kinds of uncertainty with two variants of the model: TP-AMTL (epistemic) uses only MC-dropout to model epistemic uncertainty and p θ (z d |x, ω) is simplified into N (z d ; µ d , 0) (i.e. its pdf becomes the dirac delta function at µ d and z d is always µ d ); TP-AMTL (aleatoric) uses only p θ (z d |x, ω) to model the aleatoric uncertainty, withouth MCdropout. Table 5 shows that, for this dataset, epistemic uncertainty attributes more to the performance gain. However, it should be noted that the impacts of two kinds of uncertainty vary from dataset to dataset. By modelling both kinds of uncertainty, the model is guaranteed to get the best performance.
C Quantitative evaluation on clinical time-series prediction tasks: MIMIC III-Heart Failure, Respiratory Failure
Here, we provide the experimental results of our model and other baselines on the additional dataset: MIMIC III-Heart Failure, Respiratory Failure. Table 6 and Table 7 shows that our model still outperforms other baselines, which indicates that our method can generalize well on a variety of time-series datasets. Furthermore, Table 8 shows that our model also generalize well to larger datasets. In this section, we further describe the interpretation of several example patients using generated uncertainty and knowledge transfer across timesteps.
The example patient in Figure 15 had a fever on admission and was confirmed to be positive with a bacterial infection on culture study in this specific timestep 2:57. The patient continued to have a fever, and white blood cell count increased to the state of leukocytosis. Also, both systolic and diastolic blood pressure declined over time. We can see that uncertainty of target task drops when the model can confidently infer to the patient status from feature values, and aids from source to target can decrease in that case. Figure 5 represents knowledge transfer from one timestep of a source task to multiple time steps in the target task. We examine how multiple time steps of the source task transfers knowledge to certain time steps in the target task from the same example patient of MIMIC-III Infection dataset used in Figure 5 . In the vicinity of the same timepoint where this patient was confirmed to have bacterial infection, we can see that the uncertainty of source target starts to increase, and knowledge reversely flows to source task fever. This happens in accordance with the drop of knowledge transfer from fever to infection in Figure 5 . We can infer that the knowledge from task infection becomes more useful to predict source fever in this timestep as patient condition related to this task is happening around this time step.
Additionally, we select other example patient from MIMIC-III Infection dataset (Figure 15a, 15b) . This patient had fever and leukocytosis (elevation of white blood cell as a result of the bacterial intrusion, implies the infectious status of a patient) at the earlier time point of admission but was not confirmed to have infection afterward. The first timepoint highlighted with the blue box is when this patient had a fever and started to recover from fever. At 19:00 when this patient had a fever, the uncertainty of source task fever decreases and this task transfers more to target task mortality. As the patient recovers from fever, the uncertainty of task fever increases and knowledge transfer from fever to mortality drops accordingly. The second blue box on the right denotes the time step when the complete blood count lab result showed this patient has leukocytosis, which implies the high propensity of infection. Knowledge transfer starts to drop as the knowledge from source task fever is less important as the uncertainty of target task mortality drops.
Interpretation on another example patient from MIMIC III-heart failure dataset is plotted on Figure  16 . This example patient is finally diagnosed with congestive heart failure on Chest X-ray. During the admission period, the troponin level of this patient was elevated, which is not diagnostic [26] , but implying that this patient had a cardiac event. Given cardiac events, hypotension occurred in 1 : 21 (Table 9 , Figure 16 ) can be explained to be related to final diagnosis heart failure. As the patient's SBP decreases to 90 and DBP to 30 around 1 : 21 (Table 9 ), the uncertainty of target task Heart Failure decreases in Figure 16a . Knowledge transfer starts to drop as the knowledge from the target task becomes more important than that of the source task. We can also see that the trend of knowledge transfer follows the trend of target uncertainty. Furthermore, troponin increased in 16 : 21 implies ongoing myocardial stress, which can be expressed as constantly lowered uncertainty of source task ischemic heart disease among the window period we plotted on Figure 16b . As the uncertainty of source task decreases, knowledge transfer to target task heart failure kept increasing till 23 : 21. However, as a patient condition related to heart function, especially blood pressure starts to decrease and knowledge from the target task gets important, knowledge transfer starts to decrease after 23 : 21. We denote the timesteps with noticeable changes in uncertainty and knowledge transfer with blue boxes. With the help of a physician, we further analyze how generated transfer weights and uncertainties between tasks can be used to track changes in the relationships between clinical events and (see Table 10 and Figure 17 ). We first consider an example record of a patient from the MIMIC-III Respiratory Failure(RF) dataset who was found to be in hypercarbic(high arterial CO 2 pressure (high P a CO 2 , hypercapnia)) respiratory failure. Figure 17a shows the amount of knowledge transfer from task Hypoxia and Hypercapnia at 5 : 00AM to all later timesteps of task RF. As patient experience hypoxia (low arterial O 2 pressure (low P a O 2 )) during 12 (19 : 00) to 17 (23 : 00 AM)hours of admission, the uncertainty of hypoxia (blue bar chart of Figure 17a ) drops accordingly, and knowledge transfers from task Hypoxia to task RF (dark blue line graph in Figure 17a ). As patient was hypercarbic on admission, uncertainty of the task Hypercapnia kept it's low value (dark blue bar chart in Figure 17a ) and knowledge transfer to task RF is constantly high (dark blue line chart in Figure 17a ). After 23 : 00, P a O 2 increases back to knowledge transfer drops as the uncertainty of task hypoxia increases. Change in learned knowledge graph can also be used for treatment planning, such as deciding whether the patient needs prompt oxygen infusion or not. Later in timestep 20 (01 : 00), the O 2 Saturation of this patient drops to 78 (see Figure 17b ) and uncertainty of task S a O 2 drops accordingly as a result of respiratory failure (dark green bar chart in Figure 17b ). As the target uncertainty drops, knowledge transfer drops accordingly. After treatment S a O 2 increases up to 94 and then drops again to 83 in timestep 23 (4 : 00), where knowledge transfer from source task RF increases and drops accordingly in Figure 17b (black line). 