Patient Outcome and Zero-shot Diagnosis Prediction with Hypernetwork-guided Multitask Learning (2023)

Abstract: Multitask deep learning has been applied to patient outcome prediction from text, taking clinical notes as input and training deep neural networks with a joint loss function of multiple tasks. However, the joint training scheme of multitask learning suffers from inter-task interference, and diagnosis prediction among the multiple tasks has the generalizability issue due to rare diseases or unseen diagnoses. To solve these challenges, we propose a hypernetwork-based approach that generates task-conditioned parameters and coefficients of multitask prediction heads to learn task-specific prediction and balance the multitask learning. We also incorporate semantic task information to improve the generalizability of our task-conditioned multitask model. Experiments on early and discharge notes extracted from the real-world MIMIC database show our method can achieve better performance on multitask patient outcome prediction than strong baselines in most cases. Besides, our method can effectively handle the scenario with limited information and improve zero-shot prediction on unseen diagnosis categories.

## introduction recent advances apply artificial intelligence to predict clinical events or infer the probable diagnosis for clinical decision support @xcite . those works extensively study clinical data from electronic health records (ehrs). for example, the doctor ai model for predictive modeling on ehrs @xcite and the deep patient model for unsupervised patient record representation learning @xcite is proposed by leveraging longitudinal medical data. within the patient records, we can also find various clinical notes written by general practitioners or physicians and annotated with specific diagnosis results, medical codes, and other clues of patient outcome. those clinical notes consist of meaningful health information, including health profile, clinical synopsis, diagnostic investigations, and medications, which can empower the clinical decision making @xcite . clinical text mining has demonstrated its feasibility in diverse healthcare applications, including medical code prediction @xcite , readmission prediction @xcite and diagnosis prediction @xcite . readmission and length-of-stay prediction can help with utility management, especially when healthcare service is in high demand. diagnostic prediction can support clinicians in making decisions based on many clinical reports. this study focuses on multitask patient outcome prediction from clinical notes, which classifies patients' diagnostic results and predicts clinical outcomes to empower expert clinicians and improve healthcare services' efficiency. ## related work patient outcome prediction, including automatic coding @xcite , patient severity @xcite , in-hospital mortality, decompensation, length of stay and phenotyping @xcite , can empower health practitioners to make better clinical decision. many studies have been conducted to investigate the usability of deep learning for accurate prediction using time-series, clinical text, or multimodal data. zufferey et al. (2015) compared different multi-label learning algorithms. che et al. (2015) proposed a deep phenotyping model regularized on the categorical structure of the medical ontology. zhang et al. (2019) proposed an attention-based lstm network to model the disease progression in a time-aware fashion. similarly, men et al. (2021) extended the lstm model to be attention-and time-aware for multi-disease prediction. convolutional neural networks have also been deployed in this field. for example, (bardak and tan, 2021) developed cnn-based networks for multimodal learning from patient records. clinical notes as an essential modality of ehrs have also been studied intensively. ghassemi et al. (2014) extracted topic features from free-text patient data for mortality prediction. mulyar and mcinnes (2020) extracted clinical information from clinical documents via fine-tuning large-scale pretrained language models. multitask learning learns multiple tasks simultaneously with a shared representation and transfers the domain knowledge in related tasks to improve the learning capacity @xcite . it has also attracted much attention from the research community of machine learning and natural language processing for healthcare. @xcite studied semantic similarity degrees of clinical notes by a multitask learning method with iterative data selection. @xcite proposed a multitask learning framework with feature calibration for medical coding. several multitask learning schemes have also been developed for learning from clinical time-series @xcite . hypernetworks @xcite that generate weights for other networks have also been adopted to solve the learning problem with multiple tasks, for example, the hyperformer that generates the parameters of adapter modules for fine-tuning nlp task streams @xcite . inspired by this work, our work starts from a different setting that learns different tasks simultaneously, utilizes the semantic information of clinical predication tasks, and balances the learning objective via task conditioning. ## conclusion patient hospitalization is a complex process, which needs the learning algorithm to model multiple risk indicators. multitask learning can jointly learn from patient records to empower clinical decisionmaking. to address the challenges of inter-task interference and poor generalizability to unseen labels in recent multitask learning frameworks for healthcare, we propose a hypernetwork-guided multitask learning method that learns from the task context and generates task-specific parameters for effective multitask prediction on patient outcomes. the proposed method incorporates contextualized language representations to encode clinical notes and capture the task context via the semantic embeddings of tasks. the hypernetwork-guided multitask learning framework enables task-conditioned learning and balances the joint learning objective across different tasks. experimental studies on the real-world mimic-iii dataset show our proposed method achieves better performance on early prediction and more robust performance on zero-shot diagnosis prediction.