Automatic Generation of Medical Imaging Diagnostic Report with Hierarchical Recurrent Neural Network Medical images are widely used in the medical domain for the diagnosis and treatment of diseases. Reading a medical image and summarizing its insights is a routine, yet nonetheless timeconsuming task, which often represents a bottleneck in the clinical diagnosis process. Automatic report generation can relieve the issues. However, generating medical reports presents two major challenges: (i) it is hard to accurately detect all the abnormalities simultaneously, especially the rare diseases; (ii) a medical image report consists of many paragraphs and sentences, which are longer than natural image captions. We present a new framework to accurately detect the abnormalities and automatically generate medical reports. The report generation model is based on hierarchical recurrent neural network (HRNN). We introduce a topic matching mechanism to HRNN, so as to make generated reports more accurate and diverse. The soft attention mechanism is also introduced to HRNN model. Experimental results on two imageparagraph pair datasets show that our framework outperforms all the stateofart methods.

A Study on Automatic Generation of Chinese Discharge Summary Discharge summary, which summarizes a patientu0027s health information during hospitalization, is very important for transferring information between the hospitalist and primary care physician. Discharge summary writing is a necessary but timeconsuming job for physicians. Automatically generating discharge summaries using information technology is helpful to physicians, but challenging as discharge summaries are typically long and contain amounts of information to outline patientu0027s reason for admission, labtests, examinations, diagnostic findings, treatments, and medication care plan. In this study, we propose a framework based on deep learning methods for automatic discharge summary generation. In the framework, we split information of discharge summary into two parts: 1) history information such as reason for admission, admission diagnosis; 2) outcome information such as discharge diagnosis and medication care plan, and deploy different neural networks to generate them separately. Hierarchical sequence labeling methods are proposed to select key sentences from existing documents, e.g., admission notes, progress notes, examination reports as the history information, and multitask learning methods to predict the outcomes, e.g., diagnosis and medication care plan. Experiments on a Chinese corpus show that our approach has the ability to generate effective discharge summaries.