Query: view classification radiology deep learning

title: Deep Learning for Musculoskeletal Image Analysis
abstract: The diagnosis, prognosis, and treatment of patients with musculoskeletal (MSK) disorders require radiology imaging (using computed tomography, magnetic resonance imaging (MRI) and ultrasound) and their precise analysis by expert radiologists. Radiology scans can also help assessment of metabolic health, aging, and diabetes. This study presents how machine learning, specifically deep learning methods, can be used for rapid and accurate image analysis of MRI scans, an unmet clinical need in MSK radiology. As a challenging example, we focus on automatic analysis of knee images from MRI scans and study machine learning classification of various abnormalities including meniscus and anterior cruciate ligament tears. Using widely used convolutional neural network (CNN) based architectures, we comparatively evaluated the knee abnormality classification performances of different neural network architectures under limited imaging data regime and compared single and multi view imaging when classifying the abnormalities. Promising results indicated the potential use of multi view deep learning based classification of MSK abnormalities in routine clinical assessment.

title: Automatic Radiology Report Generation based on Multi view Image Fusion and Medical Concept Enrichment
abstract: Generating radiology reports is time consuming and requires extensive expertise in practice. Therefore, reliable automatic radiology report generation is highly desired to alleviate the workload. Although deep learning techniques have been successfully applied to image classification and image captioning tasks, radiology report generation remains challenging in regards to understanding and linking complicated medical visual contents with accurate natural language descriptions. In addition, the data scales of open access datasets that contain paired medical images and reports remain very limited. To cope with these practical challenges, we propose a generative encoder decoder model and focus on chest x ray images and reports with the following improvements. First, we pretrain the encoder with a large number of chest x ray images to accurately recognize 14 common radiographic observations, while taking advantage of the multi view images by enforcing the cross view consistency. Second, we synthesize multi view visual features based on a sentence level attention mechanism in a late fusion fashion. In addition, in order to enrich the decoder with descriptive semantics and enforce the correctness of the deterministic medical related contents such as mentions of organs or diagnoses, we extract medical concepts based on the radiology reports in the training data and fine tune the encoder to extract the most frequent medical concepts from the x ray images. Such concepts are fused with each decoding step by a word level attention model. The experimental results conducted on the Indiana University Chest X Ray dataset demonstrate that the proposed model achieves the state of the art performance compared with other baseline approaches.

title: CARS 2018 Computer Assisted Radiology and Surgery Proceedings of the 32nd International Congress and Exhibition Berlin, Germany, June 20 23, 2018
abstract: Advanced atherosclerosis in the coronary arteries is one of the leading causes of deaths worldwide while being preventable and treatable. In order to image atherosclerotic lesions (plaque) intravascular optical coherence tomography (IVOCT) can be used. The technique provides high resolution images of arterial walls which allows for early plaque detection by experts. Due to the vast amount of IVOCT images acquired in clinical routines, automatic plaque detection has been addressed. For example, attenuation profiles in single A Scans of IVOCT images are examined to detect plaque. We address automatic plaque classification from entire IVOCT images, the cross sectional view of the artery, using deep feature learning. In this way, we take context between A Scans into account and we directly learn relevant features from the image source without the need for handcrafting features.

title: [Artificial intelligence and radiomics in MRI based prostate diagnostics]
abstract: In view of the diagnostic complexity and the large number of examinations, modern radiology is challenged to identify clinically significant prostate cancer (PCa) with high sensitivity and specificity. Meanwhile overdiagnosis and overtreatment of clinically nonsignificant carcinomas need to be avoided.  Increasingly, international guidelines recommend multiparametric magnetic resonance imaging (mpMRI) as first line investigation in patients with suspected PCa.  Image interpretation according to the PI RADS criteria is limited by interobserver variability. Thus, rapid developments in the field of automated image analysis tools, including radiomics and artificial intelligence (AI; machine learning, deep learning) give hope for further improvement in patient care. PERFORMANCE AI focuses on the automated detection and classification of PCa, but it also attempts to stratify tumor aggressiveness according to the Gleason score. Recent studies present good to very good results in radiomics or AI supported mpMRI diagnosis. Nevertheless, these systems are not widely used in clinical practice.  In order to apply these innovative technologies, a growing awareness for the need of structured data acquisition, development of robust systems and an increased acceptance of AI as diagnostic support are needed. If AI overcomes these obstacles, it may play a key role in the quantitative and reproducible image based diagnosis of ever increasing prostate MRI examination volumes.

title: Fast and accurate view classification of echocardiograms using deep learning
abstract: Echocardiography is essential to cardiology. However, the need for human interpretation has limited echocardiography's full potential for precision medicine. Deep learning is an emerging tool for analyzing images but has not yet been widely applied to echocardiograms, partly due to their complex multi view format. The essential first step toward comprehensive computer assisted echocardiographic interpretation is determining whether computers can learn to recognize these views. We trained a convolutional neural network to simultaneously classify 15 standard views (12 video, 3 still) based on labeled still images and videos from 267 transthoracic echocardiograms that captured a range of real world clinical variation. Our model classified among 12 video views with 97.8% overall test accuracy without overfitting. Even on single low resolution images, accuracy among 15 views was 91.7% vs. 70.2 84.0% for board certified echocardiographers. Data visualization experiments showed that the model recognizes similarities among related views and classifies using clinically relevant image features. Our results provide a foundation for artificial intelligence assisted echocardiographic interpretation.Computer aided diagnosis: Model accurately classifies heart scan dataA computer model can accurately classify what heart anatomy is depicted on an ultrasound scan. Rima Arnaout from the University of California, San Francisco, and colleagues used a machine learning technique to teach a computer to recognize different types of video and still images produced by echocardiogram tests. After training the computer with real scans gathered from patients with a range of heart conditions, the researchers showed that their model could correctly classify what heart anatomy was shown in videos with 98% accuracy. The accuracy for still images fell to 92% but that was still above the 80% average achieved by four professional echocardiographers tested on the same dataset. Now that the researchers have achieved successful view classification, the next challenge, they say, will be comprehensive computer assisted interpretation of echocardiogram results.

title: Designing lightweight deep learning models for echocardiography view classification
abstract: Transthoracic echocardiography (echo) is the most common imaging modality for diagnosis of cardiac conditions. Echo is acquired from a multitude of views, each of which distinctly highlights specific regions of the heart anatomy. In this paper, we present an approach based on knowledge distillation to obtain a highly accurate lightweight deep learning model for classification of 12 standard echocardiography views. The knowledge of several deep learning architectures based on the three common state of the art architectures, VGG 16, DenseNet, and Resnet, are distilled to train a set of lightweight models. Networks were developed and evaluated using a dataset of 16,612 echo cines obtained from 3,151 unique patients across several ultrasound imaging machines. The best accuracy of 89.0% is achieved by an ensemble of the three very deep models while we show an ensemble of lightweight models has a comparable accuracy of 88.1% The lightweight models have approximately 1% of the very deep model parameters and are six times faster in run time. Such lightweight view classification models could be used to build fast mobile applications for real time point of care ultrasound diagnosis.

title: View Classification and Object Detection in Cardiac Ultrasound to Localize Valves via Deep Learning
abstract: Echocardiography provides an important tool for clinicians to observe the function of the heart in real time, at low cost, and without harmful radiation. Automated localization and classification of heart valves enables automatic extraction of quantities associated with heart mechanical function and related blood flow measurements. We propose a machine learning pipeline that uses deep neural networks for separate classification and localization steps. As the first step in the pipeline, we apply view classification to echocardiograms with ten unique anatomic views of the heart. In the second step, we apply deep learning based object detection to both localize and identify the valves. Image segmentation based object detection in echocardiography has been shown in many earlier studies but, to the best of our knowledge, this is the first study that predicts the bounding boxes around the valves along with classification from 2D ultrasound images with the help of deep neural networks. Our object detection experiments suggest that it is possible to localize and identify multiple valves precisely.

title: Clinically Feasible and Accurate View Classification of Echocardiographic Images Using Deep Learning
abstract: A proper echocardiographic study requires several video clips recorded from different acquisition angles for observation of the complex cardiac anatomy. However, these video clips are not necessarily labeled in a database. Identification of the acquired view becomes the first step of analyzing an echocardiogram. Currently, there is no consensus whether the mislabeled samples can be used to create a feasible clinical prediction model of ejection fraction (EF) The aim of this study was to test two types of input methods for the classification of images, and to test the accuracy of the prediction model for EF in a learning database containing mislabeled images that were not checked by observers. We enrolled 340 patients with five standard views (long axis, short axis, 3 chamber view, 4 chamber view and 2 chamber view) and 10 images in a cycle, used for training a convolutional neural network to classify views (total 17,000 labeled images) All DICOM images were rigidly registered and rescaled into a reference image to fit the size of echocardiographic images. We employed 5 fold cross validation to examine model performance. We tested models trained by two types of data, averaged images and 10 selected images. Our best model (from 10 selected images) classified video views with 98.1% overall test accuracy in the independent cohort. In our view classification model, 1.9% of the images were mislabeled. To determine if this 98.1% accuracy was acceptable for creating the clinical prediction model using echocardiographic data, we tested the prediction model for EF using learning data with a 1.9% error rate. The accuracy of the prediction model for EF was warranted, even with training data containing 1.9% mislabeled images. The CNN algorithm can classify images into five standard views in a clinical setting. Our results suggest that this approach may provide a clinically feasible accuracy level of view classification for the analysis of echocardiographic data.

title: A Multi View Deep Learning Model for Encrypted Website Service Classification
abstract: This paper presents a multi view deep learning model for encrypted traffic classification which uses different neural network structures to process different types of features. Compared to previous work, our multi view deep learning model can make full use of different types of features extracted from the flow. We evaluate our deep learning model in the real world datasets acquired from the website servers of a large internet corporation. We found that the proposed multi view deep learning model gets better performance than the state of the art. Moreover, this paper focuses on a new traffic classification scenario in which the traffic under the specific website is classified on the basis of different website services. Each website service represents a specific website function which can be composed of several webpages. Therefore, the granularity of website service classification is between website classification and webpage classification.

title: Multi layer Joint Classification Metric Deep Learning for Top View Image Person Re identification
abstract: Person re identification (ReID) is the problem of identifying pedestrian images among multiple non overlapping cameras. Most of the researches are based on the front view or side view of the human body. However, there are many challenges, such as occlusion, posture changes. Therefore, based on top view images, we propose a person reidentification method, called ML CM (multi layer joint classification metric deep learning) The proposed method uses classification models and integrates them into the Siamese network, which is the unification of classification and metric learning. It is more suitable for top view images. In this way, minimum intra class variance and maximum inter class variance are achieved. We have collected the top view images of 1008 people as TVPR (top view person re identification) dataset and conducted experiments, demonstrating that the deep features obtained by our approach can get a competitive result.