ADULT BRAIN ImprovingMultiple Sclerosis Plaque Detection Using a Semiautomated Assistive Approach

BACKGROUND AND PURPOSE: Treating MS with disease-modifying drugs relies on accurate MR imaging follow-up to determine the treatment effect. We aimed to develop and validate a semiautomated software platform to facilitate detection of new lesions and improved lesions. MATERIALS AND METHODS: We developed VisTarsier to assist manual comparison of volumetric FLAIR sequences by using interstudy registration, resectioning, and color-map overlays that highlight new lesions and improved lesions. Using the software, 2 neuroradiologists retrospectively assessed MR imaging MS comparison study pairs acquired between 2009 and 2011 (161 comparison study pairs met the study inclusion criteria). Lesion detection and reading times were recorded. We tested interand intraobserver agreement and comparison with original clinical reports. Feedback was obtained from referring neurologists to assess the potential clinical impact. RESULTS: More comparison study pairs with new lesions (reader 1, n 60; reader 2, n 62) and improved lesions (reader 1, n 28; reader 2, n 39) were recorded by using the software compared with original radiology reports (new lesions, n 20; improved lesions, n 5); the difference reached statistical significance (P .001). Interobserver lesion number agreement was substantial ( 1 new lesion: 0.87; 95% CI, 0.79 – 0.95; 1 improved lesion: 0.72; 95% CI, 0.59 – 0.85), and overall interobserver lesion number correlation was good (Spearman : new lesion 0.910, improved lesion 0.774). Intraobserver agreement was very good (new lesion: 1.0, improved lesion: 0.94; 95% CI, 0.82–1.00). Mean reporting times were 3 minutes. Neurologists indicated retrospective management alterations in 79% of comparative study pairs with newly detected lesion changes. CONCLUSIONS: Using software that highlights changes between study pairs can improve lesion detection. Neurologist feedback indicated a likely impact on management. ABBREVIATIONS: CSP comparative study pairs; CSSC conventional side-by-side comparison; IL improved lesion; NL new lesion; VTS VisTarsier software Multiple sclerosis affects approximately 2 million people worldwide, predominantly young adults. During the past decade, a number of novel disease-modifying drugs have emerged that are effective during the early phases of the disease; reducing the frequency of relapses, potentially halting disease progression, and even reversing early neurologic deficits. This choice in therapeutic options allows treating neurologists to alter management strategies when progression is detected. Because most demyelinating events are asymptomatic, MR imaging has been the primary biomarker for disease progression, and both physical disability and cognitive function have been shown to have a nonplateauing association with white matter demyelinating lesion burden, as seen on FLAIR and T2-weighted sequences. Recent advances in imaging, including 3T 3D volumetric T2 FLAIR sequences, allow better resolution of small demyelinating lesions, resulting in better clinicoradiologic correlation. Despite advances in imaging techniques, conventional side-by-side comparison (CSSC) is often subject to a reader’s expertise. The sensitivity of detecting new lesions is also likely to be reduced when the section number is increased and scan planes are unReceived November 30, 2014; accepted after revision February 3, 2015. From the Department of Radiology (J.v.H., P.M.D., F.G.), The Royal Melbourne Hospital and University of Melbourne, Parkville, Victoria, Australia; Department of Electrical and Electronic Engineering (D.R., A.M.Z.), School of Engineering, University of Melbourne, Parkville, Victoria, Australia; IBM Research (R.C.), Melbourne, Victoria, Australia; and Melbourne EpiCentre (M.A.T.), The Royal Melbourne Hospital and Department of Medicine, University of Melbourne, Parkville, Victoria, Australia. Please address correspondence to Jolandi van Heerden, MD, Department of Radiology, The Royal Melbourne Hospital, 300 Grattan St, Parkville, Victoria, Australia 3050; e-mail: jolandivh@hotmail.com Indicates article with supplemental on-line tables. Indicates article with supplemental on-line photo. http://dx.doi. org/10.3174/ajnr.A4375 AJNR Am J Neuroradiol ●:● ● 2015 www.ajnr. org 1 Published June 18, 2015 as 10.3174/ajnr.A4375 Copyright 2015 by American Society of Neuroradiology. matched; however, to our knowledge, this reduction has not yet been investigated. In an attempt to facilitate accurate lesion-load and lesion-volume detection, much research has been devoted to fully automated computational approaches with unsatisfactory results. Robust lesion segmentation has been identified as a critical obstacle to widespread clinical adoption for several reasons: difficulties specific to MS, problems inherent to segmentation, and data variability. A review of fully automated MS segmentation techniques concluded that basic data-driven methods are inherently inaccurate; supervised learning methods (such as artificial neural networks) require costly and extensive training on representative data; deformable models are better; and statistical models are most promising, though these also require training on representative data. An alternative to total automation is to assist manual reporting with partial automation. A few semiautomated lesion-subtraction strategies have been used in the research setting on small patient populations with good lesion detection and interreader correlation. Semiautomation without segmentation is inherently easier, more robust, and less affected by data variability because the lesion count is judged manually. The software can present a number of false-positives without a negative impact on accuracy. Our aim was to design a nonsegmentation semiautomated assistive software platform that can be integrated into vendor-agnostic PACS and validated by application to a large number of existing routine clinical scans in patients with an established diagnosis of multiples sclerosis. The approach is to merely draw the attention of the radiologist to potentially new or improved lesions rather than automate the entire process, thus preserving the expertise of neuroradiologists in determining whether a finding is real. Our hypothesis was that CSSCs of volumetric FLAIR studies in patients with MS were prone to false-negative errors in the perception of both new and improved lesions and that more lesions would be identified by using the assistive software with improved interand intrareader reliability. Secondly, we hypothesized that presenting this information to clinicians would likely have changed patient management. MATERIALS AND METHODS Software Development Detecting lesion change in studies obtained at 2 time points, “old” and “new,” requires numerous steps including the following: 1) brain-surface extraction and masking (to remove skull and soft tissues of the head and neck), 2) coregistration and resectioning (to accurately align the 2 scans in all axes), 3) normalization of the FLAIR signal intensity (to remove global signal differences), and 4) calculating the difference in signal intensity between the old and new study at each point (to identify new T2 bright plaques and previously abnormal areas that have regained normal white matter signal). Changes between scans were presented as a color map superimposed on conventional FLAIR sequences (Fig 1). This was accomplished by bespoke code (given the trademark VisTarsier, henceforth VTS or “the software”) with the inclusion of a number of open-source components (Fig 2). Step 1: Brain surface was identified by conforming a reference model (by using BrainSuite from the University of Southern California, http://brainsuite.org/) to the FLAIR images. This brain surface was then used to mask out the skull and extracranial soft