# MILXView: A Medical Imaging, Analysis and Visualization Platform

Neil Burdett, Jurgen Fripp, Pierrick Bourgeat, Olivier Salvado

► To cite this version:

Neil Burdett, Jurgen Fripp, Pierrick Bourgeat, Olivier Salvado. MILXView: A Medical Imaging, Analysis and Visualization Platform. First IMIA/IFIP Joint Symposium on E-Health (E-HEALTH) / Held as Part of World Computer Congress (WCC), Sep 2010, Brisbane, Australia. pp.177-186, 10.1007/978-3-642-15515-4_19. hal-01054862

HAL Id: hal-01054862

https://inria.hal.science/hal-01054862

Submitted on 8 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# MILXView: A medical imaging, analysis and visualization platform.

Neil Burdett¹, Jurgen Fripp¹, Pierrick Bourgeat¹, Olivier Salvado¹,

¹ The Australian e-Health Research centre, Level 5 – UQ Health Sciences Building, Royal Brisbane and Women’s Hospital, Herston, Queensland 4029, Australia {Neil Burdett, Jurgen Fripp, Pierrick Bourgeat, Olivier Salvado}@csiro.au

**Abstract.** Medical imaging is a vital clinical and research tool that has made significant strides in the last few decades with impressive advances in acquisition technology. However, image analysis and interpretation is still primarily performed manually, and is difficult to automate. In recent years, major advances have been made that allow more automatic image analysis, understanding, and interpretation. In this paper we introduce the 3D medical imaging analysis and visualization platform, MILXView, and present some of its current and potential usage for medical image analysis, understanding and interpretation. The paper describes the design of the plugin architecture of the software application, a general overview of the plugins and the improvements and additions to the latest version of MILXView. As an example the cortical thickness estimation plugin (MILXcte) is described for performing automatic atrophy estimation of brain MRI.

**Keywords:** MILXView, plugins, AEHRC, ITK, VTK, WxWidgets, voxel, image, DICOM, CTE.

## 1 Introduction

In this paper we introduce the MILXView 3D medical imaging analysis and visualization platform, and its plugin architecture illustrated using Cortical Thickness Estimate (CTE) MILXcte plugin.

Alzheimer's disease is associated with cortical atrophy, which is monitored via MR imaging. The MILXcte plugin automates the analysis of the cortical atrophy of a brain.

The etiology of dementia is not well understood, and early diagnosis is difficult as only subtle biological changes occur in the brain many years before cognitive symptoms are evident. By the time a loss of cognitive function occurs and a patient is clinically diagnosed with dementia, irreversible neuronal dysfunction has already occurred. Obtaining early objective evidence of Alzheimer's disease (AD) pathological lesions could allow the development and use of appropriate treatments and care that could result in delayed onset or prevention of AD.

The aim of the Alzheimer's disease project is to develop algorithms and software to process MRI and PET scans automatically to obtain both qualitatively and quantitatively analysis of changes in the brain for use in cross-section and longitudinal studies. Structural information from MRI is combined with functional and molecular information from PET to allow early detection of changes related to disease progression. Our goal is to provide a complete assessment of individual
patients and to allow the development of a better characterisation of Alzheimer's progression, bench-marked against healthy typical brains computed from large longitudinal study databases.

The Biomedical Imaging team at the Australian e-Health Research Centre (AEHRC) is developing a library of image processing algorithms that can be called from our core software platform: MILXView. MILXView is a 3D medical imaging analysis and visualization platform. Individual or large batch of image analysis tasks can be scheduled and run automatically from MILXView.

MILXView is designed and developed to support internal research efforts, and provide a viable and robust environment for clinical applications. It is designed to provide a familiar and intuitive user interface for radiologists and clinicians.

Both analysis and stand-alone visualization packages are usually customized solutions developed by a site to address their specific requirements. The e-Health Research Centre has a diverse set of requirements, from multimodality neuroimaging, musculoskeletal imaging, prostate radiotherapy to small animal imaging. Each of these areas requires specific requirements and analysis methods.

<figure><img src="image_2.png" /><figcaption>**Fig. 1** - Viewing an MR image in MILXView showing the orthogonal views (coronal, sagittal, axial) and 3D MESH view illustrating the grey matter cortical thickness calculated for this subject</figcaption></figure>
## 2 MILXView Architecture

### 2.1 Subversion and repository structure

All MILXView source code and related resources are written in C++ and maintained in a subversion (SVN) repository. This is available for external access by collaborators. The source and header files are located in separate directories, so that certain areas of the code can have restrictive access. Under this scheme, collaborators can checkout include files from the MILXView repository, link against existing software libraries thus allowing to create new plugins that can be incorporated in later versions of MILXView, without distributing proprietary software source code.

Fig.2. shows part of the MILXView subversion repository structure, with a limited set of directories. All components of the MILXView application have the include and source files separated. Underneath each source and include directory, the source code (cxx) or include files (txx and h files) can be found, respectively. The core GUI directories contain the core framework that includes viewer components and basic user interface. The library directory contains the processing code and the plugins directory contains the GUI code of additional visualization tools, user interface panels and complex image analysis functions.

It is important to note that the “plugins” directory holds only the GUI code of the plugin. The “library” directory holds all the algorithmic and processing code. So CTE, for example, has its user interface implemented using the code in “cte” directory of the plugin directory, but all the processing of input files is done within the library directory, “libcte”. Thus the “cte” library found in the “plugins” directory links against the “libcte” library, so that information can be passed.
<figure><img src="image_3.png" /><figcaption>Fig. 2 – Section of the MILXView repository structure.</figcaption></figure>

## 2.1 Framework

The MILXView comprises of:

* A core framework that includes viewer components and basic user interface

* A large number of plug-in components that add visualization tools, user interface panels, image analysis functions, complex image or other functionalities. (Fig 3)

* Advanced image processing, segmentation and registration pipelines i.e. co-registration (Fig 3), spatial normalization, partial volume correction, atlas creation, groupwise statistical analysis

* Customised plugins for particular applications and anatomical regions i.e. cortical thickness estimation, deep grey-matter segmentation, knee cartilage segmentation, lower-back muscle segmentation, prostate segmentation, breast density estimation.

The framework has been designed this way to enable multiple projects to run simultaneously, developing their own plugins, without interrupting one another. Essentially, plugins are developed as autonomous components, with their own intellectual property (IP). This allows for a large collaborative team of scientists to implement their own algorithms within the same software application, whilst still
being able to manage IP. It also enables students to actively participate in the
addition of new features.

New plugins GUIs are easily added to the MILXView UI. The following files are
used to create a plugin;

* A plugin XML file containing the name, id and library of the plugin. The munu location on the MILXView application, its dock location (Left, right or centre), priority (its place in the menu list) and any parameters that need to be passed in.

* A plugin class that instantiates the plugin panel class.

* A plugin panel class that sets the display and position of buttons and windows using WxWidgets libraries.

Plugins are accessed via the menu options on the toolbar at the top of the MILXView application.

<figure><img src="image_4.png" /><figcaption>**Fig. 3** – Example of co-registered CT and MR image in MILXView and surface renderings of automatically obtained segmentations.</figcaption></figure>

## 2.3 Third Party Libraries

MILXView is built upon three open-source, third-party libraries:

* The Insight Segmentation and Registration Toolkit (ITK) [1] is a cross-platform system software toolkit that provides an extensive suite of tool for image analysis, performing registration and segmentation.

* The visualization Toolkit (VTK) [2] which supplies comprehensive rendering and visualization libraries.

* The MILXView GUI and plugins are built using WxWidgets cross platform library.

The above libraries were chosen as they allow MILXView to be freely distributed into the community without affecting the IP present in MILXView. They are also all cross-platform, thus allowing MILXView to be developed and distributed on Linux, Windows and Mac operating systems.
## 3 MILXView 3.0 overview

MILXView 3.0 is the latest release of the MILXView platform and includes, but is not limited to, the following core features, improvements and additions:

* Support for import and export of 2D and 3D medical data in all major formats: including Analyze, DICOM, INR, MHD and Nifti.

* Fully user-customizable layout and user interface

* Improved and increased the multi-threaded architecture of the platform including preferences plug-in where the total number of threads and the maximum number of threads per task can be altered. Task manager plug-in showing the status of any running process, their queued, start and running times and their current progress.

* Standard medical imaging functions: windowing, histogram inspection, panning, slicing, zooming, and metadata inspection using a configurable multi-panel and multi-tab viewer

* 3D visualization tools: maximum intensity projection, volume and surface rendering, 3D multi-slicing screenshot, volume rendering, color map, overlay, blending, checkerboard. Loading, saving, visualization, registration and manipulation of 3D polygon meshes.

* Data manipulation in voxel or scanner space: manual rotation scaling, zooming and translation. Navigation through multiple datasets simultaneously. Manual linking and registration of two or more volumetric images.

## 4 A typical plugin - Cortical Thickness Estimate (CTE) plugin

MILXView has forty plugins (core and others). The CTE plugin (MILXcte) is one such plugin.

Neurodegenerative diseases such as Alzheimer's are often associated with loss of gray matter in the cortex of the brain. It is therefore necessary to quantify the volume loss of the tissue and/or the cortical thickness. This can be used for one patient as a diagnostic step, or for a database of images during longitudinal studies to assess the effect of a therapy, or the progression of a disease.

Estimating the thickness of the cortex involves many image processing steps such as interpolation, filtering, registration, etc. Such a processing pipeline requires some parameters to be set and then be run on all the images (typically several hundred) present in a database. We have implemented a method to perform this computation and designed a specific plugin: MILXcte.

Clinicians are thus meant to use MILXcte, and ideally they need to select one or multiple images as inputs and run the computation as a batch processing task, which could take several hours per individual image. The results are multiple:

* New images generated after some of the processing steps, either as visual check, or because the resulting image has new information;
* Spreadsheet with numbers, in this case the thickness of the cortex for each anatomical regions of the brain;

* Custom visualization to display information hard to visually analyzed, in this case the display as a color map of the cortical thickness over the surface of the brain.

Plugins used with MILXView have a similar user interface and follow similar conventions. At the top of each plugin there are a number of tabs that hold configurable values.

A clinician, in most cases, will want to add one or more images and then retrieve the result. However, scientists may want to set a variety of values. Plugins incorporated into MILXView are designed to handle both of these cases.

Generally a user will need only to add one or more image input files (see Fig. 3 below left) and then click on the “Process” button. Default values will be submitted for processing, and a short time later a result will be shown. This is the case for the MILXcte plugin. However, a scientist (or clinician) may wish to change the default values in which case they will select a tab on the plugin and change the value accordingly.

In the CTE plugin, for example, a number of selections can be altered before the CTE plugin is executed. The user can add and remove files individually using the “Add File(s)” and “Remove” buttons on the “Input Files” panel or files can be imported from a list and exported to a list, using the appropriate buttons. The same can be done for the required templates [3, 4] from the Atlas, White Matter Priors, Grey Matter Priors, Cerebrospinal Fluid (CSF) Priors, Anatomical Automatic Labelling atlas (AAL) [7] and Hippocampus mask as shown below:

<figure><img src="image_5.png" /><figcaption>**Fig. 4** - CTE Input (left) and template (right) Files</figcaption></figure>
The “Results” tab allows the user to specify the output directory for the CTE results, and the “Advanced” tab (shown below) allows the user to select a number of options including:

* Preprocessing

* Segmentation

* Partial Volume Estimation

* CTE Topology Correction

* Thickness Estimation

* 3D rendering

The user can view the results they are interested in using the “View” tab (shown below).

<figure><img src="image_6.png" /><figcaption>Fig. 5 - Advanced (left) and View tab (right) options</figcaption></figure>

As previously stated the user selects the results they want to view from the view tab and clicks on the “Display button”. Sample results are shown below.
<figure><img src="image_7.png" /><figcaption>**Fig. 6** – View Thickness Map (left) and View PVE map [11] (right)</figcaption></figure>

The CTE process also generates a mesh view (Fig. 6). The two brain renderings below show the average thickness for an AD patient compared to a healthy patient for different brain areas. They use an average atlas built from N AD and M healthy subjects by groupwise averaging CTE of each subject's brain after surface registration to an atlas.

<figure><img src="image_8.png" /></figure>
3 mm

<table><caption>**Figure 2** - Comparison of MILXView output of patients with AD and those without</caption><thead><tr><th></th><th></th></tr></thead><tbody><tr><td>Healthy Patient</td><td>AD Patient</td></tr></tbody></table>

**Fig. 7** – Comparison of cortical thickness of a healthy patient and an Alzheimer's Disease patient

## 5 Conclusions

Our vision is to provide clinicians with a software tool easy to use and fully automatic. The physicians would read the scans from a patient, and we would compute several quantitative measurements from the images otherwise hard or impossible for a human to obtain. From our study on the Australian Imaging, Biomarker and Lifestyle cluster study (AIBL) patients we will be able to benchmark each individual patient against the typical age-matched individual and provide to physicians relevant statistics. This kind of information should be valuable not only to help in treatment planning of individual patients, but also to help design therapies and test scientific hypothesis.

Ongoing work includes increasing unit test code coverage, optimizing the code and checking for memory leaks, improving performance, validating the framework for research suitability and adding to the visualization techniques and plug-ins. Also on the road map are Microsoft Windows and Apple Macintosh versions.

## 6 References

1. Luis Ibanez, Will Schroeder, Lydia Ng and Josh Cates, The ITK Software Guide Second Edition, Kitware, 2007

2. Will Schroeder, Ken Martin and Bill Lorensen, The Visualization Toolkit, 4th Edition, Kitware, 2006

3. Oscar Acosta, Pierrick Bourgeat, Maria A. Zuluaga, Jurgen Fripp, Olivier Salvado, "Automated voxel-based 3D cortical thickness measurement in a combined Lagrangian-Eulerian PDE approach using partial volume maps", Medical Image Analysis, doi:10.1016/j.media.2009.07.003;--.2009

4. Andrea Rueda, Oscar Acosta, Pierrick Bourgeat, Jurgen Fripp, Nicholas Dowson, "PARTIAL VOLUME ESTIMATION OF BRAIN CORTEX FROM MRI USING TOPOLOGY-CORRECTED SEGMENTATION", In 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Boston, Massachusetts, USA, pages 133-136, June
5. Rousset OG, Ma Y, Wong DF, Evans AC, "Pixel- versus region-based partial volume correction in PET", In: Carson R, Herscovitch P, Daube-Witherspoon M, editors. Quantitative Functional Brain Imaging with Positron Emission Tomography. San Diego, CA: Academic Press; 1998. p. 67-75

6. P. Bourgeat, G. Chételat, V. Villemagne, J. Fripp, P. Raniga, K. Pike, O. Acosta, C. Szoeke, S. Ourselin, D. Ames, K. Ellis, R. Martins, C. Masters, C. Rowe, O. Salvado, and the AIBL Research Group. AB burden in the temporal neocortex is related to hippocampal atrophy in elderly subjects without dementia. Neurology, 74(2):121--127, January 2010.

7. Oscar Acosta, Pierrick Bourgeat, Maria A. Zuluaga, Jurgen Fripp, Olivier Salvado, and Sebastien Ourselin. Automated voxel-based 3D cortical thickness measurement in a combined Lagrangian-Eulerian PDE approach using partial volume maps. Medical Image Analysis, 13(5):730-743, October 2009.

8. P. Raniga, P. Bourgeat, J. Fripp, O. Acosta, V. Villemagne, C. Rowe, C. Masters, G. Jones, G. O'Keefe, O. Salvado, and S. Ourselin. Automated [11]C-PiB Standardized Uptake Value Ratio. Academic Radiology, 15(11):1376-1389, November 2008.

9. J. Fripp, P. Bourgeat, O. Acosta, P. Raniga, M. Modat, K.E. Pike, G. Jones, G. O'Keefe, C.L. Masters, D. Ames, K.A. Ellis, P. Maruff, J. Currie, V.L. Villemagne, C.C. Rowe, O. Salvado, and S. Ourselin. Appearance Modeling of 11C PiB PET images: Characterizing amyloid deposition in Alzheimer's disease, mild cognitive impairment and healthy aging. *NeuroImage*, 43(3):430-439, Novembre 2008
Note: CD-ROM paper number 234 TH-PM.

10. T-M. Diep, P. Bourgeat and S. Ourselin. Efficient Use of Cerebral Cortical Thickness to Correct Brain MR Segmentation. In *IEEE International Symposium on Biomedical Imaging: From Nano to Macro*, Washington DC, USA, pages 592-595, April 2007.
IEEE

11. Maria Zuluaga, Oscar Acosta, Pierrick Bourgeat, Olivier Salvado, Marcela Hernandez, and Sebastien Ourselin. Cortical Thickness Measurement from Magnetic Resonance Images Using Partial Volume Estimation. In *Proceedings of SPIE: Image Processing*, volume 6914, San Diego, USA, pages 8p, February 2008