Patent ID: 12198348

Throughout the description and the drawings, like reference numerals refer to like parts.

DETAILED DESCRIPTION

The present disclosure provides ways of training a machine learning image segmentation algorithm to segment structural features of a blood vessel in a computed tomography (CT) image, and further discloses methods for establishing a training set used to train the machine learning image segmentation algorithm to segment structural features of a blood vessel in a CT image. Whilst various embodiments are described below, the invention is not limited to these embodiments, and variations of these embodiments may well fall within the scope of the invention which is to be limited only by the appended claims.

A computerised tomography (CT) scan uses computer-processed combinations of multiple X-ray measurements taken from different angles to produce cross-sectional images (virtual “slices”) of specific areas of a scanned object. This allows visualisation inside the object without cutting it open. Since the invention of the first commercially available CT scanner in 1972, the use of CT scans for the diagnosis and management of disease is extensively embedded in every field of modern medicine. In the NHS alone, ˜6 million CT scans were performed in 2018-2019.

Visualisation of blood vessels on a routine CT scan is challenging. Blood vessels consist of vessel wall structures, and the contents within the vessel lumen (blood, clot, plaques, etc). These components have similar radio-densities (measured in Hounsfield Unit, HU) to the adjacent soft tissue structures. Injection of intravenous contrast enhances the radio-density within vessel lumens and enables its reconstruction. The produced CT angiogram is routinely utilised to diagnose medical problems related to blood vessels.

CT angiograms are widely used in all fields of cardiovascular surgery/medicine. When treatment of an artery, for example the aorta, is being considered, a medical/surgical professional usually requires a detailed view of the artery to differentiate the morphology/anatomy of the arterial structures. In the case of abdominal aortic aneurysms (AAAs), there is usually luminal thrombus within the aneurysm sac and full visualisation of the thrombus morphology, and its relation to the artery wall, is important for planning surgical intervention, for example by stenting or open repair.

Pathological changes can be present in the blood lumen, vessel wall or a combination of both.

FIGS.1A and1Bshow axial slices of an abdominal aortic region obtained from a non-contrast CT scan and a contrast CT scan respectively. The aneurysm lies within region110ofFIGS.1A and1B. As described above, to enable a surgeon or other medical professional to monitor the growth of the aneurysm and/or plan for surgery, full visualisation of the thrombus morphology, and its relation to the artery wall is important. The CCT image ofFIG.1Bclearly shows the interface between the aortic inner lumen130, and the intra-luminal thrombus (ILT)120, where the thickness (double-headed arrow) of the ILT is defined as the distance from the outer aneurysmal wall150to the inner aortic lumen wall140. These structural features120and130of the targeted region110are very difficult to distinguish in the NCT image with the naked eye, as is apparent from viewingFIG.1A.

FIG.2shows further axial slices taken from a non-contrast computed tomography (NCT) scan and a contrast-enhanced CT scan (CCT). In particular, panel A (left-hand panel ofFIG.2) shows an NCT axial slice. Panels B and C (middle and right-hand panels ofFIG.2respectively) show contrast images of the region shown in the box in panel A. In the example of aortic aneurysms (panel A, indicated by the arrow), there is usually a blood clot or thrombus adherent to the aortic wall within the aneurysm sac (panel B, red arrow points toward the aortic aneurysm). In Panel B, the lumen and thrombus are indicated. In Panel C, the diameter of the aneurysm is indicated. As explained above, the diameter of the aneurysm is important for the clinical care and research of patients with abdominal aortic aneurysms. Existing automated methods to reconstruct the angiogram would isolate the inner lumen but are unable to extract the thrombus and the complex thrombus-lumen interface. As such, there is no automated method to assess the aneurysm diameter (FIG.1C) or thrombus volume.

As described above, semi-automatic segmentation methods using open-source software are limited to the segmentation of the aorta with the inner aortic wall140, and segmentation of the aorta including the outer aneurysmal wall140is done manually.FIGS.3A-3DandFIG.4below describe an example of manual segmentation of the structural features130and120of an abdominal aortic CCT scan.

FIG.3Ashows an axial slice of an abdominal aortic region obtained from a CCT scan of a patient, and corresponding image masks,FIG.3B-3D, obtained from the CCT scan shown inFIG.3A. The image masks shown inFIGS.3B-3Dare generated by manually segmenting the structural features320and330from the targeted region310of the CCT image based on an analysis of the Hounsfield units of the voxels in the CCT image.FIG.3Bshows the segmentation of the aortic inner lumen330from the targeted region310.FIG.3Cshows the segmentation of both the aortic inner lumen330and the thrombus320from the targeted region310.FIG.3Dshows the segmentation of solely the thrombus320, which is obtained by subtracting the inner aortic mask, shown inFIG.3B, from the inner aortic and thrombus mask, shown inFIG.3C.

FIGS.4A and4Bshow axial slices of an abdominal aortic region of a patient obtained from a CCT scan.FIG.4Bshows overlying manually derived segmentation masks, where the segmentation masks demarcate the boundary between structural features of the targeted region410. The demarcated regions display the thrombus420, and the inner aortic lumen430.

The inventors have developed a method for establishing a training set, as described below in relation toFIG.5, to train a machine learning segmentation algorithm to segment structural features, such as the intra-luminal thrombus120, of a blood vessel. This mitigates the need for manual segmentation of structural features of a blood vessel and in the case of an aneurysmal region such as a AAA allows for the volume of the aneurysm to be calculated.

A method for establishing a labelled training set for training a machine learning image segmentation algorithm to segment structural features of a blood vessel in a CT image will now be described in relation to the flowchart shown inFIG.5. The method may be performed by any suitable computing apparatus, such as the computing apparatus600described in relation toFIG.6below.

At510, the method comprises receiving a plurality of CCT images, each CCT image showing a targeted region of a subject, such as the targeted region110shown inFIG.1Awhere the targeted region comprises an aortic aneurysm.

At520, the method comprises segmenting the plurality of CCT images to generate a corresponding plurality of segmentation masks, where each segmentation mask labels at least one structural feature of the at least one blood vessel of the targeted region in the corresponding CCT image.

At530, a labelled training set is established, wherein the labelled training set includes pairs of CCT images and the corresponding segmentation masks.

The method for establishing a labelled training set used to train a machine learning image segmentation algorithm, as described inFIG.5, is suitable for performance by a computing apparatus such as computing apparatus600as shown inFIG.6and described below.

Computing apparatus600may comprise a computing device, a server, a mobile or portable computer and so on. Computing apparatus600may be distributed across multiple connected devices. Other architectures to that shown inFIG.6may be used as will be appreciated by the skilled person.

Referring toFIG.6, computing apparatus600includes one or more processors610, one or more memories620, a number of optional user interfaces such as visual display630and virtual or physical keyboard640, a communications module650, and optionally a port660and optionally a power source670. Each of components610,620,630,640,650,660, and670are interconnected using various busses. Processor610can process instructions for execution within the computing apparatus600, including instructions stored in memory620, received via communications module650, or via port660.

Memory620is for storing data within computing apparatus600. The one or more memories620may include a volatile memory unit or units. The one or more memories may include a non-volatile memory unit or units. The one or more memories620may also be another form of computer-readable medium, such as a magnetic or optical disk. One or more memories620may provide mass storage for the computing apparatus600. Instructions for performing a method as described herein may be stored within the one or more memories620.

The communications module650is suitable for sending and receiving communications between processor610and remote systems.

The port660is suitable for receiving, for example, a non-transitory computer readable medium containing one or more instructions to be processed by the processor610.

The processor610is configured to receive data, access the memory620, and to act upon instructions received either from said memory620or a computer-readable storage medium connected to port660, from communications module650or from user input device640.

The computing apparatus600may receive, via the communications module650, data representative of a plurality of contrast CT scans of a targeted region of a subject. The data received via the communications module650relating to a contrast CT scan may comprise information relating to the measured intensity of the x-rays impinging the targeted region of the subject. The processor610may be configured to follow instructions stored in one or more memories620to use the received data to reconstruct the corresponding contrast CT image using various CT reconstruction techniques.

The processor610may be configured to follow further instructions stored in the memory620to segment the plurality of CCT images to generate a corresponding plurality of segmentation masks, each segmentation mask labelling at least one structural feature of the at least one blood vessel of the targeted region in the corresponding CCT image. The reconstructed CCT image comprises voxels/pixels, and the generated plurality of segmentation masks may be binary segmentation masks, where the voxels/pixels comprising structural feature of the blood vessel of the targeted region may be labelled with a 1 and the voxels/pixels comprising features in the image which are not structural features of the blood vessel may be labelled with a 0 (for example).

The processor610may be configured to follow instructions stored in the memory620to pair a generated segmentation mask with a corresponding CCT image.

Based on the above description, computing apparatus600can be used to establish a labelled training set for training a machine learning image segmentation algorithm, where the established labelled training set includes information relating to pairings of CCT images and their corresponding segmentation masks. The skilled person would appreciate that other architectures to that shown inFIG.6may be used.

FIG.7shows a flowchart of a method for training a machine learning image segmentation algorithm to segment structural features of a blood vessel in a CT image. The method uses a training set, which may be a training set such as that described above in relation toFIG.5. However, the skilled person would appreciate that the method may be used with NCT images provided an adequate training set is used.

At step710, the method comprises receiving a labelled training set. The labelled training set comprises information relating to a plurality of CT images, where each CT image of the plurality of CT images shows a targeted region of a subject which includes at least one blood vessel. The training set further comprises a corresponding plurality of segmentation masks, where the segmentation masks are generated from a CT image and each segmentation mask labels at least one structural feature of a blood vessel in a corresponding CT image of the plurality of CT images.

At step720, the method comprises training a machine learning segmentation algorithm using the plurality of CT images and the corresponding plurality of segmentation masks, to learn features of the CT images that correspond to structural features of the blood vessels labelled in the segmentation masks.

At step730, the method comprises output of a trained image segmentation model usable for segmenting structural features of a blood vessel in a CT image.

The method for training a machine learning image segmentation algorithm, as described above in relation toFIG.7, is suitable for performance by a computing apparatus such as computing apparatus600as shown inFIG.6.

The processor610may be configured to train a machine learning image segmentation algorithm to learn the features of CT images that correspond to structural features of blood vessels of the targeted region using the plurality of CT images and the corresponding plurality of segmentation masks. For each CT image and the corresponding segmentation mask, the processor610may follow instructions stored in one or more memories620to compare the segmentation mask with the corresponding CT image and adjust the internal weights of the image segmentation algorithm via backpropagation. Several iterations of the comparison between the CT image and the corresponding segmentation mask may be performed for each CT image from the plurality of CT images and the corresponding segmentation masks until a substantially optimized setting for the internal weights is achieved. The processor610may follow further instructions stored in one or more memories620to perform image transformations at each iteration for each CT image of the plurality of CT images to diversify the input data set and maximise learning.FIG.8below shows a single axial slice augmented ten times to diversify the training data set and maximize learning. The labelled training set may comprise such augmented images and corresponding (augmented) masks.

The processor610may be configured to follow further instructions to output the trained image segmentation model and store the trained image segmentation model in one or more memories620. The trained image segmentation model may comprise for example the weights and biases established during training, along with any selected hyperparameters such as minibatch size or learning rate.

FIG.9shows a flowchart of a method for segmenting structural features of a blood vessel in a CT image.

At step910, the method comprises providing the CT image to a trained image segmentation model which may be trained according to the method described above in relation toFIG.7. The trained image segmentation model is trained to learn features of CT images that correspond to structural features such as the inner aortic lumen or in the case of an aortic aneurysm the trained image segmentation model is trained to learn features of a CT image that correspond to structural features such as the thrombus120, inner aortic lumen130and outer aneurysmal wall150.

At step920, the method comprises segmenting, using the trained image segmentation model, at least one structural feature of a blood vessel in the provided CT image.

The method for segmenting structural features of a blood vessel in a CT image, as described above in relation toFIG.9, is suitable for performance by a computing apparatus such as computing apparatus600as shown inFIG.6.

The computing apparatus600may receive, via the communications module650, data from a CT scan of a subject. The received data may comprise information relating to the measured intensity of the x-rays impinging the targeted region of the subject, for example pixel/voxel intensity.

The computing apparatus600may store a trained image segmentation model in one or more memories620of the computing apparatus600, where the trained image segmentation model is trained to learn features of CT images that correspond to structural features of blood vessels of a targeted region. The processor610may be configured to input the received data from the CT scan to the trained image segmentation model.

The processor610may follow further instructions stored in memory620of the computing apparatus600to generate, using the trained image segmentation model, to segment at least one structural feature of a blood vessel in the provided CT image.

The inventors sought to use a modified version of a U-Net with attention gating and deep supervision to predict the inner lumen and outer wall from a given CT scan. The model was trained and tested over multiple iterations to achieve a particular task, in this case to extract the entirety of an aorta from the aortic root to the iliac bifurcation and automatically differentiate the outer aneurysmal wall150from the inner aortic lumen130.

The machine learning image segmentation architecture used in this experiment is shown inFIG.10. The algorithm utilises deep learning and is based on a modified 3D U-Net architecture. U-NET is very good for biomedical image segmentation tasks. The general U-Net architecture used for the experiment comprises two components: a downsampling/contraction path (as shown on the left inFIG.10) and an upsampling/expanding path (as shown on the right inFIG.10). The contraction path serves to extract information and capture the context of the image at the expense of losing spatial information. This is achieved through a series of algebraic manipulations (convolutions and max-pooling/averaging) as shown inFIG.10. During this process, the size of the input image is gradually reduced. In particular, in this example, the input image is of size 512×512×Z pixels where Z is the number of axial slices. The image is then sectioned into patches of size 160×160×96 and analysed. With each step for the downsampling path, the size of the input image is decreased by a factor of two. Accordingly, the downsampling path results in a resolution of 64×64×Z/4. This is followed by an expansion path, where the size of the image gradually increases and a predictive binary segmentation mask is output. This is accomplished using similar algebraic methods. The lost spatial information is restored using skip connections that connect the output of the down-sampling path with the feature maps/input of the up-sampling path at the same level. After each skip connection, two consecutive convolutions are used to integrate the spatial and contextual information to assemble a more precise output.

Attention gating is also used to train the machine learning image segmentation model to suppress irrelevant regions in an input image and to better highlight regions of interest. Attention gates are used to focus on target structures without the need for additional training/supervision. The attention gates filter along both the forward and backward directions. Gradients originating from the background regions are down-weighted during the backward pass allowing model parameters to be updated mostly based on spatial regions relevant to the given task. Accordingly, the attention gates reduce the need for hard attention/external organ localisation (region-of-interest) models in image segmentation frameworks.

Deep supervision is also used to ensure the feature maps are semantically distinctive at each image scale. This helps to ensure that the attention gates across different scales and not predictions from a small subset are more likely to influence foreground content.

Each axial CT slice and their respective image masks/segmentation masks were augmented through image transformations (shear, divergence) to diversify the input data set and maximize learning. The initial learning rate and weight decay were set to 1.0×10−4and 1.0×10−6, respectively. Training consisted of a total of 700 epochs with a batch size of 2 3D Images. Of the 143 images available, the training, validation and testing datasets consisted of 137, 3 and 3 images, respectively.

FIG.11shows several graphs demonstrating the improvement in accuracy as training progressed across the epochs. In particular, the top left image shows the overall accuracy, as calculated using the DICE score metric, for the validation data (blue) and test data (orange) of the model as the epochs progress to 700. The overall DICE score metrics reflects the similarity of the inner mask prediction to the ground truth inner mask, and as can be seen in the top left graph ofFIG.11, increased from 22.1% to approximately 98% for the test data set.

The top right graph inFIG.11shows the ability of the model to pick out the background in the validation (blue) and test (orange) images. Once again, the DICE score improved greatly with the number of epochs.

The bottom left graph inFIG.11demonstrates the ability of the model to pick out the inner lumen in the validation (blue) and test (orange) images. The bottom right graph inFIG.11demonstrates the ability of the model to pick out the outer wall in the validation (blue) and test (orange) images. Within the first 100 epochs the ability of this model to identify the inner lumen increased from 7.1% to 95%. For the remaining 600 epochs, this percentage increased to ˜98% and plateaued. On the other hand, within the first 100 epochs, the ability of the model to identify the outer wall increased from 4.0% to 35.0%. For the remaining 600 epochs, the percentage continued to rise to ˜70%. These sequential changes in learning rate pattern, suggests that this 3D network learns to decipher the aorta by first determining the location of the lumen and working outward to define the outer wall.FIG.11illustrates the results of this training loop. These results indicate for the first time the ability to extract the entirety of an aorta from the root to the iliac bifurcation and automatically differentiate the outer aneurysmal wall150from the inner aortic lumen130.

A study performed by the inventors will now be described, with reference toFIGS.12-26. Subtitles and headings in what follows have been included for readability only and are not intended to limit the scope of the invention, which is defined by the claims.

In this study, a modified U-Net architecture, as perFIG.10, was implemented to achieve high-throughput, automated segmentation of blood vessels in CT images acquired with or without the use of contrast agent. In contrast enhanced CT images, the modified U-Net architecture further enabled simultaneous segmentation of both the arterial wall and blood flow lumen to enable characterization of the pathological contents. The efficacy of this U-Net architecture is demonstrated further below by reconstructing the thoracic and abdominal aorta, which is the main artery bringing blood supply from the heart to the rest of the body.

Methods

CT Images from a Clinical Cohort

Computerized Tomographic scans of the chest and abdomen were acquired through the Oxford Abdominal Aortic Aneurysm (OxAAA) study. The study received full regulatory and ethics approval from both Oxford University and Oxford University Hospitals (OUH) National Health Services (NHS) Foundation Trust (Ethics Ref 13/SC/0250). As part of the routine pre-operative assessment for aortic aneurysmal disease, a non-contrast CT of the abdomen and a CT angiogram (CTA) of both the chest and abdomen was performed for each patient. CTA images were obtained following contrast injection in helical mode with a pre-defined slice thickness of 1.25 mm. Non-contrast CT images included only the descending and abdominal aorta and were obtained with a pre-defined slice thickness of 2.5 mm.

Paired contrast and non-contrast CT images were anonymized within the OUH PACS system before being downloaded onto the secure study drive.

Manual Segmentation of CT Images

Twenty-six patients with paired non-contrast and CTA images of the abdominal region were randomly selected. In the CTA, both the aortic inner lumen and outer wall were segmented from the aortic root to the iliac bifurcation using the ITK-Snap segmentation software.

Semi-automatic segmentation of the aortic inner lumen was achieved using a variation of region-growing by manually delimiting the target intensities between the contrast-enhanced lumen and surrounding tissue. Segmentation of the aortic outer wall was performed manually by drawing along its boundary using the previously obtained inner lumen as a base. Removing the inner lumen from the larger outer wall segmentation results in a segmentation mask highlighting the content between the arterial wall and blood lumen (in this case, thrombus). In the non-contrast CT image, the aorta was manually segmented.

Panels A-F ofFIG.12depict axial slices from the CT of chest and abdomen. Panels A and B are CTA images depicting the ascending thoracic aorta1210(yellow arrow, panel A), descending thoracic aorta1220(blue arrow, panel A), and abdominal aorta1230(red arrow, panel B) which is aneurysmal and contains crescentic layers of thrombus. Panel C shows the corresponding cross section of the abdominal aorta1230(red arrow, panel C) in the non-contrast CT scan. The aortic contour is manually segmented using IKT snap. Panels D, E and F, show respectively the CT images of panels A, B and C with the overlying manually derived segmentation. Views of the 3D volumes of the aorta derived from the manual 2D segmentations are depicted in Panels G and H. Panel G shows the reconstructed aorta from the segmentations of the contrast CT images, and panel H sows the reconstructed aorta from the segmentations of the NCT images.

Assessment of Intra- and Inter-Observer Variation of Manual Segmentation

A subset of these scans was selected randomly for intra- and inter-observer variability evaluation (n=10). This directly assessed the validity and accuracy of the manual segmentations used for subsequent analysis. For the intra-observer assessment, manual segmentation of the 10 scans was performed for the second time after a gap of 2 weeks. For the inter-observer assessment, a trained clinician performed the segmentations independent of the primary observer. In both instances, segmentation masks were compared against the ground truth (observer 1).

Data Augmentation

Of the 26 patients, 13 patients were randomly allocated to the training (ntrain=10) and validation cohorts (nvalid=3). Following manual segmentation, the original CT images and their corresponding image masks of only patients in the training/validation cohorts were augmented using divergence transformations. In order to diversify the training data set, divergence transformations employ nonlinear warping techniques to each axial slice, which manipulate the image in a certain predefined location. In panel A ofFIG.13, the aneurysmal sac can be seen towards the base of the gaussian peak and is noticeably stretched. By selecting the shape of the 3D surface, different warping affects can be created. In order to create localized stretching (Panel B ofFIG.13) a 2D gaussian curve is used:

g⁡(i,j)=exp[-(i-IC)2+(j-JC)22⁢σ2]

Here (IC, JC) is the centre from which the image is locally stretched. The images were augmented in this manner with gaussians at 5 locations adjacent to the aorta (indicated in panel A). Panel C ofFIG.13shows an augmented axial slice following a divergence transformation. This method was extended to achieve both congruent and divergent local transformations. Therefore, each patient's scan in the training cohort was augmented 10:1 to obtain a total of 143 post-augmented scans. During training, each 3D image was augmented further using random rotation (0-15°), translation and scaling (0.7-1.3).

As seen in the table ofFIG.14, post-augmented scans were split based on the original pre-augmented images into training (ntrain=10 patients, 110 augmented scans), and validation (nvalid=3 patients, 33 augmented scans) groups. This was done to avoid data leakage or the intermingling of patients and their augmented scans in the training/validation groups. Here, the validation group was used at the end of each training epoch to gauge model performance and fine-tune model hyperparameters. The remaining 13 patients formed the testing cohort (ntest=13).

U-Net Architecture

Panel A ofFIG.15shows the high-resolution segmentation pipeline for the simultaneous detection of the aortic inner lumen and intra-luminal thrombus/outer wall. In this study, a variation of the U-Net was utilized for both the Aortic Region-of-interest (ROI) detection and segmentation tasks. As discussed previously in relation toFIG.10, the general architecture of the U-Net comprises two components: the contraction path and expansion path (FIG.15, Panel B). The contraction path serves to extract information and capture the context of the input at the expense of losing spatial information. Here, as the input CT image (CTA/Non-Contrast) is deconstructed, it is able to extract more complex features relevant to the aortic segmentation task. This is followed by an expansion path, where the size of the image gradually increases to produce a predictive binary mask. The lost spatial information is restored using skip connections and is merged via concatenation. These connect the output of the down-sampling path with the extracted feature maps/input of the up-sampling path at the same level. This serves to integrate the spatial and contextual information to assemble a more precise prediction of the aortic structure.

Attention Gating

The use of a 3D U-Net with attention gating was evaluated against a generic 3D U-Net for segmentation of the aorta. Information extracted from the coarse scale is used within this gating mechanism to filter out irrelevant and noisy data exchanged via the skip connections before the concatenation step. The output of each attention gate is the element-wise multiplication of input feature-maps and a learned attention coefficient [0-1]. Given that we are simultaneously predicting the location of the aortic inner lumen and outer wall, multi-dimensional attention coefficients were used to focus on a subset of target structures. The gating coefficients were determined using additive addition, which has been shown to be more accurate than multiplicative addition.

FIG.15, panel B illustrates the 3D U-Net architecture with the attention gates utilized in this study. The architecture is the same as that described above in relation toFIG.10. The initial learning rate and weight decay for both models were set to 1.0×10−3and 1.0×10−6, respectively. Training to segment the aneurysmal region was carried out for a total of 1000 epochs with a batch size of 2 3D Images.

Loss Function

To quantify the performance of the algorithm at each step, the DICE score was utilized. The DICE score is a well-known performance metric in image segmentation tasks. This metric gauges the similarity between two images (A and B) and is defined as follows:

DICE(A,B)=2⁢❘"\[LeftBracketingBar]"A⋂B❘"\[RightBracketingBar]"❘"\[LeftBracketingBar]"A❘"\[RightBracketingBar]"+❘"\[LeftBracketingBar]"B❘"\[RightBracketingBar]"
Aortic Segmentation Pipeline: Aortic ROI Detection

Following data augmentation, all images were pre-processed. Pre-processing steps included isotropic voxel conversion and image down-sampling by a factor of 3.2 (512×512×Zi→160×160×Zf). This was performed to allow for increased efficiency during model training. The next step in this automatic aortic segmentation pipeline is Aortic ROI detection. This was performed on both the contrast and non-contrast CT images to isolate the aortic region for subsequent segmentation.

Attention U-Nets A and D (labelled “Attn U-Net A” and “Attn U-Net D” in the table ofFIG.16) were trained for a total of 600 epochs to segment the aorta from these decreased resolution, isotropic CTA and non-contrast CT images, respectively. The initial learning rate, weight decay, and batch-size for model training were set respectively to 1.0×10−3and 1.0×10−6and 2 3D Images. Aortic bounding boxes were generated from the predicted segmentation masks. Two bounding boxes were generated from the contrast CT image (1. Aortic Arch and 2. Descending Aorta and AAA) and one was generated from the non-contrast CT image (1. Descending Aorta and AAA). Regions of interests (144×144×[ZAAor ZAAA]) centred around the defined bounding box were isolated and served as the input data for aortic segmentation.

Aortic Segmentation Pipeline: Aortic Segmentation

U-Nets B and C (labelled respectively “Attn U-Net B” and “Attn U-Net C” in the table ofFIG.16) were trained for 1500 epochs on the CTA CT ROIs. They were tasked to simultaneously segment the aortic inner lumen and ILT/outer wall regions of the aortic arch and descending aorta/AAA, respectively. On the other hand, U-Net E (labelled “Attn U-Net E” inFIG.16) was trained for 1000 epochs on the non-contrast ROIs and was tasked to segment the descending aorta/AAA. The learning rate, weight decay, and batch-size for all U-Nets were set to 1.0×10−3and 1.0×10−6and 2 3D Images, respectively.

The table ofFIG.16summarises all the U-Nets trained and evaluated in this study. Model training was performed simultaneously on a workstation with 2 11 gb NVIDIA RTX 2080 TI graphics cards. Following training, in order to assess model performance and generalizability, models were evaluated on an external test cohort of non-augmented CTA and non-contrast images (next=13 scans). This cohort of scans was obtained from the same patient population and was independent of the scans used during training.

Results

CT Image Characteristics

Of the cases (n=26) included in the study, 13 were used for model training and the remaining 13 used for model testing. Details regarding the CT image characteristics between these groups are summarized in the table ofFIG.17.

Intra- and Inter-Observer Variability Assessment

There were strong agreements for both inter- and intra-observer measurements (intra-class correlation coefficient, ‘ICC’=0.995 and 1.00, respective. P<0.001 for both). The table ofFIG.18summarises the DICE score metrics for the intra- and inter-observer variability assessments performed on CTA and non-contrast CT images. The inter-operator variability is greater than intra-operator variability for all regions, as seen by the lower DICE scores. These data supports the accuracy of the manual segmentations used for model training.

Attention-Based 3D-U-Net Vs 3D-U-Net for AAA Segmentation

To assess the benefit of attention-gating for AAA segmentation, the performance of an attention-based 3D U-Net was compared against that of a generic 3D U-Net.FIG.19illustrates the evolving DICE score metric for the validation group during model training. Specifically, the left hand graph ofFIG.19relates to the overall accuracy of the models as quantified by the DICE score, the middle graph relates to the DICE score for determining the inner lumen, and the right hand graph relates to the DICE score for determining the ILT and outer wall only. During the training of the Attention-based U-Net, the overall DICE score increased from 24% at epoch 1 (Inner Lumen: 36.7%, Outer Wall: 11.9%) to approximately 95.3% at epoch 1000 (Inner Lumen: 97.4%, Outer Wall: 89.2%). On the other hand, the performance of the control 3D U-Net increased from 23.0% at epoch 1 (Inner Lumen: 38.2%, Outer Wall: 7.7%) to approximately 91.8.0% at epoch 1000 (Inner Lumen: 96.4%, Outer Wall: 87.2%).

Segmentation of the testing cohort was used to evaluate model performance. Model output was compared against the manually segmented ground-truth images utilizing the DICE score metric. The results of this analysis are found in the table ofFIG.20. The accuracy of the Attention-based U-Net in extracting both the inner lumen and the outer wall of the aneurysm is superior to that of the generic 3D-U-Net. This rationalizes the incorporation of the attention-gating unit into the segmentation pipeline.

Panels A and C ofFIG.21show attention-based 3D-U-Net outputs for two patients within the test set along with their respective ground truths (labelled segmentations masks) and DICE similarity scores. Areas of discrepancy arise at the iliac bifurcation and at other points of high curvature (panels B and D ofFIG.21).

Aortic Segmentation from CTA Images

Panels A-C ofFIG.22illustrate the evolving DICE score metric for the validation group during training of Attn U-Nets A-C involved in the segmentation of the aorta from contrast-enhanced CTA images. Attn U-Net A was trained for 600 epochs on down-sampled CT images. Attn U-Nets B and C were trained for 1500 epochs on the ROIs derived from Attn U-Net A—aortic arch (panel B) and descending aorta/AAA (panel C). The table ofFIG.23displays the performance of Attn U-Nets A-C on the ability to segment CTA images within the testing cohort via the DICE score metric. Merging the outputs of Attn U-Nets B and C to generate the entire aortic volume prediction results in an overall DICE Score accuracy of 93.0±0.6% (Inner Lumen: 96.4±0.3%, Outer Wall: 87.3±0.9%).FIG.24show the model predictions for Attn U-Nets A-C for a patient within the external test cohort compared against their respective ground truth annotations. Panel A ofFIG.24shows the ground truth and model prediction for Attn U-Net A. Attn U-Net A identified the aortic structure from down-sampled images and was the basis for ROI detection. Attn U-Nets B and C (panels B and C respectively) identified both the inner lumen and outer wall predictions for their indicated region (aortic arch for Attn U-Net B and descending aorta and AAA for Attn U-Net C). Panel D ofFIG.24shows the merged/combined volume prediction of U-Nets B and C along with the ground truth.

Aortic Segmentation from Non-Contrast CT Images

Panels D and E ofFIG.23illustrate the evolving DICE score metric for the validation cohort during training of Attn U-Net D and Attn U-Net E, which are involved in the segmentation of the aorta from non-contrast CT images. The table ofFIG.25displays the performance of Attn U-Net D and Attn U-Net E on the ability to segment non-contrast CT images within the testing cohort via the DICE score metric.FIG.26shows the results of the automated aortic segmentation pipeline (FIG.15) of a non-contrast CT image for a patient within the testing cohort (Attn U-Nets D and E). Panel A ofFIG.26shows the manually segmented aorta (ground truth). Panel B ofFIG.26shows a visualization of the aortic segmentation results as predicted by Attn U-Net E. Two regions within the (1.) thoracic region and (2.) aneurysm have been highlighted in the images.

The above discussed study has demonstrated a fully automatic and high-resolution algorithm that is able to extract the aortic volume from both CTA and non-contrast CT images at a level superior to that of other currently published methods. The extracted volume can be used to standardize current methods of aneurysmal disease management and sets the foundation for subsequent complex geometric analysis. Furthermore, the proposed pipeline can be extended to other vascular pathologies.

Furthermore, the above study has demonstrated the ability to use a deep learning method to isolate the aorta from a non-contrast CT scan. This will allow for the extraction of complex morphological information from non-contrast images and subsequent longitudinal analysis. The same methodology underpinning this work can be extended to enable automatic segmentation of other hollow or solid organs (such as the kidneys, veins, liver, spleen, bladder, or bowel) with or without the use of intravenous contrast agents.

FIG.27shows a flowchart of a method for obtaining at least one segmented structural feature from a CT image.

At step2710, the method comprises sending a CT image to a server, where the CT image comprises a targeted region of a subject including at least one blood vessel. The server may contain instructions for segmenting structural features of a blood vessel in a CT image.

At step2720, the method comprises receiving, from the server, receiving, from the server, at least one segmented structural feature of the at least one blood vessel.

The method for obtaining at least one segmented structural feature from a CT image, as described above in relation toFIG.27, is suitable for performance by a computing apparatus such as computing apparatus600as shown inFIG.6.

FIG.28illustrates a computer readable medium1200according to some examples. The computer readable medium2800stores units, with each unit including instructions2820that, when executed, cause a processor2810or other processing/computing device or apparatus to perform particular operations.

The computer readable medium2800may include instructions2820that, when executed, cause a processing device2810to, train a machine learning image segmentation algorithm, using a plurality of CT images and a corresponding plurality of segmentation masks, to learn features of the CT images that correspond to structural features of blood vessels labelled in the segmentation masks, and output a trained image segmentation model which is usable for segmenting structural features of a blood vessel in a CT image.

The machine readable medium2800may additionally or alternatively comprise instructions2820to provide a CT image to a trained image segmentation model, the trained image segmentation model trained to learn features of CT images that correspond to structural features of blood vessels and to segment, using the trained image segmentation model, at least one structural feature of a blood vessel in the provided CT image.

The machine readable medium2800may additionally or alternatively comprise instructions2820to receive a plurality of CT images, each CT image showing a targeted region of a subject, the targeted region including at least one blood vessel. The machine readable medium2800may additionally comprise instructions2820to segment the plurality of CCT images to generate a corresponding plurality of segmentation masks, where each segmentation mask labels at least one structural feature of the at least one blood vessel in the corresponding CCT image.

It will be appreciated that embodiments of the present invention can be realised in the form of hardware, software or a combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage such as, for example, a storage device like a ROM, whether erasable or rewritable or not, or in the form of memory such as, for example, RAM, memory chips, device or integrated circuits or on an optically or magnetically readable medium such as, for example, a CD, DVD, magnetic disk or magnetic tape. It will be appreciated that the storage devices and storage media are embodiments of machine-readable storage that are suitable for storing a program or programs that, when executed, implement embodiments of the present invention. Accordingly, embodiments provide a program comprising code for implementing a system or method as claimed in any preceding claim and a machine-readable storage storing such a program. Still further, embodiments of the present invention may be conveyed electronically via any medium such as a communication signal carried over a wired or wireless connection and embodiments suitably encompass the same.

Many variations of the methods described herein will be apparent to the skilled person. For example, the methods described herein can be used to identify/segment features in other blood vessels besides the aorta (e.g. other arteries or veins). Furthermore, the methods described herein can be used to analyse the behaviour of other organs, for example in the liver, spleen, or kidney.

Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features. The invention is not restricted to the details of any foregoing embodiments. The invention extends to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed. The claims should not be construed to cover merely the foregoing embodiments, but also any embodiments which fall within the scope of the claims.