A Must-Know New Clustering Algorithm for Disease Modelling
By: Lucy Rothwell
Tags: Biomarker, Unsupervised Learning, Clustering, Editors Pick, Machine Learning

Classic clustering algorithms like K-Means and Gaussian Mixture Model (GMM) are great for modelling data when we want to find cross-sectional subtypes (aka clusters). This kind of subtyping is used a lot in medicine. A well-known general example is that of subtyping diabetes into “Type I” and “Type II” using a single blood sugar measurement. This can help doctors decide whether to prescribe insulin injections or lifestyle changes.

Figure 1. Subtypes of a disease by phenotype using cross-sectional biodata; for example, a single measurement of blood sugar or a medical image. An example would be diabetes which is subtyped into “Type I” and “Type II”. Image created by author.

Grouping a disease by stage is also very useful in medicine, this time for modelling disease progression. For example, a model for grouping cancer into stages 1–4 was developed using longitudinal data (multiple measurements from the same person over time). The model itself was developed using longitudinal data but once developed, allowed doctors to determine which stage a patient is at using only a single cross sectional measurement (i.e., tumour size in millimetres). Knowing the stage of cancer may help doctors decide whether radiotherapy or chemotherapy is needed.

Figure 2. Grouping disease by stage, using longitudinal data which measures progression over time. An example would be cancer which is usually grouped into stages 1–4. Image created by Author.

The downside of these kinds of staging models is that they assume all patients come from the same type of the disease. I.e., they account for disease progression, but there is no account of disease subtypes.

Conversely, the “Cross-sectional subtypes” mentioned previously explain subtypes but not progression. I.e., they assume all patients are at the same stage.

So what if we want to do both, i.e. find subtypes of a disease based on how it progresses over time, and create that model using only cross-sectional data?

Introducing the Z-Score SuStaIn (Subtype & Stage Inference) Algorithm

The Z-Score SuStaIn is an unsupervised machine-learning technique that identifies population subgroups (clusters) with distinct patterns of disease progression based on biomarkers. This is shown abstractly in figure 3 below.

Figure 3. The circles represent groups of biomarkers. The colours (green, orange, blue) represent different subtypes i.e., different progression patterns. Note: the subtypes produced in SuStaIn are not based on clinically defined phenotypic disease subtypes (such as Type I and Type II diabetes), but rather on how they each progress over time. Image created by author.

Biomarker: “Any substance, structure or process that can be measured in the body or its products and can influence or predict the incidence of outcome or disease” (WHO, 2011). This includes everything from blood sugar measurements and heart rate to sweat levels and electrical signals in the brain.

To illustrate this on a real example, we can look at how 3 subtypes of Alzheimers were found based on how each progresses in terms of neural biomarkers (see image below). The biomarkers (features) here are atrophy (shrinkage) levels across various brain regions, measured by MRI scans. The colours are explained in the figure caption.

Figure 4. This image shows 3 subtypes of Alzheimers disease based on how each type progresses (measured using MRI brain images). The colour coding represents biomarker degeneration where white indicates no pathology, red shows moderate pathology, pink shows medium and blue shows maximum pathology. Image adapted from Young et al (2018).

As we can see, 3 distinct patterns of disease progression have emerged, based on different patterns of neurological decay. For example, as highlighted in Figure 5 below, subtype 2 starts to show shrinkage in the occipital lobe (the small, far left corner on the brain image) at stage 9, yet subtype 3 doesn’t show atrophy in that area at all, even by stage 25. These are very distinct types of disease progression which may require different medical interventions.

Figure 5. Subtype 2 shows atrophy in the occipital lobe at stage 9 but subtype 3 doesn’t show atrophy in this area even by stage 25. Image adapted from Young et al (2018).

But how can disease progression over time be modelled using cross-sectional data?

SuStaIn is a clustering algorithm that works in tandem with a disease progression model, in this case the Z-Score model (hence the name Z-Score SuStaIn). Z-Score SuStaIn models disease progression using z-scores of the biomarker data with which it measures the extent to which each biomarker deviates from normality, i.e. how the biomarker differs from that of healthy individuals (the control group). This can be seen in Figure 4 above where the different colours represent the number of standard deviations each biomarker (in this case brain shrinkage level) is from the control mean. Therefore the temporal stages (and the subtypes) are essentially hidden variables that we draw out from the data. This is how we measure progression using just cross-sectional data.

The Z-Score SuStaIn algorithm produces an output similar to that in Figure 4 in two alternating steps. First it selects the sequence of events with the maximum likelihood given the (z-scored) data. Secondly it subtypes individuals and estimates progression patterns in each subgroup. The algorithm then alternates between subtyping individuals with similar progression patterns and re-estimating the progression patterns in each subgroup until it converges, producing several subtypes with different progression patterns (as shown in Figures 3 and 4).

The key advantages of this approach are:

It can allow classification of patients based on both disease subtype and disease stage. If developed into an application, this could essentially allow clinicians or researchers to take an MRI scan of a patient, run it through the trained model, and be told which subtype and stage the patient is at in their given disease. Disease progression can be measured using only cross-sectional data which is much (much!) easier to obtain than longitudinal data, especially in medicine. This means it can be used to understand a wider range of diseases. It gives us a much more fine-grained model of disease progression than existing models. The model that underpins the Z-Score Model, called the EBM (Event Based Model; explained in “The Maths” section), provides the key information doctors need; i.e. whether a person is ill or not. This is shown when biomarker measurements transition to abnormal levels. However, by using z-scores, the SuStaIn model shows how people transition through multiple intermittent stages of normality to abnormality, giving us a more fine grained insight into progression. This fine grained insight allows us to also develop more accurate progression-based subtypes from the data which could lead to more targeted drug development. For example, instead of developing one blanket drug for all types of a disease (which may come with maximum side effects) drugs can be developed specific to each subtype, and effects can measured in line with the progression pattern of that subtype. This could potentially minimise unnecessary side effects as well as being altogether more effective in treating or alleviating symptoms.

The main drawback is that Z-score SuStaIn only works on data with normally distributed variability in the control population. If the data isn’t Gaussian, results will be poor. (In this case it would be better if the recently developed nonparametric Kernel Density Estimation Mixture Model (KDEMM) by Firth et al, (2018) was used instead of the Z-Score Model).

The Data

The input data for Z-Score SuStaIn could be any kind of biomarker, measured by for example, blood tests, MRI scans, cardiograms etc.

Figure 6. Image created by author.

After standard data cleaning, 3 things need done to the data before it can be put into the algorithm:

Adjust for the effects of covariates. In brain volume measurements for example, a covariate could be age as this variable affects brain volume independent of disease (our brains tend to naturally shrink a little as we get older). The effects of non-disease related factors like this should be stripped out so we can measure only the effect of the disease on the biomarker measurements. (Code for doing this via regression is in the GitHub repo at the end of the article). Z-score the data. This entire algorithm is based on measuring disease progression based on how biomarkers deviate from the mean. Therefore we need to input the data in the form of z-scores; i.e. the number of standard deviations a data point is from the mean (in this case, the control mean). Make sure all features increase with disease. If any features decrease with disease multiply the data in these features by -1. For example, the biomarker brain fluid will likely increase as the biomarker frontal lobe volume decreases (this is because when parts of the brain shrink, the body fills the empty space with fluid). If there is a mix of increasing and decreasing features, the algorithm will get confused and produce inaccurate results.

The python code for all these steps and the analysis itself is in the SuStaIn Jupyter Tutorial, also linked at the end of this article.

The Maths

An extra section for those interested in the details of SuStaIn.

SuStaIn is a likelihood based subtyping model which works in tandem with a disease progression model —in this case the Z-score model.

The Z-score model aims to predict the order of events that maximises the likelihood of the data, where an event is the transition of a biomarker from one stage to the next (based on z-score).

The Z-score model is built upon the Event Based Model (Fonteijn et al, 2014), therefore to understand the Z-score model, it’s easier to first understand the EBM. The EBM measures disease progression in a binary manner by measuring the point at which biomarkers transition from normal to abnormal.

Figure 7. The Event Based Model (EBM). Image taken from Young et al (2018).

The equation above shows the Event Based Model (EBM) where:

X is the data set of raw biomarker measurements (not z-scores)

is the data set of raw biomarker measurements (not z-scores) S is the sequence in which biomarkers transition from normal to abnormal.

is the sequence in which biomarkers transition from normal to abnormal. p(X|S) is the most likely order of events that maximises the likelihood of the data.

is the most likely order of events that maximises the likelihood of the data. p(k) is the prior likelihood of being at stage k at which the events E1,…Ek have occurred (a uniform prior is used).

is the prior likelihood of being at stage k at which the events E1,…Ek have occurred (a uniform prior is used). p(x|E) and p(x|¬E) are the probability the event has occurred (the biomarker has transitioned to abnormal) and not occurred given a data point x within X. These are modelled as normal distributions.

When only a small number of biomarkers is used, all possible orderings can be enumerated and the order of events, Ŝ, is the sequence which maximises P(X|S) as in Figure 7, above.

However, because the number of possible sequences grows factorially as the number of biomarkers increases, Markov Chain Monte Carlo (MCMC) sampling of p(S|X) is used to find Ŝ when the number of sequences to sample is too large to enumerate. (This process is explained on page 4 of this article).

Once we have Ŝ, it can be used to estimate a disease stage for individuals given their biomarker measurements. The disease stage, k, is defined as, the stage (i.e. the number of events that have occurred so far) that has the highest probability given the data and our sequence. This is calculated by maximising the equation in Figure 7 above, as shown in Figure 8, below:

Figure 8. Optimising in relation to disease stage, k. (Image taken from Firth et al, 2018).

The Z-score Model then builds on the EBM model as shown below:

Figure 9. The Z-score SuStaIn model

There are two key innovations in the Z-score model (compared to EBM). The first is that X now equals the z-scores of the data rather than the raw data. The second is that S now equals the sequence in which biomarkers move through multiple stages rather than just from normal to abnormal. So we are now able to measure progression, not by a single transition from normal to abnormal, but by transitions between multiple stages of disease progression. This gives us a much more fine grained model of how a disease develops.

SuStaIn is then fitted by simultaneously optimising subtype membership, subtype trajectory and the posterior distributions of both (see Figure 10 below). It is fitted hierarchically, whereby the number of subtypes that can be supported by the data is selected via model selection criteria CVIC (cross-validation information criterion) which is obtained from ten-fold cross-validation.

Figure 10: The SuStaIn model

Where:

C is the number of clusters (subtypes)

is the number of clusters (subtypes) f is the proportion of subjects assigned to a particular cluster (subtype)

is the proportion of subjects assigned to a particular cluster (subtype) M is the overall SuStaIn model

A full explanation of these mathematical models can be seen in the SuStaIn intro paper (Young et al, 2018).

In conclusion

If you are looking to subtype disease or model disease progression using biomarkers, it could be very worthwhile including SuStaIn in your model evaluation.

I hope that’s useful! Feel free to direct message me on LinkedIn if you have any questions. You can also subscribe to my posts here.

Key Resources for Implementation

The SuStaIn method was developed at UCL by Dr Alex Young and her colleagues, with senior author Professor Daniel Alexander (Young et al, 2018). It was implemented in python by Dr Peter Wijeratne with support from Dr Leon Aksman and Dr Arman Eshaghi.