3D single-cell shape analysis using geometric deep learning
Abstract 3D cell shape is linked to disease.
Despite this, there is a paucity of methods to quantify 3D cell shapes.
Current methods utilise pre-defined measures of cell geometry as opposed to data-driven approaches.
Furthermore, we do not fully understand the cell shape landscape of cancer cells in 3D and the different shape classes that exist in a dataset or how these are correlated with drug treatments.
To address this, we have developed a geometric deep learning method to learn 3D cell shape features and classes simultaneously.
This technique combines an existing dynamic graph convolutional neural network encoder with a foldingnet decoder and improved deep embedded clustering.
We use this model to learn meaningful shape representations of over 70 000 drug-treated melanoma cells imaged by 3D light-sheet microscopy.
We propose describing cells and treatments by a 3D quantitative morphological signature, representing a cell’s similarity to the learned shape classes in the dataset.
This led to the insight that drugs treated with similar inhibitors share morphological signatures.
Introduction
Aberrations in cell geometry are linked to cell signalling and disease [1].
Traditionally, we understand shape using predetermined or “classical” features.
These are pre-selected, handcrafted measures of cell geometry that often require a preconceived idea of which features are important.
These descriptors include region-based shape descriptors such as area, perimeter and eccentricity [2, 3, 4, 5, 6].
In the context of studying living cells in 2D images, these geometric approaches have been successful at identifying major categories of cell shape and linking cell shapes to the molecular signalling pathways that control geometry [7, 8] or respond to changes in geometry [9, 10, 11].
2D shape understanding has been illuminating, however, cells exist in 3D environments, and their 3D geometry is a rich source of information about the inner workings of cells and tissues.
For example, recent pioneering work has demonstrated that shape motifs extracted from 3D images can be linked to lipid membrane signalling and cancer signalling pathways [12], and changes in 3D nuclear morphometry are linked to chromatin organisation [13].
At the scale of developing tissues, the discovery of new 3D geometries has explained how cells pack stably into curved organs [14].
Now, the development of high throughput light-sheet imaging [15, 16] has made it possible to image large numbers of cells in 3D with sub-cellular resolution, leading to a growing need for accurate 3D shape representations.
We have addressed this need using unsupervised geometric deep learning (GDL) to learn shape representations of cells from 3D microscopy images of metastatic melanoma cells embedded in collagen tissue-like matrices.
We have also used improved deep embedded clustering to learn the shape features and classes simultaneously.
Our methods are implemented as a Python package for the ease of use by the biological research community
Methods 2.1 Dataset
WM266.4 melanoma cells were suspended in a collagen matrix and treated with several different drugs.
We imaged over 70 000 cells in 3D using oblique-plane microscopy [16].
We then segmented these 3D single cells, using active contours and Ostu’s thresholding, and used marching cubes to create mesh objects of vertices faces and normals.
We then sampled 2048 points from the surfaces of the meshes to obtain point cloud representation of each cell and nuclei.
2.2 Dynamic graph convolutional foldingnet
The dynamic graph convolutional foldingnet (DFN) follows the design of the FoldingNet [17] with a dynamic graph convolutional neural network (DGCNN)
[18] as the encoder (Step 1 in Figure 1).
Our reason for the amendment of the encoder was from results on ModelNet40 [19], where we achieved higher transfer learning accuracies when using a DGCNN encoder as opposed to original FoldingNet encoder.
Step 1 of our model training is done by minimising the Chamfer Distance used in [17].
We have implemented several encoder-decoder combinations in our Python package for further experimentation.
2.3 Improved deep embedded clustering
Deep embedded clustering (DEC) [20] is a specialised clustering technique that learns feature representations and cluster assignments using autoencoders, simultaneously.
This involves an addition to an autoencoder in the form of a clustering layer.
DEC [20] proposed pre-training an autoencoder and then disregarding the decoder part and replacing it with the clustering layer, but [21, 22] showed this weakens the representitativeness of the features learned.
Improved deep embedded clustering (IDEC) [23] proposed keeping the decoder both and the clustering layer in the second stage of training and jointly optimising the reconstruction loss and the clustering loss simultaneously (bottom right corner of Figure 1).
We followed
IDEC [23].
After an autoencoder, such as the DFN, is sufficiently trained, the clustering layer takes, as input, the lower-dimensional feature vector output from the encoder and maps this to the cluster centres of the feature space.
The cluster centres of this feature space are initialised through k-means clustering and then are trainable weights of the resulting neural network.
The clustering layer assigns soft labels ( q ij ) to each input by mapping features to clusters based on the Student’s t-distribution [24] as a kernel that represents the similarity between a feature vector ( z i ) and a cluster centre ( µ j )
(Step 2 in Figure 1).
This can be interpreted as the probability of assigning input cell shape i to cluster j , hence why this is a soft assignment.
Soft assignments with high probabilities are considered trustworthy, and thus DEC designs a target distribution, p ij , which raises q ij to the second power to place more emphasis on these confident assignments (See Figure 1 target distribution).
We then minimise a clustering loss defined as the Kullback–Leibler divergence between the cluster and target distribution.
This target distribution is updated every T steps and training is stopped when a divergence tolerance is reached.
This divergence tolerance is the proportion of samples that change cluster assignments during the next update.
We set the divergence tolerance to 0.001.
The features at cluster centres become representations of ‘example’ shapes of each shape class.
If there are k clusters in the dataset, each cell is assigned k soft labels (one for each cluster), and each soft label contains scores from zero to one, reflecting how similar the cell is to each of the different cluster centres (with a value of one being most similar).
Since the 3D shape of cells in a fixed assay is a continuous distribution with an indefinite number of distinct classes (cells may be very similar in shape, yet almost never the exact same), it is more natural to describe a shape by a signature of how similar it is to a number of different exemplar classes.
For example, we may wish to describe the 3D shape of a cell to be 30% similar to shape cluster 0, 10% similar to shape cluster 1, 5% similar to shape cluster 2 for shape classes j = 0 , 1 , ...
k , such that the sum of similarity percentages add to 100.
This can be defined as a quantitative morphological signature for the 3D shape of a cell (3DQMS).
This is a direct output of the clustering layer.
Step 2 in our model training is defined by minimising the clustering loss and the reconstruction loss simultaneously.
Results 3.1 Learned shape features predict drug treatment with greater accuracy than previous approaches
Initially we trained our DFN autoencoder without the addition of the clustering layer to extract 128 shape features from point cloud representations of cells and nuclei.
Many studies in computer vision and 3D shape representation learning test the effectiveness of their representations with downstream classification tasks.
This involves using a trained model to extract features from a dataset and then using these features to train a linear support vector machine (SVM) classifier [25, 26, 27, 28].
These tasks generally involve classification between objects from distinct semantic classes [19].
Our dataset is comprised of a single cell type and did not have such distinct semantic classes, so to test our shape representations, we predicted our drug treatment labels.
Following similar procedures to previous researchers [25, 26, 27, 28], we used the trained DFN without the addition of the clustering layer to extract features from each cell.
These features were then used to train a linear SVM.
For all experiments, we use one-versus-one SVM classifiers with an L2 penalty (C = 1), balanced class weights, and intercept scaling during training.
We performed 10-fold cross-validation for each experiment and report mean accuracies.
We found the greatest prediction accuracy resulted from combining both cytoplasmic and nuclear shape features, with predictions up to 89% for cells treated with myosin inhibitors and microtubule inhibitors (Figure 2 A).
This is opposed to an accuracy of 86% when using the original FoldingNet encoder and 82% when using classical features.
Furthermore, we tested whether prediction accuracies were affected by a cell’s environment (closer to the coverslip - in a rigid environment - or further away from the coverslip - in a more tissue-like environment).
Higher accuracies were seen in cells that were closer to the coverslip.
3.2 Improved deep embedded clustering learns a 3D quantitative morphological signature that generates new biological insights
After training the DFN autoencoder to learn 3D cell shape representation, we performed improved deep embedded clustering (IDEC) [23] to learn the 3DQMS of every cell.
We chose 5 clusters as a hyperparameter.
However, this could also be estimated by prior knowledge, or through more formal methods such as the elbow, average silhouette or gap statistic methods when undertaking the initial kmeans clustering.
Deep clustering found 5 shape clusters (Figure 2 B).
We extracted the 3DQMS for every cell and grouped this by treatment, to show the average 3DQMS for cells treated with different inhibitors.
Blebbistatin treated cells had high scores for protrusive shape classes, with Nocodazole treated cells having high scores for the round shape class (Figure 2 C).
Furthermore, we noted that two distinct inhibitors of myosin activation (Blebbistatin and H1152) grouped together, indicating that the 3DQMS may be a powerful tool for using cell shape to identify drugs that act on similar pathways.
Experiments on large drug screens is needed to determine if this is the case in general.
The other classes of treatment effects on shape included Nocodazole, which inhibits the polymerisation of microtubules, CK666, which prevents actin branching and then control cells grouped with those treatments that do not have significant effects on cell shape.
Conclusion
Here, we have utilised a combination of state-of-the-art computer vision techniques and applied them to cancer biology.
We have learned the shape representations of 3D cancer cells using GDL and have shown that these outperform traditional measures of cell shape when predicting drug treatment.
We propose defining cell shapes by their similarity to shape modes in the dataset - known as their 3DQMS - and show that this is useful in linking treatments which affect the same pathway.
Further experimentation on where
GDL techniques will have a significant edge over other methods such as spherical harmonics is needed.
Finally, we implemented our methods as a Python package for ease of use by the research community.