Deep Learning for Reconstructing Continuous Processes
By: Aseem Kashyap
Tags: Cell Biology, Deep Learning, Pytorch, Dimensionality Reduction, Convolution Neural Net

Reconstructing a continuous process

Dynamic processes like the cell cycle are made up of a continuum of gradually varying stages. Precise demarcation of the beginning and end of these stages can be rather challenging, partly due to significant overlap between the stages. The cell images used for training the CNN however were categorised into three distinct stages — G1, S and G2. This however should not mislead us to assume that the images within a particular stage are homogeneous.

In this section, we combine deep learning model predictions with dimensionality reduction techniques for a fine-grained understanding of classes by simultaneousely visualizing both the differences within a class and the relative ordering of classes.

We do this by taking predictions from the trained model, reducing their dimensions and visualizing them in a two dimensional space.

Let us focus our attention on the final layer of the ResNet101 model. The cells images in this layer are organized in a linearly separable manner with similar images closer to each other. The distance between different images in this imagenary hyperplane can be thought of as differences in the features extracted by the deep learning model i.e the outcome of the learning process. We could take these model prediction on the validation dataset i.e. the output of this final layer and feed them into a non-linear dimensionality reduction algorithm like t-SNE or UMAP to visualize the contained information in a 2D space.

Visualization of the validation dataset based on dimensionality reduction of the model predictions using PCA. [Images generated by author]

Principle Component Analysis (PCA) can detect linear relationships between features by transforming data into a new lower dimensional space, where each dimension is orthogonal to the other. The components arising from this analysis are sorted from the ones that explain the highest to lowest variance in data and only components with most interclass variance are used in the visualization. The underlying assumption that interclass variance is larger than the intraclass variance makes PCA inadequate for our data set as we are also interested in understanding differences within each class. Despite these limitations, PCA was able to capture the basic ordering of the interphase stages(see image above).

Visualization of the validation dataset based on dimensionality reduction of the model predictions using t-SNE. [Images generated by author]

Visualization of the validation dataset based on dimensionality reduction of the model predictions using UMAP. [Images generated by author]

While there are significant differences between t-SNE and UMAP with respect to their ability to preserve global structures, both algorithms do a reasonable good job of charting the progress of interphase stages from G1 to S and G1. Unlike PCA which merely separated the three classes, t-SNE and UMAP were successfully able to chart the variations within each class.

This approach of coupling a deep learning model with dimensionality reduction provides a unique way of understanding classes in continuous system. As is evident from the above two visualizations, images of a particular class do not clutter together, instead they exist over a continuum of space that denotes gradual variations in shape. The images of a particular class merge into the subsequent class, thereby reconstructing the transition of cells from G1 to S and eventually to G2 phase. There is also a considerable variation in shapes of G1 stages a denoted by its significantly broader spread(in blue).

At no point during the training process was the ordering of three interphase stages explicitly or implicitly specified. At no point did we say that S was the intermediate phase and that cells transition from G1 to G2 via S phase. The ability of the model to arrive at this ordering of stages by learning gradual variations in size is an immensely significant result for biological systems. Imagine not knowing the actual ordering of cell cycle stages and being able to reconstruct them simply by training a model.

Subcellular Image Classification

While cells exhibit dramatic structural rearrangements during the M-phase, the subcellular components undergo extensive re-organization in both interphase and during cell division. The S phase of the interphase entails DNA replication, resulting in a doubling of chromatin content and corresponding change in nuclear size. Other cell organelles like Golgi apparatus and endoplasmic reticulum too exhibit cell cycle dependent dynamics.

It turns out we can also track changes in these subcellular components for classifying interphase stages. In this section I present results from training a simple resnet18 model to classify microscopic images of fluorescently labeled human cells into G1/S or G2 phases. Two different model are trained to classify DNA and golgi matrix protein (GMP) pattern in the interphase.