Visualizing Topic Models with Scatterpies and t-SNE | by Siena Duplan | Towards Data Science

For better or worse, our language has not yet evolved into George Orwell’s 1984 vision of Newspeak (doubleplus ungood, anyone?). But had the English language resembled something like Newspeak, our computers would have a considerably easier time understanding large amounts of text data. Present-day challenges in natural language processing, or NLP, stem (no pun intended) from the fact that natural language is naturally ambiguous and unfortunately imprecise.
Text data is under the umbrella of unstructured data along with formats like images and videos. According to Dama, “unstructured data is technically any document, file, graphic, image, text, report, form, video, or sound recording that has not been tagged or otherwise structured into rows and columns or records.” The label “unstructured” is a little unfair since there is usually still some structure. Images break down into rows of pixels represented numerically in RGB or black/white values. Text breaks down into sentences, paragraphs, and/or chapters within documents and a collection of documents forms a corpus.
For a computer to understand written natural language, it needs to understand the symbolic structures behind the text. Using some of the NLP techniques below can enable a computer to classify a body of text and answer questions like, What are the themes? Is the tone positive? Subjective? How easily does it read?
Below are some NLP techniques that I have found useful to uncover the symbolic structure behind a corpus:
Topic modeling (themes, clustering)
Word/phrase frequency (and “keyword searching”)
Text visualization (Word Trees, Word Clouds both by @Jason Davies)
Embeddings (e.g. Word2Vec)
Sentiment analysis (positive/negative, subjective/objective, emotion-tagging)
Text similarity (e.g. “cosine similarity”)
TF-IDF (term frequency/inverse document frequency)
Part-of-speech tagging
In this post, I am going to focus on the predominant technique I’ve used to make sense of text: topic modeling, specifically using GuidedLDA (an enhanced LDA model that uses sampling to resemble a semi-supervised approach rather than an unsupervised one). One of the difficulties I’ve encountered after training a topic a model is displaying its results. The output from the topic model is a document-topic matrix of shape D x T — D rows for D documents and T columns for T topics. The cells contain a probability value between 0 and 1 that assigns likelihood to each document of belonging to each topic. The sum across the rows in the document-topic matrix should always equal 1. In optimal circumstances, documents will get classified with a high probability into a single topic. With fuzzier data — documents that may each talk about many topics — the model should distribute probabilities more uniformly across the topics it discusses. Here is an example of the first few rows of a document-topic matrix output from a GuidedLDA model:
Document-topic matrices like the one above can easily get pretty massive. Unless the results are being used to link back to individual documents, analyzing the document-over-topic-distribution as a whole can get messy, especially when one document may belong to several topics. This is where I had the idea to visualize the matrix itself using a combination of a scatter plot and pie chart: behold the scatterpie chart!
But not so fast — you may first be wondering how we reduced T topics into a easily-visualizable 2-dimensional space. For this, I used t-Distributed Stochastic Neighbor Embedding (or t-SNE). Taking the document-topic matrix output from the GuidedLDA, in Python I ran:
from sklearn.manifold import TSNEtsne_model = TSNE(n_components=2, verbose=1, random_state=7, angle=.99, init=’pca’)# 13-D -> 2-Dtsne_lda = tsne_model.fit_transform(doc_topic) # doc_topic is document-topic matrix from LDA or GuidedLDA 
After joining 2 arrays of t-SNE data (using tsne_lda[:,0] and tsne_lda[:,1]) to the original document-topic matrix, I had two columns in the matrix that I could use as X,Y-coordinates in a scatter plot.
For the plot itself, I switched to R and the ggplot2 package. The dataframe data in the code snippet below is specific to my example, but the column names should be more-or-less self-explanatory. x_tsne and y_tsne are the first two dimensions from the t-SNE results. row_id is a unique value for each document (like a primary key for the entire document-topic table). x_1_topic_probability is the #1 largest probability in each row of the document-topic matrix (i.e. the topic that document is most likely to represent). topic_names_list is a list of strings with T labels for each topic.
library(ggplot2)library(scatterpie)# high resolution tiff imagetiff(“scatterpie.png”, units=”in”, width=12, height=9, res=500)# plotp <- ggplot() +  geom_scatterpie(aes(x=x_tsne, y=y_tsne, group=row_id, r=x_1_topic_probability), data=data, cols=topic_names_list, color=NA, alpha=0.7) +  coord_equal() +  geom_label(...) +  ggtitle(“Scatterpie Chart”) +  xlab(“”) + ylab(“”) + labs(subtitle=”t-SNE Representation of Guided-LDA Topics Colored and Sized by Topic Probability”) + scale_fill_manual(values=colors) +  theme_minimal() +  theme(text = element_text(color=”white”), legend.position = “none”, panel.background = element_rect(fill = “gray17”, colour = “gray17”),  plot.background = element_rect(fill = “gray17”), panel.grid.major = element_line(colour = “gray25”), panel.grid.minor = element_line(colour = “gray25”), axis.text = element_text(color=”white”))# shut down graphics devicedev.off()
And voilà, there you have the nuts and bolts to building a scatterpie representation of topic model output. In the future, I would like to take this further with an interactive plot (looking at you, d3.js) where hovering over a bubble would display the text of that document and more information about its classification.
If you’re interested in more cool t-SNE examples I recommend checking out Laurens Van Der Maaten’s page.