GSoC Week 3
By: Sarit Adhikari
Tags: Clustering, Gsoc, Sentiment Analysis, NLP, Topic Modeling

When we think of clustering at the repository level, the document would be a collection of descriptions and comments across all issues and pull requests. Currently, only the most active repositories (in terms of the total number of comments)are considered. The bar chart below shows the number of comments for each repository.

As described in previous blog posts, the clustering is done using tfidf features derived from all the text messages in the repositories. The result below shows 4 clusters with each point denoting a repository.

Topic Modeling

To get a sense of what each document is about, we can perform topic modeling. The common approach to topic modeling is latent Dirichlet analysis (LDA) which assigns probability distribution over topics for each document. The topics, in turn, are probability distribution over words. In our case, the top words in each topic are

topic 1 : bitnami, mariadb, backup, store, mongodb, resource, pivot, tracker, docker,restor

topic 2: tabl, vmware, sign, Concours, pipelin, command, row

topic 3 : rabbitmq, pivot, channel, rabbit, amqp, cash, integr, spring,

topic 4 : spring, java, boot, springframework, class, method, secure, framework, jar

topic 5 : src, coverage, harbor, diff, tree, vmware, badg, relev

One important point to be noted here is that we cannot use tfidf feature matrix for topic modeling using LDA. Instead, we use count vectors as the features for LDA.

The table below combines results of clustering and topic modeling. The last column denotes the cluster number assigned to the repo. topic* columns denote the percentage combination of topics for the document. For e.g. repo 25432 is mainly composed of topic 3 which includes words like rabitmq, pivot, amqp, cash, etc.

The aforementioned approach captures the topic of discussion in each cluster. But another perspective would be clustering repositories based on the conversation mechanism. For this, we use the count of words belonging to certain parts of speech (noun, adverbs, adjectives,etc.)and words associated to certain emotions (like anger, joy, sadness,etc.). This will hopefully capture the conversation mechanism and culture across different repositories.

Clustering based on POS and emotion lexicon

The words in a sentence can be categorized into different parts of speech (POS). For e.g. NN (singular noun), JJ (adjective), IN (preposition), RB (adverb), etc. The relative frequencies of these categories in the document can be taken as features.

NRC emotion lexicon is a collection of words into categories like anger, surprise, anticipation, disgust, and trust. For e.g.

abandon => sadness, fear

success => anticipation, joy

wait => anticipation

helpful => joy, trust

The relative frequencies of words denoting these emotions will be another set of features. The following exploratory analysis shows the relative frequency of words denoting a particular emotion across repositories. (all repositories are not shown to avoid clutter.). For e.g. repo 2552 has a high percentage of words denoting ‘trust’ compared to other repositories. Repo 25441 is high on ‘anticipation’ and ‘joy’.

After k-means clustering on the new set of features, the cluster assignment can be visualized as follows

The repositories which belonged to the same cluster contentwise necessarily won’t belong to the same cluster when it comes to conversation mechanism. For e.g. repos ‘30067’ and ‘25857’ are assigned to the same cluster in both methods but repo ‘27962’ which was on same cluster as previous two according to the first approach, now is assigned to a different cluster. This means ‘276962’ is likely to be similar to ‘30067’ and ‘25857’ contentwise but has a different conversation structure.