Can LIWC help predict Jungian cognitive functions?
By: Mattias Östmar
Tags: Umap, Xgboost, Data Science, Carl Jung, Liwc

In my previous post Towards Jungian Typology with Data Science I drew the outlines of a novel way to assess Carl Jung’s original personality types using data science. Here is how I’ve gone about with an initial experiment using quite up to date technical methods such as XGBoost and UMAP and the well-researched psychological word categories LIWC. The code used in this experiment is published here for reference.

The website typealyzer.com (R.I.P. 2008–2019) allowed people to post a blog URL and get back the four-letter Myers-Briggs type. Digging deeper into how the MBTI works shows that each of the 16 types have a dominant cognitive function. For simplicity those Jungian cognitive functions can be reduced into four basic functions that are either extraverted or introverted i.e. applied to a persons inner world or thoughts and feelings or outer world of things and other people.

In comparison to ordinary business data such as sales or stock data view thorough the lens of business logic, natural language data viewed through the lens of Jungian theory is way more complex and challenging. That’s why I find it so fascinating. However, as in any data science project one of the first things to do is to reduce complexity, so that’s where we start.

Let’s start by reducing the eight original cognitive functions to four, by just disregarding that each of them should also be either extraverted or introverted.

Sensing ( S )

) iNtuition ( N )

) Thinking ( T )

) Feeling (F)

Reducing linguistic complexity by applying LIWC

The dataset I use if full of noice since it is based on all the text found on an URL put into the typealyzer.com. Lots of irrelevant words and numbers, such as time stamps, link text etc can be found in every sample. Since it’s the psychologically meaningful words where interested in it makes sense to create a structured dataset by counting the words in the LIWC (2007) categories first. We actually end up with with 64 new columns that we’ll later use to predict the four Jungian basic types.

The structured dataset with Jungian types and cognitive functions (left) and LIWC results to the right

Most of the people that took the time to post the actual MBTI type of the posted blog URLs seems to have been of dominant iNtuitive type judging by the distribution of samples for each class. That probably makes sense from a typlogical perspective, since intuitive types are supposed to be more interested in abstract things such as typology systems.

Dominant iNtuitive types are about three times as common as sensing types

There is a handy Python library called imbalanced-learn that nicely complements and integrates into the very popular machine learning library Scikit-Learn. It contains several ways of handling class imbalances such as these and I decided to use the RandomUnderSampler to get about the same amount of samples in each class. After balancing the classes we have about 2500 samples in each class. That will do!

Predicting the four functions with XGBoost

The technical description of XGBoost is that it is a decision-tree-based ensemble Machine Learning algorithm that uses a gradient boosting framework. It suffices to know that it has performed very well in many machine learning competitions the last years and is considered a good first choice for general machine learning tasks today.

Here is a visualisation of how it managed to predict the four Jungian functions based on the 64 LIWC categories.

Almost no predictive power at all

A quick look at the confusion matrix tells that there seems to be almost no predictive power at all. This is confirmed by looking at sklearn’s more detailed classification report.

No better than random

We could just as well have thrown a dice to guess the Jungian functions from the LIWC categories. Absolutely no indication of predictive power what so ever. It’s nice when you get experimental results that leaves no room for doubt, isn’t it!

Too many columns makes the algorithm overfit?

Let’s see what happens if we reduce the 64 LIWC-category columns into just two. Since we’re dealing with features derived from natural language we might not want to use good-old PCA. And since UMAP is faster and newer than t-SNE (just kidding, even though it’s true) we’ll go with UMAP. Let’s reduce the 64 dimensions (LIWC-category columns) to only two so we can visualise them nicely.

A beautiful starry boomerang in a night sky signifying nothing

Each of the dots here represent a row in the dataset and the four Jungian functions a colour-coded to see how well they spread out. They don’t actually. It’s all just a chaotic mess where dots of each colour are spread all over the boomerang-like cluster. The little mostly blue satellite of dots in the upper part of the chart as well as the other small clusters are most likely outliers that adds nothing to the overall understanding.

Let’s again do the same classification with XGBoost, but this time with only two features (i.e. columns). We should probably be reducing the dimensions a little less aggressively, let’s say 5–10 dimensions would be more reasonable. But since we now know that we calmly can keep our hopes down to find anything in the data anyhow, let’s just use the same embedding that we used for plotting.

Even worse, which shouldn’t come as a surprise. Especially iNtuition is misclassified as Sensing

If we have a look at the classification report we can conclude that we’ve gone from a result exactly in par with random throws with a dice to even worse than trowing dice!

Accuracy below 25 per cent with four classes. That sucks.

What is to blame for the poor predictions?

LIWC, XGBoost and UMAP are all well-researched and battle-tested methodologies. That, unfortunately, can not be said about how the original data for the dataset was collected. Here are some of the short-comings of how the data was collected.

Anyone could post any URL, not only blogs.

Anyone could state that the posted website author had a certain MBTI type

Any type of shallow online test could be the grounds of the stated type

The text data from each URL is full of noice such as boiler plate text

I would say that even though we’ve used pretty standard and reasonable choice of machine learning algorithms and evaluation metrics here. But any data science project fundamentally comes down to one paramount limitation; sh-t in — sh-t out.

Let’s see if we can get hold of a better dataset later on and repeat the experiment! All the code is published here.