Personality Prediction from Myer Brigg’s 16 Personality Types Dataset
By: T G Narayanan
Tags: Kaggle, Iiitd, Machinelearning2020, Personality Types, Mbti

— by Abhishek Madaan, Neha Rana, and T G Narayanan

What’s it about?

As provided by The Myers-Briggs Type Indicator® (MBTI®) tool, there are 8 personality types, and every individual can be categorized into a particular combination of these personality types based on their behaviour, thinking, decision-making skills, etc. This project concentrates on an area of Machine Learning where given certain twitter posts by 8675 individuals, we predict the personality of that individual. We have used word embedding techniques (tf- idf, Word2Vec, GloVe) with Machine learning and Deep learning algorithms such as SVM, Logistic Regression, BiLSTM, Naive Bayes, and AdaBoost. It was found that SVM + SMOTETomek is best for this classification. XGBoost with resampling also gave quite promising results.

Introduction

Personality is the manner in which one person’s traits and characteristics are defined. Each person is special and unique in their own way and it’s the personality that defines a person. It is also an indicator of how a person sees and interacts with the world and her/his environment. Based on Carl Jung’s theory of psychological type, MBTI (a world renowned personality indicator) indicates ones’s personality preferences in four dimensions: 1. Where you focus your attention — Extraversion (E) or Introversion (I), 2. The way you take in information — Sensing (S) or INtuition (N), 3. How you make decisions — Thinking (T) or Feeling (F), 4. How you deal with the world — Judging (J) or Perceiving (P). Using these four dimensions, a total of sixteen personality types exist:

Importance of the project

The project focuses on analyzing the “Myers Briggs Type Indicator Dataset” and visualizing the data distribution with various plots.

Preprocessing of the dataset is done by splitting of instances into sentences, performing removal of sentences with URLs, etc. Then the occurrences of sentences for each label (indicator) are recorded. The count of the words in the dataset comes from using Count Vectorization and the importance of those words is determined by the use of Tf-idf. Word2Vec and GloVe are used to learn word embeddings.

To handle the imbalance in the dataset, we perform resampling. Resampling can be done either by performing oversampling or undersampling or a combination of the two. The resampling techniques create fairness among the dominating and recessive classes.

Oversampling techniques are used to populate the dataset with minority class instances. Undersampling techniques on the other hand reduces instances of the dominating class and bring it down to the count of the recessive class instances.

Although, the oversampling techniques tend to cause overfitting and the undersampling techniques tend to cause loss of useful information. In our work, SMOTETomek is used which performs oversampling using SMOTE (Synthetic Minority Oversampling Technique) and cleaning using Tomek links.

Now the machine learning algorithms such as Logistic Regression, SVM, XGBoost, AdaBoost, Naive Bayes are implemented to generate models.

A comparison study is done using the performance scores attained using those models.

Dataset Used

https://www.kaggle.com/datasnaek/mbti-type

Methodology

Data Analysis

This study uses the Myers Briggs Type Indicator Dataset. It consists of 50 statuses of 8675 users with personality labels based on Myers-Briggs Personality Types. The dataset consists of two columns. The first column consists of the MBTI personality trait of each user and the second column consist of the statuses of each user. Each status is separated by three pipe characters. The distribution of the dataset based on each personality trait is presented in Table 1 below.

Table 1: Distribution of Personality Traits

According to the table, the distribution of personality traits, ’INTJ’, ’INTP’ and ’INFJ’ and ’INFP’ is much higher than other personality traits. Figure 1 and 2 shows the number of occurrences of each MBTI personality trait in the dataset; in the form of a bar graph and pie chart, respectively.