Personality Analysis Using Machine Learning

The goal of this project is to develop a software capable of performing sentiment analysis on bodies of text with a given author. This tool will mimic IBM Watson's Personality Insights Service - a popular online resource and application programming interface (API) for personality classification. 
==Problem Definition== 
===Background===
There are many sentiment analysis tools available for public use, however, many have significant drawbacks, are not well documented, or perform classification with relatively poor accuracy. We plan to produce the initial framework needed to create a new sentiment analysis tool that remedies many of these problems. Specifically, we will focus on web mining Twitter tweets, storing tweets in a database with relevant statistical information, and training a machine from the collected data. In time, the results of this project may develop into a mobile app.
===Requirements===
Data/Web Mining Requirements	Data/Web Mining Requirements 	Data/Web Mining Requirements	Data/Web Mining Requirements	Data/Web Mining Requirements	Data/Web Mining Requirements
No. 	Need 	Metric 	 	 	
1 	Text Sample 	1200 - 3000 words 	 	 	
2 	Quality of Text 	Raw/Unprocessed 	 	 	
3 	Author 	Singular/Separable 	 	 	
4 	Retain Information 	Copyable to Repository 	 	 	
5 	Semi-Automated 	10000's tweets/crawl 	 	 	
6 	Location Retention (i.e., the URL)	Scraper Pulls URL 	 	 	
Database Requirements	Database Requirements	Database Requirements 	Database Requirements	Database Requirements	Database Requirements
No. 	Need 	Metric 	 	 	
1 	Text Storage 	1,000,000's of entries 	 	 	
2 	Links to local copy 	Cataloged 	 	 	
3 	Author of text 	Cataloged 	 	 	
4 	IBM Watson Output 	Numerics/Classifications indexed	 	 	
Trained Machine Requirements	Trained Machine Requirements 	Trained Machine Requirements	Trained Machine Requirements	Trained Machine Requirements	Trained Machine Requirements
No. 	Need 	Metric 	 	 	
1 	Process entries 	1,000,000's to train 	 	 	
2 	Approximate IBM Watson Output	Mean Average Error 	 	 	
==Deliverables==
*Twitter mining tool capable of collecting and combining numerous tweets from verified users/authors into text samples
*Text samples passed through IBM Watson Personality Insights Service
*Database with large collection of authors, indexed IBM Watson output, and statistics
*Machine trained on collected data
==Design==
===Database===
For this project we are using a MySQL database. Each entry for the database contains the author’s name, the original text file, the URL of the source, the JSON objects from our scrapper (Tweepy), and the JSON objects from IBM Watson’s output. Eventually we will add the Five Factor Model information results from our machine. Until then we will collect data in order to use it to train our machine.
===Twitter Scraping===
Sample is based on Twitter user profiles. A Python package called Tweepywas used to extract data from Twitter. Twitter restricts the number of API calls that can be made, so the scraper is be set up to run without intervention. Tweepy API information can be found HERE 
Scraper	Flow
 	
===Text Preprocessing===
Input text will be processed using an algorithm called GloVe*[8]. GloVe uses a strategy in which relationships between words can be found by analyzing their co-occurrence. Specifically, GloVe optimizes a global log-bilinear language model with a weighted least squares objective such that the difference between the dot product and co-occurrence probability between any two word feature vectors is zero. The resulting trained word vectors can be used as features in various machine learning algorithms. 
GloVe Model	Explanation
center 	
; Weighting function 
: Prevents overemphasis of large co-occurrence counts
; Dot product
: Used to encode co-occurrence probabilities in vector space
; Biases 
: Maintains symmetric properties and absorbs independent variables
; Log of co-occurrence count
: Interacts with the bias bi to produce the co-occurrence probability of j in the context of word i
===Machine Learning===
Two machine learning (ML) approaches, Gaussian Process (GP) and Convolution Neural Network (CNN), were considered. Comparisons were made between the two to determine which approach would best suit the needs of the project. Due to the statistical complexities of GP [9], it was decided a CNN would be implemented. 
Approach 	Pros	Cons
Gaussian Process 	
* Uses GloVe output as features	
* Statistically heavy
* Difficult to determine best co-variance function
Convolution Neural Network	
* Used for five-factor model personality classification
* Identifies most relevant layers (performance)	
A CNN personality classifier based on Majumder et al. [6] implemented by SenticNet was chosen. The package includes text pre-processing, which can take the place of GloVe. Test will have to be conducted to determine whether GloVe provides any meaningful benefit.
===Training===
The CNN was trained on Watson's output, which includes both percentile and raw scores for each personality trait. Mean absolute error was used as a metric to measure how far off our machine was from Watson. Below are the MAE values for each trait based on percentile and raw scores and data set sizes of 1000 and 6000 text samples. For comparison, IBM Watson achieved a MAE score of 0.12 averaged across all five traits. 
850px
==Project Learning==
===References===
[1] Adedoyin-Olowe, Mariam, et al. “A Survey of Data Mining Techniques for Social Media Analysis.” 2013, Journal of Data Mining \& Digital Humanities, 2014 (June 24, 2014) jdmdh:18. 
[2] Arnoux, Pierre-Hadrien, et al. “25 Tweets to Know You: A New Model to Predict Personality with Social Media.” 2017. https://arxiv.org/ftp/arxiv/papers/1704/1704.05513.pdf 
[3] Digman, John M. "Personality structure: Emergence of the five-factor model. Annual Review of Psychology" 1990. 41.1: 417–440. 
[4] Kalghatgi, M. P., et al. “A Neural Network Approach to Personality Prediction based on the Big-Five Model.” International Journal of Innovative Research in Advanced Engineering, vol. 2, no. 8, 2015, pp. 56–63. 
[5] Kaur, Arvinder, and Deepti Chopra. “Comparison of Text Mining Tools.” Institute of Electrical and Electronics Engineers, 7 Sept. 2016, pp. 186–192. ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7784950. 
[6] Majumder, Navonil, et al. “Deep Learning-Based Document Modeling for Personality Detection from Text.” IEEE Intelligent Systems, vol. 32, no. 2, 2017, pp. 74–79., doi:10.1109/mis.2017.23. 
[7] Mnih, Andriy, Hinton, Geoffrey. "Three New Graphical Models for Statistical Language Modelling." 24th International Conference on Machine Learning, 2007 
[8] Pennington, Jeffrey, Socher, Richard, Manning D., Christoper. "GloVe: Global Vectors for Word Representation." 2014. https://nlp.stanford.edu/pubs/glove.pdf. 
[9] Rasmussen E, Carl, Williams K. I., Christoper. "Gaussian Processes for Machine Learning." 2006. http://www.gaussianprocess.org/gpml/.
===Important Links===
Tweepy Documentation 
Watson Personality Insights 
GloVe & GitHub Repository 
Gaussian Processes 
WEKA 
Intro to CNN's 
Mairesse Feature-Based Personality Recognizer 
==Team Information==
Picture	Bio	Discipline
 	Paden Rumsey:
I come from a small town in Idaho called St. Anthony. I came to the University of Idaho almost three and half years ago and I’m finally graduating with a B.S. in Computer Science next spring. I initially got into computer science because of my interest in artificial intelligence and “smart” computers. The trends appeared to be moving in that direction and it seemed like a field I would really enjoy. The classes I’ve taken at the university have only strengthened that conviction. This senior project is a great opportunity to learn more techniques in machine learning and display the ones I’ve already learned.	Computer Science
 	Lise Welch:
I was born and raised in Boise, Idaho and started attending the University of Idaho in the Fall of 2013. I dabbled in Mechanical and Electrical Engineering my first couple years at the university but have since found my passion in Computer Science and am looking forward to graduating with my B.S. in the Spring. Outside of school you can find me at the Alumni Office helping them plan the university’s biggest events such as Homecoming and Parents’ Weekend.	Computer Science
 	Austin Sass:
I've lived in the area of Moscow, Idaho for most of my life. Most of my time is currently being spent in the Neurophysiological Imaging and Modeling Laboratory at the University of Idaho, where myself and team members are studying Space Flight-Associated Neuro-ocular Syndrome as it pertains to visual impairment in Astronauts. On the side, I develop flight simulation software and play music with the "Sass Brothers" band.	Computer Science
 	Seth Forrest:
I am a Technophile, I love exploring the application of technology in our daily lives. I am studying Mechanical Engineering and Computer Science in order to further investigate how tech can be integrated into our culture. I have a particular passion for the the union of the cyber and the physical, and how technology can be used to improve people's lives.	Computer Science/Mechanical Engineering
==Document Archive==
* 
*