Utilizing Natural Language Processing Methods With Supervised Learning on Reddit Data
By: Christopher Kuzemka
Tags: Machine Learning, Python, Naturallanguageprocessing, Reddit, Data Science

One of the largest schools of interest in the vast world of data science is machine learning. Machine learning is a field of study focusing on having a computer make predictions as accurately as possible, from data. Today, there are two main types of machine learning used: supervised and unsupervised learning. In supervised learning, a programmer will have access to data that depicts an outcome from a certain pattern of features. The “outcome data” can then be used as a prediction measure for modeling a machine learning process, through training. In supervised learning, classification scoring metrics such as accuracy and precision are unlocked for a data scientist to estimate how well their model performs where as in regression learning, scoring metrics such as mean absolute error and mean squared error are often utilized to complete the same objective of measuring the effectiveness of a machine learning algorithm. Many times a programmer may not have access to “outcome data” or prediction data and will have to resort to unsupervised learning. In unsupervised learning, different approaches are taken to help a data scientist determine useful information from data. Often, unsupervised learning will encompass the analysis of the relationship between features. Methods such as “clustering” and “principal component analysis” are used to classify and determine significance of features for other models. Ultimately, both types of machine learning are a subset of AI development and have their roots embedded in computational statistics. Each type of machine learning is powerful and utilizes unique methodologies for uncovering pattern trends in data.

Today, we are going to focus primarily on supervised machine learning models and their use with natural language processing practices. While data science may intuitively seem like a field for only analyzing numerical data (which is correct in a literal sense), a big focus in industry is placed on interpreting human language data in a programmatic way. Businesses thrive from learning in what their users are saying about them (particularly in sentiment analysis), and machine learning is an excellent way to tackle the challenge of “how does one understand what people are saying from a large data set of human language?” Let’s explore that by going through a classification process involving Reddit data…

Consider This Problem Statement…

Reddit, the popular, “community run” social media website, has a dilemma in which two subreddit content are mixed together into one subreddit. You as a data scientist are tasked with creating a machine learning model to classify which content comes from which subreddit to help the Reddit developers and subreddit moderators clean up this blend of content.

This form of a problem statement can be interpretable for different situations which may occur in the world across various domains — imagine the case scenario where a hacker maliciously blends categorical content for pleasure or where a developer may want to create a classification application for users to navigate through. This problem statement can also be transferable for situations where sensitive and valuable information is accidentally merged with data that does not associate.

The key takeaway here is that this is a binary classification problem, where an outcome can be determined with a simple question of asking whether the data associates with one class or another. For this particular problem statement, we are going to analyze whether submission content belongs to the subreddit “r/aww” or “r/natureismetal.”

What are These Subreddits and Why is Researching Them Important for Our Problem?

Each subreddit has a vast difference in type of content it showcases. This is useful to understand when considering how well our model will perform. Similar content may be more difficult for a computer or a person to classify, where as polarity in content is more intuitive to identify. In “r/aww”, textual submission content is relevant to posts that nurture feelings of positive sentiment; content is not graphic nor intimidating to experience. For “r/natureismetal”, posts are more closely related towards negative sentiment, where NSFW (“not safe for work”) content is allowed; content is often more graphic and more intimidating to witness.

In any given data science problem, a good portion of one’s time should be spent on researching the involved subject. A lack of subject matter knowledge will more often yield a poor result and ultimately may spread misinformation to anyone who chooses to study from your work. Furthermore, research on a relevant topics may also help one discover more about the different and more efficient kinds of approaches one can make to solve a problem. In this case, it was discovered through the subreddits’ rules and guidelines that each subreddit expresses a different image for what it chooses to popularize — this affects how post titles are made. For instance, r/natureismetal requires that submission titles are descriptive while r/aww does not have such requirement. Furthermore, r/aww prohibits any “sad” content whereas r/natureismetal endorses the showcasing of graphic animal content or grand acts of nature. It was also noticed that each subreddit domain has a vast difference in size in userbase: r/aww showed about 24.4 million users while r/natureismetal showed about 1.4 million users.

A portion of our time was also spent on analyzing Reddit’s platform in general. It was discovered that subreddits undergo a judicial system of sorts when it comes to posting content. Each subreddit has moderators and auto-moderators (robotic moderators) which patrol the subreddit for content not appropriate for the community’s image. Many times, ill-suited posts are removed from a subreddit, but some posts manage to squeeze by the judicial system. The next filter is a userbase reaction where more popular submissions will often receive more upvotes, comments, and virtual awards to increase a post’s popularity — which often yields more upvotes, comment, and virtual awards as if it were a positive feedback loop. Such applauded submissions will serve as better choices for a user to discover what image a subreddit expresses.

All of the above aforementioned regarding the subreddits and Reddit as a platform must be taken into consideration when analyzing results. For now, we will continue with analyzing the data.

What Data are We Analyzing?

Luckily, we have access to 2,500 submission texts from each subreddit before their fictional merge, where each post in our dataframe has an associated label showcasing what subreddit the post is from. The data was scraped, prior to the merge, and may be enough for us to make a working model. With access to such data, we can very easily integrate a supervised machine learning model.

Let’s Explore What We are Working With…