Here’s How I Use Natural Language Processing In Stock Price Analysis
By: Trist'N Joseph
Tags: Machine Learning, Artificial Intelligence, AI, Data Science, Stock Market

Here’s How I Use Natural Language Processing In Stock Price Analysis

Using NLP and Granger causality to analyze the relationship between the sentiment of a written article and a stock’s price

Image by Trist’n Joseph

Being able to accurately predict the stock market is like being able to see into the future. Stock market prediction refers to the act of attempting to determine the future value of a company’s stock that is traded on an exchange. However, being able to accurately predict the stock market is like being able to ride a purple unicorn; that is, it probably is not possible.

There are way too many factors to consider which can affect a stock’s price and building a model that consists of all these factors will likely produce poor predictions in the long run. Because of this, I do not think that the goal should be to accurately predict the stock market. Rather, it should be to determine and understand the factors which have the greatest influence on the stock market’s fluctuations.

Image by Trist’n Joseph

It is quite common to hear that a stock’s price is nothing more than a random walk and thus, it cannot be predicted. Whether this is true or not, I believe that it is possible for key external events to have a significant impact on a stock’s price. Understanding and investigating these events could lead to understanding a stock’s movement better; even understanding just 1% more of these factors could imply significant returns.

A potential issue, however, is that is it not usually immediately clear what information is relevant to a particular problem. This kind of information can be referred to as alternative data. In the world of finance, alternative data is referred to as information that is beyond the typical company filings, earnings calls, or fundamental data sets.

The use of alternative data in the finance industry is rapidly growing as investors are looking for new signals which can give them an edge over their competitors. In other words, analysts might just be able to ride a purple unicorn if they can effectively utilize an appropriate set of alternative data.

Image bt Trist’n Joseph

Since June 2020, I have been working on a project to understand the power of natural language processing (NLP) in the context of the stock market’s movements. My motivation for this comes from the fact that the stock market is a forward-looking instrument. This means that it does not reflect a financial market’s current situation, but it reflects an investor’s outlook on that financial market. If investors generally believe that Apple will be successful in the future, they will invest in Apple now and the stock’s price will increase over time (granted that Apple is successful and holding all other factors constant).

This project has definitely been a work in progress, and I have previously published two articles where I outlined my findings as they developed. I suggest looking at those articles for greater context on this one (links in the reference section). As a summary, I collected articles that were published about Apple and Exxon Mobil, calculated the daily sentiment of those articles, and then used these values (along with stock volume data) to predict the stock price for the following day.

I found that by using this method I was able to somewhat predict both the stock price and the day-to-day absolute value price change of the stock.