Algorithmic Trading using Sentiment Analysis on News Articles | by Jason Yip | Towards Data Science

Well I was supposed to study for my exams but I was very tempted to explore this while I was taking breaks. I realized that I could use elements from my previous projects to work on this. I previously wrote about Automating Social Media Contests with Web Scraping which is relevant to news scraping and I also wrote about Generating Singlish Text Messages where I worked with text data. Isn’t it interesting how it could all overlap?
Algo trading automates the trading process in financial markets by rapidly and precisely executing orders based on a set of defined rules. They remove human error (provided the algorithms were developed without them) and they also remove the dangers of acting on emotion. The algorithms that are used in production can be fairly complex and heavily optimized with low-latency systems.
I’ll be implementing a very basic strategy (based on trend) on a single stock. Trend-following strategies are generally easy and straightforward to implement using simple technical indicators such as moving averages. The interesting thing that we are going to include in our algo is the qualitative element from news related to the company (ironically acting on emotion).
Sentiment Analysis or Opinion Mining refers to the use of NLP, text analysis and computational linguistics to determine subjective information or the emotional state of the writer/subject/topic. It is commonly used in reviews which save businesses a lot of time from manually reading comments.
Just like Algorithmic Trading, Sentiment Analysis could also go very deep as a field. Besides just giving a positive/negative sentiment, we could understand how subjective a text is, the intensities of different emotions (excitement, frustration, etc.), how Shakespearean or Trump-like a text could be, and much more.
Facebook has recently been facing a lot of backlash and I thought it would be interesting to see how the stock moves with its news sentiment.
The idea is pretty straightforward. We crawl the Business Times for any article related to Facebook, mine the texts, get the overall sentiment for each day, and buy 10 shares of its stock if the sentiment increases by 0.5 or sell if it decreases by 0.5. Bear in mind that the sentiment ranges from -1 to 1 with 0 being neutral and we are using the previous day’s sentiment to trade in the current day.
Similar to my previous post on web scraping, I used the same idea of extracting URLs from the search page and visiting each article to mine its sentiment for the previous day.
The sentiment is stored in a dictionary e.g. {datetime.date(2018,7,5):-0.59,...,}
VADER (Valence Aware Dictionary for sEntiment Reasoning) is a pre-built sentiment analysis model included in the NLTK package. It can give both positive/negative (polarity) as well as the strength of the emotion (intensity) of a text. It is rule-based and relies heavily on humans rating texts via Amazon Mechanical Turk — a crowd-sourcing e-platform which utilizes human intelligence to perform tasks that computers are currently unable to do. This literally means that other people have already done the dirty work of building a sentiment lexicon for us. These are words or any textual form of communication generally labelled according to their semantic orientation as either positive or negative) for us.
The sentiment score of a text can be obtained by summing up the intensity of each word in the text and then normalizing it. The human raters of Vader used 5 heuristics to analyze the sentiment:
Punctuation — I love pizza vs I love pizza!!Capitalization — I’m hungry!! vs I’M HUNGRY!!Degree modifiers (use of intensifiers)— I WANT TO EAT!! VS I REALLY WANT TO EAT!!Conjunctions (shift in sentiment polarity, with later dictating polarity) — I love pizza, but I really hate Pizza Hut (bad review)Preceding Tri-gram (identifying reverse polarity by examining the tri-gram before the lexical feature— Canadian Pizza is not really all that great.
Punctuation — I love pizza vs I love pizza!!
Capitalization — I’m hungry!! vs I’M HUNGRY!!
Degree modifiers (use of intensifiers)— I WANT TO EAT!! VS I REALLY WANT TO EAT!!
Conjunctions (shift in sentiment polarity, with later dictating polarity) — I love pizza, but I really hate Pizza Hut (bad review)
Preceding Tri-gram (identifying reverse polarity by examining the tri-gram before the lexical feature— Canadian Pizza is not really all that great.
VADER however is focused on social media and short texts unlike Financial News which are almost the opposite. I included a chunk in my notebook to update the VADER lexicon with words+sentiments from other sources/lexicons such as the Loughran-McDonald Financial Sentiment Word Lists.
With a simple code like this, we could easily get the sentiment from a passage.
from nltk.sentiment.vader import SentimentIntensityAnalyzernltk.download('vader_lexicon')sia = SentimentIntensityAnalyzer()polarity_scores(passage)['compound']
Finally, with the use of the backtrader package, we have a convenient framework to backtest and write our trading strategy. I used the Quickstart code in the docs as a base and modified it to include our sentiment scores.
I specified the “FB” stock feeds to be pulled from Yahoo Finance, set an initial amount of $100,000, a fixed size of 10 lots per trade, a commission of 0.1%, as well as a simple strategy to buy if the previous day’s sentiment score increases by 0.5 from the last day and sell if it decreases by 0.5.
I started out with $100,000 and ended up with $99,742. Haha. Not gonna say anything about that.
Okay, well the results were sorta expected because reality is so much more complex than our model. Notice how the trades were executed as the sentiment score swings. The model / I was definitely very stupid actually, we were always behind and executed trades right after the “hype” moments. But you can see where this goes, we could definitely build on the strategy by including more technical indicators or even improve the sentiment analysis by training our own model (could use NLTK) on more relevant Financial News.
I hope you all enjoyed this, I was definitely more cautious of the readability and conciseness this time. Please leave comments, feedback and ideas if you have any. I would really appreciate it. Till next time!
Link to project repo
Discuss further with me on LinkedIn or via jasonyip184@gmail.com!