Detecting Trends on Twitter
Tags: trends, machinelearning
Published: 2022-04-27 08:36:37

In today’s society most people communicate through social media every day, ranging from ordinary people to politicians and corporations. Social media forums can be utilized to reach extensive crowds around the world if used strategically.

An example of such a social media is Twitter, which is a widely used platform with a large worldwide user base of 310 million monthly active users. When a topic or an event becomes trending on Twitter, it reaches out to an international crowd of 1 billion unique people per month.

Twitter is a social media platform where users post short messages. Other users can then like and retweet their tweets, which means that they share the other person’s tweet with their followers. Users can follow each other and no mutual following is required. This permits certain users to have millions of followers, whereas other people only have a dozen, if any at all. To enable users to find tweets related to a certain subject, a short sequence of characters starting with a “#” is used. These sequence of characters are called hashtags and allow users to participate with and view all related tweets.

In 2015, the trends ranged from world wide events and tragedies to social phenomenons[21]. These trends included #FIFAWWC (FIFA World Cup) and #ParisAttacks as well as #TheDress1. When a hashtag becomes [trending on Twitter](https://wotrends.com) it happens fast, the Paris attacks is an example of how a previously nonexistent hashtag’s activity suddenly goes through the roof. However, a group of hashtags can behave differently and still all be classified as trends.

Finding and detecting trends on Twitter is not only useful for companies or people that wish to pick up on the latest trends, but also to Twitter themselves as a company. Twitter makes approximately 85% of its revenue based on selling advertisements. Twitter has developed an algorithm that finds trending hashtags, both global and local trends. Then it is possible for Twitter to strategically use the trending hashtags to place its advertisements such that it reaches out to a large crowd.

Previous work has been done in the area of trend detection in Twitter’s feed. In 2012, Nikolov et al. developed an algorithm that found trends in the Twitter data feed, before Twitter’s algorithms was able to do so. The research team was able to do this by experimenting with different machine learning methods and arrive at an algorithm that used the frequencies of hashtags to predict an upcoming trend.

Machine learning is a relatively new area within computer science that uses data-driven learning to train programs without literally programming them to do so. Methods within machine learning are used for image recognition and selfdriving cars among others[13]. In the context of detecting trends, machine learning algorithms are trained to look at existing Twitter data and then be able to find trends in new data.

Within machine learning, there are two categories of methods. The first one is supervised learning that trains the algorithm by providing a training set of input data and the expected outcome. The second one is unsupervised learning that
takes in a training set of input data, but no data of the expected outcome. This makes unsupervised learning different, as the algorithm has to find a structure and patterns in the given data without knowing what the output is supposed to be.

Unsupervised learning methods can also be helpful in other contexts, such as unsupervised pre-training. Supervised training methods aim to find a global minimum (fitting the problem with the smallest error), but the global minimum
is not guaranteed to be reached every time. Unsupervised pre training can help initialize the supervised method such that when it begins, it is more prone to reach the global minimum.