NewsLens
Tags: octograd2020
Published: 2020-05-21 17:55:06

It is no secret that news organisations receive a lot of abuse regarding their content, especially when an individual’s opinion conflicts with that of the news outlet. Currently, there exists no standardised method to automatically determine the differences between news articles regarding the same event. Thus, a system that measures the bias of media content automatically and visualises the result would offer a clear benefit for members of the general public, as well as provide a defence mechanism for journalists that are often subject to groundless claims of biased content.

Thus, for my final year dissertation I created NewsLens, a system that measures the bias of media content automatically and presents a visualisation of related articles would offer a clear benefit for those involved in such conflicts. 

NewsLens provides journalists and their news organisations with a defence mechanism to protect against the aforementioned accusations, and provide a counterpoint for them to confirm their content is indeed neutral.


## Demo
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/3tafd836dcv6d5d9jwyp.png)


[> Here](https://people.bath.ac.uk/hi263/study)

## How I built it 
Over 20,000 online news articles were analysed using a variety of machine learning methods, and clustered together to form groups of articles that represented single events within the news.

The demo above reveals the capabilities of the system when viewing one such cluster.

### Backend (Python)
- spaCy - General language processing tools including NER, Sentence Segmentation, Tokenisation and Lemmatisation.
- Sklearn - HAC, TF-IDF and feature extraction.
- Gensim - LDA topic modelling implementation.
- NLTK - VADER sentiment analysis package.

### Frontend (JavaScript)
- React JS - Base web framework.
- React Redux + Saga - Application state management and web re-
quests.
- React Material UI - Front-end components and theming.
- eCharts JS - Interactive data visualisations.

## Components 
The NewsLens UI consists of two main components, the document similarity chart, and the sentiment analysis chart.
### Document Similarity
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ko03qzbdao2nc1uif4fi.png)
The topic distribution vector obtained from LDA topic modelling was used to create a distance metric for the documents, so that it could be visualised in 3D. The more similar the documents topics are, the closer they will appear.

The plot allows for users to explore the similarities between the news articles, thanks to WebGL.

### Sentiment Analysis
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/rdwfywve22pk1z7xs8zh.png)
The entities within the text were extracted and the words surrounding them analysed using VADER sentiment analysis package. 

The dropdown box in the top left can be used to select the entities appearing within the chosen cluster. A 2D plot then displays the aggregated sentiment for that entity, per article.

In order to maintain transparency an accountability of the algorithm, the sentences used in the calculations are shown below the plot.

Highlighting the emotive words surrounding the entities turned out to be a particularly challenging task...

## Final thoughts

This post is somewhat vague due to time constraints with my exams! If you would like to know more, I can release the code and ~100 page dissertation explaining every single detail of the project, but we'll have to wait until I graduate for that. :)