Auto Generated FAQ with Python Dash, Topic Analysis and Reddit Praw API
By: Vincent Tatan
Tags: Reddit, Topic, Python, Dash, Data Science

Auto Generated FAQ with Python Dash, Topic Analysis and Reddit Praw API

Dash Topic Modelling with Reddit API and Python

Problem Statement

“Damn, after my 3 days leaves, I’m already lost of what’s important on my Slack group” “There should be a quick access to the posts which are most relevant and get me updated quickly”

The biggest problem with peer reviews or forums is the copious amounts of available information on the site. Often times, we feel frustrated by the number of comments that are unrelated to what they have been searching for. Take this Reddit for example, in here, there are so many posts within our home page. All of the information clutters are very hard to keep track of.

These Reddit Posts show how much clutters a forum could bring just by a few days of inactivity

In this article, we are going to learn more on how we can make the extractions of information from forums such as Reddit easier and more intuitive. One way to do this is to build a dashboard page designed for extracting critical topics out of forums and package them in a filterable dashboard for quick overview — I will call this Auto-Generated FAQ as it goes through text corpus and extract the topics to form trends and patterns to create a Frequently Answered Question (FAQ)/Posts. This will help us keep in touch with the right information at the right time.

Disclaimer: This disclaimer informs readers that the views, thoughts, and opinions expressed in the text belong solely to the author, and not necessarily to the author’s employer, organization, committee or other group or individual. References are picked up from the list and any similarities with other works are purely coincidental

This article was inherently picked up from the author’s paper as a part of Final Assignment for Master of Computer Science in Georgia Tech.

Why Auto-Generated FAQ?

While there is always the search engine for us to find the information we need on these forums, the functionality is limited especially for users who are not active or lagging behind the flow of the discussions. Ultimately,Search engine would be useful for us who already know what they should be looking for (e.g. the latest machine learning paper). However, its function may not be useful for us who want to contribute to a trending topic or want to discover new ideas/information related to a topic.

Consider the following matrix of knowledge

Matrix of Knowledge and the solutions in between

Four domains of knowledge exist during a learning process. The first involves the known knowns, knowledge that is known and is readily accessible to help tackle problems one is familiar with. Next would be the known unknowns, or knowledge one is currently unaware of but is accessible. An example would be finding information on a python programming syntax, which I currently have no knowledge of. The third form is the unknown knowns. This is defined as knowledge that one knows exists but have no idea how to access/obtain. An example would be the completion of a task one has no idea how to go about starting/researching into. The final case is unknown unknowns, knowledge one does not have which he/she is unaware of. In this case, one is actually clueless of what is going on and what is important to know. This may happen when a person is too busy at work and lost track of the discussion flow in the forum.

Our solution will help the area of C and D, where users might not know what is important topic at a given time. Our goal is to provide a knowledge dashboard for users to have a quick glance of forum discussion.

Purpose and Github Code

The purpose for this Proof Of Concepts (POC) was created as a part of assignment submission for Master of Computer Science in Georgia Tech that the author is managing currently. Some of the contents in this article is modified to better reflect general audience’s needs. Please refer to our paper here.

We aim to build this application as a web page application with Python and Flask/Dash. Other tools such as Github for Version Control will also be implemented. Please refer to our Github Code here

Workflow

Let’s get our hands dirty now. There are 3 steps that we need to consider to create the Auto Gen FAQ

The workflow of creating Auto-Gen FAQ

Extracting Reddit with Praw Python Library

How do we extract the Reddit Corpus?

Authentication starts with using the praw library for Reddit. I won’t talk in details on how to get your authentication ready as there are many resources available. Feel free to access it here

Reddit praw code to input authorization profiles

Then we would get the subreddit information which would be exported to.

Reddit Code to gain a certain subreddit channel

We can then open this hot_python which reveals JSON data with the following metadata

Metadata retrieved after extracting posts from Reddit Praw

Topics Extraction

This section illustrates how to do approximate topic modeling in Python

We will use a technique called Non-negative Metrics Factorization (NMF) which is used to find extract topics from bags of words (list of words). NMF introduces a deterministic algorithm to create a single representation using the text corpus. Due to this reason, NMF is is characterized for ML Algorithm.

For more information on NMF and other topic modelling techniques, please refer to this research paper here

Using sklearn Count Vectorizer to vectorize words

This will return 500 words in the bag upon 1115 posts in the topics.csv.

We will use NMF to get a document-topic matrix (topics here will also be referred to as “components”) and a list of top words for each topic. We will make analogy clear by using the same variable names: doctopic and topic_words

doctopic and topic words created

We will then generate the topics and visualizations as followed

DocTopic Created

This will create the visualizations for 5 topics, each topic would group 5 closely related words together based on Euclidean distance managed by NMF. The calculations of appending the argmax given the length of the topic also helped us define the underlying dominant topic for each of the document in the corpus.

The following are the 5 topics found

Topic 1: omscs program students job course

Topic 2: cs undergrad degree non reviews

Topic 3: georgia tech online master program

Topic 4: courses classes semester students new

Topic 5: time did job offer commitment

Analysis and Visualization of Topics Analysis

We then tried to map the topics to each relevant posts, and we will count the number of posts that are relevant to certain topics and visualize them in pie chart and line chart. The first pie is polling based on the dominant topics on each post.

Analysis

From here we can see that top topics talk about job offers for master and non master students. Followed by the time commitment for omscs students. Then by programs and undergrad classes.

Analysis

1. In May and August, we can see that there are rising topics in terms of jobs and offers regarding master students

2. In October there is raising trends in the interest for the time and workload of OMSCS students (probably due to the beginning of the new sems)

3. For the overall activities, we can see decline in April but increasing activity in October

Dash Visualizations

We will use Dash Python, which is Python Data Visualization Framework that is built on top of Plotly and Flask. This will be the foundation for our visualization and deployment in local. Please check our presentation and demo below for a more animated view of the application.

Through our application, users will be able to select the topics which mattered the most recently, filtered on them and revealed the timeline. Furthermore the table below will showcase the most relevant posts given their chosen topics. Therefore, rather than clicking all of the available posts , users could just take a quick glance at this dashboard to catch up with the discussions on certain topics.

All in just a few clicks to filter selected topics and reveal the relevant posts.

Dash Python Visualization

Survey: How useful is our Application?

After building the following prototype, we built a survey to indicate confirm the app’s usefulness. From this result,we have found 93% of our respondents regarded the app as useful to very useful. Users also find that the application were very intuitive which potentially saved time from manually scour through uncategorized posts. This marks the success of our application in adding values to forum goers.

Usefulness of the application

Suggested Improvements

Demo

Demo for Auto Generated FAQ for Online Master of Computer Science Educational Tech

Conclusion and Future Work

This project gave us opportunities to solve a practical and relevant problem which many of us seem to face on forums such as Reddit. There are many functionalities and user experience we could add on to improve this application, but for now, this application has successfully handled the mentioned core problem of extracting relevant topics and trends from forums with cluttered posts.

Acknowledgments

We would like to thank you our fellow students and my mentor, Jace Van Auken, for their supports and helpful comments on the work products we have culminated in this paper. Lastly to Ranon Sim who oversees the general project management of this tool

Finally…

Whew… That’s it, about my idea which I formulated into writings. I really hope this has been a great read for you guys. With that I hope my idea could be a source of inspiration for you to develop and innovate.

Please reach out to me via my LinkedIn and subscribe to my Youtube Channel

Comment out below for suggestions and feedbacks.

Happy coding :)