Quora Question Pairs Case Study
By: Amit Moralwar
Tags: Xgboost, Logistic Regression, Python3, Machine Learning, Svm

Description :: Quora is a place to gain and share knowledge about anything. It’s a platform to ask questions and connect with people who contribute unique insights and quality answers. This empowers people to learn from each other and to better understand the world. Over 100 million people visit Quora every month, so it’s no surprise that many people ask similarly worded questions. Multiple questions with the same intent can cause seekers to spend more time finding the best answer to their question, and make writers feel they need to answer multiple versions of the same question. Quora values canonical questions because they provide a better experience to active seekers and writers, and offer more value to both of these groups in the long term.

2. Real world business constraints :: Problem definition of Quora question similarity is given two questions let say q1 and q2 represents in the English text, we have to determine whether two questions are duplicates of one another.

Let’s understand the business context here. Suppose there is a question on quora and multiple people answer that question. Now let say another person ask the same question but with slightly different worded language but let say context is same. As soon as someone ask the question it make sense to quora to understand it and connect it to first question if they are similar so that answer given to first question is applicable to second question. This enhance the user experience and provide the solution to the problem where lot of people put hundreds of man hours to provide the answer to each question.

3. Data Overview ::

Data looks something like this::

As shown above, we have given 6 fields contains id which represents unique id for each record. qid1 and qid2 are the unique id for each question in the pair, question1 and question2 contains actual data and is_duplicate is our dependent feature which we are trying to predict.

4. EDA (Exploratory Data Analysis)

Number of duplicate and non duplicate pairs of question present in the data shown below which clearly tells us that non duplicate question pairs are comparatively high than duplicate pairs

I used histogram to check Number of occurrences of questions to Number of questions. You can see above Maximum number of times a single question is repeated is 157 as shown above.

After that I checked information about data using info() given below..

As you can see above the encircled value indicates that there are two null values present in question2 which I addressed in data preprocessing.

5. Featurization

Features which used for this case study with there use mentioned below along with existing features

cosine_distance = (When we vectorize Question 1 and Question 2 then we can use cosine distance to check the angle between 2 vectors which gives us sense of similarity between two vectors)

(When we vectorize Question 1 and Question 2 then we can use cosine distance to check the angle between 2 vectors which gives us sense of similarity between two vectors) word_common = (Number of common unique words in Question 1 and Question 2)

= (Number of common unique words in Question 1 and Question 2) word_total =(Total num of words in Question 1 + Total num of words in Question 2)

=(Total num of words in Question 1 + Total num of words in Question 2) word_share = (word_common)/(word_Total)

= (word_common)/(word_Total) freq_q1+freq_q2 = sum total of frequency of qid1 and qid2

= sum total of frequency of qid1 and qid2 freq_q1-freq_q2 = absolute difference of frequency of qid1 and qid2

Bunch of provisions are available in NLP to convert textual data into vectors such as BOW, TFIDF, W2Vec. In this article TFIDF and W2Vec were preferred as BOW creates unexpected sparsity.

Now if you ask why we avoid sparsity? the reason is…

Sparse representations are harder to model both for computational reasons (space and time complexity) and also for information reasons, where the challenge is for the models to harness so little information in such a large representational space.

Now as a part of data preprocessing we will do some data cleaning which includes following operations:

Remove Punctuations

Remove html tags

Perform Stemming : Stemming converts words to its stem.

For example playing and played are the same type of words which basically indicate an action play. Stemmer does exactly this, it reduces the word to its stem. I used a library called porter-stemmer for stemming.

For example playing and played are the same type of words which basically indicate an action play. Stemmer does exactly this, it reduces the word to its stem. I used a library called porter-stemmer for stemming. Remove Stopwords

6. Machine Learning Models:

I tried with logistic regression first

After training the model I got the following as my output which contains very high log-loss value.

For performance metric I used confusion matrix, precision matrix and recall matrix which is given below…

As I got high log loss in logistic regression I decided to explore other models, so I tried SVM and checked the log loss

As you can see above log loss for SVM is worse than that of logistic regression so I tried one of the ensemble technique which is xgboost with hyperparameter tuning. xgboost code mentioned below..

After train and test data on xgboost, I got train-logloss:0.38776 and

test log loss : 0.38989 which is upto some extent considerable compare to linear models like logistic regression and SVM as we saw above.

Confusion, Precision and recall matrix for xgboost model is as follows::

8. Conclusion::

The metric that we tried to optimize is the log-loss. This is the metric upon which the competition was judged and thus it is the one I tried to optimized.

Techniques used in this case study to optimize log loss and there corresponding log-loss value.

Logistic Regression :: 0.59

SVM :: 0.61

Xgboost :: 0.38

We can conclude that xgboost performed well with the kind of data that we used in this case study.

Github repo of this code is::

https://github.com/amitmoralwar/Quora-Question-Pair-Similarity

9. Sources ::

https://www.kaggle.com/anokas/data-analysis-xgboost-starter-0-35460-lb/comments

https://towardsdatascience.com/finding-similar-quora-questions-with-bow-tfidf-and-random-forest-c54ad88d137

https://www.kaggle.com/c/quora-question-pairs/discussion/30340

https://www.kaggle.com/c/quora-question-pairs/discussion/31796

https://www.kaggle.com/c/quora-question-pairs#evaluation