🤖 Grab Your Wine. It's Time to Demystify ML and NLP 🍷
Tags: machinelearning, ai, beginners, python
Published: 2019-07-23 13:09:24

---
title: 🤖 Grab Your Wine. It's Time to Demystify ML and NLP 🍷

published: true
description: In this tutorial you will learn how to use ML to predict wine price, points and variety from the text description. We will process the wine description text with the library SciKit Learn to create a Bag-of-Words Logistic Regression Multi-Classification model. If that didn't make sense then you are in the right place! If it did make sense continue reading because wine.

tags: ml, ai, beginner, python,
---

![graph](https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/wordcloud.PNG)

In this tutorial you will learn how to use ML to predict wine price, points and variety from the text description. 

We will process the wine description text with the library [SciKit Learn](https://scikit-learn.org/stable/) to create a Bag-of-Words Logistic Regression Multi-Classification model. If that didn't make sense then you are in the right place! If it did make sense continue reading because wine.

## Configure your ML dev environment

### There are two options: cloud or local.

Azure Cloud Setup - Use Azure Machine Learning Workspace with the integrated cloud Notebook VMs. Workspace gives you a LOT of functionality and I would highly recommend this.
  1. Create a free [Azure account](https://azure.microsoft.com/en-us/free/?WT.mc_id=ca-0000-cassieb) if you dont already have one.
  2. Follow the [Create Workspace](https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-run-cloud-notebook?WT.mc_id=ca-0000-cassieb) tutorial through the "Create a notebook VM" step
  3. Click "Jupyter Lab" from the Notebook VM navigation
  4. Then run the below command in the terminal to clone the notebook from GitHub.

<img src="https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/launchnotebook.png" width="800" height="500"/>

```bash
#CTRL+SHIFT+V to paste
curl https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/winereview-nlp.ipynb --output winereview-nlp.ipynb
```

Local Setup
* [Download Jupyter](https://jupyter.org/) notebooks 
* [Download Anaconda](https://www.anaconda.com/distribution/) 
* Run Jupyter notebook


Once you are set with one of the above environment configurations its time to start building! 

## Import packages and data
If you get an error "No module named" install it with the command `!pip install wordcloud`. Replace `wordcloud` with the module name in the error message.

```python
#This package need to be installed first
!pip install wordcloud
```
```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import math
from sklearn.model_selection import train_test_split
from sklearn.metrics import precision_recall_curve
from sklearn.linear_model import LogisticRegression
from sklearn.feature_extraction.text import CountVectorizer
from joblib import dump, load
```

We need data! I used a dataset I found on Kaggle. Kaggle is an online community of data scientists. Download the dataset from the github repo or kaggle.
* [Wine Dataset from Repo](https://github.com/cassieview/intro-nlp-wine-reviews/blob/master/dataset)
* [Kaggle Dataset](https://www.kaggle.com/zynicide/wine-reviews)
![data](https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSwvLv12Qt9SOXvdGwlqQP0ORHhvO1OI7hAxqAvXbf3tpRl4t2Isw)

Import the data as a [Pandas](https://pandas.pydata.org/pandas-docs/stable/) DataFrame
```python
#File path to the csv file
csv_file = "https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/dataset/winemag-review.csv"

# Read csv file into dataframe
df = pd.read_csv(csv_file)

# Print first 5 rows in the dataframe
df.head()
```
![graph](https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/dfhead.PNG)

## Visualize the data
Once we have the data then its time to analyze it and do some [Feature Selection and Engineering](https://docs.microsoft.com/en-us/azure/machine-learning/team-data-science-process/create-features?WT.mc_id=ca-0000-cassieb). We will visualize our data using [Seaborn](https://seaborn.pydata.org/). This will allow us to see if there is a strong correlation between different data points and help us answer questions about our data. Since our initial question was around predicting `price`, `points` or `variety` from the `description` we already know that our Feature will be the `description` and our Label will be `price`, `points`or `variety`. Features are the data we use to make predictions and Labels are what we are predicting. Each label will be a separate model so there will be three models in total if you choose to build all three predictive models.

For fun, let's ask some questions about the data and answer them by graphing it with Seaborn.

Is there a correlation between price and points?
```python
sns.barplot(x = 'points', y = 'price', data = df)
```
<img src="https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/priceandpoints.PNG" width="500" height="500"/>

```python
sns.boxplot(x = 'points', y = 'price', data = df)
```

<img src="https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/priceandpoints2.PNG" width="500" height="500"/>


Does one wine critic give higher ratings than the others?

```python
sns.catplot(x = 'points', y = 'taster_name', data = df)
```
<img src="https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/tasterpoints.PNG" width="500" height="500"/>

Let's look at a WordCloud of the `description` Text

```python
from wordcloud import WordCloud, STOPWORDS
import matplotlib.pyplot as plt
text = df.description.values
wordcloud = WordCloud(
    width = 3000,
    height = 2000,
    background_color = 'black',
    stopwords = STOPWORDS).generate(str(text))
fig = plt.figure(
    figsize = (40, 30),
    facecolor = 'k',
    edgecolor = 'k')
plt.imshow(wordcloud, interpolation = 'bilinear')
plt.axis('off')
plt.tight_layout(pad=0)
plt.show()
```
![graph](https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/wordcloud.PNG)
<small>I like to think of this WordCloud as a cheatsheet of discriptive words to use when tasting wine to make yourself sound like a wine expert 😃</small>


What other questions could you ask and answer by graphing this data?
![graph](https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/dfhead.PNG)

## Create Calculated Columns for Labels

This is going to be multi-classification for the price points or grape variety of the wines reviewed by the wine critics. Right now our points and price are number features. This needs to be updated to a text feature category, to do this we will create a couple functions to generate calculated columns based on the values in the points and price columns to use as are our labels.

Create `quality` column from points values to classes of bad, ok, good, and great. Below is a function to return string `quality` based on the points value.

```python
def getQuality(points):
    if(points <= 85):
        return 'bad'
    elif(points<=90 ):
        return 'ok'
    elif(points<=95):
        return 'good'
    elif(points<=100):
        return 'great'
    else:
        return 'If this gets hit, we did something wrong!'
```

Next apply the function to the points column of the dataframe and add a new column named `quality`.

```python
df['quality'] = df['points'].apply(getQuality)
```
Visualize the new column against the price column like we did above.

```python
sns.catplot(x = 'quality', y = 'price', data = df)
```
<img src="https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/pricequality.PNG" width="500" height="500"/>

```python
sns.barplot(x = 'quality', y = 'price', data = df)
```
<img src="https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/pricequality2.PNG" width="500" height="500"/>

Create priceRange column from price column of `1-30`, `31-50`, `51-100`, `Above 100` and `0` for columns with NaN. Below is a function to return string priceRange based on price value.

```python
def getPriceRange(price):
    if(price <= 30):
        return '1-30'
    elif(price<=50):
        return '31-50'
    elif(price<=100): 
        return '51-100'
    elif(math.isnan(price)):
        return '0'
    else:
        return 'Above 100'
```
Apply the function to the points column of the dataframe and add a new column named `priceRange`.

```python
df['priceRange'] = df['price'].apply(getPriceRange)
```

Print totals for each priceRange assigned to see how the labels are distributed

```python
df.groupby(df['priceRange']).size()
Output: priceRange
        0             8996
        1-30         73455
        31-50        27746
        51-100       16408
        Above 100     3366
        dtype: int64
```
We now have our labels for  models to predict quality, priceRange and grape variety.

## Process description text with the library SciKit Learn to create a Bag-of-Words using the [CountVectorizer](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html) functionality.

The docs do a great job of explaining the CountVectorizer. I recommend reading through them to get a full understanding of whats going on, however I will go over some of the basics here.

At a high level the CountVectorizer is taking the text of the description, removing stop words (such as “the”, “a”, “an”, “in”), creating a tokenization of the words and then creating a vector of numbers that represents the description. The text description is now represented as numbers with only the words we care about and can be processed by the computer to train a model. Remember the computer understand numbers and words can be represented as numbers so the computer can "understand".

This is an example of how the words become numbers. We will go over this in more detail with an example from the dataset as well.

![graph](https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/vectorchart.PNG)

Before we jump into the CountVectorizer code and functionality. I want to list out some terms and point out that CountVectorizer _does not_ do the Lemmatization or Stemming for you.
 
* StopWords:  A stopword can be a word with meaning in a specific language. For example, in the English language, words such as "a," "and," "is," and "the" are left out of the full-text index since they are known to be useless to a search. A stopword can also be a token that does not have linguistic meaning.
* [N-Gram](https://docs.microsoft.com/en-us/dotnet/machine-learning/resources/glossary#n-gram?WT.mc_id=ca-0000-cassieb): A feature extraction scheme for text data: any sequence of N words turns into a feature value.
<img src="https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/ngram.PNG" width="500" height="500"/>

* [Lemmatization](https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/preprocess-text#module-overview?WT.mc_id=ca-0000-cassieb): converts multiple related words to a single canonical form ("fruity", "fruitiness" and "fruits" would all become "fruit")
* Stemming: Similar to Lemmatization but a bit more aggressive and can leave words fragmented.

### Let's take a look at how to use the CountVectorizer.

These are all the properties that you can set within the CountVectorizer. Many of them are defaulted or if set override other parts of the CountVectorizer. We are going to leave most of the defaults and then play with changing some of them to get better results for our model.

```python
CountVectorizer( 
  input='content', encoding='utf-8', decode_error='strict', 
  strip_accents=None, lowercase=True, preprocessor=None, tokenizer=None, 
  stop_words=None, token_pattern='(?u)\b\w\w+\b', ngram_range=(1, 1), 
  analyzer='word', max_df=1.0, min_df=1, max_features=None, 
  vocabulary=None, binary=False, dtype=<class 'numpy.int64'>
)
```
Create the function to get the vector and vectorizer from the `description` feature.

There are different CountVectorizer configurations commented out so that we can play with different configs and see how it changes our result. Additionally this will help us look at one description and pick apart what is actually happening in the CountVectorizer.

```python
def get_vector_feature_matrix(description):
    vectorizer = CountVectorizer(lowercase=True, stop_words="english", max_features=5)
    #vectorizer = CountVectorizer(lowercase=True, stop_words="english")
    #vectorizer = CountVectorizer(lowercase=True, stop_words="english",ngram_range=(1, 2), max_features=20)

    #vectorizer = CountVectorizer(lowercase=True, stop_words="english", tokenizer=stemming_tokenizer) 
    vector = vectorizer.fit_transform(np.array(description))
    return vector, vectorizer
```

For the first run we are going to have the below config. What this is saying is that we want to convert the text to lowercase, remove the english stopwords and we only want 5 words as feature tokens.

```python
vectorizer = CountVectorizer(lowercase=True, stop_words="english", max_features=5)
```
```python
#Optional: remove any rows with NaN values. 
#df = df.dropna()
```
Next let's call our function and pass in the description column from the dataframe. 

This returns the `vector` and the `vectorizer`. The `vectorizer` is what we apply to our text to create the number `vector` representation of our text so that the machine learning model can learn.

```python
vector, vectorizer = get_vector_feature_matrix(df['description'])
```
If we print the vectorizer we can see the current default parameters for it.

```python
print(vectorizer)
Output: CountVectorizer(analyzer='word', binary=False, decode_error='strict',
        dtype=<class 'numpy.int64'>, encoding='utf-8', input='content',
        lowercase=True, max_df=1.0, max_features=5, min_df=1,
        ngram_range=(1, 1), preprocessor=None, stop_words='english',
        strip_accents=None, token_pattern='(?u)\\b\\w\\w+\\b',
        tokenizer=None, vocabulary=None)
```
Examine our variables and data to understand whats happening here.

```python
print(vectorizer.get_feature_names())
Output: ['aromas', 'flavors', 'fruit', 'palate', 'wine']
```
Here we are getting the features of the vectorizer. Because we told the CountVectorizer to have a `max_feature = 5` it will build a vocabulary that only considers the top feature words ordered by term frequency across the corpus. This means that our `description` vectors would _only_ include these words when they are tokenized, all the other words would be ignored.

Print out our first `description` and first `vector` to see this represented.

```python
print(vector.toarray()[0])
Output: [1 0 1 1 0]
```

```python
df['description'].iloc[0]
Output: "_Aromas_ include tropical _fruit_, broom, brimstone and dried herb. The _palate_ isn't overly expressive, offering unripened apple, citrus and dried sage alongside brisk acidity."
```

The vector array (`[1 0 1 1 0]`) that represents the vectorization features (`['aromas', 'flavors', 'fruit', 'palate', 'wine']`) in first description in the corpus. 1 indicates its present and 0 indicates not present in the order of the vectorization features.

Play around with different indexes of the vector and description. You will notice that there isn't lemmatization so words like `fruity` and `fruits` are being ignored since only `fruit` is included in the vector and we didn't lemmatize the description to transform them into their root word.

## Train the Model

Update the function so that the second vectorizer configuration is being used.

```python
def get_vector_feature_matrix(description):
    #vectorizer = CountVectorizer(lowercase=True, stop_words="english", max_features=5)
    vectorizer = CountVectorizer(lowercase=True, stop_words="english", max_features=1000)
    #vectorizer = CountVectorizer(lowercase=True, stop_words="english",ngram_range=(1, 2), max_features=1000)
    #vectorizer = CountVectorizer(lowercase=True, stop_words="english", tokenizer=stemming_tokenizer) 
    vector = vectorizer.fit_transform(np.array(description))
    return vector, vectorizer
```

And call the function to update the vectorizer

```python
vector, vectorizer = get_vector_feature_matrix(df['description'])
```

Now create our feature matrix. If you get a MemoryError here reduce the max_features in the CountVectorizer.

```python
features = vector.todense()
```
We have three different labels for three different models. Let's assign the label variable next and use the `quality` label first.

```python
label = df['quality'] 
#label = df['priceRange']
#label = df['variety']
```
We have the features and label variables created. Next we need to split the data to train and test. 

We are going to use 80% to train and 20% to test. This will allow us to get an accuracy estimation from the training to see how the model is performing.

```python
X, y = features, label
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
```

Train the model using a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) algorithm and print the accuracy.

```python
lr = LogisticRegression(multi_class='ovr',solver='lbfgs')
model = lr.fit(X_train, y_train)

accuracy = model.score(X_test, y_test)
print ("Accuracy is {}".format(accuracy))
Output: "Accuracy is 0.7404885554914407"
```

This is an ok accuracy but I am sure it can be improved! For this tutorial we are going to call it "good enough" which is a decision that needs to be made with every model you ever build! 

## Test the model

When you select a candidate model it should always be tested on unseen data. If a model is [overfitted](https://en.wikipedia.org/wiki/Overfitting) to its data it will perform really will on its own data and poorly on new data. This is why its very important to test on unseen data.

I grabbed this review from the wine mag site. Its a 95 point and $60 bottle of wine review.

```python
test = "This comes from the producer's coolest estate near the town of Freestone. White pepper jumps from the glass alongside accents of lavender, rose and spice. Compelling in every way, it offers juicy raspberry fruit that's focused, pure and undeniably delicious."


x = vectorizer.transform(np.array([test]))
proba = model.predict_proba(x)
classes = model.classes_
resultdf = pd.DataFrame(data=proba, columns=classes)
resultdf
```
![testresult](https://raw.githubusercontent.com/cassieview/intro-nlp-wine-reviews/master/imgs/testresult.PNG)

# This is a correct prediction! 🎉 

Another way to look at the result is to transpose, sort and then print the head resulting in a list of the top 5 predictions.

```python
topPrediction = resultdf.T.sort_values(by=[0], ascending = [False])
topPrediction.head()
```
## Other things to try
1. Change the label and run again for the price bucket prediction or grape variety.
2. Try to use different algorithms to see if you can get a better result
3. Add additional features to the description text to improve accuracy. There was a strong correlation between price and points. Maybe adding those would improve the accuracy score?
4. Add lemmatization to the text to improve score using the [NLTK](https://www.nltk.org/)
5. Try doing a text classification on a different dataset.

Data science is a trial and error process. I am sure there are ways to improve this model and accuracy. Play around and see if you can get a better result!


## Other helpful links
[The Machine Learning Algorithm Cheat Sheet](https://docs.microsoft.com/en-us/azure/machine-learning/studio/algorithm-choice#the-machine-learning-algorithm-cheat-sheet?WT.mc_id=ca-0000-cassieb)

[How to choose algorithms for Azure Machine Learning Studio](https://docs.microsoft.com/en-us/azure/machine-learning/studio/algorithm-choice?WT.mc_id=ca-0000-cassieb)

[Full GitHub repo](https://github.com/cassieview/intro-nlp-wine-reviews)

<center>CHEERS! 🥂</center>

<img src="https://thepracticaldev.s3.amazonaws.com/i/5uzwtn3ltbn9jxi4nfj5.jpg" width="350" height="450"/> <center>🍷🤖💚Drinking wine and hanging out with bots. 🍷🤖💚</center>