End-to-End Wine Quality Modeling with a Boost from Watson NLP Services
By: Elliott Botwick
Tags: Machine Learning, Natural Language, Pandas, Data Science, Wine

Three questions I will seek to answer in the following sections are -

How can we use high cardinality categorical data effectively in model training? How can we improve the efficiency of compute-heavy functions against pandas dataframes? Can we extract meaningful information from the description field to improve model performance?

1. First things first — EDA

For the scope of this article I wanted to share just a few findings from exploring the data.

I. Wine prices vary… a lot.

I don’t even want to include the histogram of price here its so ugly. Against my better judgement you can see it below.

Yuck — very skewed

You can also see how nicely a log transform helps get this to better mirror the distribution of points in this data.

For more of a fun look — here’s a breakdown of the average price of the 20 most common wine varieties in this data set —

A bottle of Nebbiolo typically costs over $60 but Rose usually runs under $20

II. What the people are saying

Looking at the length of description, as expected, wines that got higher scores had longer descriptions in their reviews. Not quite as obviously to me, people have more to say about Cabernet’s than about Chardonnay’s.

On average reviews were ~30 characters longer for cabs than chardonnays

III. Virginia is for wine lovers

Being from Virginia and being a fan (by no means a connoisseur) of visiting vineyards, I was curious to see where I could get a good, cheap bottle of wine.

Wineries in VA with at least 10 bottles of wine reviewed in this dataset

If you’re ever around Albemarle County there’s several good wineries to try like King Family vineyard.

2. Data Preparation

There were several key data preparation steps that needed to be taken before modeling this data.

I. Duplicate Rows:

One of the first steps I took in assessing the quality of this data was to inspect the proportion of unique values in each categorical variable. The results I found provided a good few challenges down the road. Here is a grape-themed visualization showing the breakdown of unique values in categorical feature:

The greater the sliver of green on the chart, the greater the number of values that category has

Naturally the description field makes sense as the most unique column here — there’s only so many countries or wine varieties in the world but essentially limitless permutations of an open text field. However, as can be seen above, the description field is still a long way from being 100% unique. This helped lead to the discovery that around 1/3 of this data were duplicate records.

I used the pandas drop_duplicates() function for a quick and easy solution to the replicated rows.

wine.drop_duplicates(subset = ['description', 'winery', 'variety', 'country', 'province'],inplace = True)

II. Missing Values

In the spirit of keeping a common format, I created a similar visualization for the proportion of null values in each column. Here are some more grapes:

Fortunately several of the fields were entirely populated, and some of the scarcely populated fields were found to have low associations with the target (points) were likely going to be dropped anyways.

Before explaining how missing values in region_1 were handled I’d like to clarify the location variable hierarchy a bit in case it may be unclear:

country > province > region_1 > region_2

So to deal with the missing values in region_1 I created a dict that contained each unique value of province and the highest occurring value of region_1 for that province. Then with a lambda function, the missing values in region_1 were imputed with the most common region in that province. You can see the code below:

regionDict = {} for p in wine.province.value_counts().index:

if len(wine[wine.province ==p].region_1.value_counts()) < 1:

regionDict.update({p: 'NoListedRegion'})

else:

regionDict.update({p: wine[wine.province == p].region_1.value_counts().index[0]})

wine['region_1_imputed'] = wine[['province', 'region_1']].apply(lambda row: row.fillna(value = regionDict.get(row['province'])), axis = 1).loc[:, 'region_1']=

Some provinces didn’t have any listed regions, if this was the case the null value was replaced with ‘NoListedRegion’.

III. Dealing with Categorical Variables with high cardinality

This ended up being the area of data prep I probably spent the most time on — partially because it was difficult and partially because it was interesting. With some help from a few very clever friends and colleagues, I came up with a methodology I believe worked out well.

For categorical variables with > 100 levels, the first N levels to make up 95% of all values in the column were identified and all other levels were cast to ‘other’. Those columns were then one-hot encoded. If it took more than 100 levels to make up 95% of all values in the column I decided to either take the first 100 values and one-hot encode or just to index the column (I know, I know) on a case-by-case basis.

A screenshot of the notebook showing how we found that the 98 most popular varieties make up 95% of all varieties

Some features, such as winery, needed over 1000 levels to make up 95% of all values. That column was simply indexed here but in future analysis an approach that may be worth taking is doing some clustering of those columns to form logical grouping of winery values and reduce dimensionality by replacing the original value with a cluster value.

At this point we did have our data ready to train some models and created a gradient boosted model with an Adjusted R² value of 0.586 and a MSE of 4.29. Not bad but lets see what we can do to improve!

3. Natural Language Understanding

Again, a primary inspiration here was to build out a good baseline regression model and then see if applying some NLP to the description column could push the model even further. Obviously there’s a lot of NLP libraries and functions out there and you could spend months trying on various methods. I wanted to keep things relatively short and sweet which is something Watson APIs across the board can be very handy for.

First step here was to provision a Watson Natural Language Understanding (NLU) lite service. This is free for anyone who wants to try it and is a really valuable API for no cost. If you’d like to read up more about the API and sign up for a free NLU service or subscribe to the full-fledged service, check out the home page:

There’s a lot that developers can pull from natural language including basic to advanced entity extraction, concept recognition, sentiment analysis, and even relationships between entities. Language models can be tuned to recognize particular sets of terms and be fully customized to the language of specific businesses and domains.

In our case, I just used an out-of-the-box model as those can provide great value as well. Seeing as we’re essentially looking to gauge the quality of a product based on the review, looking at the scored emotion results from the model was theorized to be very helpful in prediction.

Here is some of the code I used to extract the emotion from a wine review —

As you can see I used a few helper functions I was able to construct thanks to some really helpful articles I found online (namely https://towardsdatascience.com/make-your-own-super-pandas-using-multiproc-1c04f41944a1) around optimizing lambda functions. Also using the tqdm package and replacing .apply() with .progress_apply() gives a live status bar for your function and updates against a projected time to completion. For the lite version of NLU there is a concurrency limit of 30 calls, but for the standard version you can parallelize even further and really speed up these operations. Still, I went from an ~8 hour job to a ~20 minute job on a pretty large data set — 30 threads are better than 1!

So with a few quickly defined functions I was able to produce the level of joy, fear, anger, disgust, and sadness in every wine review. I was happy to see much more joy present in reviews rather than anger or sadness here. This makes sense as all the wines in this data set were rated from 80–100 on an 100 point scale.

Emotion is scored on a scale from [0,1] with 0 being no trace of that emotion detected to 1 being totally representative of that emotion

Using the scores as additional inputs to our ML model with all else held constant — we went from an R² of 0.586 to .605 and an MSE of 4.29 down to 4.10. Certainly an impactful improvement.

In addition to emotion I also used NLU to extract Keywords from the descriptions. Keywords are essentially the most relevant tags that NLU finds within the text. Several keywords are detected for each piece of input text and are ranked in order of relevance. I thought it would be interesting to revisit the cabernet vs. chardonnay comparison to see how the top keywords differed for a popular red and white wine.

Word clouds of keywords for Cabernet Sauvignon (top) and Chardonnay (bottom)

You can certainly see the difference in the flavor profiles between the two wines and some other interesting like ‘tannin’ being a popular keyword in cabs and ‘acidity’ being a popular keyword in chardonnay.

I would highly recommend anyone interested in exploring the NLU app to check out this hosted demo. You can see example output (entities, concept tags, sentiment, emotion etc.) for a variety of industry examples (financial, legal, media) or input your own text or URL. It’s a great way to explore how you could use NLU to help extract meaning from your business data — friendly reminder IBM never collects or stores your data!