The Discovery of Wine’s Structural Form
By: Stefano Ioannucci
Tags: Editors Pick, Algorithms, Data Visualization, Graph Theory, Wine

The Discovery of Wine’s Structural Form

Image by the author.

Hello there. Today I will present a guided tutorial for applying Kemp & Tenembaum’s brilliant “form discovery” algorithm to a wine dataset. Ultimately, this provides a data-driven map to choose wines from, based on our tastes. If you are, like me, fond of data science, machine learning, cognition and/or a wine lover, then you might find this post interesting.

[disclaimer: I am by no means implying that the method employed here is the best. Actually, if you know of ways it could be improved I’d love to hear them!]

First of all, like every recipe, we’ll start with a list of things we need:

✓ Wine dataset ( freely available here)

✓ Python (I use PyCharm)

✓ Kemp & Tenenbaum’s algorithm (here)

✓ Graphviz (here: make sure to download a version that contains neato)

✓ Matlab

Essentially, in their work Kemp & Tenenbaum created an algorithm which finds the best structural representation for a dataset, without any assumption nor indication about this dimension. It achieves this feat by organizing input data in 8 “fundamental forms” and assigning a sort of score to each, in order to find the one that fits the data best.

These domain-aspecific forms are: partition, order, ring, hierarchy, chain, tree, grid and cylinder. Each protoform, depicted below, is derived from graph theory and has its own specific rules guiding how the data clusters are related. If this sounds confusing don’t fret! Further below we’ll link these somewhat abstract words with a hands-on example (we won’t be going into the details of the paper though).

Figure adapted from “The Discovery of Structual Form” in PNAS, published under Creative Commons.

In the paper, the authors provide several worked examples with different datasets, however I felt it’d be fun to try to use it on my own on new data. To look for an appropriate dataset, I headed over to kaggle where one may find exactly this type of thing. The ‘wine reviews’ dataset caught my attention because it contained the type of information one might use as features to provide to the algorithm, so wine not?

This dataset contains 260.000 bottles of wine, with info for each wine such as the name, variety, country of origin, price and, importantly, a review written by a sommelier. I figured one could find a way to extract the recurring words inside all these reviews and then use those as features to feed to the algorithm

To do so, one could use part of the process used to generate one of those “wordclouds” that seem to be everywhere nowadays.

Here’s an example of what a wordcloud looks like. You should have seen at least one in your life. Image by Alberto Frigo, public domain

Imagine the disbelief when I searched ‘python wordcloud tutorial’ and the first result was a guide employing exactly the same dataset I had chosen to work with! Beginner’s luck at its finest.

Anyhow, we’ll use this method to extract the most common words that the sommeliers use to describe wine tastes, assign such features to a list of wine varieties, feed it to Kemp & Tenenbaum’s algorithm and see how it arranges them for us. Easy.

Alrighty, we can begin with the step-by-step breakdown of the pipeline.

points 1 to 10 for basic data manipulation and wordcloud set up, 11 to 19 setting the matrix for the algorithm and feeding it, point 20 for the result

1.As is the custom in Python, we begin the script by importing the necessary packages (called like this since they literally contain codes that do things). Shockingly, before they can be imported, packages need to be installed. The way you do this depends on how you’re using Python. I’ll just briefly mention how to do it in PyCharm. Basically, each package installation involves a single line of code written in the terminal, with the general structure of “pip install nameofthepackage” (as reference, here you can find instructions to install pandas for various builds). The needed packages are matplotlib to help us make graphs, numpy and pandas to deal with manipulating data, scipy.io converts a Python output to use it in Matlab and tkinter allows us to make a menu to pick stuff from. I guess you can figure out what wordcloud is useful for.

import matplotlib.pyplot as plt

import numpy as np

import pandas as pd

import scipy.io as sio

import tkinter as tk

from wordcloud import WordCloud, STOPWORDS

When we import a package as *something*, it means we can reference that *something* to access the codes from the package, hence why it’s usually an abbreviation of the name itself. Just to type a bit less.

2. Now we can load the dataset into Python. I’m assuming it was already downloaded from kaggle, saved in the directory we’re working in and named winedata1 and winedata2, since for some reason the dataset is split into two halves.

df1 = pd.read_csv("winedata1.csv")

Like that, we’ve assigned to the variable called “df1” all that is contained in the file “winedata1.csv”. This is done through a pandas function (read_csv), which we access by calling the abbreviation we chose for pandas when importing it (pd). If you’re wondering what a csv file is, think of an excel sheet. If you’re wondering what an excel sheet is, think of numbers and words arranged into tables with rows and columns. If you’re wondering what numbers, tables, rows and columns are… well, uh.. nevermind.

Anyways, we’ll do this also for the second half of the dataset:

df2 = pd.read_csv("winedata2.csv")

Then, we combine these two separate dataframes (yay! a new word) by using another pandas function that does exactly this, and restore the whole dataset:

dataset = pd.concat([df1, df2])

3. As I mentioned earlier, this dataset contains wines from all over the world. It’s probably best if we narrow down the wines we’ll use by their country of production, since a Cabernet produced in someplace may differ in features from a Cabernet produced elsewhere.

Is there a better way to do this than a pop-up window which you can scroll through and pick a country from? I don’t think so.

Before doing that, it’s a good idea to know how many countries there are in the dataset. We’re lazy, so we let the computer count this (number of unique words in the country column of the dataset), and tell us (print).

print("There are {} countries producing wine in the whole dataset, please choose one by selecting it from the list"

" and pressing enter".format(len(dataset.country.unique())))

Next, we’ll set up the pop-up window, using the tkinter package (tk).

window = tk.Tk()

window.title(‘Wine producing countries’)

window.geometry(‘100x150’)

tiny_list = tk.Listbox(window)

tiny_list.pack(expand=True, fill="both")

countries = dataset.country.unique() for item in range(len(countries)):

tiny_list.insert("end", countries[item])



tiny_list.select_set(0)

tiny_list.focus_set()

We also need to write a tiny function that allows us to choose a country from the pop-window, and close it once we do so.

def exit_gui(event):

global choice

choice = tiny_list.curselection()

window.destroy()

Finally we can run the window, letting the machine know that our choice will be signalled by pressing return, and have it inform us of the choice we made.

window.bind("<Return>", exit_gui)

window.mainloop()



print("Wines from", countries[choice], "were chosen.")

Image by author

In this tutorial, I chose Italian wines, but the code should run fine with most choices. However, if a country with few wines is chosen there may be issues, so don’t scroll too far down.

4. Now let’s make a variable with all the data of wine bottles from the country we chose. We’ll call it ‘wines’, because how else would we name it? This is what we’ll work with from now on.

wines = dataset[dataset.country == countries[choice]]

Also, there’s multiple bottles for each wine variety, so it’s a good idea to print out what those varieties are.

print(wines.groupby("variety").describe())

In the case of Italian wines, there are some ‘blends’ among the varieties. Since the first goal is to match the wine varieties with their taste features, it is advisable to just go ahead and remove these wines, as their flavor will vary too much, depending on the ‘blend’, to be grouped in a single variety.

wines = wines[wines.variety != "Red Blend"]

wines = wines[wines.variety != "White Blend"]

wines = wines[wines.variety != "Sparkling Blend"]

It’s also a good idea to remove the varieties that have less than 20 entries in the dataset, as there won’t be enough data to decently estimate their flavors.

wines = wines.groupby("variety").filter(lambda x: len(x) > 20)

5. In this part we take care of another issue in the dataset: some varieties are named slightly different, but are actually the same. For example, some wines are labelled as ‘Sauvignon’ and others as ‘Sauvignon Blanc’ when they refer exactly to the same variety. This type of issue is normal with large datasets, and it needs to be fixed.

The following bit of code looks for similarities in the name of all the varieties and asks what the user wants to do if they partially match, giving 3 options to choose from :

[1] to merge the two varieties with a similar name (and choose which name to keep).

[2] to remove one of the two from the dataset (and choose which one).

[3] to just ignore them and carry on.

Once finished, the program tells us the number of remaining varieties.

varietylist = sorted(wines.variety.unique(), key=len) for first_index in range(len(varietylist)):

try:

for second_index in range(first_index + 1, len(varietylist)):

if varietylist[first_index] in varietylist[second_index]:

first_rep = varietylist[first_index]

second_rep = varietylist[second_index]

print('

detected a repetition between', first_rep, 'and', second_rep)

choice = input('How would you like to proceed? [1]Rename (merge), [2]Remove, [3]Ignore')

if int(choice) == 1:

subchoice = input(

'Would you like to [1]assign ' + first_rep + ' to ' + second_rep + ' or [2]viceversa?')

if int(subchoice) == 1:

wines.loc[wines['variety'] == first_rep] = second_rep

elif int(subchoice) == 2:

wines.loc[wines['variety'] == second_rep] = first_rep



if int(choice) == 2:

menu = list([first_rep, second_rep])

subchoice = input('Which type would you like to drop?' +

' [1]' + first_rep + ' has' + ' ' +

str(len(wines.loc[wines['variety'] == first_rep])) + ' entries' +

' or ' + '[2]' + second_rep + ' has' + ' ' +

str(len(wines.loc[wines['variety'] == second_rep])) + ' entries')



wines = wines[wines.variety != menu[int(subchoice) - 1]]

if int(choice) == 3:

continue

finally:

continue print("there are", len(wines.variety.unique()), "varieties of wine left in the data")

6. Okay, it’s about time we make a plot to have a rough look at the data we’re working with. To do so, we use matplotlib which will output a figure like the one below.

plt.figure(figsize=(15, 10))

wines.groupby("variety").size().sort_values(ascending=False).plot.bar()

plt.xticks(rotation=80)

plt.xlabel("Type of Grape")

plt.ylabel("Number of Wines")

plt.show

A histogram giving us a quick feel of how many wines (vertical axis) each wine variety (horizontal axis) has. Image by author.

7. If you had a look at the dataset, you’d noticed there’s a lot of other information I haven’t even mentioned. In fact it is useless for our purposes, and we’ll drop it from the ‘wines’ dataframe. How? Well, with the drop command and a list of the column names we deem no longer desirable.

wines.drop(

["country", "winery", "designation", "province", "region_1", "region_2", "price", "points", "taster_name",

"taster_twitter_handle", "title"], inplace=True, axis=1)

8. Here the wordcloud process begins, which will look for the most common words in a given text. First of all, all the sommelier reviews of each bottle of wine are joined into a single, very large (> 7 million words in this case) text. Since many of these words aren’t tasty taste features, we make a filter to block some (stopwords) and avoid getting them in the result. The wordcloud is then generated.

text = " ".join(review for review in wines.description)

print("There are {} words in the combination of all reviews.".format(len(text))) stopwords= set(STOPWORDS)

stopwords.update(["drink", "now", "wine", "flavor", "flavors", "show", "hint", "sangiovese", "expression", "delivers", "of",

"open", "made", "quality", "end", "layer", "generous", "aromas", "occasion", "nose", "carry", "over", "a",

"offer", "chardonnay", "offers", "make", "nose", "cabernet sauvignon", "merlot", "through", "with", "now",

"close", "palate", "texture", "feel", "mouth", "mouthfeel", "year", "boast", "touch", "the", "elegant",

"well", "one", "pretty", "pinot grigio", "lead", "vintage", "personality", "boasts", "savory", "and", "is",

"imported", "along", "load", "years", "give", "intensity", "made", "followed", "whiff", "elegant", "has",

"pair", "note", "aroma", "this", "suggest", "with", "after", "aged", "dominate", "opens", "fragrance",

"those", "who", "enjoy", "that", "recall", "pair", "time", "come", "around", "completed", "by",

"but", "find", "delicious", "blast", "still", "nebbiolo", "glass", "already", "opens", "doles", "note",

"less", "which", "such", "it", "despite", "care", "gone", "framed", "in", "drinks", "packed", "packs",

"bouquet", "local", "italian", "finish", "aromas", "assertive", "delightful", "evolved",

"evolves", "italy", "would", "intensity", "genuine", "wonderful", "smelling", "manifested", "bursting",

"modern", "solid", "qualities", "more", "than", "terms", "here", "ambitious", "version", "unique", "into",

"southern", "northern", "enough", "really", "very", "plenty", "touch", "particularly", "probably",

"opulent", "attractive", "should", "be", "drunk", "any", "doesn't", "marked", "excellent", "bloated",

"steady", "on", "crowd", "packed", "grapes", "rich", "beautiful", "presents", "approachable", "shows",

"you", "although", "need", "along", "alongside", "accompanied"]) wordcloud = WordCloud(stopwords=stopwords, max_words=1000).generate(text)

This part gets a bit manual, in the sense that I haven’t found an automatic way of extracting the taste features from our wordcloud in a satisfying manner, since no matter how many stopwords you put, some irrelevant words are still going to pass through. So, you can either trust me, or have a look yourself at wordcloud.words_ and check which are the most common/relevant taste features that sommeliers assign to wines (I did this across countries of production). Anyhow, here’s the taste features list:

features = list(['description', 'variety', 'sweet', 'black cherry', 'tobacco', 'coffee', 'clove', 'chocolate', 'rose',

'cinnamon', 'oak', 'leather', 'peach', 'red berry', 'thick', 'blue flower', 'white flower', 'licorice',

'plum', 'vanilla', 'almond', 'wild berry', 'alcohol', 'blackberry', 'cola', 'honey', 'toast', 'prune',

'pear', 'spice', 'black pepper', 'white pepper', 'apple', 'creamy', 'soil', 'nectarine', 'truffle',

'smooth', 'citrus', 'stone fruit', 'acidity', 'tannin', 'simple', 'floral', 'mint', 'mocha',

'red cherry', 'orchard fruit', 'raspberry', 'wood', 'menthol', 'exotic', 'blueberry', 'yellow fruit',

'bitter', 'mineral', 'crisp', 'violet', 'herb', 'anise', 'apricot', 'spring flower', 'espresso', 'dry',

'forest', 'lemon', 'tangerine', 'nut', 'astringent', 'thyme', 'acacia', 'saline', 'kiwi', 'bodied',

'structured', 'dried', 'cantaloupe', 'rosemary', 'earth', 'cocoa', 'sage', 'cigar', 'currant', 'sour',

'barrique', 'hazelnut', 'maple syrup', 'mushroom', 'balsamic', 'nutty', 'toasted', 'syrupy', 'cedar',

'smoky', 'tropical', 'gentle', 'delicate', 'easy drinking', "strawberry", "orange", "figs", "silky"])

Not bad. In a matter of seconds, we squeezed over 7 million words to get precisely 100 out. If you noticed ‘description’ and ‘variety’ at the beginning, it’s a workaround I did for dataframe manipulation and will soon be dropped.

9. As mentioned, Data Camp has a tutorial on generating a wordcloud from this same dataset and how to put it in some internet-downloaded image through Python. However, one may decide to go over at WordArt with the taste features list and get a faster and nicer result, like this one:

A prettier wordcloud. Image by author.

10. We’re done with the first part of the plan, i.e. extracting the flavors of wines from the dataset. Now, we need the program to assign these taste features to the varieties of wine. We start by making the extracted features columns in the dataframe, and then for each single bottle of wine we check if its sommelier’s review contains any of those features, which will give us a nice “logical”, yes/no (1/0), matrix indicating if a bottle of wine has a certain taste feature.

wines = wines.reindex(columns=features)



for feats in range(2, len(features)):

wines[features[feats]] = wines['description'].str.contains(features[feats])

That was fast. Now we can drop the sommelier’s descriptions from our data, ’cause we’re done with them. Hate the game, not the player.

wines.drop('description', inplace=True, axis=1)

11. Finally, we make a new dataframe that sums all these 1s and 0s, collapsing the data by variety of wine! Meaning that, if our dataset had 2 bottles of Sangiovese wine, and both had ‘tannin’ as a feature, the new matrix would have a single Sangiovese row with a 2 in the ‘tannin’ column, while if only one of the bottles had ‘chocolate’ now we’d have a 1 in the ‘chocolate’ column and if neither had ‘peach’ then there would be a 0 in that column, and so on for all features and varieties. We’ll call this tasteoccurrences.

tasteoccurrences = wines.groupby("variety").sum()

12. Thanks to this project, I learned that sommeliers have one trait in common with academics: when writing, they open a thesaurus and look for fancy synonyms, to gain sophistication and reduce repetitions. That is unless they actually intend that ‘coffee’, ‘espresso’ and ‘mocha’ are different flavors, which I doubt. So, we need to sum and drop all these redundant features.

tasteoccurrences["coffee"] = tasteoccurrences["coffee"] + tasteoccurrences["espresso"] + tasteoccurrences["mocha"]

tasteoccurrences["mint"] = tasteoccurrences["mint"] + tasteoccurrences["menthol"]

tasteoccurrences["floral"] = tasteoccurrences["floral"] + tasteoccurrences["spring flower"] + tasteoccurrences[

"white flower"]

tasteoccurrences["citrus"] = tasteoccurrences["citrus"] + tasteoccurrences["lemon"] + tasteoccurrences["tangerine"]

tasteoccurrences["structured"] = tasteoccurrences["structured"] + tasteoccurrences["bodied"]

tasteoccurrences["soil"] = tasteoccurrences['soil'] + tasteoccurrences["earth"]

tasteoccurrences["chocolate"] = tasteoccurrences["chocolate"] + tasteoccurrences["cocoa"]

tasteoccurrences["tobacco"] = tasteoccurrences["tobacco"] + tasteoccurrences["cigar"]

tasteoccurrences["alcohol"] = tasteoccurrences["alcohol"] + tasteoccurrences["barrique"]

tasteoccurrences["nut"] = tasteoccurrences["nut"] + tasteoccurrences["nutty"]

tasteoccurrences["toasted"] = tasteoccurrences["toasted"] + tasteoccurrences["toast"]

tasteoccurrences["syrupy"] = tasteoccurrences["syrupy"] + tasteoccurrences["maple syrup"]

tasteoccurrences["wood"] = tasteoccurrences["wood"] + tasteoccurrences["cedar"]

tasteoccurrences["delicate"] = tasteoccurrences["delicate"] + tasteoccurrences["easy drinking"] +

tasteoccurrences["simple"]

tasteoccurrences["smooth"] = tasteoccurrences["smooth"] + tasteoccurrences["silky"]



tasteoccurrences.drop(["espresso", "menthol", "mocha", "spring flower", "tangerine", "lemon", "bodied", "white flower",

"earth", "cocoa", "cigar", "barrique", "nutty", "toast", "maple syrup", "cedar", "simple",

"silky",

"easy drinking"], inplace=True, axis=1)

13. For the final data manipulation part, we’ll normalize the summed scores. This is done by subtracting the mean and dividing by the standard deviation. First, we need to replace the zeros with a null value or it will affect the mean.

tasteoccurrences.replace({0: np.nan}, inplace=True)

Thanks to the normalization (or standardization) here below, the values in our new matrix (the very originally named datamatrix) will indicate how much the features for each wine variety are distant from the mean, in terms of standard deviations. This is known as Z score, and most stats boil down to it. We can use it as a guide to pick which features to retain per variety.

I set a very lax half a standard deviation threshold (> = 0.5) in the code, to reject the features that are rarely associated with some variety. For reference, what we call ‘statistical significance’ consists in a distance of 2 standard deviations from the mean, so we are at 1/4 of that here.

datamatrix = pd.DataFrame(columns=tasteoccurrences.columns) for grapes in range(len(tasteoccurrences)):

datamatrix.loc[grapes, :] = (tasteoccurrences.iloc[grapes, :] - tasteoccurrences.iloc[grapes,:].mean()) / tasteoccurrences.iloc[grapes,:].std() >= 0.5

14. Then, the names of the rows (i.e. the wine varieties) are transferred from tasteoccurrences to datamatrix, and specify its contents as integers (aka numbers), otherwise Matlab is going to unnecessarily annoy us by considering them strings (aka text) when we go over there (soon).

datamatrix.index = tasteoccurrences.index

datamatrix = datamatrix.astype(int)

And get rid of all flavors that haven’t been assigned to any wine variety.

datamatrix = datamatrix.reindex(columns=datamatrix.columns[datamatrix.sum() != 0])

15. Also, the algorithm ideally needs more features (flavors) than entitites (wine varieties) (i.e. more columns than rows). So, we make a quick loop that gets rid of some varieties until this condition is met. Here I drop the varieties with fewer features, just another arbitrary choice.

while datamatrix.shape[0] > datamatrix.shape[1]:

lowfeat_wines = datamatrix.sum(axis=1) < (datamatrix.sum(axis=1).mean() - (datamatrix.sum(axis=1).std()))

datamatrix.drop(datamatrix[lowfeat_wines].index, inplace=True, axis=0)

16. We set up the matrix, with the row and column names saved separately, to be converted to Matlab. Obviously the path where they’ll be saved needs to be changed if you want to run this.

datamatrix = datamatrix.reindex(columns=datamatrix.columns[datamatrix.sum() != 0]) names = np.sort(list(datamatrix.index))

mdic = {"names": names}

sio.savemat("wine_names.mat", mdic)



datamatrix.to_csv(r'C:\Users\stempio\PycharmProjects\AdditionalSources\winedatapyt.csv', index=False)

All is set for the final stage in Matlab, and the algorithm is pretty easy to set-up. In my case, the trickiest part was getting the right graphviz version and installing it correctly.

17. Through Matlab, we move our Python output files to the directory where Kemp & Tenembaum’s algorithm data folder is located.

kemp_path = 'C:\Users\stempio\Documents\MATLAB\formalgo\data';



movefile('C:\Users\stempio\PycharmProjects\AdditionalSources\winedatapyt.csv' , kemp_path)

movefile('C:\Users\stempio\PycharmProjects\AdditionalSources\wine_names.mat' , kemp_path)

18. We teleport to that directory, and perform some minor manipulations on the data. The meal for the algorithm is ready, and this is the salt at the end.

Kindly made for this article by Massimo de Antonis.

cd(kemp_path)

load('wine_names')



names = cellstr(names);

temp = readtable('winedatapyt.csv');

features = temp.Properties.VariableNames';

data = table2array(temp); save('italian_wines', 'data', 'features', 'names')



cd ..

We need to make a small edit to the algorithm’s masterrun.m file, by changing line 35.

thisdata = 21

Similarly, we modify setps.m, by adding the name of our data file (‘italian_wines’ in this case) to the list at line 12 and 27, following the format that is already there.

Just like that, we can run the algorithm in the command window by typing:

masterrun

It’ll work its magic trying to discover the ideal structural form in which to arrange the data it was fed with. The best part is that it will actually show us how it’s trying to figure it out, and if you don’t find this fascinating I doubt you’ve made it this far, but kudos to you regardless!

19. Once it’s done (may take a while), the output we’re interested in will be in the variable modellike, specifically in its 21st column (the number of thisdata that we set up above). We’re interested in the structural form with the highest score, so we look for the max across rows. Then we plot it with its names.

[~, idx] = max(modellike(:,21)); draw_dot(structure{idx,21}.adj, names{21})

And voilà. The first time i managed to run this, it gave me a ring structure as result, which is particularly remarkable given that graphs dealing with taste and flavour are usually depicted as wheels. It also followed the colours of the wines, without being given any indication about it!

“Italian wine ring”, sounds slightly dodgy. Image by author

However, as I refined the code and added features, the algorithm also changed its output, and in the final script it outputted a chain. So the results will vary according to the data, but still following the colour and taste dispositions.

“Italian wine chain”, now that could be a liquor store name. Image by author.

20. Admittedly though, the output of Kemp & Tenenbaum’s algorithm does not look as good as its theory, and also doesn’t convey much information about the entities (wines) and features (flavors). So I asked a good friend of mine for suggestions to help improve its looks. Without further ado, here’s the final form: