Clustering on Mixed Data Types in Python
By: Ryan Kemmer
Tags: Data Science, Mixed Data Types, Clustering, K Means, Python

Image by the Author

During my first ever data science internship, I was given a seemingly simple task to find clusters within a dataset. Given my basic knowledge of clustering algorithms like K-Means, DBSCAN, and GMM I thought that I could easily get this task done. However, as I took a closer look into the dataset, I realized the data contained a mixture of categorical and continuous data, and many common methods of clustering I knew would not easily work.

Categorical data consists of multiple discrete categories that commonly do not have any clear order or relationship to each-other. This data might look like “Android” or “iOS”. Continuous data consists of real numbers that can take any value. This data might look like “3.14159” or “43".

Many datasets contain a mixture of categorical and continuous data. However, it is not straightforward how to cluster datasets with mixed data types. So how do we cluster on data that has both categorical and continuous features? Lets take a look at two simple ways to approach this problem using Python.

Dataset

In this post, I am going to cluster a small dataset I created that has a mixture of categorical and continuous features. My fake data represents customer data that might be used to understand customers of an E-commerce website/app. Our fake dataset will have 4 features:

OS — operating system of our fake customer (Categorical)

— operating system of our fake customer (Categorical) ISP — internet service provider of our fake customer (Categorical)

— internet service provider of our fake customer (Categorical) Age — customer age (Continuous)

— customer age (Continuous) Time Spent — time that our fake users spent on our website (Continuous)

Here is the code used to generate our fake data.

#create dataset

import numpy as np

import pandas as pd



operating_systems = ["Android","iOS"]

isp_names = ["Cox","HughesNet","Xfinity","AT&T"]



data = []

for i in range(100):

row = []

row.append(np.random.choice(operating_systems)) #OS

row.append(np.random.choice(isp_names)) #ISP

row.append(np.random.poisson(lam=25)) #Age

row.append(np.random.uniform(low=0.5, high=1000)) #Time Spent

data.append(row)



customers = pd.DataFrame(data, columns = ['OS', 'ISP','Age','Time Spent'])

Here is what our fake dataset looks like. Now lets get our hands dirty and do some clustering!

Method 1: K-Prototypes

The first clustering method we will try is called K-Prototypes. This algorithm is essentially a cross between the K-means algorithm and the K-modes algorithm.

To refresh our memory, K-means clusters data using euclidean distance. Meanwhile, K-modes clusters categorical data based off the number of matching categories between data points. A mixture of both of these: the K-prototype algorithm, is just what we need to cluster our fake customers!

First, lets normalize the continuous features in our data to ensure that one feature is not interpreted as being more important than the other.

from sklearn import preprocessing customers_norm = customers.copy()

scaler = preprocessing.MinMaxScaler()

customers_norm[['Age','Time Spent']] = scaler.fit_transform(customers_norm[['Age','Time Spent']])

Now, lets use the K-prototypes algorithm to cluster our data. My implementation of the algorithm is very simple, with 3 clusters and the Cao initialization. When training the model, we specify which columns in the data are categorical (columns 0 and 1). More information on implementing and fine tuning this algorithm can be found here: https://github.com/nicodv/kmodes.

from kmodes.kprototypes import KPrototypes kproto = KPrototypes(n_clusters=3, init='Cao') clusters = kproto.fit_predict(customers_norm, categorical=[0, 1]) #join data with labels

labels = pd.DataFrame(clusters)

labeledCustomers = pd.concat((customers,labels),axis=1)

labeledCustomers = labeledCustomers.rename({0:'labels'},axis=1)

Finally, lets make some swarm plots to see how our clustering performed.