AI, Cluster Analysis of Categorical Data (Part II) | by Michelangiolo Mazzeschi | Towards Data Science

Entire Procedure
Full code available at my repository. The notebook is called: CRL_Clustering.
Downloading match datasets with APIPre-processing: personalized sorting of every rowPre-processing: label encoding of the entire datasetPre-processing: one_hot encoding of the entire datasetPerform cluster analysisTrain K-Node algorithmTuning number of clusters to get an accurate resultEstimate the top meta-decks
Downloading match datasets with API
Pre-processing: personalized sorting of every row
Pre-processing: label encoding of the entire dataset
Pre-processing: one_hot encoding of the entire dataset
Perform cluster analysis
Train K-Node algorithm
Tuning number of clusters to get an accurate result
Estimate the top meta-decks
So far, Part I covered up to step 3: label encoding the entire dataset. In this final part of the tutorial, I will be covering the next five steps.
In step 3 we have labeled each element in the dataset, obtaining the following result:
To feed our dataset into the AI we will need to use one_hot encoding. Only in this way, the model will be able to process categorical data. All our columns are interchangeable: this means that we will need a single giant chunk of one_hot variables.
If we had to apply one_hot encoding to every single column we would be obtaining a one_hot chunk per column: that is now what we want. With 8 existing features, the data will be considered independent between columns, and the labels will be applied to individual columns, not a single dataset:
As you can see from the picture above, this is what would happen if we had to apply one_hot encoding to each column. In the example above, b is in both columns, but it will be decomposed in 1_b and 2_b, and b will be split into two separate variables. What that would mean is that a card, ex. Giant, in column 1 and Giant in column 2 will be completely different. Because this is not the case (the order of the decks should have no impact on the way meta-decks are chosen), we have to find a way to avoid the separation of the same variables in the dataset when they are situated in different columns.
print(max(one_row_labeled+1), len(X_labeled))95 664
Our dataset has a total of 94 cards used among players in any possible deck. This means that for every row there have to be 94 columns, each one representing a dummy variable.
#  one_hot of multiple columns at the same time#create an empty dataset of equal lenght of max. label (94), but equal width of our X (664)m = [[0 for x in range(max(one_row_labeled)+1)] for x in range(len(X_labeled))]
I have first created this empty dataset as a multidimensional array:
I will now iterate through each row of X. Each row will contain 8 different numbers, I will find the corresponding dummy variable row and set it to 1.
#turn each corresponding label to 1for row in range(len(X_labeled.values)):for num in range(len(X_labeled.values[1])):m[row][X_labeled.values[row][num]] = 1m = pd.DataFrame(m)m
After having added the 1 in the right spots, I will convert the list into a DataFrame. This is what we end up with:
Now we can feed it to the model.
Now that we prepared our data, we will need to identify the optimal number of clusters:
#graph the number of clustersfrom sklearn.cluster import KMeansimport matplotlib.pyplot as pltwcss = []for i in range(1, 100):km = KMeans(n_clusters = i, init = 'k-means++', max_iter = 300, n_init = 20, random_state = 0)km.fit(m)wcss.append(km.inertia_)plt.plot(range(1, 100), wcss)plt.title('The Elbow Method')plt.xlabel('No. of Clusters')plt.ylabel('wcss')plt.show()
We are trying to assess the number of optimal clusters through the elbow method. Usually, we would look for the spot where the line in the graph bends the most. Unfortunately, we can immediately see that there are no clear signs of a defined number of clusters (maybe around 20 clusters there is a possible bend). Especially in K-Mode, it is very hard to estimate the best number of clusters due to the categorical nature of the features. We will need to find the optimal number of clusters by using several iterations and comparing the results.
!pip install kmodes
***Because I am using Google Colab I will install the package kmodes using the line above, however, depending on if you are running your algorithms on your local machine or remotely, it may vary.
Let us now run an experiment using K-Mode on our dataset. I will need to indicate the number of clusters to group our dataset. Essentially, the output is an array of 664 numbers. Each number maps one row with its corresponding cluster.
#mapping each row to its correponding clusterfrom kmodes.kmodes import KModestest=KModes(n_clusters=20, init='Cao') #Huang in another possibilcluster=test.fit_predict(m, categorical=list(m.columns))clusterarray([11,  0, 11,  4,  5,  3,  3, 11, 11,  9, 11, 11, 16,  5,  5,  0,  6,         1, 11,  3, 15,  0,  8,  1,  8,  0, 15,  5,  7, 15,  8, 17,  1, 15,         5,  2, 19, 13,  1,  9,  7, 16,  7,  1, 16,  3,  5,  1,  8,  7,  6,        16, 18,  7,  0,  3,  3,  3, 13, 19, 15, 11,  8,  4, 14,  4,  2, 14,         5,  7,  6,  1,  8,  1,  3,  8,  4,  3,  5, 19,  2,  1,  5,  0,  1,         6, 14,  8,  3,  4,  7,  4,  1, 14,  1,  5, 11, 18,  5,  1,  0,  2,        12,  3,  8,  9,  2,  4,  7,  7, 16, 11,  4, 11, 11,  1, 16,  6,  0,        16, 12,  0,  3,  7, 18,  1, 12,  0, 16, 18, 16,  3,  4,  7,  1,  1,         1,  2,  3, 19,  5, 11,  4, 16,  4,  0,  3, 12,  0, 12,  0, 12,  5,         2, 14, 11, 12, 15,  1, 11,  7,  2, 12,  0,  3,  3, 12,  7,  7, 18,         1, 16,  7, 17,  1, 12, 11, 14, 15,  3,  5,  0,  8, 14,  2,  2,  0,         4,  7, 16,  8, 11,  0,  6,  1,  5, 19,  7,  7, 16,  0,  2, 11,  4,         7,  5,  1,  8, 14,  0,  0, 19, 11,  7, 15,  5,  9,  2, 11,  7, 19,        14,  3, 11, 11,  1,  9,  5, 12, 12,  7, 12, 14, 14, 11,  0,  8,  3,         4, 11,  4,  5,  8, 15, 15, 14,  7,  4,  3,  6,  3,  5,  7,  4,  6,        15,  5, 13, 11,  4, 16,  7,  0, 14, 16,  7,  3,  9, 15,  0,  7,  0,         1,  8, 15,  5, 11,  3,  6,  4, 12, 15,  1,  2,  7,  1,  6,  1,  3,         8,  1,  0, 17,  5,  5, 14, 11,  3,  4,  2,  3,  7,  7, 11,  3,  0,         4,  1,  3,  7,  7,  4,  8,  3,  0,  3,  3,  9, 12,  3,  7,  7,  8,        12,  2,  3,  2, 19,  5,  7,  2, 14,  7,  2,  4,  1,  5, 19,  3,  2,        13,  5,  5,  3, 19,  4,  2, 15, 19,  1,  0, 12, 13,  8, 13,  0, 17,         7, 14,  3,  4,  0, 14, 12,  0,  1,  8, 12,  4,  3, 17, 15,  7,  4,         3, 14,  0, 12,  0,  9,  1, 14,  4, 12,  7,  4,  3,  2,  3, 15, 16,         2,  4,  0,  2, 15, 12,  7, 15,  2,  1,  1,  4,  0, 19,  6,  5, 11,        12,  8,  3, 16,  2,  3, 19,  0,  0,  1,  3, 17, 11, 10,  2, 15,  2,         0,  3,  7,  4, 17,  5,  5,  1,  8,  6,  6, 12,  5, 13, 19,  2,  6,         3,  3,  4,  3, 12, 17,  5, 15,  0, 19,  6,  0,  3,  2,  3,  0, 12,        11,  2, 14,  1,  1,  0,  7,  7, 12,  8,  4,  1, 12, 14, 17, 17,  8,         0,  2,  3,  2,  0, 16,  2,  6,  2,  0,  6,  3,  1,  0,  8,  3,  7,        11, 15,  0, 11, 11,  7,  2,  6,  5, 11,  4,  3,  5, 17, 15,  2,  0,        11, 11,  4,  3, 16, 19,  8, 15, 14,  0,  5, 13,  2,  0,  2, 14, 19,         8,  0,  1,  1, 14,  6, 15, 19, 19,  2,  0,  1,  1, 11,  5,  3,  3,         6,  8,  8,  1,  0, 19,  7,  2,  8,  4,  0, 12, 12, 12, 14,  4,  2,        16,  9,  0,  0,  4,  0,  2,  8,  4,  0,  2, 15,  4, 13, 15,  3,  0,         0,  4,  5,  6,  9,  0,  4, 14, 19,  1, 15, 15, 14,  2, 17,  4,  0,        15,  2, 12,  8, 14,  2,  1, 15,  2,  2,  2, 11, 12,  6,  0,  0, 12,         3, 17, 14, 11,  4,  0,  2,  4, 17,  0,  3,  5, 16,  0, 12,  0, 17,        17,  6,  4,  0, 16, 16,  0, 14,  3, 12, 11, 19,  6,  8,  2,  0,  4,         0,  7, 17,  1,  0, 15, 16, 13,  1,  7, 16,  0, 14,  6,  0,  0, 16,         5], dtype=uint16)
Now, how do we isolate the clusters and understand what characteristics they need to have?
test.cluster_centroids_array([[0, 0, 0, ..., 0, 0, 0], [0, 1, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 0], ..., [0, 0, 0, ..., 0, 0, 0], [0, 0, 0, ..., 0, 0, 1], [0, 0, 0, ..., 0, 0, 1]])
These 20 lists of one_hot numbers describe the features of every single cluster. Each one is composed of 94 elements, some of them are 1, the majority is 0. If for every single list, we will convert these 94 binary values to numbers, then the numbers to card names, we will obtain the 20 meta-decks.
For now, the results are incomprehensible to us. We need to translate them into a language we can comprehend. The algorithm below takes the output of a K-Mode algorithm and translates it into its original strings:
#convert one_hot numbers to final decksdef archetypes(knode_list):  archetypes_list = list()#for each 1 inside the one_hot cluster form return its number  #not all clusters have the same lenght  #store all number is arrays  cc = list()  for n_centroids in range(len(km.cluster_centroids_)):    for k in range(len(km.cluster_centroids_[n_centroids])):      if km.cluster_centroids_[n_centroids][k] == 1:      cc.append(k)  len(cc)#determines number of chunks per cluster  n_chunks = list()  for am in range(len(km.cluster_centroids_)):    n_chunks.append((list(km.cluster_centroids_[am]).count(1)))  n_chunks#creates the meta-decks  from itertools import islice  it = iter(cc)  sliced =[list(islice(it, 0, i)) for i in n_chunks]  slicedsliced_list = list()  #convertion numbers to card names  for o in sliced:    sliced_list.append(order_row(list(le.inverse_transform(o))))#we only conserve the win conditions  card_dictionary = dict(zip(cards_classifier[0],  cards_classifier[1]))  card_dictionary#an absurd way of making a copy of our dataset, .copy() does not work  list_tot_classification = [x[:] for x in sliced_list] #un modo     assurdo per fare copia, altrimenti non funzionafor o1 in range(len(list_tot_classification)):    for o2 in range(len(list_tot_classification[o1])):      a = card_dictionary.get(list_tot_classification[o1][o2])      list_tot_classification[o1][o2] = aonly_win_conditions = list()  #only conserve lists with win conditions  for o3 in range(len(list_tot_classification)):    if 'Win Condition' in list_tot_classification[o3]:    only_win_conditions.append(sliced_list[o3])#we store all the lists with a lenght of 8: full decks  for _ in sorted(only_win_conditions):    if len(_) == 8:      archetypes_list.append(_)return archetypes_list
Now, let us try different configurations of the algorithm we have just written and compare the results. I will run a total of 60 clusters, storing both results and the number of meta-decks for every iteration:
import numpy as npfrom kmodes.kmodes import KModesarchetypes_len = list()for clus in range (8, 60): ###FUNZIONA IN UN FOR CYCLE, MA NON IN UNA FUNZIONE  km = KModes(n_clusters=clus, init='Cao', n_init=100, verbose=0)  clusters = km.fit_predict(m)#one_hot results to archetypes sorted by win condition  archetypes_list = archetypes(km.cluster_centroids_)  archetypes_len.append(len(archetypes_list))  for _ in archetypes_list:    print(clus, _)#25 is the optimal number of meta-decks
As we can see from the last image, the higher the number of clusters, the higher the number of similar meta-decks. Essentially, at one point also the deck variations are included in the calculation.
I have now stored the number of clusters in archetypes_len. I will create a list that contains the range of clusters, so we can graph the relationship between the number of clusters and the number of meta-decks.
import seaborn as snsarchetypes_iteration = [x for x in range(8, 60)]archetypes_iterationsns.set(rc={‘figure.figsize’:(11.7,8.27)})sns.scatterplot(archetypes_iteration, archetypes_len)
How does this translate into our hierarchy of meta-decks?
If the number of clusters is too small, we end up with only a few meta-decks. They will be correct, probably the ones at the top of the list, but they only represent one small batch among the pool of meta-decks.
If the number of clusters is too big, not only we will list the top decks, but also all their variations, ending up with a list of decks that blurs the notion of meta-decks.
Only with the right amount, we can have a notion of the most used meta-decks in the current Season.
Finally, after looking at the data, I have selected a number of clusters that show the top-meta decks with minimum variations. We can conclude that these are the meta-decks for Season 11, the current Clash Royale season:
#questo funzionan_clusters = 25km = KModes(n_clusters=n_clusters, init='Cao', n_init=100, verbose=0)clusters = km.fit_predict(m)#one_hot results to archetypes sorted by win conditionarchetypes_list = archetypes(km.cluster_centroids_)for _ in archetypes_list:  print(_)