Semi-Supervised Learning with K-Means Clustering | by Yufeng | Towards Data Science

Supervised learning and unsupervised learning are the two major tasks in machine learning. Supervised learning models are used when the output of all the instances is available, whereas unsupervised learning is applied when we don’t have the “true label”.
Even though the exploration of unsupervised learning has huge potential in future research, supervised learning is still dominating the field. However, it’s common that we need to build a supervised learning model when we don’t have sufficient labeled samples in our data.
In such a case, the semi-supervised learning can be taken into consideration. The idea is to build a supervised learning model based on the output of the unsupervised learning process.
I would like to go through a simple example.
I collected the NBA players’ per game stats in the season of 2018–2019. The players’ positions are defined as the conventional basketball positions: point guard (PG), shooting guard (SG), small forward (SF), power forward (PF), and center (C).
Before the modeling process, I did some pre-processing on the dataset. First, remove the players who played less than 10 minutes per game. Then, fill NA values with 0 (For example, center players never shoot 3 pointers).
df_used = df_num.loc[df.MP.astype('float32') >= 10]df_used.fillna(0,inplace=True)
After the pre-processing, the data looks like below:
df_used.head()
Then I separated the data to the training and testing set.
from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(df_used, labels_)
If all the labels (players’ positions) are given, it’s a simple supervised classification problem. I fitted a simple logistic regression model to the training dataset.
from sklearn.linear_model import LogisticRegression from sklearn.pipeline import Pipelinefrom sklearn.preprocessing import StandardScalerpipeline = Pipeline([        ("scaler", StandardScaler()),        ("log_reg", LogisticRegression()),    ])pipeline.fit(X_train, y_train)
I evaluated the model on the testing dataset.
pipeline.score(X_test, y_test)
This procedure gives 0.644, which means 64.4% of the predictions are correct. From this result, we know that the room for improvement on the classifier is very large. However, I am not focused on the classifier development in this article.
I am going to talk about the situation when the data labels are only partially seen.
If I am only allowed to see the position information of 100 players in the training data, how will it change the performance of the model? There are a lot of strategies that can be applied to select the 100 players as the input of the classifier.
The first strategy is to randomly choose 100 players, the first 100 players, for example. I checked the performance of the logistic regression model trained only on this subset.
n_labeled = 100pipeline.fit(X_train[:n_labeled], y_train[:n_labeled])pipeline.score(X_test, y_test)
This time I got only 56.8% of the accuracy. It is as expected because I have only seen a subset of the true labels.
However, can we improve the performance by selecting a different subset of 100 labels? The answer is yes.
The second strategy is to apply the unsupervised learning procedure to cluster the data in the entire training dataset, and to expose the labels of the representative of each cluster. In this way, we can assume the data points that are close to each other in the clustering space should have a high chance to own the same label.
In another word, the players who have similar game stats should play the same position on the court.
from sklearn.cluster import KMeansk=100kmeans = KMeans(n_clusters=k)X_dist = kmeans.fit_transform(X_train) representative_idx = np.argmin(X_dist, axis=0) X_representative = X_train.values[representative_idx]
In the code, X_dist is the distance matrix to the cluster centroids. representative_idx is the index of the data points that are closest to each cluster centroid.
After selecting the representatives in the feature space, we collected the true labels of these data points. Here, I just need to extract the labels from the raw data.
y_representative = [list(y_train)[x] for x in representative_idx]
However, please note that the practical situation is that we don’t know any true label, so it’s necessary to manually label these selected data points.
Let’s check the performance of the model trained on this subset of training data.
pipeline.fit(X_representative, y_representative)pipeline.score(X_test, y_test)
I got 59.6% accuracy! Even though it is not comparable to that of the model trained on the entire training set, it’s better than that of the randomly chosen 100 data points.
With the evolution of basketball, it’s becoming harder and harder to tell the player’s positions based on their game stats. That’s why we only have an accuracy of around 60%.
Even though the performance of the model trained on the cluster centroids is better than that trained on random data points, the improvement is limited. It could be interpreted as that the game stats of the players in the same position could vary a lot in the current NBA.
hands-on machine learning with scikit-learn keras and tensorflowBasketball Reference.
hands-on machine learning with scikit-learn keras and tensorflow
Basketball Reference.
I hope you find this short article useful! Cheers!