Foundations in Analytics: The Importance of Scale
By: Nathan Pratt
Tags: R, Data Analysis, Data Science, Editors Pick

I’m sure many of the readers here are familiar with the basics of scaling data. I’ll be covering some simple intros, but this article is primarily to provide you with an example to use when introducing the uninitiated. I’m writing this as a bit of a primer for a semi-trivial project I proposed to some apprentices in the past and would like to share here in the near future. A much more detailed explanation of feature scaling can be found here.

Table of Contents

Much of machine learning involves minimizing for some value. KNN and K-means are among some of the earliest models you will be introduced to and are highly dependent on the scale of the values you pass to them. There are many others that have similar dependencies (PCA, Linear Discrimination Analysis, Hierarchical Clustering, and SVM, the list goes on). Without proper scaling, certain features can overpower the influence others can exert on your results.

I will be providing code in R. There is another article that covers the common scaling functions used in Python and goes into more depth regarding use cases and outliers. We will only cover the two most basic ones. There is some contention regarding how each should be defined, so for the duration of this article we will define them as follows:

Normalization: A variable will be scaled from 0 to 1 given a minimum and maximum.

Standardization: A variable will be scaled such that it now has a mean of 0 and a standard deviation of 1.

Wide aspect ratio for feature image. Additional animated charts at the end of the article.

Example Problem

The primary example we are going to consider can be found here. This is a very simple introduction to elbowing using a within-cluster sum-of-squares for determining an appropriate K value. The site does a great job of introducing this concept, but you’ll see that scale was overlooked. Disclaimer: Perhaps this was intentional. If not, I really hope the site doesn’t correct this. This is one of my favorite examples for apprentices and mentees getting into the field.

I encourage you to check out the example above if you have not already. We are going to follow the code very closely (but abbreviated) to recreate the solution and then discuss the importance of considering scale for a problem like this.

As mentioned in the article, you can download the data here. Unzip the file so we can read it in.

Open up your R environment, I recommend RStudio. In case you don’t have the libraries we’ll be using, the following code will install and load the libraries as necessary.

# declare packages for use

requiredPackages = c("ggplot2", "dplyr", "tidyr", "gganimate") # try to load package, if it fails then install and load

tmp = sapply(requiredPackages, function(pkg){

if(!require(pkg, character.only = T)){

install.packages(pkg)

library(pkg, character.only = T)

}

})

We will be using the function file.choose() to select our file (from the unzipped folder above). This will cause a window to pop-up and allow you to select the file for import.

testDf = read.csv(file.choose()) # check structure of the dataframe str(testDf) # remove ID column

testDf = testDf[-1] ### wss function plotWss = function(df, maxK = 20){

wss = (nrow(df)-1) * sum(apply(df, 2, var))

for (i in 2:maxK){

wss[i] = sum(kmeans(df, centers = i, nstart = 20)$withinss)

}

plot(1:maxK, wss, type = "b", xlab = "Number of Clusters", ylab = "Within Cluster Sum of Squares")

} plotWss(testDf)

Now get the models and plot the charts for k = 3 and k = 5. At first glance, k3 looks relatively decent, but you will notice some odd points classified as cluster 1 in the 40–60 range of the y-axis (TestB). Looking at k5 makes the problem even more evident. It appears that the value of TestA is much more important than the value of TestB.