Diamond price prediction based on their cut, colour, clarity, price with PyTorch
By: Vineel Sindiri
Tags: Machine Learning Course, Linear Regression, Deep Learning, Pytorch

Have you ever asked yourself, how are diamonds priced? Well, this article talks about the diamonds price prediction based on their cut, colour, clarity & other attributes and it also covers the building a simple linear regression model using PyTorch.

I have started my PyTorch journey with “Deep Learning with PyTorch: Zero to Gans” course and now we are in the second week of this course, till now we have covered the basics and implementation of Linear & Logistic regressions in PyTorch. Please refer to this article for basic PyTorch functions.

Intro to Linear Regression:

Linear Regression is one of the simplest algorithms in Machine Learning. As per statistics, the term Regression is defined as a measure of the relation between an output variable and the input variable(s), hence, Linear Regression assumes a linear relationship between the independent (input) and dependent (output) variable.

More generally, a linear model makes a prediction by simply computing a weighted sum of the input features, plus a bias (also called the intercept) term. (Source) as shown in the below equation.

Source: CS.Brown.edu

I have considered the classic Diamonds dataset which contains the prices and other attributes of almost 54,000 diamonds and this dataset is hosted on Kaggle. The dataset contains 53940 rows and 10 variables. Before jumping into building the model, let’s have a look into the variables & their definitions.

Price is in US dollars

is in US dollars Carat weight of the diamond

weight of the diamond Cut quality of the cut (Fair, Good, Very Good, Premium, Ideal)

quality of the cut (Fair, Good, Very Good, Premium, Ideal) color diamond colour, from J (worst) to D (best)

diamond colour, from J (worst) to D (best) clarity a measurement of how clear the diamond is (I1 (worst), SI2, SI1, VS2, VS1, VVS2, VVS1, IF (best))

a measurement of how clear the diamond is (I1 (worst), SI2, SI1, VS2, VS1, VVS2, VVS1, IF (best)) x length in mm

length in mm y width in mm

width in mm z depth in m

depth in m depth : The height of a diamond

: The height of a diamond table: The width of the diamond’s table expressed as a percentage of its average diameter

Let’s get started to build the linear regression model based on the following steps

Import Required Packages Load the dataset Perform the exploratory data analysis (EDA) Prepare the dataset for training Create a linear regression model Train the model to fit the data Make predictions using the trained model

Step 1: Import Required Packages

Step 2: Load the dataset

To load the dataset we will be using pd.read_csv() function and it will convert the dataset to a dataframe and have a look into the top 5 rows of dataset using pd.head() function

Drop the column “Unnamed: 0”, which is unnecessary

The dataset has 53940 rows & 10 variables

Step 3: Exploratory Data Analysis (EDA)

Look at the concise summary of the dataset

There is no null data in the dataset and the same can also be checked by using data.isnull().any . The given dataset has 6 numeric columns and three non-numeric (categorical) columns

Get the descriptive statistics of the dataset using pd.describe()

Observed that the minimum value of x (length), y(width) & z(depth) is zero and It doesn’t make any sense to have length\width\depth of a diamond to be zero.

Let’s drop these rows

Observed that the minimum values of x,y&z are non-zero values

Plot the distributions of numerical attributes by using hist() method

Let's look at the pair plot of the dataset. Pair plot allows us to see both the distribution of variables and also the relationships between two variables

Observed that x,y&z variables has good correlation with respect to the dependent variable (target) price and let’s quantify that correlation by using .corr() & visualize the same using sns.heatmap() method

We can conclude that carat,x,y & z features have strong correlation w.r.t price variable and depth has a very weak relation w.r.t price variable. Hence, we can drop the depth feature from the final list of input features to the model.

Let’s have a sense of categorical features with respect to target (price) variable by using box plots

I have used the boxplots to compare the data spread, central tendencies (Median), variability amongst the categories of a given categorical variable and these plots also helps to identify the outliers.

Till now, I have done very preliminary data exploration to understand the features and it's their relation with respect to the target variable. And the main objective of this article is to implement the linear regression in PyTorch, so let’s stop the data exploration and jump into the main core of the article.

Before proceeding to build the model, it is good to convert the categorical data to numerical data and there are two ways to convert to numerical form 1. Label Encoder or Integer Encoding 2. One-hot Encoding

In general, one-hot encoding does a good job and for further details please refer this link

Step 4: Prepare the dataset for training

We need to convert the data from the Pandas dataframe into PyTorch tensors for training. To do this, the first step is to convert it NumPy arrays

Now, we need to create PyTorch datasets & data loaders for training & validation. The first step is to create TensorDataset by converting the input & target arrays to tensors by using torch.tensor function

And now we will be dividing the dataset into training & validation dataset. The training set will be used to train the model and whereas validation set is used to evaluate the model & it also used to tune the hyperparameters (learning rate, epochs, batch size, etc..) for the better generalisation of the trained model.

We will use the random_split function to split the dataset into train & validation of the desired percent

Let’s look at a batch of data to verify everything is working fine so far.

The shapes of xb & yb indicate that everything has working fine till now

Step 5: Create a linear regression model

We will be creating the linear regression model by extending the nn.Module class from PyTorch

Inside the __init__ constructor method, we instantiate the weights and biases using nn.Linear the function. And inside the forward method, which is invoked when we pass a batch of inputs to the model, and it will pass it into self.linear . The defined neural network will look like as shown below

We have defined the model and instantiated the weights & biases. Now, let’s define the cost function.

The common cost function(J) for linear regression is mean squared error (MSE). This function calculates the difference between the predicted and target values and squares it. Cost function measures to evaluate how good the model performance is.

training_step & validation_step the method computes the predictions & MSE loss for every training & validation batches respectively. validation_epoch_end the method returns the average validation loss for every epoch. Please refer this link to know the difference between epoch & iterations.

Let’s create a model object by calling DiamondPriceModel() class

Step 6: Train the model to fit data

The main objective of linear regression is to minimize the cost function.

Gradient Descent is an optimization algorithm which helps to minimize the cost function and its main aim is to find out the global minima. Here, gradient represents the rate of change of a function and it points in the direction of greatest increase of the function. To minimize the cost function, we need to perform the gradient of our cost function (J) with respect to its parameters (weights(w) & biases(b)).

NOTE: The learning rate α is a way to control how much we update the weights by. If we choose a small learning rate, it may take a long time for our model to train. However, if we choose a large learning rate, we may overshoot and our training will never converge. The specific learning rate depends on our data and the type of models we use but it’s typically good to explore in the range of [1e−8,1e−1].

evaluate function, which invokes validation_step & validation_epoch_end methods and returns the average validation loss after every epoch.

fit function performs the model training by updating the parameters for each step with the help of stochastic gradient descent(A variant of gradient descent) optimization. It records the validation loss from each epoch and returns a history of the training process.

The initial validation loss very high, which is what one might expect from a randomly initialized model. The objective is to minimize this loss using sufficient epoch and suitable learning rate. This is a trial and error process to generate the best fit for our model.

Ran the model for 800 epochs and the validation loss has been reduced to 1276336.0000. Now, let’s test the model with validation data.

Step 7: Test the model on validation data

predict_single function returns the predicted diamond price value for a given input tensor.

Test # 1

Test # 2

The model performance can improve by introducing the hidden layers to the current model, and can further improve the optimizer to Adam optimizer.

Code Repository: You can find the notebook at my GitHub repo.

Well !! we have just completed our first machine learning alogirth using PyTorch. Thanks for Reading !! and try yourself with your own dataset.

Happy Learning ! Happy coding !

References: