Building Your First Machine Learning Model with Linear Regression Using Ordinary Least Square
Source: tai_blog

IntroductionSuppose youre on the hunt for a new apartment in your dream location  be it Thailand  Japan  or London. Youve got the money (lets skip the how for now)  but how do you decide on the right price? You dont want to just rely on the sellers word  right? How about staying a step ahead by using a machine learning model to predict the price  ensuring you negotiate like a pro?   To build this model  youll need a dataset with past prices in that area. Lets assume youve somehow acquired this elusive data. You now have features like land area  the number of rooms  the number of bathrooms  and living room dimensions  along with the all-important price column. Naturally  each of these features will influence the price in different ways.   Now  lets simplify. For the sake of understanding  well just consider the relationship between the number of rooms and the apartment price.   Understanding the RelationshipImagine plotting this relationship on a graph  price on the y-axis and the number of rooms on the x-axis. You might think that by tweaking a weight (which represents the influence of the number of rooms on price)  you can get the perfect line to predict prices.   But what if the relationship isnt just a straight line through the origin? What if its offset? Thats where bias comes in  adding a little twist to our equation to capture a more accurate relationship.   Okay you have observed that you are basically drawing a line and changing it with changing both weight and bias to best represent the relation between input and output axis.   The Fun Part BeginsLets proceed with three data points representing the prices of three different apartments based on their number of rooms. If I asked you to draw a line that perfectly fits all three points  you might struggle to get it just right. And guess what? You dont actually need to.   In reality  youre aiming to draw a line that best represents the overall trend  rather than fitting every single point exactly. This line is known as the best fit line.   Finding the Best Fit LineHow do you determine if your line is the best fit? This is where we get into the concept of error. We want to minimize the difference between the actual price (from your dataset) and the predicted price (from your model).   The error for a single data point can be calculated as the difference between these two values. But since errors can be positive or negative  simply summing them up wouldnt give us a meaningful measure. To avoid the confusion of negative errors  we square each error before adding them up  a process known as calculating the Mean Squared Error (MSE).   Now  your goal is to minimize this MSE to find the best fit line.   Enter OptimizersTo minimize the MSE  we need some help from optimizers. For now  well focus on two: Ordinary Least Squares (OLS) and Gradient Descent.   Ordinary Least Squares (OLS)Lets dive into some math.   We start with the equation of a straight line  which we use to represent the relationship between price and the number of rooms.   y = mx + b   Here  we need to determine the values of m (the slope) and b (the bias). Using calculus  specifically differentiation  we can find these values by setting the derivative of the MSE with respect to m and b to zero.   Because the MSE is a convex function  finding where its derivative equals zero will give us the global minimum  our best fit line.   Lets break it down further:   When differentiating with respect to b  treat m as a constant  and vice versa. This partial differentiation helps us isolate the effects of each variable.Once youve calculated these  you can plug in your values and draw your best fit line.   Coding OLS from ScratchNow  enough with the theory  lets get our hands dirty with some code. Heres how you can implement OLS from scratch:   import numpy as np def ols(x  y): mean_x  mean_y = np.mean(x)  np.mean(y) m = np.sum((x - mean_x) * (y - mean_y)) / np.sum((x - mean_x) ** 2) b = mean_y - m * mean_x return m  b # Example dataset used from the start x = np.array([2  3  4]) y = np.array([3  4  4.5]) m  b = ols(x  y) print(fSlope (m): {m}) print(fIntercept (b): {b}) # Plotting the best fit line import matplotlib.pyplot as plt plt.scatter(x  y  color='blue') plt.plot(x  m * x + b  color='red') plt.xlabel('Number of Rooms') plt.ylabel('Price') plt.show()ConclusionCongratulations! Youve successfully used Linear Regression with OLS to predict the price of your dream apartment based on the number of rooms. But hold onthere's a catch. OLS is great for small  simple datasets  but it struggles with large  complex data. That's where Gradient Descent comes into play. Its not just for Linear Regression but is a powerhouse in many machine learning algorithms. Stay tuned for the next blog where well dive deep into Gradient Descent.