Optimization of The Square Loss
Tags: machinelearning
Published: 2023-11-21 21:22:12

The square loss function is the average of squared differences between the actual and the predicted values. the squared error is one of the most important functions in machine learning. It is used to train linear regression problems and certain neural networks.

In this problem, there are 3 power lines and our task is to place the house at a point where it is the most cost-effective.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a9bbssy42rqb4zskmp2y.png)

Firstly, let's take the first power line. Where the house be situated to get the least cost? Obviously, where the distance is zero, so that we get zero cost.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3ucc52uthp9mtajasnko.png)


Now, let's bring another power line to the discussion. 

Let, a= the distance of pl1 from origin
     b= the distance of pl2 from origin
     x= the distance of the house from origin


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/aejfo0s424xoxiby2jyh.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a06enupak6cxmaxpavns.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bpk4ih8z4o4grhatotqk.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rhval6ioljlhq0x3mhwz.png)

So, we can see the cost of connecting both of the powerlines is quite the same when the house is placed in the middle.

We don't want any of these above situations cause the cost area is very big.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lxr99oa42yes105vor10.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tmmb41n09w61cehu3tj3.png)

We see that the tangent is zero in the lowest point. (x-a)^2 +(x-b)^2 is the cost function here, the derivate of this function is- 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mlnem01owemx9poq5gv5.png)

So, if the house is placed between two poles the cost will be minimized.

Now, let's see what happens if a 3rd powerline is in the scene,
Again, if we repeat the same procedure,

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ktm2e1m6k89u76x2c93h.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vo2u6yszku8f45htt97a.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g9w2kzo4rwszg6ikmql9.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cjj57jo7398s9tyo4thv.png)

If we sum the heights of the three parabolas, we get the black parabola.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o86jealgcy1fve4cohkl.png)

finding the derivative of the cost function-
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3juokm6ao15jh49433ur.png)

So, the generalized solution is-
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nwz6g2azf4r3jdl6vq1c.png)

Finally, the solution would be to put the house in the average of the coordinates of all the power lines. And this is the square loss function. 

**The pictures are taken from different source.