In the last discussion we saw how to solve the SVM problem in both its primal and dual forms. You might want to click on the link above to revisit that discussion. But have you ever wondered why does this model have such a funny name, SVM, support-ficked machine? Let's try to find out. In our recent discussions on SVM solvers, we always chose to hide the bias term inside the model since it simplified our algorithms. However, to unearth the secret behind the name SVM, we need to work with an explicit bias term. The calculations here are on our pretty standard. We convert all constraints to less than equal to constraints as a matter of convention, ensure that we have a minimization problem at hand, and then introduce two end dual variables to create the Lagrangian. Next, we begin simplifying the dual problem by trying to minimize the Lagrangian with respect to the primal variables. Note, we now have three primal variables, w, xi, and the bias b. To minimize with respect to w, we can use first order optimality which gives us an expression for w which helps us recover the optimal value of w once we have solved the dual problem. Minimizing the Lagrangian with respect to b requires a bit more care since the Lagrangian is a linear function of b. The only term that contains b looks like the summation over all i alpha i times y i times b. We have seen before, unless this linear term is identically zero everywhere, minimizing with respect to b will yield a negatively infinite value which is useless. Thus, at the optimum, we must have summation over all i alpha i times y i equal to zero. A similar argument tells us that in order for the minimization with respect to xi to make sense, at the optimum, we must have alpha plus beta equals the all c vector. Putting all these relations into the Lagrangian and eliminating the dual variable beta, as we have seen before, gives us the simplified form of the dual with alpha as the only one dual variable. We note that this dual is identical to the one we obtained earlier when we had a hidden bias, except that now we have a new constraint that is summation over all i alpha i times y i equals to zero. Thus, having an explicit bias causes the dual to have an additional constraint that links all the coordinates of the dual variable alpha. We are now ready to reveal the secret behind the name SVM. For sake of simplicity, let us consider a setting where the feature vectors xi and the model w are all three dimensional. Suppose, we have solved the SVM problem to get the optimal values of model w star, bias b star, slacks xi star, and the dual variables alpha star, which gives us the separating hyperplane with the equation w star transpose x plus b equals zero. Let u be the unit vector along w star. Now imagine that each data point xi is exerting a force on the separating hyperplane equal to y i times alpha i star times u. Note that since the force has a term y i in it, data points of opposite labels exert forces in opposite directions, although all forces are parallel to you. Also, the magnitude of the force depends on the magnitude of alpha star i. This setup might seem a bit contrived to you, but just go along with it for now. If we agree with this setup, then we find that the new constraint on alpha introduced by the explicit bias term tells us that the total force on the hyperplane is zero. This means that the forces will not let the hyperplane shift or accelerate in any direction. We find that the total torque due to these forces is also zero. Recall that if a force f is applied at a point r, then the torque due to the force is the cross product of the vectors r and f. The total torque in r case turns out to be the cross product of w star and u, which is zero since they are parallel vectors. This means that the forces will not let the hyperplane rotate either. This result holds for higher dimensions as well by using a more complicated definition of the cross product. So, in this sense, the data points with alpha star i not equal to zero can be said to be supporting the hyperplane as they neither let the hyperplane shift nor rotate. Unsurprisingly, these data points are called support vectors. Notice that w star is equal to summation alpha star i times y i times x i, which means that the data points with alpha star i not equal to zero, that is the support vectors, are sufficient to construct w star. The gurus of machine learning gave this model the name support vector machine because it does computations using a model that relies on support vectors that can be thought of as supporting the decision boundary hyperplane of the model. And now, we know the reason behind the name SVM. To get some more practice on constructing duels, derive the dual for the hard SCM problem with the hidden bias as well as with an explicit bias. Recall that the hard SCM formulation has no slack variables and it makes sense only when we have a classification problem that is linearly separable. If you want to learn more about SCMs and their history, check out this very nice tutorial written by Christopher Burgess. So that's all for this short video. Have a very nice day ahead and I will chat with you next time.