Hello everyone, welcome to the course of machine learning with Python. In this video, we shall learn about a very good classifier known as socket Vector Machine classifier. So it has a widespread used. So we'll see intuitively how support vector machine works. So support vectors. So, what are called support vectors.

So, in a binary classification problem, we usually want to find a line or usually the hyperplane in higher dimension that separates the two classes of the points as you can see, this is basically of class two points and these are class one points. So we want to have a hyperplane or straight line in two dimension in order to separate these two classes, okay. Now to choose a good line we have to optimize some objective function. For example, as we have seen in logistic regression, we minimize the cross entropy loss function. Usually the objective function depends on all the points Okay, so now look at this could be one of the separator or one of the decision boundary, this could be another decision boundary this could be at the decision boundary okay. So, there can be many such lines hence, finding a good line is a difficult task.

Moreover, changing the position of the training points can affect the decision play. So, primarily we want least number of misclassifications of the test points Now, consider a decision plane as shown over here which points are more likely to be misclassified these points or these points. So, the answer is the test points which are closer to the decision boundary are more likely to be misclassified right hence, when designing the classifier we need to give more emphasis on the training points which are closer to the border. The training points closer to the border which are most crucial to design the classifier are known as support vectors. Okay, so in this case, these three are all separate vectors, some mathematical prerequisite so Part A called a normal Victor, so, it is nothing but the magnitude. So, how do we calculate a normal vector So, let's say a vector v is equals to v one v two v three up to V transpose.

So, again why transpose because we denote a vector by a column vector or the column matrix format, then the norm of the vector is which is denoted as these double line bracket V is equals to square root of v one squared plus v two squared up to vn square. Now equation for n dimensional hyperplane it will be w one x one plus w two x two up to w n x n is equals to E where E is a scalar and W one w two w three Who all are nothing but scalar values okay. So, if we consider W to be a vector of all these scalar values w one w two w three, when then we can write the equation of the in dimensional hyperplane as W transpose X equals to E as we have seen W is nothing but the data of all these week.

This is also called the wave vectors. The first task of a support vector machine is to identify support vectors that is creating data points those are closer to the boundary and act as support okay. So, we are identifying the support vectors. So, these are the support vectors. Now, l one and l two are the lines or hyperplanes defined by the support vectors. So, these are the lines defined by the support vectors marching is the separation between the lines l one and l two.

Note that the separation is nothing but the perpendicular distance. So, this is basically the marching, our objective is to maximize the marching. Hence support vector machine is also called a maximum margin classifier, the decision boundary is the line or the hyperplane that passes through the middle of L one and l two. So, as we can see, this is our decision boundary over here. So, now let's say W is the vector normal to the line in even better equation W transpose X equals to eight Okay, so w is the vector not one two the given land now the perpendicular distance of the line in from any point U is given by W transpose u minus e this whole magnitude hold divided by the norm of W okay. So, this is our line this is the perpendicular vector which is perpendicular to this line.

So, his perpendicular distance of the line is from the origin is given by D of zero comma L which is nothing but magnitude of a whole divided by the norm of W okay why because you Victor here would be zero. Now, the next step is scale the W vector and V such that the lines are defined by these equations. So, l one will be nothing but W transpose X plus B equals to minus one into b W transpose X plus B equals two plus one and our decision boundary that is ill will be nothing but a Buta. Suppose x plus b equals to zero Okay. So, as we can see this is a one this is a two and this is our hence the modular the separation between L one and l two which is nothing but d of L one comma L two would be nothing but, this minus this which we ultimately give you two divided by the norm of the vector w is to maximize the margin we have to minimize the norm of the weight vector w. So, we have to find the weight vector W and V which will minimize this law of the weight vector minimizing the norm of the weight vector is same as minimizing the squared norm okay.

So, squared norm is nothing but W transpose W fun. So, let's see one is the set of all points that belongs to class one and C two is the set of all points that belongs to class two late Why is the corresponding class level of is creating point which is excited to know why you can take either plus one or minus one that means more n equals two plus one if x is a vector belongs to C one that is class one and it is equals to minus one if excited two belongs to class two, all the training points on the left of the line l one belongs to class two, where all the training points on the right of the line into belongs to class one. So, what does it mean? So, these all these blue points as you can see, it belongs to the right of these lines and this belongs to class one similarly, all these triangular points are left to the line l one and this belongs to class two.

Okay, so, here's our constraint to the optimization problem is W transpose X i plus b should be less than equals to minus one for all x, y belongs to C two and W transpose X i plus b, should we get a third equal to one for all excited belongs to C? Well, okay, this is very much straightforward because we know this is basically W transpose X plus B equals to minus one this line So, these points must have W transpose X plus B less than equals to minus one. And these points must have W transpose X plus B greater than equals to plus one. Now, we can combine these two constraint into one constraint, if we multiply these by y, we know that for x i belongs to C to y is nothing but minus one. So, if we might multiply this whole thing by y, what we actually achieve that minus one into minus one, this will become plus one and this sign will be changed to rather equal to.

Now again, if x is belongs to C one, then y equals two plus one, then multiplying this equation with class one, just do no harm to this particular equation right. So simply we can say that y multiplied with W transpose X vector plus b should be greater than equals to one for all is equals to one to a to m. Note that there are enough Printing samples. Hence, our overall optimization problem for SVM is to find the value of a filter and the bias V, which will minimize the function hapa W transpose W subjected to the constants y multiplied with W transpose X plus B should be greater than equals to one for all. Now, the W vector and V thus often defines our classifier for predicting the class of a new place point is Victor, we can use the following equation. So, the class of x should be equals to signum of W transpose X plus B as we know that signal value will give me either plus one or minus one depending upon the size of its argument.

Note that this is for binary classification for multi class classification we can again use decomposition techniques like one versus all or one versus rest. So, this is called hard machine support vector machine classifier as opposed to soft margin support vector machine class. effect which we shall introduce shortly in our next video. So see you in the next lecture. Thank you.