Hello everyone, welcome to the course of machine learning with Python. In this video, we shall learn about how to implement k nearest neighbor classifier in Python. So we start from importing the necessary libraries. So we are importing NumPy. And from SK learn dot data set, we are importing load IDs because we will be going over classification on Iris data set. So let's go ahead and press Shift Enter on the cell to run this particular set.

Okay, now we'll be loading the data set to the variable x and y. Next step would be splitting the entire data set into train and test. So from SK learn dot model underscore selection will be import crane underscore test underscore split. So this function will enable us to divide the entire data set into create and test. So we'll run this particular cell and see that our training data set contains 120 data points. And test data set contains 30 data points now we'll be standardizing the data set.

That means we'll be normalizing the dataset. So from SK learn dot p processing will import standard scaler. So standard scalar is a class. So if c is the instance of the object of this class, and we'll be fitting extreme and estates separately to that particular class. So now our columns in explain will have unit variance and zero mean. Similarly, the column index test will also have unit variance and zero mean Now we shall look upon an important concept called pairwise distance and how it works.

So from SK learn dot matrix, we'll be importing a function called pairwise underscore distances. Let's say we have a set of points which is denoted by zero comma one and one comma minus one and we have set up points which is another set of points which is noted by zero comma zero and two comma zero if we want to compute the pairwise distances between the set of points one and set of points two, we can simply call this function pairwise underscore distances and pass these set of points and see what are the results. Now we shall interpret the result look at the first point in the set of points one. So the first point is zero comma one. And the first point in the set of points to is zero comma zero. So the first element in this adding is the distance between the first point of set of points one with the first point of setup points to similarly the second element of this ad is the distance between the second point of the set of points one that means one home minus one with the first point of setup points to okay.

So similarly, we will have two into two total for distances. So pairwise distance actually computes all the distances between one set of points to another set Points okay. I hope this is clear. Now we should implement our K nearest neighbor algorithm. So, we have defined our kts algorithm in this particular cell look at what are the attributes or what are the variables it takes. So, it takes x underscore train y underscore train number of classes in underscore class x underscore test that is the test data points and number of nearest neighbor which is identified by key.

Now, by default the value of k is three okay. So, number of test points here is nothing but how many points are there in x test right we compute the pairwise distance matrix which is denoted by capital D between the test points and the training points. So, what will be our predicted variable or y predict we initialize this with an empty matrix. Okay. Now, for each of the base points We'll find the neighbors by the function NP dot art sought. So art sought his argument sort of which metrics that is the pairwise distance matrix.

And look at the index. So I am selecting row I and all the columns, okay, starting from one to k plus one, so y one to k plus one, because if the distance between the point let's say x to the point x would be zero, and this would be the minimum distance, so we have to exclude that. So that is one I am not taking from zero I'm starting from one and I am taking k plus one Now we shall label the neighbors. So what will be the label of the neighbors which is nothing but from the Y train, we'll be taking all the necessary levels of the neighbors. Now the implement majority 14 Let's introduce another variable count count is initialized with a zero vector now for j in levels underscore neighbor count within bracket t plus equals to one and why underscore credit I should be NP dot arg max of count okay.

So, I am taking the label of that particular test sample to be nothing but which one is occurring most that is the majority 40 and then from the function will return the predicted values of y. So, this is a vector will be returned Okay, so, let's go ahead and run this particular cell. Now, we should create it using our K nearest neighbor classifier. So, here I am calling our K nearest neighbor classifier here external we have already specified it contains 120 data points. Similarly y train number of classes identified as three now instance is equals to x underscore test which we have already Specify these countries, two data points and key we have specified three, okay, we can also specify these as five or one or seven. Okay, now let's go ahead and run this particular cell file.

Now we shall define from fusion metrics function. I hope that you have already seen these in the logistic regression video. So I'm not going to explain this again. So let's go ahead and run this particular cell. Now we should evaluate the model okay. So, we shall call this confusion matrix function, we should have passed our original TEST TEST levels predicted class levels and the number of classes it was two three.

Okay, so let's go ahead and on this particular cell, and this is our confusion matrix. So now to calculate the accuracy it will be nothing but sum of all the diagonal elements of the confusion matrix divided by the total number of elements in the confusion matrix and total correctly identified is nothing but some of the dialogue metrics and there are total 30 data points as you can see, so let's go ahead and run this particular cell. So currently identified 29 and there are total 30 data points. So accuracy of Kanan classifier with K equals to three is 96.67%, which is quite good now Kalan classification using a scanner. So from SK learn dot neighbors will input key neighbors classifier. So this is a class so now we shall create an instance or object of this class and specify number of neighbors is equals to three, we can always play around with his number of neighbors and see how changing the number of neighbors will affect the performance of the classifier.

Okay, so let's go ahead and run this particular cell. Note that After defining this object, I have fitted these with the training data set. Okay, now as you can see, here, there are so many Any parameters mentioned, so I will explain two of them. So here matrix Minkowski and P equals to two minutes, we are actually using Euclidean distance to measure the distance between the neighbors. Now we should test the model. So the scaling object has a method called credit.

So I will pass x test as the argument of this predict function of the predict method and the output will be all the predicted values of y that is all the levels. Okay, let's go ahead and run this particular cell and the similar format. We'll be constructing a confusion matrix and we comment on the accuracy. Okay, so here, the correctly identified is 30 and total is 30. So the Kenyan classifier in this case has achieved 100% accuracy on the test dataset which is quite remarkable. So thank you for watching this video.

In the Next video, we should introduce another classifier called support vector machine classifier. So see you in the next lecture.