Now we have trained the model. Let's see, let's evaluate the model by running them on the test data set right. So test data set what we have done is we are giving the image So, once we train the model, the model has a predict method to which we just to feed Third, the test input dataset. So, here we just feed the test data set without any labels, and the model will predict certain data, right. So this is the actual output the model will predict. And we will compare with the test accuracy expected output.

And then we can plot the confusion matrix and the model can tell you what is the procedure and recall the f1 score for the test. So let's do that here. Let me run this again. So it's predicting now it will take few moments. Okay. So the model has been prediction has been done.

And the model is printed the evaluation report where it has outputted procedure recall f1 score and other metric right. The most important metric is precision recall the f1 score that we want to understand here. Now understand it came can And one important thing to note is, unlike a traditional ml model, it's not learning anything during training time. It actually does the calculation during the prediction time because as we saw in our model right algorithm here What it does is given a test it as it as a code, it computes the distance between each of the available training data set. And then it will, and then it sorts, the nearest neighbor and then based on the majority voting, it will compute the output, right? That's all it's doing.

So for each label, like zero is dog, one is cat, and two is panda. It outputs the matrix right? So as you can see, for Panda, the precision is really high. I think that's because if you think about it, pandas are really easy to identify because they just can be of two color red, black and white. But if you take example of a dog or an image, the precision would be a little bit lower because because you know there are variations rights. Cats and Dogs are different colors.

They have a different type of four and they will be different positions, different background. So in that case, the procedure is little bit lower for dogs and four cats, right? And the recall is high for cats, but it's lawyers for better, right. Now, as you know, in the evaluation matrix study, we know that our goal is to maximize the f1 score. And here we observe that the f1 score is good for cats, and it's lower for dogs and then is lower for panda. So this is the ad if you just average our accuracy, this is also the accuracy about 55%.

Right? So So our procedures overall our f1 score, if it were random guess right our parameter prediction mean you can get a 33% probability, right? So as long as our model predicts, with There are three different outputs and all of them are of equal value. So random prediction will give you one third which is point three 3% 10% accuracy. So our definitely model is doing definitely better, right? As compared to the random prediction.

Now, how you can hyper tune the parameters of this model and see how the model metrics would vary, that will definitely vary because, as you can see here, number of ports we considered as to in this particular case, right. So, at least the two levers have to vote consistently to say that it's a dog. What would happen if we lower the number of neighbors instead of two? Let's say if we do one, what would happen do you think? Can you guess? Let's try it out.

So I'm going to change the number of neighbors to one and see Output change, as you can see, this is remember with a number output score with the number of neighbors to Let's run, I'm going to run the whole program again. So you can see the kernel and restart and run all. Let's see what happens in this particular case. This reading the entire code again now instead of doing cell by cell running, and let's see how it changes. So they attend the training and now it's doing the prediction Alright, so here the maximum precision that you can get is 70%. Right and the maximum score we can get is point four, five.

So as you can see the full score is lowered as you change the value of neighbors from two to one, right? So I think so. So that's how you play with the score edit, because our goal is to maximize f1 score. So number of neighbors two was definitely a better choice, right. So as you can see, then recall has increase in this particular case, because as you reduce the number of neighbors, you will get more and more true positive at the same time, you will get more and more false positive as well. So this is definitely not a good choice number of neighbors one, let's see what happens you should change to three, right?

And then run the whole program again. So this is how you should do this kind of hyper parameter of you. model, this is how you should keep playing with the different hyper parameters and choose the value that gives you maximum f1 score. So the training is done in leads to the prediction. So as you can see prediction. So the position is very good for Panda, but the f1 score isn't changed significantly.

So what we observed is so when the number of papers were two, we got a maximum of f1 scores. So that should be the choice for running your code. So you saw the typical process that you go through when you're training and after doing a prediction. You First pre process your data set, make sure you flatten your data set, then you split into train and test dataset, then you use to train the model and then you use to do the actual prediction and then you evaluate your score. And then choose the model with the x maximum f1 score. Now in practice, this is good lesson, you know, just for beginners.

Now in practice, you know, this much of FIFO score is not good enough. So we need to use more sophisticated algorithms like CNN, or LST M's. But if you start doing that, right from the beginning, you will get turned off because of the complexities into the algorithms. So by learning about kn and most of the basic stuff, you can get out of the way, and all you can actually focus on just selecting an actual algorithm. Now you've actually understood the skeletal structure of choosing any algorithm in your code, and you can easily swap in K nearest neighbor with some other algorithm Engines run the entire code in a similar fashion right? So that basically is out of the way and you could keep playing with new algorithms and see what best f1 score you get and then you can arrive at that algorithm.

So this is how you evaluate a model.