Hello everyone, welcome to the course on machine learning with Python. In this video, we shall see how to implement the gradient descent for logistic regression in Python. So first we shall import the necessary libraries. So here the libraries we have used is NumPy library, as well as for loading the data set will be using a function called load underscore IDs, which is defined under a scalar data sets package. So let's go ahead and run this particular cell. Now, we should load the data set.

So to load the data set, we have used the following command. So x comma y is equals to load underscore IDs return x, y is equals to true that should be passed as an argument to the function node IDs. So if we go ahead and run this particular cell, then all the features value will be stored into eggs and all the target value or the class labels will be stored into one note that we are using it is data set over here. So what is x dot shape? So the shape of the x matrix is 150. Come up for so that means it has 150 rows and four columns what is wild ocean so it is only 150 so it is a flat array of 150 elements okay.

So you can see why is basically coded in 01 or two. So zero means setosa one means versicolor and two means virginica, so first we will binarize the data that will say Tosa is equals to zero and non setosa is equals to one. So non setosa means both are similar and virginica. So, this is what binary classification. So, that is why we are classifying the data set into either seducer or non setosa. So, all the synthesia will be labeled as zero and non setosa will be labeled as one.

So, we are creating another error why underscore mod or y modifier is nothing but the copy of the original y and then we are changing y mod that means all the two values, we are converting it to one and if we done this particular set, we can see that now All the virginica and the Mertesacker has been converted to one but the seducer are all seniors Okay, fine. Now, let's define the function to compute the cost. So, for the simplicity, I have also written the cost function over here. So, first what we shall do, we have to identify how many arguments will be there, as we can see, there shall be two arguments, one is y underscore credit that means, a vector of predicted outputs and y underscore actual which is a vector of cool levels. So, first we shall assert that if the shape of the actual and the predicted output or the variable is not equal, then we shall throw our dimension mismatch error that means the shape of the actual output and the two levels should be same.

Okay. Now, how many samples are there that is identified by y underscore pretty dot Ship within that it's zero. So that was go over him. So AMD was a number of training samples. So as you can see in the roster number of training service fine. So first let's do not cause too close to zero.

For i in range of him, we define the cost function as printed over here. Note that Sam equals two. So first we have defined cost equals to zero. So some equals two means cost equals two cost plus these intervals okay. So, as we can see we have made use of NumPy dot log function to predictor over them or to have the value of the logarithm. And in order to find the total cost we have so as you can see, this is basically one upon aim is there so we have to divide this by him.

Then we can return the final cost. So, that is how we have defined the cost function of logistic regression okay. So, we can also name it as binary cross entropy loss or log loss whatever now defining the function to compute the output of the hypothesis function. So, the output of the hypothesis function depends on the theta value and as well as the x so, that we compute as x at theta so, ad here implies matrix multiplication So, it is nothing but x inputs into these we have already seen why it is executed in the lecture of advanced regression the output will be close to one upon one plus exponential of minus it okay. So, this is nothing but the logistic output, then we shall return Okay, so, we define the function like this, this is nothing but the output of a legitimate patient function. Now, let's go ahead to the main part of our program, which is the gradient descent algorithm for logistic regression, okay.

So, I have repeated the equation of the logistic regression over here, you can see the same equation in the slide as well as in the lecture video. So, I have reproduced it here in order to have a quick glance okay. So, we have defined a gradient descent logistic regression, as you can see, we have to pass the training samples. So, both extend and whiten has been passed then the alpha value that is alarming rate 34 of the learning rate is 0.01. So, the number of iteration is also passed as you can see there are total thousand number of iteration as the default iteration value and the tolerance means it is basically to stop the written descent algorithm, what should be the required amount of tolerance and the default tolerance is 0.2 to 01. Okay, so again here if the dimensions are the number of training samples does not match with the number of samples in the white cream white print means the training label produced or the trading label given then we will again throw a dimension mismatch data that means the number of samples as present in the future my fix should be same as the number of samples as presented in the level output or the class levels.

Okay, so how many training samples are there? There are a million samples that we can opt in from x underscore train dot shape with a bracket zero, okay, fine. Now we shall club a column of all ones to the extreme so that we can do by column step five. So keys basically total number of features note that key here is nothing but the total number of features present in the data set plus one because we have already added one called external Column all right fine, then we shall define a random theta value. So, there will be total key or in this case k plus one number of model parameters. So, all the model parameters are initialized to the unknown values.

So, that what we have done in this step okay. And cost is basically initially empty list and count is equals to one find why you predict So, far each iteration what we shall do we shall predict the value of y or from the output of the rasterization function theta comma extreme we shall pass then we'll compute the cost okay. And in this empty list we keep on appending the cost value. So, in the cost least we are keep on appending the computed cost for each iteration and then we compute the change in the amount of the model parameters using this equation. So, this equation as you can see, it is nothing but this equation okay and then we change Hello Tita now we check for the tolerance if the change in density is not significant, that means it is less than equal to the tolerance, then we break the loop.

Okay? And then finally we return the model parameters to the cost list and the total number of iterations. Okay, nice. Now we define the function for predicting the class. Okay, so the output that the last integration will produce is a probability it is a continuous valued output between zero to one. Now if the output is more than point five, we'll give the output as one and it is less than perfect and it is zero just to identify this is two separate classes, okay?

Or in other words, if as improbability or the output of the luxury tuition function is more than point five, then we can predict this as non setosa. And if the output is less than point five, can we predict these as sacrosanct? Okay, so let's go ahead and on this particular board, Of these cells, let's check how the gradient descent works. So, we are we have already defined X and Y mod and we run this okay. So, we can see how the cost is varying for each iteration okay. So, for that we have used the matplotlib function okay.

So, for this we have to use matplotlib in okay. So, as you can see the cost is gradually decreasing So, that means the gradient descent is working fine now created speed of the data set. So, as we can see from this column dot model selection, we have imported this fleet and we are splitting the data as you will that this size is the 20% of the data is 0.2. Now, the extreme in contents 120 samples and extend exist contents 30 samples that means the training data set contains what will disappear Out of those 150 samples and the test dataset contents random practice samples out of those 150 samples Okay. Now we can estimate the model parameters using the training data set fine and then we can predict the output of the model on the test data set. Now, we can define a function to generate a confusion matrix.

So, if class is correctly predicted as So, all the values of the confusion matrix is first initialized to zero. Now, if the value or if the observation that belongs to classify is predicted correctly as class file in the ICA is empty of the confusion matrix will be increased by one else if it is predicted as j then the i comma j in three will be increased by one. So, in that logic we have created the confusion matrix Okay. Now, what is the confusion matrix of in this case we have predicted and the actual test value or the ground truth then So, this is our confusion matrix okay and what is the accuracy as you can see there are no false positive or false negative. So, we can have accuracy of hundred percent which is quite excellent given that the model is very simple okay. This is how we can actually implement the gradient descent for the logistic regression by 10.

Hope you have enjoyed this lecture. In this in the next lecture we shall discuss how to implement the logistic regression in pythons in built function using the escalon library. Okay, so, see you in the next lecture. Thank you.