Hello everyone, welcome to the course of machine learning with Python. In this video, we shall see the implementation of gradient descent algorithm for linear regression in Python. So what is the problem statement when petrol is pumped into the tanks the hydrocarbons escape to evaluate the effectiveness of pollution controls experiments were performed the quantity of hydrocarbons escaping was measured as a function of the following variables the tank temperature the temperature of the petrol pumped in the initial pressure in the tank and the pressure of the petrol pumping. Now, the data set description there are 32 rows of the data that data includes six columns. The first column is the index the second column is the tank temperature in degrees Fahrenheit. The third column is the actual temperature in degree Fahrenheit.

The fourth column is the initial temperature in pound par in square the fifth column is the pickup pressure impound part in square and the last column is the amount of the escaping hydrocarbons in graphs. So first we shall import the necessary libraries, which is NumPy and pandas. Now we shall import the data set. So data equals to PD dot read underscore CSV. So these particular CSV file gas underscore tank underscore time underscore patient dot CSV is stored under the data folder. So we'll be calling this particular file and we load the data from this particular file.

And we know that the index column equals to zero because it is already indexed. So we'll be using that index for our data frame index. Okay, so let's go ahead and read the data. Now. If we print the head of the data that means first five rows, it will look like this okay. So this is the tank temperature in degree Fahrenheit, temperature, different height, tank pressure, petrol pressure and the hydrocarbon escaping in grant.

Okay, now we shall pre process the data set. So first, we'll take our feature And the target variables. So here the feature variables are tank temperature, cold temperature, temperature and the petrol pressure, and the target variable is the hydrocarbon escaping. So here x will be equals to data, which is nothing but from 0123 column. Okay, so first four columns, we'll be taking all the values of the first four columns and y will be the value of the last column fine, we should feel the shape of x and y and also print the x. So this is the shape of x that we had 32 rows and four columns and y has only 32 rows and this is nothing but x A now we'll do feature scaling.

So as I have said that feature scaling is an important step for gradient descent algorithm. So we'll be doing feature scaling first. So from SK learn dot pre processing, we are importing a function called standard scalar. Okay, so standard scalar is very Basically module or it is a class. So, if c is equals to tender scaler So, it sees the object to that particular class standard scalar and I am GUI is C dot fit underscore transform x. So, the x now will be scaled by standardization method, okay.

So if I go ahead and print the value of x now, so it will look like this Okay, so what does it really do if we print the mean value of the x along all the columns we can see that the mean value is close to zero it is zero then the tricky part minus 17 which is close to zero is also close to zero there is also close to two So, the mean value of all the data columns are basically zero and what about the standard deviation So, the standard deviation we can see that the standard deviation of all the data columns is equals to one so, we have converted the data into zero mean new standard deviation. So, this is what is called the standardization process. Now we'll do a train test split of the data set. So for that we are importing a function called Korean test speed from SK learn dot model selection library.

Okay, and we'll be passing the original data set x and y within the strengths test speed, and we'll specify the test size to be point to five at least 25% of the data will be kept aside as a test data and the remaining 75% data we'll be using for training we are using shuffle equals to true and when the data sheet will be shuffled, and the random state equals 240. So this is just for generating these. So whenever we are actually randomly shuffling it, we have to make sure that each time we shuffle we did the same kind of shuffling Okay, so for that we have specified random state equals 240. And it will return code values x train, x test y train and whitest so we started coding that values in the variables as described over here will paint the shape of x train y train x test and y test.

So let's go ahead and run these two cells. So, we can see that 24 data has been selected as training data. So, extreme dot shape has the dimension 24 comma four why train has the dimension only 24 similarly, SDS has the dimension eight comma four and whiteness has the dimension eight now we should implement the cost function. So here note that we have mean squared error cost function, it depends on the actual value and the predicted value. So here the cost function takes only two arguments are NumPy one dimensional array of y true for the two values of the target variable, and another NumPy array of y predict for the predicted values of the target variable, it returns a scalar which is mean squared error cost function we are defining a function mec underscore cost which will take only two arguments. So these y two and y predict are nothing but the NumPy arrays.

So first we'll assert the shape of both true and the predicted value of the target variable must be same, otherwise it will raise a dimensional mismatch error. Okay, now, how many data points are there that is stored inside aim and D will be equal to NP dot Sam within NP square y two minus y credit so, we are actually computing the mean squared error cost function here, okay and as you can see this highlighted part is nothing but the residual sum of squares as we divided it by twice aim, we'll be getting the cost function Okay, and we'll return that cost function. Now we'll be going to implement the gradient descent algorithm But before that, we have to implement the function to calculate the predicted output. So the output of the linear regression is nothing but the ad of the feature variable x and the model parameters theta and the output will be a multipad with Cheetah okay.

So, it is nothing but matrix multiplication and it will return the value eight multiplied with 3d it is matrix multiplication not now equal to the width in this upper two I we have already seen in the last video I have repeated that just to make sure that we when we implement, we can always cross check and see that whether we have implemented it correctly or not note that this particular thing theta zero underscore t plus sum over j equals to one to k theta j Xj is nothing but our predicted value of y. So, we can simply file these and write it in the form of this equation as written over here okay. Now, note that the gradient descent will take the arguments of the entire data set x the target of the predicted target variable y and alpha value number of iterations and the tolerance okay.

So, what is the tolerance is for Tolerance is to take the difference between the value of the model parameters into successive iteration if it falls below the tolerance will stop the algorithm. So, this is for the stopping criteria. So, again we are asserting the shape of X and Y if these are not innocent and will, again through a dimensional mismatch error. So, in is the total number of training samples, we are actually combining column of all ones, which is a dummy column to the x variable, or this is basically the array of the predictor variables x we are inserting a column of all ones at the left of it and key is nothing but the number of predictors including So, it is basically the origin and it was the number of predictor is equals to four but now it will be five because we have already included a dummy column of all ones Fine, then we'll be initializing the value of the theta with the random number Okay, we shall initialize cost value and the count value equals to zero then for each iteration will be predicting the value of y as the function of x comma theta.

So, this function I have already implemented output of the linear regression. So it will be x comma theta of the output linear regression, so, it will give me the predicted value of the variable y and the cost function will be nothing but ABC underscore cost y come away credit so, we'll append the cost function and we'll compute the change in the value of theta Okay, and we'll update the value of theta. Now, we shall see if the dimension or not that means the magnitude of the change in the value of theta is less than equals to tolerance will break is we keep on increasing the count value and after it has completed the iteration or it passes through this particular practice. Via finally we'll return the value of theta. That means all the model parameters the cost least and the count. Okay, so let's go ahead and run this particular session.

So now run the gradient descent algorithm on our training data set, okay sorry. So we are getting an error. So MSE cost is not defined. So we have forgot to run this particular cell. So we have to run all the cells in order to get the output of now fine, we will run this particular cell and see though, so the model parameters are these so this is theta zero, theta one, theta two, theta three and three to four. As we have total for number of predictor variables, we'll be having five number of model parameters and the number of iteration is to 6064 in order to complete these iterations, okay, now we'll be plotting cost versus iterations.

So okay for that as the past Okay, so this is the cosmos tradition function and as we can see that the cost is reducing Okay, I will predict the model output on the test data set. So as you can see this is the actual value and this value on the test data set and the MSE cost or the mean squared error caused on the test dataset is 7.468. So, we have seen how to implement gradient descent algorithm in Python from scratch. In the next video, we shall delve deeper into the regression analysis we should learn some advanced and efficient techniques. See you in the next lecture. Thank you.