Hello everyone, welcome to the course of machine learning with Python. We will be starting a new module called regression analysis, and in today's video, we will learn about something called linear regression. So what is linear regression? So first we'll begin our study of simple linear regression consider the scatterplot of weight versus height of adults as shown in this figure, the creator or the form of the relationship is strongly positive, as you can see from the scatterplot. Now, suppose we wish to estimate the weight of a person just by knowing his or her height. In order to do so, we first feed a straight line through our data points like this, then the form the graph knowing the height we can find the weight of the corresponding person Hence, we are intending to find out the equation of the straight line that best describes the relationship between weight and height.

Now there is only one predictor or the input variable that is the height and one target variable that is the weight and we are in To find out a relationship of the form y equals to theta naught plus beta one x here y is the target variable and x is the predictor variable here, y will be nothing but the weight and x will be nothing but the height, we have to find out who doesn't want to one such that the straight line y equals to theta zero plus theta one x fits into our data set best This is called simple linear regression, because it has only one predictor that is x and the relationship among the target variable and the predictor variable is linear. It is nothing but a straight line right now, simple linear regression model with a single predictor will describe our model. So y equals to theta zero plus theta one x plus epsilon here, theta zero and theta one are called the model parameters, y is called the target variable.

Theta zero is called the intercept, theta one is called the slope. X is called the predictor variable and epsilon is called the random residuals. Later, we use our sample data to find estimates for the coefficients of the model parameter theta zero and theta one. We call the estimates theta zero as theta zero hat and the estimate of theta one as theta one hat, we can then predict what the value of y should be corresponding to a particular value of x by using the least squares prediction equation, also known as our hypothesis function, which is denoted by Y hat equals to theta zero hat first theta one hat X. Here, y hat is our predicted variable. So y is basically the true value and Y hat is our prediction, no residuals and residual sum of squares for the sample x comma y.

The predicted value of y is y hat, which we obtained from the equation y hat equals to three times zero hat plus two to one had expired. Now, there is some error associated with this estimate. The error for this is of sufficient will be y minus y hat that is actual minus predicted this represents the year residual, we define this it was sum of squares as addresses or the residual sum of squares is equal to sum over all Ei square items from one to m. So, we can expand the value of Ei as shown over here as Ei is nothing but y minus y i hat and replacing the value of y i had we get the residual sum of squares in this format note that there are total in number of training samples no mean square error cost function we can define the cost function as g of so g is the cost function, which is a function of the model parameter theta zero hat and theta one hat will be equal to half of the residual sum of squares divided by the number of printing samples is one by twice m multiplied with the number of cases In some sort of multiplied with the residual sum of squares is the total number of training samples here a factor half is multiplied just for computational simplicity, otherwise, the cost function g is nothing but mean or average of 30.

So it was off squares, which is also known as mean squared error or m s E. Now, our objective is to find suitable values of theta zero hat and theta one hat such that the cost function g is minimized. In other words, the residual sum of squares is minimized, then the straight line will fit our data set best This is called the least square feet. Now, we intuitively understand what the cost function mean. Consider the example of a single predictor variable where the hypothesis function is of the form y hat is equal to beta zero hat plus beta one hat into x and the cost function g is having this form. Now, we keep one parameter fixed so, there are two per model parameters theta It'll hurt and fit a one hat we will keep one parameter fixed and very another and let's see how the cost function J varies.

So let's say this is our data set. So first we fixed t does you know hack and we've had E to one hat t does you know hat is nothing but the intercept part and Tito one hat is nothing but the slow part of the straight line. So, this is one straight line that we can fit and corresponding cost function would be this one, this is another straight line and note that as the straight line waiter feeds than the previous state land the cost will be less similarly this is another straight line, this is another straight line, okay. So as we increase the value of theta one hat, the cost function g will first decrease and then increase again. So there is one sweet spot of theta one hat where the cost function is minimized. Okay, so this is the particular theta one hat now we'll see See 321 had is fixed but we've had in the beta zero hat.

Okay, so we'll fix the slope but will vary the interesting let's see this is for zero intercept we'll get some cost value for another intercept we'll get some other cost value, if we keep on changing the theta zero hat, we also see that there is a curve like that of cost function passes to zero what it means here also there is a sweet spot or there is a minima where the cost function is minimized for a certain value of theta zero hack okay. So our objective is to find out these particular frita one hat and theta zero hat for which this cost function is minimized. No solving for this feat, in order to minimize the residual sum of square or the cost function with respect to theta zero hat on T dot one hat, what we shall do we will take the partial derivative of the ratio square both respect to theta zero hat and theta one hat and set them individually to equal to zero.

Okay, so we'll take the partial derivative of this it was off square with respect to theta zero hat set it to close to zero and again we'll take partial derivative of this it was sum of square with respect to theta one hat and set it equal to zero by solving the M of two equation we get the following value of theta one hat and beta zero hat for simplicity, I am not showing the entire derivation, but I will recommend students to do it by their own okay. So theta one hat is nothing but sum over x i minus x bar multiplied with y minus y bar where it goes from one to M divided by sum over excitement as expert whole square items from one to n. Okay, if we expand the value and we rearrange, we can see that it is nothing but correlation coefficient between x and y multiplied We standard deviation of y divided by standard deviation of x and theta zero hat will be nothing but a mean of Y minus theta one hat that we have already obtained in this equation multiplied with x mean okay or x bar.

So x bar and y bar are nothing but the mean values of x and y respectively. Okay, so first we opt in to tawan hat and putting the value of theta one hat over here, we'll get the estimated value of theta zero hat oh where expert is the mean of the filter variable as already stated and why what is the mean of the target failure? Okay, sigma x is the standard deviation of x and sigma y is the standard deviation of y and our x y as I have already stated is the correlation coefficient between x and y. In the next video, we shall see how to implement the Y variate linear regression in Python. See you in the next lecture. Thank you.