Hello everyone, welcome to the course of machine learning with Python. In this video, we shall start a new module called artificial neural network. And in this particular video, we shall discuss about perceptron model. So this is the brief history of neural learning system. In 1940s McCulloch and Pitt's come up with the concept of electronic brain in 1950 Rosenblatt came up with the concept of perceptron. In 1960, we have come up with the concept of atoms.

However, in 1970, Minsky and pat pepper showed that perceptron model is unable to solve certain classes of problem known as XOR problem or nonlinear the separable data set problem. So, this creates a huge setback in the AI community. And this is what he calls the dark age or the AI winter from 1970 T two t 1986. In the 1986 rule heart, intern and Williams come up with the concept of backpropagation learning algorithm in multi layer perceptron, which will eventually solve this XOR problem and give these field a boost. However, because of the lack of computational instruments at that point of time, this was not a huge success in 1995, they actually come up with the concept of support vector machine. And in 2006 onwards, deep neural network was a trend because now people have that much of computational power to train deep neural network and people started getting amazing results.

Today, every research in the field of artificial intelligence is extremely concerned with deep neural network. So let's see from where we have got the inspiration to build an app. neural network. The biological neuron is basically the building block of our nervous system. This is called the cell body of the biological neuron and from the cell body there are so many extensions. So these small extensions are known as dendrites, and there isn't long extension that is known as axon.

So, what does this did right or dangerous do it the impulses from the other cell bodies of the other neurons is actually transferred to another neuron by the means of these dendrites, okay. And the dendrites of other neurons or the axon of other noodles is connected with the dendrite of this neuron and the connection is known as the signups. So through axon the impulse is basically carried away from the cell body, okay, this is the basic concept of violence you can move on. So, what we have understood so there is As his body, the cell body basically get impulses from the other neurons by means of dendrites, and it passes through the acceleration or the impulse through axon to some other neuron. Okay? Now, artificial neuron.

So this is what he called the cell body, and these are basically the dendrites. So, through dendrites, we're actually taking the sensation from the outside world or the actual from another neuron. So these are nothing but numbers or the values of the features. Okay, and they are synetic weights, denoted by W zero, W one or w two. In cell body, we compute the weighted summation of all the inputs from the other neurons. This weighted solution then passed through a function called the activation function.

And finally, we get the output So, this is how we have modeled our original biological neuron to some mathematical model known as artificial neuron. So, B here is known as the bias, why is that called a signup tick weights are simply weights and this is a very crude approximation of original biological neuron. Actual biological neurons are even more complex, this is known as the activation function. For example, the activation function could be a sigmoid activation function or a logistic activation function. So, perceptron So, this model of neuron is also called macula fits model. However, Rosenblatt extended this idea and give a learning rule to update the weight parameters from a given data set.

This is called perceptron. The activation function if in this case is a hard limiter or signum function, let V is nothing but a weighted summation of all the inputs Then output Y hat is calculated as it is plus one, if v is greater than equals to zero, it is minus one if E is less than zero, okay, so that is how the activation function if map's v to Y hat, so if he's value is non negative, then y hat or the output from the cell body, we will be one or plus one. If V is negative, then the output from the cell body will be minus one. So this is the scale of the activation function as we can see V is greater than zero, then the output is plus one. If he's less than zero, the output is minus one. Now comes perceptron learning rule.

So there are a few assumptions. First, we considered these a binary classification problem. The data set is x, and there are two classes denoted by C one and C two respectively. class level of a data point x vector is basically denoted by y. So, x vector is a data point inside the data set capital X, it is denoted as following. Now, y is equals to plus one if these expected belongs to class one and y is equals to minus one, if expected, we have two plus two.

So, that is how we have label the classes the waves are denoted by the vector w, the bias is denoted by B. Now, this weight and bias could be the lead can be considered as a weight vector w, if we consider another input x zero of Hello always plus one, then this whole wi x i plus b can be written as the dot product of this w vector and this is nothing one W transpose X no algorithm. So first, we should initialize the word vector with some small random number and the iteration count. P is equals to one. Now, we shall repeat for x vector in the data set capital X we shall compute y hat which is nothing but signal function of W transpose X or the dot product between w vector and x. This white hat is also called the computed output or the label.

Now, in t plus one is iteration the weight vector is nothing but the winter active iteration plus eater times y minus y hat or multiplied with the X Factor. What is why why is called the actual output or the level the actual output is obtained from the label training data. And it is platform if expected belongs to plus one and it is minus one if x vector belongs to plus two. So we deplete these tapes until, in two successive iterations, the award victory is not changed. So, how do we measure that a weight vector has not changed. So, let's say we compute the norm of the difference between the weight vector at t plus one iteration and with vector A t iteration, and if this norm is less than some predefined tolerance or very small number epsilon, then we shall see that our algorithm has fun first.

Now, we shall discuss about this update rule a little bit if the actual output that is y is same as the predicted output Y hat, then there is no way to update However, if the actual output and the predicted output differ from each other, then there is a word update, okay. However, if the algorithm is not converged, then we increase the iteration count by one and repeat the whole process. Okay, here he typed in the wind upgrade rule is called a learning rate and it is usually used defined and it is easily a small positive number perceptron decision boundary is a hyperplane defined by W transpose X equals to zero consider binary classification problem data set is shown in the figure, no random initialization of weight vector at the starting of the learning algorithm randomly feeds a hyperplane as shown beside as we keep updating the weights using the learning tune, the decision boundary keeps on changing till all the training data points fall correctly in the either side of the decision boundary Okay, does perceptron acts as a linear classifier and can classify linearly separable data set with high accuracy the perceptron with sigmoid activation function, not we have used hard limiter or the signal function as activation function.

However, we can choose sigmoid function as our activation function as well. And in this case, the single noodle call response exactly to the input output mapping defined by the logistic regression okay. So, this is how the sigmoid function looks like and with the sigmoid activation function perceptron is nothing but a logistic regression. So, limitations of perceptron the limitation of perceptron is not able to classify the nonlinear data set. So, here we shall demonstrate the problem using something called the XOR problem. So, this is XOR data set Okay, the input one and input two that means this x and y if both are zero then the output is zero.

If the input is zero and input two is one, then the output is one if input one is one and input two is one, then the output is zero. And if input one is one and input two is zero, then output is again one. Okay, so that is how they saw data set looks like. And this is our perceptron. Note that the perceptron works fairly well for linearly separable data. said it failed to solve the exact problem and other nonlinear data sets because there is no straight line that can feed the data points or that can form a decision boundary between the two classes.

So, let's say this is a possible decision boundary, but however, we can see that there is one data point of Class Zero in this side of the decision boundary and another data point of Class Zero in this side of the decision boundary, okay. So, this single region boundary is not able to classify this data set properly. Similarly, if we place the decision boundary anywhere in this plane will be unable to solve this problem, okay. So, this is known as exhort problem or non linearly separable data set problem, and this has traveled the AI community for many years until in 1986, Geoffrey Hinton, and his to come up with the concept of backpropagation learning algorithm, which eventually solve the problem using multi layer perceptron. In the next video we shall discuss about multi layer perceptron. So see you in the next lecture.

Thank you