Hello everyone, welcome to the course of machine learning with Python. In this video, we shall discuss about multi layer perceptron or MLP. So, these are the connections of biological neuron, this is having a very complex connection packets, now in the similar fashion neurons in a neural network organized into regular layers for computational efficiency. So, this is what he called input layer and this is what he called output layer. And in between there are hidden layers. These types of architectures are called multi layer perceptron.

So, as you can see this is what he call it two layer network okay or single hidden layer neural network. However, this network we have two hidden layers So, any layer other than input layer and output layer are called hidden layer. So, in this particular neural network, there are three layers that is hidden layer one hidden layer two and output layer. We usually do not mention input layer as the layer While defining the multi layer perceptron This is also defined as two hidden layer neural network. So, input layer and output layer will anyways be there the number of hidden layers we have to specify. So, the neural network actually specified in terms of number of hidden layers or number of hidden layers and output layer.

So, this hidden layer as you can see, accepts connections from the previous hidden layer or the input layer. And as you can see a particular node in a particular layer is connected with every other node in the previous layer, but not connected in between. So, the node in one layer are not interconnected, but those are connected with the previous most of the previous layer and the nodes of the next layer. So, these type of connections are known as fully connected layers or dense layers, different kinds of activation functions found in the neural network, the first one being sigmoid function or a sigmoid activation function, these we have seen in the context of logistic regression that comes tannish activation function. So, Xs is nothing but e to the power x minus e to the power minus x divided by e to the power x plus e to the power minus x.

So, 10 is basically ranges from minus one to plus one y sigma ranges from zero to one then comes radio radio is nothing but maximum between zero or x. So, it is nothing but x y equals two x when x is positive and it is zero when x is negative, then comes liquid au which is nothing but max of point one x comma x so, in the negative direction of x or in the third quadrant, that is a straight line of small slope as you can see that comes max out and exponential linear unit or EU. So, Ian is x when x is greater than equals to zero and it is some alpha times e to the power x minus one, when x is less than zero riilu is a good practical choice for most of the problems relu stands for rectified linear unit. However, in the output layer, we do not specify the new we specify their softmax activation function, which we shall discuss later softmax regression or softmax activation function.

So, softmax regression is a generalization of logistic regression for multi class classification problems. So, let's say we have a training set x one y one x two y two up to x m y m. Note that x is boldface that means x is nothing but a vector. So, x is called the feature vector y is called the corresponding level. So, n is the number of creating data and the feature vector x belongs to yn plus one dimension note that we have added and extra input features Which is zero, which is always one for all i and the labels y can be 123 up to k. So, as this is a multi class classification problem, hence k is greater than equals to two. So, if k equals to two then it is called the binary classification problem, if key is more than two, then it is multi class classification problem the now for softmax efficient the hypothesis function looks something like this.

So, this is nothing but a vector. So, as you can see this is e to the power three to one transpose x i, then the next time is e to the power theta transpose excite up to e to the power theta k transpose excite and in the denominator we have sum of all these values. Okay. So, this theta one theta two theta k is nothing but the victor of dimension in plus one of the parameters of our model and the denominator is actually to normalize the distribution, so, what we shall get over here we shall get up probability distribution and to normalize it, we are dividing the entire vector by the sum of all the values of the victim okay. So, in turn the sum is equal to one, so, that is a legitimate probability distribution. So, usually softmax addition is used at the output of the classification layer of the artificial neural network, where it is also referred to as softmax activation function.

Now, multi layer perceptron architecture the number of nodes in the input layer should be close to the number of features or attributes of the patterns or the data points. The number of nodes in the output layer should be equals to the number of classes of the data set. Number of hidden layers and number of nodes in each hidden layer is user choice and there is no general guideline for the choice of number of hidden layers and numbers. notes in each hidden layer there should be at least one hidden here. Usually we use riilu that is rectified linear unit activations for all nodes in the hidden layer and softmax for output layer. However, there are exceptions where we use different activation functions.

Now, for example, to classify the simple ID data set using artificial neural network, the number of input nodes is equals to four as the data set has four features or attributes, number of output nodes is equals to three as there are three classes, we can build an artificial neural network with one hidden layer containing four nodes set so, we have a two layer neural network or one hidden layer neural network, but we have input layer output layer and one hidden layer containing four or five nodes Okay. Now, advantages of multi layer perceptron non linearity so multi layer perceptron is very suitable to model a physical phenomenon, which is in general nonlinear activity, it can cope with the change in data set distribution parallelism, it can be implemented in multi core parallel architecture for faster computation. robustness, it has inherent ability to handle confusing, noisy or missing data points fault tolerance, it has the ability to work to some extent, even the case of component or neuron failure complex decision boundary it has the ability to have a complex decision boundary for classification VLANs are implemented at it can be implemented quite easily in VSI chips.

We should use multi layer perceptron with supervised learning for complex classification tasks, know how it will be solved exact problem. So, this is our well known exert problem. Now, this is our monitoring perceptron architecture. So, we have only two input nodes as there are on two inputs available x one and x two then we have a hidden layer which contains only two nodes h1 and h2 and then we have an output layer while Okay. Now linear classifier like perceptron cannot solve these we have already discussed these in the last video. Now, let us consider that there are biases that V equals to minus 10 v goes to 30 v goes to minus 30.

So, in the hidden layer each one the input will be receiving is 20 times x one plus 20 times x two minus 10 So, minus 10 is for the bias. Similarly in the each hidden unit will be receiving input minus 20 x one minus 20 x two Plus turkey Okay, and we have applied sigmoid activation function over here. So let's see what happens. So when both the input x one and x two are zero, then the output of the hidden unit is also zero, that is output of H one is also zero. However, output of h two will be much one, then if both the input is equals to one, then the output of the hidden unit one is one and output of the hidden unit two is zero. Similarly, when the input is 01, then the output of the hidden unit one is one, the output of the hidden unit two is one and when that input is one zero, the output of the hidden unit one is one and output of the hidden image two is also one.

Okay now, we have combined these two the layer in the output the input received to the output unit is 20 h1 plus 20 h2 minus 30 Okay. Now if we just put the values we have obtained in the hidden units, so, when both This is zero and one then the output is zero when both of them is one and one now police one okay. So, if I go up there is zero or one now two will be zero if either of them is one now this one so that is how the multi layer perceptron has solved the exact problem. Now, the question is how do we know what are the weights of these connections are Oh what are the values of these biases. So, when we present the input data to the network, the network will learn these parameters by itself Okay, so this is what he calls the learning of audible.

So in the next video, we shall discuss about the backpropagation learning algorithm of multi layer perceptron. So see you in the next lecture. Thank you