Hello everyone, welcome to the course of machine learning with Python. In this video, we should learn about Bayes classifier. So, we should now revisit the Bayes rule. Bayes theorem is simply a consequence of conditional probabilities of two events A and B. So, this is a sample space, this is even a this is even we will note that A intersection B is not now, the probability of A given B is equals to probability of B given A multiplied with probability of a hole divided by probability of V here, probability of A is called the prior probability of event a. probability of b given a is the likelihood of event v given event a probability of V which is in the denominator is called the evidence of event B and probability of A given B is equals to the posterior probability of event a given event B. So, P A is called the prior probability and P a given these called the posterior probability, Bayesian classification, the classification problem is posed in probabilistic terms, we create models for the distribution of objects of different classes.

Probabilistic framework is used to make classification decisions. So, we consider our two dimensional data set each object is usually associated with multiple features or predictors here, it is associated with only two features. However, we will look at the case of just one feature for now. So if we map all these training samples, who only to the feature one, then we shall get a distribution like this, we are now going to define two key concepts. So the first concept is class conditional programming. Distribution patterns for each class is drawn from class conditional probability distribution of ccpd.

So, let's say this is our distribution along the feature x. So, clearly there are two classes, these numbers new class is class one and this brown class is class two. So, this distribution actually gives probability of an object or principle x given class one and this distribution actually gives probability of the object or the training sample, given class two. Okay. So our first goal will be to model these distribution. So we model prior probabilities to quantify the expected a priori chance of seeing a class lead there are total in many creating samples and out of each each one number of signs.

Belongs to class one, and a minus m one which is called into number of samples belongs to class two, then prior probabilities of class one and class two are calculated as two higher probability of class one is equals to m one by M. And prior probability of class two is equals to implement. Now, we have tires defining a priori probability of classes. That means p, or probability of class one a probability of class two. And we also have models for the probability pattern given each class. That means probability of X given class one and probability of X given class two, usually, these probability of X given class one and probability of X given class to our model using some standard probability distribution function such as Gaussian distribution, we want the probability of the class given a pattern. So, if a pattern comes We want to identify in which class it belongs.

That means we want to classify the pattern. And we want to estimate the probability of the class given a pattern x that means probability of class one given pattern x or probability of class two given pattern x. So we have probability of X given class one and probability of X given class two. We also have probability of class one and probability of class two, then how do we get probability of class given x knowing probability of X given class and probability of class. So this is a classic Bayes rule application. So we will apply Bayes rule to obtain probability of class given x how so this is the formula.

So probability of class given x is equal to probability of X given class multipack the probability of class hold divided by four with p of x. So here, probability of classes called the prior or belief before evidence, probability of X is called the evidence. probability of X given class is called the likelihood of the evidence and probability of class given eggs is called the posterior or belief after evidence. Now comes Bayes decision rule. If we observe an object x, how do we decide if the object is from class one or class two. So Bayes decision rule is simply choose class one if probability of X given class one multiplied the probability of class one whole divided by probability of x is greater than probability of X given class two multiply the probability of class two whole divided by two d of x, okay.

So we are actually calculating posterior probability of class one given x and posterior probability of class two given XE the posterior probability of plasma given x is more than posterior probability of class two given x, then we can say that the object or the pattern x belongs to class one. This is called maximum a posteriori rule or MEP rule, so we can cancel p of x from the denominator, and we can simply write this form, then, just by dividing this right hand side term, we can say that we will have a given class form multiplied with probability of class one full divided by probability of X given class to multiply the probability of class two is greater than one. Okay? We can take log on either side, log of one is equal to zero. And we can reload this entire log expression as gx. Okay.

So we can say that if g x is greater than zero, we classify as class one. Otherwise we classify as class two. This is called maximum a posteriori, or MVP rule. Now these decision boundary, so Bayesian boundaries obtained when g of x is equal to zero, that is probability of X given plus one multiply the probability of plasma is equal to probability of X given class two multiplied with probability of plus two. Now, if we assume that the prior probability of the classes are identical, that means a balance distribution, then this rule simplifies to probability of X given plus one is equal to probability of X given plus two. Okay, so this is our decision boundary.

So this is our Bayes decision boundary where probability of X given class one is equal to x cubed plus two and this is the DGN where nice specification occurs. So, this is what is called the Miss classification region, this shaded region. Now, instead of Bayes decision boundary if we choose some other decision boundary like this, then the Miss classification region is more widened, or in other words, Miss classification error we'll be more similarly, in the other side of the Bayesian decision boundary, we will also see an increase in Miss classification error. So, changing the decision boundary other than the Bayesian decision boundary increases the Miss classification error. So, Bayesian decision boundary gives minimum misclassification error. Now, for multiple features, we have a feature vector comprises of in many features.

So, the feature vector is actually denoted as x vector which comprises of in many features x one x club two exam. Now, this classification is probability of Class C given X Factor is proportional to probability of expected given class c multiplied with the prior probability of class C. Now, probability of expected given c is equals to probability of x one x two up to XM given c, this is called a joint probability distribution or joint conditional probability distribution as it is conditioned on Class C. What is the difficulty here it is to learn the joint conditional probability probability of x one x two up to x given c me this classification. So, whenever there is a difficulty, we will do something to simplify it. So, here the assumption is that all input features are conditionally independent, then, this joint probability or joint condition probability becomes Kota of this simple conditional probabilities.

So, probability of x one x two x given c is equal to probability of x one given c multiplied with probability of X given c, up to t of x and given the maximum a posteriori rule, in this case would be for a sample x vector, it belongs to class C one is probability of x one given c one multiplied with probability of X given c one up to probability of X given c one whole multiplied with probability of c one is greater than probability of X, given c two multiplied with probability of X given c two up to an extent given c to whole multiple of C two advantages, the training of neighbors classifier is very first, he's just required to consider each attribute in each class separately. Taste is straightforward just looking up tables or calculating conditional probabilities with normal distribution. Performance of neatest classifier is competitive to most of the state of the art class.

Suppliers many successful applications for example, spam main filtering. So there are a few issues in this classification. The violation of independence assumption for many real world tasks, probability of x one x two x given c is not equals to probability of x one given c multiple of x two given c multiply the probability of X given c. However, this works very well even when independence assumption is violated, zero conditional probability problem, if no example contains the attribute value x equals two e JK, the distributed probability we had x equals to a JK given c equals to C will be equals to zero. In this circumstances to distributed probability of x one given ci multiplied we're going to have Excel given CL b equals to zero Pause this time is equal to zero and as this is under product, so if one time is zero then the whole equation the whole expression will be zero, okay.

So for remedy conditional probability is calculated with following formula. So, festival probability of X equals to a Gk given c equals to C i will be equals to NC plus MP divided by n plus m. where NC is number of training examples for which x t is equals to a JK and C equals to ci in is equal to the number of training samples for which equals to ci. Small T is the prior estimate usually small p equals to one upon t for t possible values of exchange. Small n is with two prior that is number of virtual examples, usually small n is greater than or equal to one In the next video, we shall implement named this classifier in Python. So see you in the next lecture. Thank you.