Model Training

14 minutes
Share the link to this page
Copied
  Completed
You need to have access to the item to view this lesson.
One-time Fee
$49.99
List Price:  $69.99
You save:  $20
€46.59
List Price:  €65.24
You save:  €18.64
£39.95
List Price:  £55.93
You save:  £15.98
CA$68.27
List Price:  CA$95.58
You save:  CA$27.31
A$76.51
List Price:  A$107.13
You save:  A$30.61
S$68.03
List Price:  S$95.25
You save:  S$27.21
HK$391.33
List Price:  HK$547.90
You save:  HK$156.56
CHF 45.60
List Price:  CHF 63.85
You save:  CHF 18.24
NOK kr550.06
List Price:  NOK kr770.13
You save:  NOK kr220.06
DKK kr347.49
List Price:  DKK kr486.51
You save:  DKK kr139.02
NZ$84.03
List Price:  NZ$117.65
You save:  NZ$33.61
د.إ183.60
List Price:  د.إ257.06
You save:  د.إ73.45
৳5,484.82
List Price:  ৳7,679.19
You save:  ৳2,194.37
₹4,167.13
List Price:  ₹5,834.32
You save:  ₹1,667.18
RM238.42
List Price:  RM333.81
You save:  RM95.39
₦65,297.93
List Price:  ₦91,422.33
You save:  ₦26,124.40
₨13,918.71
List Price:  ₨19,487.31
You save:  ₨5,568.59
฿1,846.97
List Price:  ฿2,585.90
You save:  ฿738.93
₺1,626.69
List Price:  ₺2,277.50
You save:  ₺650.80
B$257.91
List Price:  B$361.10
You save:  B$103.18
R942.93
List Price:  R1,320.17
You save:  R377.24
Лв91.05
List Price:  Лв127.48
You save:  Лв36.42
₩68,777.09
List Price:  ₩96,293.43
You save:  ₩27,516.34
₪190.37
List Price:  ₪266.53
You save:  ₪76.16
₱2,884.27
List Price:  ₱4,038.21
You save:  ₱1,153.93
¥7,839.25
List Price:  ¥10,975.59
You save:  ¥3,136.33
MX$862.02
List Price:  MX$1,206.90
You save:  MX$344.87
QR182.32
List Price:  QR255.26
You save:  QR72.94
P689.82
List Price:  P965.80
You save:  P275.98
KSh6,748.65
List Price:  KSh9,448.65
You save:  KSh2,700
E£2,394.60
List Price:  E£3,352.63
You save:  E£958.03
ብር2,846.33
List Price:  ብር3,985.10
You save:  ብር1,138.76
Kz41,775.97
List Price:  Kz58,489.70
You save:  Kz16,713.73
CLP$47,415.51
List Price:  CLP$66,385.51
You save:  CLP$18,970
CN¥362.26
List Price:  CN¥507.19
You save:  CN¥144.93
RD$2,936.27
List Price:  RD$4,111.01
You save:  RD$1,174.74
DA6,712.91
List Price:  DA9,398.61
You save:  DA2,685.70
FJ$112.97
List Price:  FJ$158.17
You save:  FJ$45.19
Q388.83
List Price:  Q544.39
You save:  Q155.56
GY$10,456.12
List Price:  GY$14,639.41
You save:  GY$4,183.28
ISK kr6,993.60
List Price:  ISK kr9,791.60
You save:  ISK kr2,798
DH505.76
List Price:  DH708.11
You save:  DH202.34
L888.32
List Price:  L1,243.72
You save:  L355.39
ден2,869.11
List Price:  ден4,016.98
You save:  ден1,147.87
MOP$402.79
List Price:  MOP$563.94
You save:  MOP$161.15
N$957.24
List Price:  N$1,340.22
You save:  N$382.97
C$1,839.29
List Price:  C$2,575.16
You save:  C$735.86
रु6,654.43
List Price:  रु9,316.74
You save:  रु2,662.30
S/186.47
List Price:  S/261.07
You save:  S/74.60
K192.50
List Price:  K269.52
You save:  K77.01
SAR187.49
List Price:  SAR262.50
You save:  SAR75.01
ZK1,315.66
List Price:  ZK1,842.03
You save:  ZK526.37
L231.86
List Price:  L324.63
You save:  L92.76
Kč1,171.86
List Price:  Kč1,640.70
You save:  Kč468.84
Ft18,288.87
List Price:  Ft25,605.88
You save:  Ft7,317.01
SEK kr544.90
List Price:  SEK kr762.91
You save:  SEK kr218
ARS$43,678.56
List Price:  ARS$61,153.48
You save:  ARS$17,474.92
Bs346.09
List Price:  Bs484.56
You save:  Bs138.46
COP$197,009.16
List Price:  COP$275,828.59
You save:  COP$78,819.43
₡25,122.12
List Price:  ₡35,172.98
You save:  ₡10,050.86
L1,233.97
List Price:  L1,727.65
You save:  L493.68
₲371,261.88
List Price:  ₲519,796.34
You save:  ₲148,534.45
$U1,917.12
List Price:  $U2,684.12
You save:  $U767
zł201.38
List Price:  zł281.95
You save:  zł80.57
Already have an account? Log In

Transcript

Let's look at the code that does more training. To do that, first I'm going to launch My anaconda. So I'm going to launch my navigator. Let me close the existing one so that I can restart. So this is my navigator. Okay, so now I started up.

So I have created a virtual environment called p3 seven as we saw in the institute So I'm going to go to that environment and I'm going to open my Jupiter notebook. OK, so it opens a Jupiter notebook in the browser. And I'm going to go it shows you all the folders in your machine. I'm going to the folder where I downloaded my code. And I'm going to use the iPython Notebook file, which happens to be this one. Okay, so if you have never used iPython Notebook before, you don't need to worry.

It's very simple interface where you can run a bunch of Python lines one by one. So this gives you interactive feedback. And you can see So this code is the main code that we are going to run through. And this code will reuse the data set loader class in the preprocessor class that we saw earlier. So the goal here is, again, we have 3000 images that is consists of a cats, dogs and pandas. And we want to classify them into appropriate images.

So this does the steps are here. First, we are going to split the training and test set. And you're going to train the model and then we go to evaluate the model with the testing. Right. So the first thing that we need to do is we need to import few libraries, right. So these libraries have been installed as a part of open sim installation tutorial that we saw earlier.

Right. And so we have any psychic libraries like SK learn library, and the image utils library, right. And these two classes we already have source code in our four folder, okay, so and just to make sure Every package is installed, we can run this particular lines of code this section by pressing Shift and enter. So when the stars in indicates your core is running and it's importing the file, and when successfully imported, it will show you why that's important. Sometimes you might get an error if the package is not installed properly okay and it will tell you which package is not installed. When that happens, what you usually do is that a package is not installed on your machine.

So, what you do is you go to the Anaconda command prompt, right. And then you activate you go to that environment, where you have created a virtual environment, like this was the environment and then you just run pip install. Iam utility utils example this is a package that is that gives you an And then this will install you run to you just run this command and install the package for you. I'm not going to do that because it will have the package already installed. But if you run into that error, just go ahead and install that package is giving me an error, close this window and restart Anaconda and then it should work. Okay, so the next thing we are going to do is we are going to initialize our data set path, which is in our current folder, which is into the data sets in animals folder, right?

So let me show you how it's organized. So our code is here. My code is over here. Okay, my Python notebook is this one and ready to this part, the data folder that contains images and data sets. And inside that animals right here, it contains all the images And that's the part I have specified here. Then in my KNN, as we saw in the Kanan tutorial, I need to specify the number of neighbors who are going to vote for me.

And then I'm assuming n is two. And then I'm going to say how many cores on my machine that should be used by this program. And I'm saying one, so if you have more than one CPU on your machine, you can increase that. So your program runs faster, right? So let's run this piece of code and make sure there are no errors. Cool, it's running fine.

So press Shift, enter, whenever you want to run this say, then we want to grab all the images from our image folder into this variable, right. So let's do that. So I'm going to again, our highlight this particular sale which does that and to print some messages, I will show you how we are doing it. So I'm going to press Shift and enter. So now it Loading all the images. Okay, so we loaded all the images.

Now the next thing we are going to do is we are creating a preprocessor class that we saw earlier. And we are passing a parameter of width and height, because we want to resize all of our image to 32 by 32 size. Then we pass the preprocessor classes the input to data set loader class right, because the preprocessor is called Big Data Loader as it enumerates through the files that we saw in the previous class. Right, then what we do is we call the load method of data set loader, which is going to go through all the images and it's going to create the two lists that we saw right data which is nothing but the actual binary images and the labels which are nothing but the folder name for Each of these image, right, and then we will print the shape of the each of those data outputs and then see what what it kind of look like.

And then we are going to flatten the input data set into two dimensional data set. Okay, and we will show you what that means when we see the output. So I'm going to run this by pressing Shift Enter. So this is an example of one class. So it's loading is running this code. The image path is saying this and it is saying, once it is placed, this label is this folder packed into by by backslash, it creates a list.

And it's going to extract this list is this folder name as a label here, right? So it's showing the progress of how many images it has processed 500 or 3000 1000, or 3000. And it has processed all the images at this point. Post all the images. So, so this is the dimension of each image class that it loads right. So basically it means there are 3000 images, right?

And each image has 32 pixels, right? So we'll do height and width and there are three channels right if you look at the image basics section earlier, then you will remember that for color image, they described as 32 by 32, which is height and width. And there are three channels one for red, one for green, and one for blue. So the total dimension of the image data set is 3000. There are a number of images 1000 for cat 1000 for dog and 1000 for Panda, and each image is of the size 32 by 32. And there are three channels right?

For processing we need to flatten this out. into a single dimension. And we saw earlier how we do that right. So, let me go to that lecture again and show you how we are going to do that. So, so, this is how each image would look like right there are three channels each condensing 10 by 10 right. And then what we are going to do is we are going to flatten out each of these dimensions like here each of this RGB channel into a single dimensional vector right.

So, here are the sizes in certain button is 32 by 32 and there are three channels. So, there are 3072 elements will be there into single dimensional vector right and there are 3000 such images. So, that's why our input sizes this output size 3000 because it doesn't labels and once you flatten this is the input size before flattening. So we call this function to flatten our input, and it is converted into 3002 3072, or 3072 is number opted by multiplying 32 is a two to three, right? So this is what we do pre processing for each image where we flatten the three channels into a single channel, right, that is required before we process the image. Now, let's talk let's understand what is the data?

What is the memory occupied by our data because data consists of 3000 images. And if you want to express that size in bytes, then you just divide by one zero to four, again into one zero to four. That will give you data in megabytes. Great Getting Started. So that because I want to clear out what I want to start with a clean output. Let me execute those steps again.

Can you see processing the image? Okay, the next thing that we do is we want to use our labels are nothing but simple plain strings. Like we have cats, dogs and pandas, we need to convert them into integers and that is done by this label encoder class, which is obtained from scikit learn, okay. So we just can work our label into integers then All it is now what we do is we need to split our data set into train and test. So psychic learn basically provides a convenient method which takes data as input labels as input. And then we specify what should be the size of test data, which is 25%.

And the remaining size would be trainings a training session with 75%, right. And it splits into the input train data, input test data, output, train data and output test data right. And then you can also print the size of each vector. So, we have an input data set in data set, two to five zero by 307 to size and this is what we are going to use for training the model and then we are going to start the next data set is 750 by 3072 is a test data set that we are going to feed to Train model to predict the expected labels and we are going to compare the expected labels with the actual labels that is this data set has right and that we will see in the next lecture. So let's understand how the training is done now. So, we have split that train and tested it into 75% in 25%, respectively.

So what we do is we train the model using the K nearest neighbor classifier class which is built in the psychic learn, we just pass it the number of neighbors and the number of jobs which are be configured. And then we ask the model to fit the tree next and train why so we pass in the training training data. So where the train x is nothing but our image matrix, right, and the train Why is our labels so we pass it to the cane and didn't care. In a model, and we saw how the Kanan model learns, right? So, so we saw it here. So what it does is for each model, basically, for each image, it's going to compute the distance.

And it's going to just keep those vectors in memory, right? So it doesn't learn anything but just want to keep those vectors in memory, right? That's all it does during the training. And then we will also see the next lecture, how we are going to use it to predict though tested asset, right, okay.

Sign Up

Share

Share with friends, get 20% off
Invite your friends to LearnDesk learning marketplace. For each purchase they make, you get 20% off (upto $10) on your next purchase.