so of course don't take this as being the best way to do it all if you have a better way I'm


Code with me (live): How to make your first Kaggle submission from scratch!

always open to feedback or or advice or whatnot but we're on to our last feature then we'll run


some models and then we'll get a submission all right how many missing values does embarked have


two not too bad over 891 so 800 889 total values so what kind of values are we don't want that


we don't want we don't want some so there's 644 s 168 C and 77 Q what does that mean so Southampton


was the most popular point at when the passenger boarded the Titanic Queenstown does this influence


whether you survived or not who knows we'll we'll see we'll see we'll run a little count plot over


there to just look at this in a different way these count plots are handy because a lot of people


might not understand this sort of stuff like if you're doing a presentation people can quickly


interpret that right they might not be able to go look at that I mean there's only three here so


so that's quite easy but definitely if you wanted if you wanted more more people to have a look more


people to be understand it quickly you want to you want to visualize as much as you can I know


that's that's a big request from any of the clients we deal with can you make it visual yes of course


we can we're just we're just if you if you don't you're not communicating properly how can we deal


with the two missing guys I've embarked one option is to drop the two rows so remove that row where


we're embarked is missing completely so if we go back up to our missing number so we could just


drop these two get that get that whole line so take this draw it across drop that and draw that


across and drop that out of the outer frame do we want to do that maybe maybe not depends on what


it is but we go back to our our little our little embarked column I think we have yeah we've removed


the values which are missing what's another way to do that how can we do it another option would be


to randomly assign a value of C Q or S how would you do that how what are the pros and cons of each


for now because we're just rushing through this this EDA I don't know it seems it might be going


slow but believe me if you were to code this whole pipeline it might go a bit slower we're just trying


to get get a model out to see to see what kind of predictions we can we can try and make all right


so we remove the two rows missing guys from embarked we can now add embark to our sub data


frames which we have done so now we have two sub data frames DF bin go ahead beautiful we'll view


that and we'll do the F con dot head now of course we're missing some features from the training


data set which is which is here we got that boom all right look at that it's a lot smaller but why


because we've gotten rid of some things we've got rid of name we've gotten rid of ticket we've got


rid of cabin and now as I said there's some challenges for you in this notebook to figure


out a different way to use the name column to figure out a different way to use the ticket


column to figure out a different way to use the cabin column these are all projects that that


are that that you would have to work on as a machine learning engineer right and I've left


them as challenges because it's good practice all right it's good practice to try and think


about those things and and visualize new ways and then running the rest of the notebook to see how


it affects affects your models all right before we get on a feature encoding we have in here


hey yash nice to see you brother welcome to the chat if you have any questions please ask otherwise


we're gonna we're gonna keep going pushing on with our exploratory data analysis eventually


leading into a Kaggle submission all right feature encoding we now have our two subset data frames


ready we can encode the features so they're ready to be used with our machine learning models so


we're gonna one hot encode our binary variable I'm sorry our binned variable so this is our bin


data frame here we're gonna one hot encode that what does that mean now one hot encode means


giving creating a new data frame that's going to give things a zero for what it isn't and a one for


what it is now example so we take this first row let's have a look here the FB in dot head one hot


encode so we have survived zero it's given it at zero because it's not it's not up it's a zero


here because it's give it member one hot encoding gives us a zero for what it what it isn't and a


one for what it is now we have P class so it should be a one for P class three bomb it is


sex should be a one for sex zero yes all right now what can you think about what the disadvantages


are of doing this right so we have our our our data frame here it's one two three four five six


seven seven columns and this one is 28 so what have we done right we've we've got the same it's


just gonna be the same length but we've just added a whole bunch of columns hmm what what


could that do all right I'll let you think about that what's what's what's the disadvantage there


now for the continuous data frame we're going to label encode all of the values using label encoder


so this is a SK learn right function we imported that right up the top so if we go label right


there so there we go from SK learned up pre-processing input one auto coder label encoder


all right so we go we'll find the next incident there we go that's what we want all right now we


look at that so what what is that done what is this label encoder done so we have DF con okay


so now it's changed all of these into different labels so we have sex has been turned into ones


and zeros automatically p arch and Sib SP hasn't been changed fairs been changed hmm do we need to


do that maybe maybe not we'll see how the models go but usually with with these kind of values fair


the numeric you don't usually have to change them but we'll we'll leave this for now it's just what


it's done is it's given every every fair that $7.25 will have the code 18 okay and then same for for


for every fair that $7.92 and 92 and a half cents I don't know how how that happened how someone got


charged 92 and a half cents that'll get a number of 41 and then embarked has simply been changed to


zero one or two so maybe if we blow this out to head and then we have 20 boom zero one two so


anything with an S has been given a two anything with the C is zero and anything with I can't see


that one so maybe and anything with a Q is a one all right so that's just replaced our variables


with with numbers because what's the goal of of getting before we run machine learning models


it's to it's to it's to turn all of our values into numbers right because what do computers work


best with they work with numbers best so we can't necessarily feed it where is it information like


this like name it's not gonna understand that so we need to turn it something into this right so


now what our what our models can do is search through this search through actually it's gonna


search through these columns missing out the survived and figure out what pattern in these


numbers gives a zero gives a one gives a one gives a one gives a zero all right can it work out a


pattern in that we'll see if you were to guess if someone survived or not what do you think you


would get think about that if you had a guess just by looking at this guess 0 1 how accurate


do you think you could get if you did it across the whole data frame think about a coin heads or


tails all right we won't answer that yet so we're gonna start building some machine learning models


yes all right so first of all we're gonna separate the data into a train and test I mean sorry labels


and data now what what does that mean so the selected data frame is this one here the one


we've encoded we're going to use the continuous data frame for this for this EDA you might want


to use the bin bin data frame in your EDA but for the sake of simplicity we're going to use


the encoded version of the continuous data frame subset which is which is this bad boy here so


select data frame so we don't change this we're going to change it put it into another another


variable could select data frame so split this into data and labels so X train so our X variables


because this is a machine learning model you're gonna have a whole bunch of X variables and you're


gonna try and predict the Y all right so X X train equals selected data frame dot drop survived


access equals one so this is our data so what is this doing it's taking the selected data frame


okay so selected data frame go ahead it's taking this data frame dropping that column and just


using these variables so these are our X variables so let's have a look at X train


run this all right so see how this we've dropped the survive column and now we're gonna make our


Y so this is this is the labels of our data data frame so this is the just 0 or 1 so it's practically


this just this column Y train equals DF select the DF dot survived so it's this column so now


we're gonna use this and these to build our model so this is the data and this is the labels all


right that's what we've done we've just just gone through each feature turned it into a set of data


and a set of labels now we can run some machine learning on it so we've built a little function


here because we're going to be running multiple different machine learning algorithms called fit


ML algo now this is it takes an algorithm a type of algorithm it takes a set of data it takes some


labels it takes a number of times you want to cross validate right so cross validate is important


we'll get on to that in a second it's gonna do it's gonna create the model then it's gonna fit


fit the algorithm to the training data and the labels it's gonna give us an accuracy accuracy


score it's gonna perform cross validation automatically and it's going to give us a


cross validation accuracy metric accuracy cross validation and then it's going to return training


predictions the accuracy and the cross validation accuracy so we're gonna start off with logistic


regression then we're gonna move through through a bunch here okay now why would you run so many


different different types well you're trying to figure out which model is best suited to the data


so you're gonna try and see run a whole bunch of models and it's not hard to run these look how


quick it runs right oh we didn't define our function I say this now and it takes a little


bit while a little while there we go 4.5 seconds not that not that long all right doing tenfold


cross validation so 78% accuracy on cross fold validation that's beautiful now we've tracked the


time here by creating start time all right we've done our function here fit ml algo okay and then


we're logging the time right that's just there and then we've got just some some print statements


for the for the metrics we've we've returned here okay so then we've got accuracy 79 that's just on


one pass and then accuracy of cross fold validation on 78.5% now why would you want to do cross-validation


validation we'll get to that we'll get to that in a minute we're gonna run all the other algorithms


first so k nearest neighbors now these are what actually what are these what algorithms are we


running so we go up into sklearn and if you if you look here we've imported a bunch of different


algorithms here okay now these are some of the most common machine learning algorithms that


you'll see all right now if you google most common machine learning algorithms these will probably


come up linear support vector classifiers gradient boosting classifiers k nearest neighbor classifiers


now we're not going to go through what each of them does all right now at because of our our


need for speed with our EDA we're just gonna run them see which ones the best and then we'll choose


the best one and then you can improve that from there all right so we've run our list of regression


by the way if you want to find out which algorithms and what they do I would highly suggest looking at


the sklearn documentation so just google sklearn machine learning algorithms and you'll find a


bunch of stuff there so we have here look how quick that ran under a second bomb 70 73.68 percent


accurate again under a second because there's not much data right there's only 891 rows and these


these are great algorithms computers are faster doing math now we have support linear support vector


machines under a second again stochastic gradient descent under a second what's it a 92 percent


accuracy oh but it's fallen down to a 79 percent accuracy in the cross fold validation so stochastic


gradient descent didn't do too well gradient boosted trees 81.1 on tenfold that's the best


so far all right so these are some of your most common algorithms but I want to introduce you to


to one of my favorite one I've discovered recently and it does an amazing job right cat boost the cat


boost algorithm so it's by a Russian company Yandex who created as their in-house algorithm


but now they've open sourced it it's it's similar to well it's similar to I'll just read you the line


it's a state-of-the-art open source gradient boosting on decision trees library so this one


was a gradient boosted trees algorithm and that got 81 percent right so this is a state-of-the-art


gradient boosted on decision trees well maybe this is this is cat boost is a combination of


these two so wonder how it's gonna go okay so it's a decision tree and a gradient boosted tree


combined all right it's simple and easy to use and is now one of regularly one of my regular one


of my go-to algorithms for any kind of machine learning tasks if you want to check out more


encourage you to check out the cat boost docs you can find that at cat boost AI all right but


we're not gonna dive too much into the algorithm because we just want to try and run it so this


one's gonna have its own special kind of implementation because it the fit ml algorithm


won't won't work on it there's a website with cat boost check that out we have here we'll view the


training data one more time we'll view the the training labels excellent now cat boost has a


little section where you define the categorical features now that that might be what cat boost


stands for categorical boosting I'm not entirely sure what do we have here I think you need test


data for testing algorithms yep yes you're completely right we're gonna test our data


we do have a test data set so we have a we right back up here so we have a training and a test


data set this notebooks actually bigger than I thought so we have training this is our training


data here and then we have test test data so we're gonna train on this and then we're gonna


test it on that which is what our submission will be our submission will be our predictions


on the test data set and yes Zach the the video will be up to see later it will be live in my


channel once we've finished here all right let's go back to cat boost okay so cat boost has a


really cool feature where you can tell it what categorical features you have of your data set


let's go right back down to cat boost I should probably just search cat boost anyway we're here


excuse me okay so categorical features for the cat boost model in the cat boost documents it


says feed cat boost the categorical features you have right so what's a categorical feature


alright so these are none of these are categorical features but a categorical feature might be might


be something that's non-numerical okay so let's think about it so say you had cars cars a category


might be the different type of make of car so you could have a category for Toyota a category for


Ford a category for Mercedes something like that okay so all of our data is numerical so if we run


this little function here that says cat boost equals NP where X train so this is this is our


X train here does not equal NP float so what's a float so float is what is a float so we have a


look here so if we have actually you know what a float is let's just let's just let's just put it


there this is not to tell you what type of variables are so a float is a number like that


right so it's got it's got decimals and in in the integer is probably something like this okay


it's just a six so int no decimals float decimals okay so none of ours have decimals right because


they're all integers so we don't have the cat boost is seeing these as all categorical features


so what's it going to do cat boost has its own methodology of of encoding these so I'm not going


to dive too much into that essentially what it does is cat boost all values sorry all values


except fair what's happening I'm doing this wrong see this is the beauty of doing it live let's see


there we go okay so zero one two three four five all right all of them are categorical features so


what cat boost is going to do is when you when you create your training data all right you tell it


what the categorical features are okay so these are going to treat them all as categories all right


because we've encoded them all into categories and then cat boost is going to figure out its


own way of best dealing with those categories so member like up here we one-hotted our data in


data set right now that might not be the best way so cat boost has its own way and you can read


about more in the docs of one-hotting a data frame but in a different method I'm using don't even


think about one-hotting because cat boost does it in a different way okay just think of it as you


tell cat boost what your training data is what your labels are and what the categorical features


are of your training data set and it will create a training pool which means cat boost has its own


way of manipulating this data into a way that its algorithm can best explore it okay that's that's


how I understand it there is some some some math you can find in the docs about how it actually


does that but for the sake of this of this video we're just going to leave it at that cat boost


figures out based on the information you feed it the best way for its models to run over your data


and let's let's create the training pool and now we're going to fit our cat boost model so we have


cat boost model equals cat boost classifier so classifier what does that mean so classifier


means zero oh well it's going to classify something now in this case cat boost classifier


means it's it's going to it's it's zero one classification now that is our that is our goal


do we want zero one yes for survived all right so it's going to classify the training data the


training data into zero or one that's what we want and we're going to give it a loss function


of accuracy so I want to figure out how accurate this cat boost model is and then the log the loss


function it uses is log loss all right we won't dive too much into that but that's the default


that comes with it you could actually comment this line out and it will it will still run but


we'll leave that there so you can see what it does and then so we've created our cat boost


classifier model and then we're going to fit it by calling cat boost model dot fit on the training


pool that we just created and we're going to plot so we can see what happens and then we're going to


save out the accuracy much in the same way as we have for our previous machine learning algorithms


so round machine dot model dot score x train y train same same way we've done that the cat boost


okay let's go down here we're going to run we're going to watch this beautiful graph this is what


you can see with with cat boost check that out you can actually watch the accuracy climbing so


this is what this is doing is running over our training data set so going through this and


figuring out which label it should have right and this is this is how accurate you can see it in


live time cat boost is beautiful I actually love cat boost a lot so um we'll see how this goes


gonna take a little bit while to run now we're gonna run cat boost cross validation after after


this is done because this is just fit remember right so so in our fit ml algo function we did


fit and then we did cross validation so up here while cat boost is running we go up here fit ml


algo bomb so we did fit algo dot fit and then we did cross validation so we need to do the same for


cat boost so we're doing cat boost dot fit down here and then running cross validation all right


so cat boost dot fit and then running cross validation down here so cat boost has a function


built-in called cross validation it's just CV we imported that right at the top there we go from


cat boost import cat boost classifier that's the classifier model that's running right now pull


that's how we created the training pool data and cross validation from cat boost which you'll be


able to see in a second so now that's running apparently it's gonna take a little while


hopefully it goes a bit quicker than that but in the meantime while we're waiting for that we'll


check out the chat oh sorry just joined the stream any clips from Silicon Valley a lock how are you I


haven't actually doubt divided into a training data set so so the the Kaggle competition gives


a training and test set so what we're doing is we're going to we'll model the training data and


predict on the test data so we've already got a training and test set from Kaggle usually if we


didn't have a train test set already we'd we'd split we'd split it yep you're right 80% training


and 20% 20% test or a little different depending on your problem so cat boost is running cross


validation it's going to take a little bit longer than our than our previous algorithms because of


what it does and now I actually can't couldn't explain to you what what it does on the inside


but just in my experience of using it multiple times for multiple projects it does take a little


bit to run because I think I think it's doing a combination of different types of models so it's


not just doing one one one single way of fitting on the data it's doing it's doing multiple different


ways so it's doing a decision tree classifier and a gradient boosted tree all all rolled into one


so we're fitting that here now why would you want to perform cross validation all right so I'm going


to explain cross validation and how I understand it I would I would look into it into it yourself


as another piece of homework but if you imagine if you imagine let's let's do this let's use a


piece of paper while cat boost is running we've got a little countdown here so you've got some


time to play with imagine you have your data here right just as one simple piece of paper right and


you do one your model goes down and it fits all the rows in one hit okay it might have done a


really good job right because some of what it does is randomly initialized what does that mean so


it's used it could use random random numbers it might have done a really good guess right so if


you actually say you had this piece of paper and you had to guess zero or one for each each different


line say you went down your first try your first shot you did really well all right you got you got


80% accuracy or 85% accuracy you got all the all the zeros and ones pretty much spot-on but then


you reshuffle you shuffle all the lines you move the piece of paper around and then you guess again


and you only got 60% accuracy right that's essentially what cross-validation is doing


instead of just running one one time over the data it's it's running multiple times over different


versions of the data so it goes once all right boom we got 80% mix the data around again we got


65% mix the data around you got 79% and then we're gonna do that do that ten different times


to really get a good idea of how well our model is actually doing on ten different types of the


same data not types but ten different ways of looking at the same data that's how I understand


cross-validation and why why is that important well as I said look at this this accuracy without


cross-validation is way higher than that with cross-validation so see how we've done tenfold


that's that's going over the data in ten ten different ways so this way that way do it again


that way all right not exactly how this piece of paper is looking but just imagine reshuffling the


rows in different orders so this is a lot higher look at this one 92% so it got really lucky on


that one hit and then on the cross-validation not as high right because it's cross-validation


and much more robust metric because you know it's gone over the data in several different shapes and


sizes all right how's our how's our cross-validation cat boost model going all right we still got a


little bit of time how we have here hey just arrived don't know if you spoke about that


already we're trying to build a model how do you pick your features great question let's get into


that speed speed trem maga all right so how do you pick your model your features we're gonna answer


this question while our cat boost cross-validation model is running so let's look at our initial


training data frame we have some features thank you for the question we go up here we have this


is our training data frame so if you imagine each column is a feature so we have survived that's our


target feature p-class name sex age now for this one for this training data frame there aren't that


many columns right so there's only let's say 10 doesn't really matter might be whatever so how do


you pick which ones to use in this case we've decided to use as many as we can because there


isn't that many to go over but what happens if you had this and it just kept going and kept going


off the page you've got maybe maybe a thousand a thousand different columns that that could be hard


that could be hard to go through them one by one all right so what you would probably do is go


through them do something like this do something like this figure out where all the missing values


are get rid of all the ones with missing values and just just run it on on your on your best ones


or you go through it step by step you go through each each feature individually work out which


one's the best or so so this we've got a few different ways here so this is this is going


through each feature step by step the other option where are we going we've got to go right down to


the bottom there's two more two more options that that really stand out to me so we've run some


models we made some predictions I'm jumping a little bit of head of where we actually are up


to so our cat boost model is still running look how beautiful this graph is by the way look at


that cat boost is awesome it shows you these plots maybe if I didn't have this plot on it


would probably run faster so how do you choose your features back to that so we'll see how it


is we're gonna get to that in a second actually feature importance so which features are the model


were most important in making predictions there's another way you could do it that I've set in the


challenges which is called PCA it's a principal component analysis that'll be in the extensions


it'll also be in this resources that I've listed down here Daniel Ferroso awesome name he's done


some epic work on how you can do this notebook on a bigger data set this one I built is actually


based on his his notebook so all that'll be linked in the description as well as in the in the github


version of this notebook so that's PCA feature importance iteratively going through each feature


one by one or another big one is talking to a subject matter expert now what does that mean


now say you had a project right so you had a project and you want to predict the prices of


houses and you have a data set like like we do you have you have the data set like we do you have


you have let's say a thousand different columns right and you all of them go from number of


bathrooms to to suburb to to what color the floor is and stuff like that to work out which features


of those columns you might want to use you might talk to a real estate agent and say hey I'm trying


to do some data analysis on this bit why why would you talk to the real estate agent because


they sell houses they know the value of what contributes to two different house prices or


they at least have some insight the data might be able to tell us more but to get started they might


be able to say okay you really want to pay attention to the number of bedrooms you want to


pay attention to the the suburb you want to pay attention to how close it is to schools because


these are the things that my customers really ask for you might say okay I'll start with those you


start with those and you build a pretty good model but you want to make it better and that's when you


can start to go through each of the features and start adding more and seeing if it improves your


model so so step by step you can do feature importance once you've made some predictions


we'll show you that in a second or you can go through see where the missing values are and just


drop all the ones that have missing values because because they're because you just want to work


through it fast or talk to a subject matter matter expert to get their advice and now there's more


ways but these are just the ones I can think of all right there's some of the ways that that are


that we literally do day to day at at Max Kelsen so for some of the projects we'll talk to the client


like hey you've got all this data but just some information what is most important when you you


the expert the people who've been there for 20 years because there's a data scientist I may come


along and I've never seen this problem before but you have someone who's been doing it for 20 years


maybe I had to tell you straight away oh don't worry about the color of the floor no one cares


about that all right of course the diet will tell you more once you figure that out but that might


be a way to eliminate the amount of features you have to deal with otherwise there's this PCA


different ways of analyzing features with code but we won't go through that in this notebook that'll


be in there in the extra resources down below if you want to look it up just Google sk loan


classification notebook by Daniel Ferraso all right our cat boost model has run beautiful look


at that look at that all that green and you can run over it and see where where it's most accurate


and stuff like that let's do this all right cat boost metrics as I said it's a higher accuracy


all right but we did some cross-file validation and it's lower so that first guess it ran through


the the our piece of paper for it ran through it got really good results but then we decided to go


you know what I don't trust that first result I'm gonna run it multiple times and it averaged out


to 81.2 and it took 11 minutes so I'll be a lot longer than what our other models took so this is


where we can just make a little table so we've just gotten the accuracy scores from all of them


so look at that decision tree right up there 92.46 but as I said we're not going to pay attention


to these regular accuracy scores we care more about the cross validation accuracy scores and


there we go our baby boy cat boost is right up the top I seriously love cat boost not partnered


with the index or anything I just I just love their their algorithm because it's so easy to implement


and it gets incredible results so cat boost got the best correlation cross validation accuracy


score so we're going to use that and now it got close to 82% now if you're trying to guess zero


or one what do you think is a baseline maybe 50% right because heads or tails so if we were to


guess who survived and who didn't we could probably get around 50% maybe a little bit less or more


because the data set is not evenly not evenly spread it's not 50 survived 50% survived 50% did


not survive so 82% is is a pretty good score in my opinion just for this exploratory data analysis


and remember there's always a challenge how could you improve the cat was boost model so that will


involve you checking out the cat boost docs I put a little tutorial down here on cat boost by the


the authors of cat boost themselves you can check that out it's really great they work on the Titanic


data set as well so we're gonna run feature importance that's what I was talking about


before this is just a little function to create a feature importance graph by by sucking out the


the feature importance is parameter of the cat boost model and then we go here so if we look at


this this is this is how this this axis here is how much it contributed to the model and this is


our feature what what contributed so we have this is in reverse order so sex is number one so male


or female is really important to to whether you would have survived in the Titanic according to


our model so sex is number one P class is next so does this make sense right have you seen Titanic


have you seen the movie this is where you can ask yourself is my model talking bullshit or talking


real is the data showing us the right thing right and you can you can use your own intuition here


to kind of spot check to whether your results are okay so P class what was that that was the class


so that was first class second class third class so you it makes sense maybe people actually let's


go through sex so member in Titanic the movie women and children had priority that makes sense


and a lot of in our analysis right back up in sex the feature more males if you're a male your chance


of survival was way less look at that survived not many did not survive lots remember zero is male


all right so that makes sense that sex is a big contributor and then P class is if you were in the


first class maybe you get an advantage because you're you paid more for your ticket then you're


a higher class person or a citizen maybe they see you as more important I'm not entirely sure


fair is there as well P arch all right how many parents you had on there so if you're a parent


you that it valued more these are all things that you can do to spot check your model right and what


you may do is why would you why would you want this feature importance well what you could do


for your next version of running a model is maybe remove this bottom one all right ideally we want


to add a few more because we missed out on the age and the name remember right up here we didn't


use the name and the age but say you did use the name and the age and you find out that see the SP


really doesn't give you much it's under 10 all right so well in this case 10 is pretty significant


you might be able to remove the bottom feature and run your model again and see if it gets better


results all right but in our case in this smaller data set you could probably use more more more


features the better because there really is only 10 all right but if you had if you had a hundred


of these and you have the bottom 50 are giving you a little sliver so absolutely nothing you


might want to get rid of those and rerun your model okay another option is trying to improve


your best features you can't really improve sex because it's just 0 or 1 you can't really improve


p-class there aren't many here that you could you could really change because as I said these are


already ideal parameters but you might want to say you did add add name here as it was without


altering it and it just didn't give you anything you could change it to how we discussed before by


reducing it to like misters, missus, miss, doctor and it might might boost it up might give you a


bit more value all right but that's something you'll have to explore yourself but yeah what


feature importance does is it it it gives you an insight into what information is most important


when making a prediction okay and you can you can use it to improve the number of the features


you're using or use it to remove the number of features you're using okay precision and recall


I'm gonna mention that here we won't go through it in entirety I have linked down below a great


article actually no it's it's up here on what precision and recall is so when would you use


precision and recall so we've used accuracy as our as our measurement here right so maybe accuracy


is not the best metric right because if you had a hundred thousand people and only one of them gets


a certain type of disease this is another example not Titanic and you in your model predicts that


every single person doesn't have that disease because only one person gets that disease your


model is 99.9 99% accurate so that's a really accurate model but it's not that helpful right


because your model all it does is just predicts that everyone doesn't get this disease but there's


one person in a hundred thousand that gets it so this is where precision and recall come in now we


can run precision and recall on our cat boost model pretty easily I'll let you read up about


precision and recall and why that's a challenge actually what are some ways to visualize precision


and recall right you may want to look into what a confusion matrix is now remember here's where


you would use precision and recall instead of accuracy if you had cases where you have an


in-balance classification problem right so ours is pretty balanced we have although there are more


zeros than ones so did not survive versus did survive they are pretty balanced so they're not


like they're not like this they're pretty close right but if you had a problem like this where


you have a hundred thousand people and only one of them only one of them has a disease accuracy


is not your best measure right so if you go into your own problem and you have you have an in-balance


class so you say you have a hundred hundred thousand examples and you're trying to predict


ten which ten of those hundred thousand are the ones you're looking for right so you have a hundred


thousand yeah ninety nine thousand nine hundred ninety zeros and ten ones you want to find the


ten ones you might not want to use accuracy you might want to use precision and recall right so


so look into precision and recall there's a little little little link here that you can check it out


otherwise google beyond accuracy precision and recall really great article we won't dive into


this because of precision recall in this this notebook I just want to leave it there so so


you you think about it in the future you're not just using accuracy for every single every single


one of your models so our model got pretty good precision right so close to close to one there


recall not as great f1 combined so so these metrics are going to be between zero and one


closer to one is good I won't go too deep into it so our recall is low so this means there's a


higher amount of false negatives so recall is the ratio of true positives over true positives plus


false negatives so that means our model has a higher amount of false negatives predicting did


not survive when the person actually did survive and our precision is is higher so that means


there's less false positives and precision is a ratio of true positives over true positives plus


false positives so that means a false positive is predicting survived when it was actually did not


survived right so our models recall is not as great as its precision f1 is a combination of


precision and Reese recall and area under a curve is another another way of checking out the ratio


between false positives true false positives and true false I'm getting it all mixed up best to go


check out that article much better job at explaining it is what I do remember I'm still


learning these things as well I'll try and get better and then I teach them as I get better


alright we're ready to make a prediction on the test data set finally you're like Daniel I've sat


through this this video for two two hours whatever however long it is and just show me how to make


a submission to Kaggle alright well we're gonna do that right now so we're creating a variable


wanted test columns because let's have a look here why do we want to do that our model was


built off certain amount of columns it was built off these columns however our test data set has


not the same columns so our model if we tried to run it on this right now it would not work we need


to have our test data frame in the same version as our selected data frame actually oh no it'll


be the same as X train because we want to predict the survived column so 0 or 1 so we need to convert


test into something that looks like this and it's gonna be a lot easier because now we have our


wanted test columns right so we're gonna these are the only columns that we want so dfcon so


we use dfcon so let's let's actually change this to selected selected df because or actually we


can just change this to X train dot columns so what the same columns is X train much easier right


there we go so this is just a list of columns that are in here and then we're gonna make the


predictions by choosing cat boost model dot predict remember we saved our cat boost model


out up here and somewhere cat booze model so yep cat booze model equals this and then we use it to


fit boom we're going down here now we're going to use cat boost model to predict on the test data


frame so test we're choosing the wanted columns from that test data frame and then we're gonna


apply remember that label encoder fit transform that we did for the encoding df dfcon right back


up here same same thing we're doing because we want to predict on data that is in the same format


as our model trained on because the model is not going to be able to predict on data that's not in


the same format so we're just applying this one here so what we're saying is get me that test data


frame only the columns that I train the model on and run this function on them so that's what we're


gonna do we're gonna do that we're gonna make predictions doing that we'll go down here boom


look how quick that is our model just ran now what does it look like so it's just a bunch of


zeros and ones why because that's what we wanted our survive column to be so our model has just


output the same type of data as it was trained to predict right so zeros and ones that's all it is


zeros and ones now to create a data frame what does it have to look has to look like this so we


need passenger ID and survived so we need this this from the test column so passenger ID and


whether they survived or not based on our models prediction so let's do that create that oh look


at that but do they look the same what's wrong what's wrong with this one right what's wrong


with with these numbers compared to that remember we talked about floats and ints has to be the exact


same format otherwise I ran into this issue before otherwise the Kaggle submission won't take it so


although it's still zeros and ones we need to convert them into integers so this is what we can


do we can do submission data frame so this is this one this is our submission data frame and we need


to turn the survived column into integers so we're just going to go as type int that's a function


built into pandas bomb converted survive column into integers let's have one last look at that


before we are before we get it up there submission submission dot head wonderful now what is this by


the way this is our example right we imported an example right up the top this is what it looks


like does it do they look the same yes I think they do except for these values are a bit different


here but remember this is not a this is not um not the true reflection of what the label should be


okay our our test and submission data frames the same length why because if they're not the same


length Kaggle won't submit it Kaggle has to be the exact same length as the test data frame so


we have to use every single row and now to submit to Kaggle what do we need we're going to submit


this this is exciting submit predictions so upload files your files should be in a CSV format the same


way we downloaded them so we can do that right we can do that let's check it out bomb oh no well


submission CSV is ready so what have we done we've just taken our submission data frame said to CSV


we've chosen where we're gonna put it right and we we we were not including the index and that so


we don't we don't care about this all right because that's just that's just how Kaggle likes it so our


submission CSV is ready let's do a quick check on that just to make sure what it looks like how do


we check that we're just gonna do the same thing submission check reading our CSV file file and then


we got it up here now it's time to submit your Kaggle dot CSV are you ready are you ready look


what we've got here we've got we go into our Titanic folder we've got our notebook in here


that's it there kaggle Titanic data analysis this is our submission file right here how are you


everyone to the chat what did you do right out at Andrew Ong's Coursera course you can see you


can see that in the AI masters degree curriculum how long does it take to get started sir get


started with what it depends it takes as long as you need all right let's submit our Kaggle CSV


our predictions submission dot CSV file to Kaggle where we going we're going to Chrome we're gonna


upload a file we're going to desktop Titanic oh we don't want to upload that submit all right now


this is live hopefully it works what do you reckon we'll get uploading file complete now


how do we how do we check our score my submissions to submission submissions


all right it might may have a little bit of a wait time


oh wait did that go up capu submissions dot CSV we're trying this live boom we got a tick


excellent bites fingernails hey are you sure are you after Andrew Ong's Coursera course which one


the deep learning AI or the machine learning we're waiting for our Kaggle submission to be


to be processed it takes a little bit where is this well as you can see my submission six days


ago was was was successful but I'm not sure where why this one hasn't been maybe I have to refresh


the page well let's just take that 76% accuracy thank you very much cuz kushik so what have we


done here let's go back we have gone through step by step downloaded a data set from Kaggle done a


full exploratory data analysis so we've got the data set here done a full exploratory data analysis


built several machine learning models used our best machine learning model to predict whether


people would not would have survived on the Titanic then we've we've submitted our data


frame we've we've we've output our predictions right and then we've submitted them to Kaggle


right all in the space of however long this video is gone for but as I said before this is not an


exhaustive list of the things that you could do right there are I've listed some possible


extensions that you could do of this notebook if you want to go through this one yourself things


like what could you do with the age feature what can you do with the name feature how about the


cabin can you can trick combine Sib SP and P arch features to see if the person was alone or not


right can you do a PCA that was the person's question before how do you choose what features


to use or is there a way for you to improve the cat boost model I would suggest looking at the


hyperopt library and then there's a bunch of references that I use to create this notebook


SK learn classification notebook by Daniel Ferraso this is what this one was inspired by really you


should probably check out his it's a high quality of work I just decided to make my own version and


go through it for you his goes a bit more in depth on a lot of different things so be sure to


check that out there's a great blog post on how to encode categorical features in Python by


practical Python business there's a cat boost put tutorial on github now that's that's the challenge


that's how you could complete this one the cat boost Python tutorial on github is great all all


linked in there and then shat library is another alternative to viewing feature importance which is


this one here but as I said this all the code will be available on github there will be a blog


post that will be about a five to seven minute read of talking through what we've talked about


in this video I'll link it all below and last but not least if you do want to sign up to the


Coursera did sponsor this video so if you do want to sign up to a computer science specialization


look at all this right mobile web development they're currently offering one month free up to


December 11th for anyone who wants to sign up to to any of these courses I've left a link in the


description you can check that out all other relative links will be in the description as


always if you have any questions if you find any errors in this notebook if you're having trouble


tweet me send me a message my email you can always email me is Daniel at mrd berg calm otherwise


leave a comment below if you'd like to see a similar video to this in the future how can I


improve I'd love to know but most of all thank you so much for joining in and all the best


exploring your future data sets but most of all keep learning and what do we have here hey man


can you give me best learning learning machine learning Ricardo the ones you use are using right


now Ricardo my friend great great question there are some great references in here send me an email


Daniel at mrd berg calm Ricardo I'll be sure to send you a link to everything that I've been


using in the past few days but thank you so much we're gonna end the stream now I'm gonna figure


Data scientists should be more end-to-end | Machine Learning Monthly September 2020

You know what time it is. I can't believe it's been a month already since the last episode of this.


If you're new to these kind of videos, we're going to go through machine learning monthly,


which is a little bit of an article that I write rather than talk about it. Let me just show you.


It's machine learning monthly, the September 2020 edition. And of course, we are October 10th. So


this is for last month. So that's usually what happens. I write an article collecting the best


things from machine learning, or at least in my opinion, over the last month. And it's not limited


to the last month. It's actually because there's so much out there. It's just collecting the things


that I found most valuable and potentially you might find valuable too. So without any further


ado, this is the article that we're going to go through. There'll be a link if you prefer to read


to this article. It'll be the first link in the description so you can read along. As we go through


it, I'm just going to riff on it. So let's get started. What you missed in September as a machine


learning engineer. So at the top, I usually put out my work. There was a video I did. I'm not going


to show you the video. I'll just show you three years ago. There we go. There, how I'd start


learning machine learning again. So there's a video if you haven't seen that one, go check it


out. It's got some helpful tips. If you're getting started into machine learning, I spent a lot of


time studying in my bedroom. So it's just basically all the information that I've gathered from


learning machine learning over the last three years in a, or how I would do it again. So the things


that I've learned that I would improve on. Also another one here. So this is not my own work, but


you could sort of say that it is. It's from Siobhan. He sent it to me. Now this is what I love,


love, love to see. Like seriously. So Siobhan, what he did, he actually sent me an email saying,


do you mind if I do a data science project or a data analytics project on your YouTube data?


And I'm like, sure, go for it. That'd be amazing. And he actually, he didn't ask for permission. He


was more like, I've already done this. Here's what I've done. So Siobhan, big shout out to you.


And this is his first data project, exploring Daniel Burke's YouTube channel, part one. And so


a whole bunch of different data analytics here using the, I believe it's the YouTube API. Yeah,


there we go. YouTube data API. Can you see that? We'll zoom in a bit. So a lot of cool things that


you can pull from that. I even learned a few things about my own YouTube channel here.


So go check this out, give Siobhan some, some support. There is nothing more than I love seeing


than someone getting after their own thing, whatever project it might be. And so this is


a little bit of tip for you. If you're working on your own stuff, you might look at something like


this, or you might look at anyone else's work and you might think, wow, there's no way I could do


that. But what you have to realize is that everyone starts learning whatever it is at their own point


in time. Everyone has to be a beginner at some point. And you might be thinking, oh, I'm not


going to share this because it's not that good or whatever. Right? You need to get that mentality


out of your mind. Because I know what it feels like. I had to tell myself that you know what,


I'm just going to create this, regardless of what other people think, because I think it would be


helpful. If you're ever stuck, create something that would help your past self. So what would you


have liked to have known six months ago, six months ago, that's the type of stuff that I,


I like to create. So again, big shout out to Siobhan, make sure you check it out and show him


some love. And if you have something you want to share with me something that you've worked on,


please send me an email. It's not hard to find it's on my website. I'd love to share it in a


future video slash article of machine learning monthly. Anyway, let's get into the best from


the internet. So in the article, if you want to read along, I've got a little what it is slash


why it matters. We're not going to read them out verbatim in these videos, I'm just going to sort


of talk about it. So that way you sort of it's not just repeating what exactly is in the article.


Now, advanced NLP with spaCy. So if you're not sure what spaCy is, let me just show you spaCy.io.


So spaCy is an industrial strength natural language processing library built in Python. The


actually underlying code is Cython. So CPython. It's really, really fast. When I was working at


the machine learning company I was working for, we we tended to use the same code as


spaCy.io. So it's used by a whole bunch of different companies for natural language processing


tasks. We would start with spaCy as a as a way to tokenize text. Tokenize text is take some natural


language like this and turn it into turn it into numbers. So spaCy has had a massive amount of


upgrades over the past few years. And now if you want to learn how to do NLP, you can go to


spaCy.io. Now if you want to learn how to do NLP with it, there's a course here. It's completely


free from the founder or one of the founders of spaCy. So chapter one, you'll find words, phrases,


names and concepts. You'll learn how to do large scale data and analysis with spaCy processing


pipelines, training a neural network model. And it's by Inez, I believe her name is yeah, Inez.


If I'm pronouncing that wrong, who is one of the core developers spaCy and co founder of Explosion,


one of my favorite AI companies actually, Explosion.ai. Make sure you check them out.


I take inspiration from from this company. So if you want to learn NLP, I would highly suggest at


least knowing some spaCy. If you're if you're doing NLP in Python, I would say spaCy is a


requirement. So if you want to get familiar with it, check out this free course. That's number one.


Number two, speaking of natural language processing. This is a beautiful article that I


came across, which is it compares essentially three different ways of so NLP natural language


processing, the first major step in any NLP project is converting text into numbers. So often referred


to as tokenization. And there's a few different ways that you can do it. So if we zoom in here,


so this article is brilliant text classification with NLP, TF IDF, verse word2vec verse BERT.


So TF IDF is term frequency, TF IDF, inverse document frequency. I believe what it does,


it what does it do? Where's the actual formula? So basically, it takes into account the count,


how many times a word appears in a set of texts. So suppose we'd use the word suppose, it's going


to count how many times the word suppose appears in here. And then I don't know the exact formula


off by heart, I usually just call the function. It figures out how the ratio of that that term


across the entire document corpus. So if we had 100 Wikipedia articles, how many times does suppose


appear in each article? And then how many times does appear across all of those articles? I may


be butchering that, but that's why I'm linking this article for you. Word2vec creates a word


embedding of words. So it turns each word into a vector. And then BERT also creates a word embedding


a word embedding, but it's usually now these days referred to more so as a language model.


So if you want to learn how each of these different ways to tokenize text work, I would


definitely read through this article in full if you want to see the reason I love this article


is because it hits it hits a number of different points. Not only does it talk about the theory


behind something, it shows you coded examples. So they're my favorite, favorite resources is not


only okay, here's the concept. Here's how you actually do it with code. So getting hands on


with it. Check out that article. We're sticking with a theme of NLP for number three, super


efficient NLP projection. Advancing NLP with efficient projection based model architecture.


So this is from Google. They just last year they published a neural architecture called Prado.


Now if I believe we go to here, it's going to take us to the maybe the paper. There we go.


Projection projection attention networks for document classification on device. So this is


the key word here on device. Whenever you see this, it usually means that whatever machine


learning model they're talking about has some sort of computation constraints. And what I mean


by that is if you imagine a smartphone, its processor isn't as powerful as say you had a


data center and a full blown deep learning computer set up. So although you might train a model on


your deep learning computer or data center, if your end goal is to get that to work on a mobile


phone, like on someone's native device, so they don't need an internet connection, the model just


lives on the phone's storage. It might not necessarily perform as well as it does on the


deep learning PC on the phone, because one has far more compute power than the other.


And so the the graph you really want to have a look at here is this one here, the model comparison.


So I believe they tested it on civil comments data set, which is a classification data set.


Trying to classify civil comments data set provides primarily seven labels, so seven labels,


so multi class classification. And if we go back, the results were the area under the curve for


this new model, which is called PQRRNN. So the beautiful thing about this as well is that BERT,


which is a transformer model, very popular in the NLP world over the past, I think it was released


in 2018. BERT uses a transformer architecture, whereas the cool thing is that PQRNN uses a


modified RNN, so recurrent neural network. And the main figures here you want to pay attention to is


if you look at the model size and the log scale, BERT has 440 million parameters, you can just


think of parameters as patterns or numerical patterns that a model has learned, whereas PQRNN


has 1.3 million. And if we look here at the model size, PQRNN is over just over one megabyte,


and BERT is getting close to a full gigabyte. Now a little quick story here is we were working


on a machine learning project, sorry, an NLP project to classify insurance claims. So whether


or not it was a car accident, and whether or not the person submitting the claim was at fault,


or not at fault, based on the text they said of the description. So say for example, I backed my


car out into a pole in the car park. In that case you would want the machine learning model that we


were training to classify that claim as being it was that person's fault, so they have to be charged


the excess amount. And we tried to use BERT, and BERT was getting great results for classifying the


text, but the problem was when we tried to deploy it in practice, it was far too large a model, so


it was almost defeating the purpose. Even though in our Jupyter notebook that we were getting these


amazing results, something like 98% accuracy, when we tried to deploy it, we found that it was taking


far too long to process these claims. And if you imagine when I was processing the claims, I'd do


it on just one or a dozen examples at a time. But this insurance company that we were working with


had to process upwards of 3000 per day. And so if they're waiting for them to come through,


well then it's not a great experience overall if BERT is taking forever to process these.


So we ended up going with a smaller model that performed almost just as well, but it worked far


better in practice because it was a small model. And now if this PQRNN is performing again almost


as well as BERT, but is something like 300 times smaller, well then that's pretty good, especially


when you want to work on device. So check out that article for some pretty cool advancements in NLP


for on device. Now this is a really cool one, machine learning from scratch book. As I said,


nothing impresses me more than seeing someone share their work. And what I believe it's by


DA Friedman, I'm not sure what their first name is. Where is it? I can't find it. But shout out


to DA Friedman, 97. Is that essentially, I've done a little bit of background research on what


this book essentially, this person's a student studying machine learning, and they decided to put


all of their knowledge, the things that they'd learned into a book. And the book, the beautiful


thing is, I believe it is made with Jupyter book. Yeah, powered by Jupyter book. So you can write a


whole book using Jupyter notebooks. How cool is that? And the thing is, another reason why I love


this book is because it's got, not only does it have some really popular and useful machine


learning algorithms here, it's got three parts to it concept. So it shows you here the model


structure, this has got all the math, all the theory behind the model. And then it has construction.


So how would you write it in code? And then finally, so like from scratch, like this is coding


ordinary linear linear regression from scratch in Python, and then an implementation. So scikit


learn, of course, has the linear regression algorithm implemented. So it goes through


the from the very foundations using pure math, and then writing the code of how to build that model


with Python, and then implementing it with with an existing library. So this is a phenomenal resource,


and I believe there we go, Denny Friedman. So big shout out to Denny Friedman, make sure you check


out the ML from scratch book, I definitely I'd actually bookmark this page. What you could do is


if you're learning machine learning is spend 25 minutes or half an hour a day just reading through


here. And when you get to the code parts, go through the whole thing and write out the code by yourself.


So just write out just copy this, don't copy and paste it, write it out by yourself. That would be


that would be a really cool six week project is to just re implement this entire book. Seriously,


copy it. And if you want the framework for that go to mrdberg.com slash 42 days. This is what a


six week project is. Spend a week on each of these. Where is it? Where's the book?


Spend a week on each of these and just re implement them. Give that a try. So yeah, big shout out to


Denny Friedman, inspiring work, my friend. Check out the machine learning from scratch book. It's


definitely on my my list of top machine learning books. Now we come back. Number five, speaking of


learning the math for machine learning. This one this one wasn't published in September 2020. So


we're going against the rules. So we're going against the rules. So we're going against the


trend here. But that's what I said. Again, I like to share content that is evergreen. So things,


of course, latest state of the art research. But when it comes to fundamentals, some of the things


that are 10 years old are still well and truly relevant today. So a lot of people email me asking,


Daniel, how do I learn the math for machine learning? And the truth like, there's no like set


real way to learn it. One of my favorites is, is when you run into a roadblock. And you're writing


some code. So I'm code first, I like to see things being run, I do put a notebook is like basically


built for me. Because it's it's got the it's got the motto of the data science practitioner built


into it, if in doubt, run the code, see what it does, if it doesn't work, try something else.


I really adhere to that framework. But if you want to learn the math that's going on behind the scenes


behind all of that code that you're running, I would highly suggest reading this article.


So it talks about I love some of the points in here, a note on math anxiety, it turns out that


a lot of people including engineers are scared of math. To begin, I want to address the myth of being


good at math. So if we come in zoom right in so you can see this. The truth is, if you're


good at math, you can see this. The truth is people who are good at math have lots of practice doing


math shock. As a result, they're comfortable being stuck while doing math, a student's mindset as


opposed to innate ability is the primary predictor of one's ability to learn math, as shown by recent


studies. And of course, you can check out those studies if you want. But that's it's all about


yourself. See a lot of the time the people asking me, Daniel, how do I learn math machine learning,


the answer they're really looking for is that if you want to learn something, you can, if you spend


enough time putting in the practice, testing yourself going over concepts, you'll start to


figure it out. Imagine when you were three years old, and you could barely walk, you could barely


talk, what did you do? You just kept practicing. And now I'm talking without even thinking here,


I just go for a walk down the street, and I don't even have to think about moving my legs. So math


treat it like another language, just as you were like learning to, learning to code or learning to


talk. At the beginning, you're probably terrible at it. Right? And even now, I still say some things


out loud, and they don't really make sense. And same with code, I'll look at a body of code and


be like, I don't know what's happened there. But with enough practice and going over it, you start


to build that confidence with going forward or running into a problem and getting stuck. Or you


know, hey, at least I've got this experience here. I've come up against one of these problems before,


I can figure it out. And there's another one, I think somewhere down here. I really like these


four questions. So define your system, what are the inputs outputs of your system? How should you


prepare your data to fit your system? How can you construct features or curate your data to help your


model generalize? How do you define a reasonable objective for your problem? So this is amazing.


Learning math as you need it. That's my favorite approach. Anyway, go check out the article highly


recommend if you want to figure out or have some sort of idea of how you should approach


math for machine learning. Number six, speaking of beautifully written articles, this is another one


that I think is going to be evergreen. So unpopular opinion data scientists should be more end to end.


So where let's find the definition of end to end. So we know what we're talking about here. So we


have some different definitions here of a generalist focus on machine learning. So let's


look at this. So let's say we have a generalist focus on roles, product manager, business analyst,


data engineer, data scientist, machine learning engineer, full stack in a unicorn.


There we go. And end to end data scientists can identify and solve problems with data to deliver


value. I'm biased. I really do like this approach. So I agree with this this opinion.


To achieve the goal, they'll wear as many or as little hats as required. So this is the main one here in bold.


Identify and solve problems to deliver value. And what the article goes through is it talks


about some different points here. There's a big one here. Yeah, here we go. For some data


scientists, it can lead to increased motivation and job satisfaction. So what end to end means is


imagine you've got a machine learning project, and you need a way to collect data, you need a way to


model data, and you need a way to deliver the the model slash service that you've built based off


those data, you have three points data collection, data modeling, data, model deployment. And so what


the article is arguing is that you should have knowledge of, of at least all of them, essentially,


if you want to be a practitioner. And I agree with that. And I also there's the article talks about


the other side, right, the other approach is just to be an absolute expert at collecting data and


storing it in an efficient way, in efficient way. And then you might be an absolute expert, if you've


got data that already exists, and you just know how to model it really, really well. That would


be something like a Kaggle competition expert or something a data set already exists, you're just


really good at modeling. And then finally, there's arguments for just being really good at deploying


a service. So that's more the DevOps side of things. And now, what being end to end offers you


is autonomy, mastery, and purpose. Again, these are points from the article. So by being able to


solve problems independently, so this is if you had end to end knowledge, instead of waiting and


depending on others, end to end data scientists are able to identify and define the problem,


build their own data pipelines and deploy and validate a solution. Mastery in the problem


solution outcome from end to end, they can also pick up the domain and tech as required. So again,


learning the things you need to learn as you go. And purpose by being deeply involved in the entire


process, they have a more direct connection with the work and outcomes, leading to an increased


sense of purpose. So I remember another story from the fields is that I was probably one of the


experts who could do the modeling part. So once the data was there, I think this is where a lot


of people start actually is if you're learning machine learning usually start with an existing


data set, you build a model. And then that's you just keep doing that over and over and over again,


where you sort of miss out is the collecting data phase and the deployment phase. So in another


project, similar to the NLP one that I talked about, is that I built this incredible model.


But if I wanted to deploy it, I had to wait for another person on my team who might have been


working on another project to be able to do it. Because we had I was a machine learning engineer,


but I was probably more so a, how would you say, a machine learning modeler. And then we would have


would have a DevOps engineer who would deploy our machine learning models into practice so that we


could show the clients what we built. And so I remember, we had a few projects on the go,


I built this amazing model, but then I had to wait a couple of days to get it deployed and show


show the client a proof of concept, because the DevOps engineer was working on another project.


And so if I had more end to end skills, which I kind of do now after more practice, I would have


been able to not only build the model and then deploy it myself, myself and not have to wait for


that buffer of waiting for someone else. So basically, being end to end helps you to remove


roadblocks. Now, if you want specialists, yeah, of course, if you want to build something like GPT3,


you want to be really specialized in say, a language model or something like that.


The best way I love this is to pick up, pick it up via learning by doing. If you ask


most software engineers or machine learning engineers that I've met, of course, they've


studied the right books and courses, but they've also done their own projects end to end.


So I'm going to leave that here. I highly recommend reading this section,


read the article in its entirety. It's only about a 10 minute read, it'll be well worth your time.


So yeah, great little article there. Big shout out to Eugene Yan. And so Eugene has written this


with experiment experience as being an applied data scientist at Amazon, at Alibaba,


look at that. There we go. Great article. That's something that's got, I think that's an evergreen


article. So you could read that in a year's time and it'll still be valuable.


Number seven, using GitHub actions for MLOps. Speaking of building an end to end machine


learning system, GitHub actions is probably a tool you might want to look at, especially if you're


hosting your code on GitHub. What GitHub actions allows you to do is, where's a diagram?


Here we go. Say you updated, you did a pull request to your GitHub repo. So you uploaded


some code to GitHub. What you could do is add a comment here and then it automatically triggers


GitHub actions to do some sort of ML workflow. And in this case, it's testing the new model.


So model statistics are dropped back into the PR with links back to the experiment tracking system.


So that's really cool. So say for example, you updated your model, you've got a new model,


you're like, oh, I changed a few parameters on this new model. I'm trying it out. I'm going to


upload it to GitHub, push it to the model.py file. GitHub actions automatically watches that model.py


file. And then as that model.py file gets triggered, say you add this comment here,


I'm sure you can adjust these steps as needed. If you want to test that new model.py file,


type in the comment, run full test. It kicks off some tests on your new model. And then it


drops into the PR scores of that model and whether it's better or not. And so if it is better,


if it runs faster, if it's got a higher validation accuracy, well, then you might want to


merge that new model.py into the master branch or something like that. So really cool tool.


Check out GitHub actions. I haven't personally used it enough myself, but Hamal Hussain is a


machine learning engineer at GitHub, writes some great articles. Big shout out to Hamal.


Here we go. Number eight, deep speed. Again, this is probably, actually, yeah, this does help the,


the, even if you're at a, so this one is, man, I'm sweating. And that's how hot it is in Australia.


And that's how excited I am to go through all these amazing resources, working up a sweat.


That's how you really know that we're getting things, we're going through, we're having fun.


Because when you're working up a sweat, there aren't many times that you work up a sweat that


you're not having fun. So this is from the Microsoft research blog. So deep speed, it's


a framework for extreme scale model training for everyone. The main four points are here.


So trillion parameter models. So if you imagine GPT-3, which is probably the best language model


that's ever been built so far, was 175 billion parameters. So this is saying that we're now,


we're now making it, Microsoft basically, we're now making it possible for trillion parameters,


an order of magnitude more than GPT-3. We're now making it possible to start training those type


of models, which will be absolutely crazy. 10 times bigger model training on a single GPU with


zero offload. So that's another really big point here is that, say you're someone like me, you


don't have access to massive amounts of cloud compute. I do, but it would cost a lot. I have


a deep learning box over there with a single GPU on it. So if I wanted to train a big model on it,


I would have to train a 10 times bigger model. And I believe it's, yeah, models of up to 13


billion parameters without running out of memory. So that's massive. Powering 10 times longer


sequences and six times faster execution through deep speed sparse attention. So if you imagine


you have a natural language processing model, you can process up to 10 times longer sequences. So


you say you had a sentence of 100 words, and you can process up to 10 times longer sequences. So


you say you had a sentence of 100 words, you could now process 1000 words with up to six times


faster. So that's pretty insane. And then one bit atom, Adam. So Adam is an optimizer. So if we go


TF, Kerrers, optimizes Adam. Adam is very popular. I find Adam just works for almost any neural


network. Put a comment below what your favorite optimizer is. Mine's Adam. But so essentially


what this does is it one bit Adam, if you're using the atom optimizer makes it more efficient. So


you can get five times communication volume reduction. So just some massive things. Check out


this article. It's actually pretty in depth. I read through it and kind of, there's a lot of things


in there that I didn't understand. But if you're, especially if you're wanting faster compute on a


single GPU, or if you're working on really large scale models with PyTorch, you probably want to


check out this Deep Speed article. So shout out to the Microsoft research team. This is another one,


number nine, the ultimate guide to object detection from the team at RoboFlow. I actually


really like what RoboFlow are doing. So this is a great example of how I would build an AI startup


is so their product is basically, I don't want to say basically because that sounds like I'm


simplifying it. Let's just go to product overview. RoboFlow, so it helps you pre process your image


your image data. Yeah, transform raw images into trained computer vision model in minutes. So it


helps you not only store your images, pre process them, it now helps you train computer vision


models. So I think their tagline was making computer vision accessible for anyone. Yeah,


we help innovators like you apply computer vision. And so what this guide is, is this is the link


that I was sharing in machine learning monthly is an ultimate guide to object detection October 2020.


So look how fresh this is, this is fresh off the press. This is what you get here with machine


learning monthly, you get the freshest stuff. So this is going to be if you're if you've done


image classification before, assigning a label to an image, object detection is a little bit


different because not only does it have to recognize the image, it has to figure out


things like the edges of the boxes where that boat is so sure you could take image classification,


this image and say it's the photo of water, it's a photo of forest, but where where are the boats


or something like that, that's what you're trying to figure out with object detection.


And so basically, it turns into a regression problem. Regression, because you're predicting


because you're predicting a number, you're predicting the point of where, where is it? Somewhere in


there, there we go, anchor boxes, that's what it's called. So you're trying to predict each of these


corners around not only where, where they are in the image, but or sorry, what the image is of,


but where the corners should appear. So check this out. It's got a whole bunch of different


things that you can do for object detection, a whole bunch of different models for object


detection and how to train them and how to prepare your data, how to label data.


It's just amazing. And so this the case study here for Robo flow is not only are they building


an amazing AI tool, they're teaching people how to use it. And not only just how to use the Robo


flow tool, but how object detection works. So that's what I would do if I was building an AI


startup. Not only would I build a product, I would also educate people on how to use it.


So shout out to Robo flow, amazing product. And thank you so much for your incredible guides.


Speaking of AI in the wild, how about AI in agriculture? So this is another phenomenal


article, just sometimes it's hard to understand like where if you're learning these things,


like say, for example, in your bedroom, studying in the Jupiter notebook, it's hard to understand


like where, where the code that you're writing can be used. And so this is a great example of


how computer vision models are being used to detect weeds. So if you imagine you've got


this tractor going across this whole field, and I'm sure this this field is not sorry,


this photo does not show probably even a quarter or a 10th of the field that this tractor has to


drive over. Now, what happens is as it goes over, it's got cameras on the I'm not sure what you'd


call it, but let's just say cameras on the tractor. And instead of spraying the actual crops with


pesticide, it sprays only the weeds using the computer vision. So the benefit of this is two


fold. Not only do people like us the consumers, we I don't really want pesticides on my food or on my


my cotton on the t shirts that I'm buying. So the twofold here is that the pesticide doesn't go on


the actual plant. It only goes on the weed. So again, that's saving us as end users from not


having pesticides on our on our food, as well as in our clothes. But it also uses less pesticide,


because instead of just spraying the whole field and just going gung ho with with pesticide,


it's really targeted on where it needs to go. And so I can't remember the exact. Oh, yeah. And then


this article, by the way, talks about how the they set up the pipeline to train the model,


what sort of tools they're using, weights and biases, Pytorch, shout out to weights and biases,


amazing tool, check that one out. I'm not sure there was somewhere in here where they said how


much they saved reproducible models. Yeah, this is just a great case study on how machine learning


can be used in practice. So if you're interested in not only just AI and agriculture, but just how


how you go about implementing a machine learning model into into the real world, read this case


study of an article and just copy it just just build your own system that's like this. But yeah,


there's somewhere in there of how much they improved. I'm not sure how much they saved,


but it was a very significant amount saved on pesticides every, every year or every sort of


timeframe because they weren't spraying the whole field, they were only targeting just the weeds.


Well, looks like we're done. So again, as always, September was another massive month for for


machine learning. I hope you like some of the resources there. I'm not sure what my favourite


one was because I mean, they're all so so great. I think I think it was the end to end article,


just because it's it's a whole bunch of confirmation bias. I really like that mentality


of just being able to someone who who goes and looks at a project and you know what, I can,


there's a problem here. And I'm going to try and build the whole solution myself, just give it a go.


So that's probably my favourite one. Leave a comment below on what your favourite one was.


And if you'd like to actually, I'll just leave it leave my cell phone here. If you'd like to receive


that article in your inbox, it comes out about the first of every month. And if you'd like to


receive that article in your inbox, it comes out about the first of every month. Make sure you go


to the link in the top of the description and sign up there completely free, just some really cool


resources and machine learning. But otherwise, if you have any resources that you'd like to see in


a future article slash video, because we're doing articles and videos now, feel free to either leave


a comment or send it to me in an email or something like that. But otherwise, as always, keep machine


GET-TO vs. HAVE-TO

Yo, what's going on everyone? It's episode 107 of the Daniel Burke Show.


GET-TO vs. HAVE-TO

Thank you so much for tuning in, thank you so much for watching.


GET-TO vs. HAVE-TO

Wherever you are, I hope you're having an amazing day.


GET-TO vs. HAVE-TO

It is the 14th of July 2017, here in this lovely city of Brisbane.


GET-TO vs. HAVE-TO

Well that's where I am at least, wherever you are.


GET-TO vs. HAVE-TO

I hope it's a lovely day as well.


GET-TO vs. HAVE-TO

And it's about 7pm, I just finished a workout so that's why, I know,


GET-TO vs. HAVE-TO

my muscles might look a bit bigger than usual, I got a lot of blood through them.


GET-TO vs. HAVE-TO

But I'm in a good, you know when you post workout, you're getting a really good feeling?


GET-TO vs. HAVE-TO

That's why I decided to record a podcast.


GET-TO vs. HAVE-TO

Right, and so I think I've talked about that in the past, in the last episode actually,


GET-TO vs. HAVE-TO

about public speaking, is that if you want to sort of have an engaging talk,


GET-TO vs. HAVE-TO

or you want to use some energy to talk, rather than eating or sitting down before talking,


GET-TO vs. HAVE-TO

try to do some sort of activity, whether it be like a 20 minute walk,


GET-TO vs. HAVE-TO

or whether it be, I don't know, a workout like I just did.


GET-TO vs. HAVE-TO

Something fun to get you sort of in a good mood, and then, I don't know,


GET-TO vs. HAVE-TO

those nerves of that talk or meeting someone new, will more than likely go away.


GET-TO vs. HAVE-TO

So if you're going on a first date, how about instead of going to dinner or something,


GET-TO vs. HAVE-TO

ask to go for a walk? Like, I think that would be cool, I think that would be fun.


GET-TO vs. HAVE-TO

And plus, walks are fun, you get physical activity, and you get conversation with someone,


GET-TO vs. HAVE-TO

or even just on your own.


GET-TO vs. HAVE-TO

Yeah, but where is this going on to?


GET-TO vs. HAVE-TO

Oh yes, so it is episode 107, and I was almost about to put this off to tomorrow.


GET-TO vs. HAVE-TO

It is Friday, so I said I would release an episode every Friday,


GET-TO vs. HAVE-TO

and I'm not going to do that, I'm not going to put it off to tomorrow, I'm going to do it now.


GET-TO vs. HAVE-TO

And I was about to have dinner after working out, but I decided, no,


GET-TO vs. HAVE-TO

I'm just going to get this workout, this, sorry, get this podcast done,


GET-TO vs. HAVE-TO

and then I'm going to have some food, and then I'm going to call it a night after that.


GET-TO vs. HAVE-TO

But, what is this episode about?


GET-TO vs. HAVE-TO

Well, it was originally going to be about what I've learned from Ray Dalio,


GET-TO vs. HAVE-TO

however, I'm going to save that for a later date, so look forward to that in the future,


GET-TO vs. HAVE-TO

I'm going to put that into a blog post of some sort as well,


GET-TO vs. HAVE-TO

things I've learned from his book and whatnot.


GET-TO vs. HAVE-TO

But instead, I did read an awesome passage from this book today,


GET-TO vs. HAVE-TO

and for those on the podcast who can't see this, it is called The Daily Stoic,


GET-TO vs. HAVE-TO

366 Meditations on Wisdom, Perseverance, and the Art of Living,


GET-TO vs. HAVE-TO

by Ryan Holiday, who's the best-selling author of The Obstacle is Away,


GET-TO vs. HAVE-TO

and Stephen Hanselman.


GET-TO vs. HAVE-TO

So, this book, I read every morning getting up, I'm actually about a week behind,


GET-TO vs. HAVE-TO

so I've got a few, I've got a few days to catch up on,


GET-TO vs. HAVE-TO

but every morning I get up and I read a page of this, or I might read a couple of pages,


GET-TO vs. HAVE-TO

depends how many days I've gone without reading it, sometimes I forget.


GET-TO vs. HAVE-TO

But it is called The Daily Stoic for the reason, for a reason,


GET-TO vs. HAVE-TO

because it has 366 little passages from people such as Seneca, Epictetus, Marcus Aurelius,


GET-TO vs. HAVE-TO

for those of you who don't know, these guys were all stoic philosophers back in the day.


GET-TO vs. HAVE-TO

Marcus Aurelius was actually an emperor of Rome,


GET-TO vs. HAVE-TO

and he has a really popular book called Meditations,


GET-TO vs. HAVE-TO

which is a collection of his journal entries that would just be his thoughts at the end of the day,


GET-TO vs. HAVE-TO

writing down being the emperor of Rome, and that's amazing,


GET-TO vs. HAVE-TO

like that's 2000 years old and we have access to that text today.


GET-TO vs. HAVE-TO

But I thought I'd share a passage like this one,


GET-TO vs. HAVE-TO

every couple of days, like they're all great, the passages in here,


GET-TO vs. HAVE-TO

but every few days like one will just stand out,


GET-TO vs. HAVE-TO

and I don't know whether it's like confirmation bias,


GET-TO vs. HAVE-TO

like you know you read your horoscope of whatever time of year it is,


GET-TO vs. HAVE-TO

and something in there will stick out to you, but this is similar,


GET-TO vs. HAVE-TO

I suppose the effect is similar with this book, right?


GET-TO vs. HAVE-TO

But today this one hit home with me, because my to-do lists have been getting out of hand,


GET-TO vs. HAVE-TO

right? Like I've got this, you can't see it, but it's on the whiteboard there,


GET-TO vs. HAVE-TO

and there's, I like to keep things to like three things, and right now I've got nine,


GET-TO vs. HAVE-TO

nine for today, and I didn't even allocate any time to them,


GET-TO vs. HAVE-TO

it's just like do all of these today, and I don't know,


GET-TO vs. HAVE-TO

when you sort of have a list that's that long, can cause unnecessary stress,


GET-TO vs. HAVE-TO

right? And that's what I'm all about, I'm about, anyone who knows me,


GET-TO vs. HAVE-TO

I'm like a chilled out, calm guy, I don't like creating stress,


GET-TO vs. HAVE-TO

I like creating calm, I like being the one who's calm in all types of situations,


GET-TO vs. HAVE-TO

but the things that I get stressed about are more, and I think this, this is probably,


GET-TO vs. HAVE-TO

I don't know, let's just talk about me, I don't know about other people,


GET-TO vs. HAVE-TO

but for me the things I get most stressed about are literally the things that I've created myself,


GET-TO vs. HAVE-TO

like you have a choice in every moment, like whether you want to be stressed or calm or whatever,


GET-TO vs. HAVE-TO

and every single time like I've created a stressful time,


GET-TO vs. HAVE-TO

in the past few years, I've been very lucky to sort of live a relatively sheltered life,


GET-TO vs. HAVE-TO

I don't, I haven't had any major tragedies in my life as of yet,


GET-TO vs. HAVE-TO

I do would, but most of the stress has been created by myself,


GET-TO vs. HAVE-TO

whether it be self-imposed deadlines, whatnot, workouts, losing weight, etc, etc,


GET-TO vs. HAVE-TO

whole bunch of things, diet, you name it, it's all been there,


GET-TO vs. HAVE-TO

but it's completely unnecessary, right?


GET-TO vs. HAVE-TO

It's all self-imposed, and so that's why this, and a lot of it comes from to-do lists,


GET-TO vs. HAVE-TO

like having been too long and me not considering how much time I've actually got in the day,


GET-TO vs. HAVE-TO

to sort of get all these things done, right?


GET-TO vs. HAVE-TO

We can create, like the to-do list is never-ending,


GET-TO vs. HAVE-TO

I think it's Cal Newport who talks about in his book Deep Work,


GET-TO vs. HAVE-TO

rather than sort of having a to-do list, schedule time to do activities,


GET-TO vs. HAVE-TO

so for example, say you want to work on Project Z,


GET-TO vs. HAVE-TO

or say you want to work on Project Schedule,


GET-TO vs. HAVE-TO

two hours of time to work on that project,


GET-TO vs. HAVE-TO

rather than having the goal of finishing that project or something like that,


GET-TO vs. HAVE-TO

so you can control how much time you dedicate towards something,


GET-TO vs. HAVE-TO

but you can't control whether you finish a big task or not.


GET-TO vs. HAVE-TO

So that actually worked really well for me,


GET-TO vs. HAVE-TO

I was trying to put out a blog post this week,


GET-TO vs. HAVE-TO

and instead of doing it all in one sitting,


GET-TO vs. HAVE-TO

which I've been doing lately when I do writing,


GET-TO vs. HAVE-TO

I just did it an hour each morning, and I posted it today,


GET-TO vs. HAVE-TO

and it took a bit longer than I thought, but it's out in the open,


GET-TO vs. HAVE-TO

it's out in the wild now, it's about How I'm Learning Deep Learning in 2017,


GET-TO vs. HAVE-TO

part three of the series, so I'll link that in the description by the way.


GET-TO vs. HAVE-TO

But let's get to this book, right?


GET-TO vs. HAVE-TO

Because this is the passage that I wanted to talk about.


GET-TO vs. HAVE-TO

So I'll just read the quote to you, so it's July 3rd,


GET-TO vs. HAVE-TO

so as you can see, it's 14th today, so I'm about 11 days behind.


GET-TO vs. HAVE-TO

So, turn how to into get to.


GET-TO vs. HAVE-TO

The task of a philosopher, we should bring our will into harmony


GET-TO vs. HAVE-TO

with whatever happens, so that nothing happens against our will,


GET-TO vs. HAVE-TO

and nothing that we wish for fails to happen.


GET-TO vs. HAVE-TO

Epictetus, Discourses 2.14.7.


GET-TO vs. HAVE-TO

And so, a lot of the quotes from the Stoic philosophers,


GET-TO vs. HAVE-TO

like what I just read, are kind of hard to interpret.


GET-TO vs. HAVE-TO

That's why I really like this book,


GET-TO vs. HAVE-TO

is because at the top you've got the quote there


GET-TO vs. HAVE-TO

of the actual philosopher from history, like Epictetus Marcus Radius,


GET-TO vs. HAVE-TO

and then you've got this paragraph here of what the authors,


GET-TO vs. HAVE-TO

Ryan Holiday and Stephen Hanselman,


GET-TO vs. HAVE-TO

have sort of translated into their own words,


GET-TO vs. HAVE-TO

and sort of brought it into the 21st century in terms of giving examples,


GET-TO vs. HAVE-TO

and in English that's more readable for today's age.


GET-TO vs. HAVE-TO

And so, something really I highlighted, as you would have seen on the video,


GET-TO vs. HAVE-TO

I highlighted a line of text because it really stuck out to me.


GET-TO vs. HAVE-TO

So, a long to-do list seems intimidating and burdensome.


GET-TO vs. HAVE-TO

All these things we have to do in the course of a day or week,


GET-TO vs. HAVE-TO

but a get-to-do list sounds like a privilege.


GET-TO vs. HAVE-TO

And the things we're excited about, the opportunity to experience.


GET-TO vs. HAVE-TO

This isn't just semantic playing,


GET-TO vs. HAVE-TO

it's a central facet of the philosopher's worldview.


GET-TO vs. HAVE-TO

It's a central facet of the philosopher's worldview.


GET-TO vs. HAVE-TO

And so, I'm even having trouble reading that right now,


GET-TO vs. HAVE-TO

I'm sorry, I'm too amped up after the workout.


GET-TO vs. HAVE-TO

But, essentially it said,


GET-TO vs. HAVE-TO

swap to-do lists for get-to lists.


GET-TO vs. HAVE-TO

And I've spoken about this in the past, like no more to-do lists and whatnot,


GET-TO vs. HAVE-TO

I've put it on my blog and I'm sort of contradicting myself and all this sort of stuff.


GET-TO vs. HAVE-TO

But, instead of a to-do list, a real quick concept to think about is a get-to-do list.


GET-TO vs. HAVE-TO

So, rather than thinking about I have to do the dishes,


GET-TO vs. HAVE-TO

think about you get to do the dishes, right?


GET-TO vs. HAVE-TO

So, you got to eat a meal off a dish and now it's dirty.


GET-TO vs. HAVE-TO

So, that's, I don't know, find the benefit in everything, right?


GET-TO vs. HAVE-TO

So, for example, I was putting off this podcast and it's like,


GET-TO vs. HAVE-TO

rather than me thinking of it, I have to do this podcast today.


GET-TO vs. HAVE-TO

Which of course I don't, right?


GET-TO vs. HAVE-TO

I've self-imposed this Friday deadline for me every single time.


GET-TO vs. HAVE-TO

I get to do a podcast, right?


GET-TO vs. HAVE-TO

I get to record this video and I get to record this audio and put it out into the world.


GET-TO vs. HAVE-TO

60, 70 years ago, like that, well even less, 20 years ago,


GET-TO vs. HAVE-TO

this would not have been possible.


GET-TO vs. HAVE-TO

Like, to get this sort of set up, like to get a YouTube video up,


GET-TO vs. HAVE-TO

to get a, have my own radio show,


GET-TO vs. HAVE-TO

that wasn't even conceived of in the past 20, 30 years,


GET-TO vs. HAVE-TO

let alone like when this stuff was written.


GET-TO vs. HAVE-TO

So, I don't know, that's the simple concept I've been playing with.


GET-TO vs. HAVE-TO

Tell me what you think about it.


GET-TO vs. HAVE-TO

I highly recommend this book, by the way.


GET-TO vs. HAVE-TO

Get it, I got the hard cover because it's something I want to be reading every day.


GET-TO vs. HAVE-TO

Stoic philosophy has definitely changed my life.


GET-TO vs. HAVE-TO

And look, with everything that I say, make sure you apply it to your,


GET-TO vs. HAVE-TO

well, if you do choose to apply it to yourself,


GET-TO vs. HAVE-TO

make sure you decide if it's true or not in your world


GET-TO vs. HAVE-TO

because the things that I share, and I think this goes for everything,


GET-TO vs. HAVE-TO

like if you learn something new, if you listen to someone else,


GET-TO vs. HAVE-TO

be sure to try it out and find out if it's true for you first, right?


GET-TO vs. HAVE-TO

Because if I'm preaching all this stuff, well not preaching,


GET-TO vs. HAVE-TO

but I'm just sort of, I like to share what I've learned


GET-TO vs. HAVE-TO

and if it's fun, I like to put that out into the world,


GET-TO vs. HAVE-TO

which is what I'm doing here with this YouTube video and podcast and whatnot.


GET-TO vs. HAVE-TO

And so, before you, I don't know, implement it into your life,


GET-TO vs. HAVE-TO

make sure you just think about it.


GET-TO vs. HAVE-TO

Is it true to you? And that goes for everything.


GET-TO vs. HAVE-TO

Maybe that can be a double fold lesson.


GET-TO vs. HAVE-TO

Just ask everything, ask the question of everything, sorry.


GET-TO vs. HAVE-TO

Is this true?