but in our case we don't want to always go to the end we'd rather explore more into like more branches than always go to the end and this is where the value comes in so at some point you simply say now i'm deep enough and now i'm going to ask my value v now there are slight differences with respect to alpha go and alpha 0 and so on but they all have in common that they


estimate the value of the intermediate nodes using this v model from over here um i have v s v was green so they use this v model from over here to estimate at a certain depth so v learns to look into the future so everything that can happen from here and it estimates and it says well from here you maybe have a you know a 0.5 value or maybe a negative 0.7 and so on so v learns to assign these

Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess (Paper Explained)


values to situations to states which are these nodes right here and p learns to suggest things to expand that's alpha zero and then at the end if you've expanded the tree enough and estimated well then you have a pretty good idea what's going to happen in each of the branches that you considered right in each of these branches you look into the future um from you here you look into the


future here look into the future by doing this pv play and after one second after you've done you know a couple of hundred or thousand or however many uh looks into the future then you have a pretty good idea for each of the top level actions what's going to happen in the future and you can simply pick the one that has the best future for you according to your own model so that's


what alpha zero does not so this is how you combine planning and neural networks you want to do planning but you can't because you can only go so deep so you use neural networks to first of all reduce the number of branches you consider because the neural network will tell you which ones are worthy to even look at and second of all you don't always have to plan to the end


because you can simply ask your neural network how much an intermediate state is worth in expectation and this turns out to be pretty good why don't we do this for every single problem well we do for this we do need a simulator so you may recognize that right here i said we consider all the possible actions that we have and for each action we know exactly what's going to happen


this is only possible like in a board game it's not even possible in like a board game where you have a a die to roll or a card to draw anything that is random there there is a way to include this right here but in this simple formulation we need to know exactly with 100 certainty what is going to happen if we take a particular action so this is only really applicable for


the types of full information board games where we can write simulators that are pretty fast right and even then um even though chess you know has lots of available actions and complications it's nowhere near the complexity of like a let's say a modern video game or even or the real world is is completely out of scope for now for these types of things all right so that was alphago


sorry alpha zero uh which builds on alphago of course and uh the rules of chess that we're going to consider using alpha zero are the following so there's no castling no castling for ten moves pawns can only move by one square forcing a stalemate is a win rather than a draw so you may know this in chess if you do not um checkmate the opponent's king but only


put them put the king in a situation where it cannot move that's called that's considered a draw and i think even in the chess community some people want to consider this a win there is torpedo where pawns can move by one or two squares anywhere on the board and semi torpedo where it's the same but only from the second and the third rank pawn back where pawns can move


backwards and pawn sideways where pawns can move laterally by one squares but captures are unchanged diagonally upwards and there is self capture where it's possible to capture one's own pieces so um there are you know slight slight details here with respect to the 50 move rule and so on but if you if you don't play chess simply consider these are changes minor in a lot of cases


minor changes to the chess rules that make the new rules either a superset or a subset of the original rules but they are going to have quite some changes in for the play and we're going to look at what happens so that's the entire research setup as you've seen it's alpha 0 applied to these new rule sets and under the assumption that alpha 0 will solve these will become master at these


games which we can't verify we can verify in chess because right alpha zero can beat people that have trained chess for all their life we can't verify it here so again this is an assumption so the first thing i want to look at here and this is going to play a little bit into my criticism of this paper it's a pretty cool paper but i do have some concerns right here


is the following uh the following charts so they do they do we don't consider how you train alpha zero let's just say you can train it you know to whatever pretty good performance here is how they evaluate so they evaluate for each variant they do 10 000 games played at one second per move for each different chess current so if you remember as we do our tree search right we expand


the tree according to our p and we estimate the values according to our v and we do this for one second in this first thing so in one second maybe this here is the tree so we have some sort of an understanding of what's going to happen in the future you can imagine if we have more time then we can expand this tree more and get a much more accurate picture of what


happens in the future okay so they do 10 000 games at one second per move but they also in addition to 1000 games played at one minute per move so there's 60 times more time and you can imagine that we'll add quite a number of nodes here and you know if if your p and v would be perfect then it wouldn't matter as much how much time you have as long as you sort of


have enough time but since they're not going to be perfect since they're only neural networks they're not uh god or schmidt hoover um they cannot accurately extremely accurately predict the future so this planning the the more you plan the more you actually look into the future the bigger your tree becomes the better moves you make so on the left you see the distributions


of winds losses and draws for one second per move and on the right for one minute per move so both white and black pieces here are played by alpha zero so it's not alpha zero against something else this is playing against itself and you can see in uh in classic chess it's it's quite it's quite saddening actually um that this game which is is so famous you can see that


in of 10 000 plays 8 820 end in a draw which means that if both players are super duper good and uh and and play you know play against each other it most likely is going to be a draw and this i think is the criticism even in human chess is that it's not really a decisive game in that it ends a lot of times in a draw so one of the motivations here would be can


we find a rule set that is maybe more decisive so that's one of the investigations they do in the paper but you can see that there are actually so if you consider this torpedo chess right here um there it is more decisive as you can see in more times either white or black winds right here um and there are others which are even less decisive like pawn back so


when pawns can move back then uh players may just camp they like move a pawn forward and move it back again and that will lead to a lot of closed plays and so on whereas torpedo makes you move much faster you can advance your pawns much faster and that will probably lead to the end much faster so if you consider this on the right so what changed the rules didn't change


alpha 0 didn't change it simply changed that we now let alpha 0 think for longer and you can see that the decisiveness reduces dramatically so whereas 88 resulted in a draw with one second per move now 98 result in a draw with one minute per move and this is a trend throughout these games and that's also what they say in the text it is to assume that if you let alpha zero plan for even


longer that this trend will continue and ultimately whatever rule set you make the result is going to be a draw um if two two let's say perfect players play against each other which is a bit which is a bit saddening right because um yeah that ultimately ultimately means that all of these rules aren't decisive it's only they're only decisive due to the fact that either


um one or the other players is way better or or that in general that they are not they are not perfect um which is an appeal of the game but there are certainly games that are decisive even though both players are pretty high level i mean think of every every competitive video game um so yes so that's a bit of my criticism all of this all of this needs to be analyzed in the background


that what's actually happening here is that we're dealing with imperfect decision making due to a limit in resources okay and this assumption now is already a little bit invalid right the assumption we made at the beginning why i pointed this out is that alpha zero can solve these games let's say to perfection and here when we analyze the decisiveness and so on it seems to be


purely or largely a factor of um how much time alpha zero has to think about the moves and these two things to me they don't really go go together because we don't know if for a different rule set um you know the training is harder or might take longer and so on or that this exact one second makes a difference or not it's it's just um there are so many variables here and


when you're dealing with let's say imperfect systems that are not trained to the end or evaluated in their full potential you're always dealing with the fact that you stopped each thing at some intermediate point and that intermediate where that intermediate point is can influence the results drastically now here it seems at least the ordering isn't changed by much but um


yeah this is one let's say one criticism the other criticism here uh that that i would have again is the fact that if you consider something like torpedo where you can move much much faster then yes of course uh let's say i don't know is it more interesting that's that's the question right here so they look at a lot of things like decisiveness diversity and so on


but the question is is it more or less interesting to play and i think that's what humans are really after and they're sort of trying to find proxies to this um i would argue if you play something like torpedo the game's maybe much faster and um so you you get to the end faster but also maybe might not be as interesting even though it's it's faster uh because


your the complexity is is less and with respect to the decisiveness here so if you have a game that's faster um you also need to take this to into account because here is another thing that is sort of an arbitrary choice as moves are determined in a deterministic fashion given the same condition diversity was enforced by sampling the first 20 plies in each game proportional


to their mcts visit counts so what does that mean that means that if you run alpha 0 on the same situation on the same tree sorry on the same board position it will always come up with the same move except for parallelism inconsistencies and so on but it will in you know in in a lot of times it will come up with the same move so how do you play 10 000 games because you can just play one


game because each game will be the same because you simply tell alpha zero give me your best move right so it will just play its optimal strategy and all the games will be exactly the same so there's no reason why these should come out different so they enforce diversity by saying okay okay in the first 20 moves of a game we don't actually take the best move


right usually you have you have this distribution at the end of the tree search you have a distribution where you say okay this move right here is clearly the best move i'm going to play this however if this is one of the first 20 moves of the game they say no we need a bit of diversity uh so we're going to sample according to this distribution rather than just play the best one


now this number 20. it's just sort of decided arbitrary right and if you consider something like torpedo it's a faster game so you're faster in opening faster make and you're faster to the end game maybe even though they say well the game length isn't affected this much it could just be that um you're faster in a situation where um you're kind of forced to do


certain moves and maybe the difference in decisiveness here is simply a result of the combination of the faster uh moves in torpedo together with this the fact that they just keep the 20 plies for each game again this is something that you need to consider when analyzing these results right here and there are a number of these choices um right here like the one second or one


minute per move we sample for the first 20 plies before we play the max move that where i think the the results of the study right here they have rather limited interpretability if you if you ask me because um because of these of these choices now of course they're still the results are quite plausible believable and the idea is really cool to explore these rule sets but this


was this is just my criticism right here so we'll go through the rest of the results pretty pretty quickly because a lot of people aren't chess enthusiasts and we'll just pick out kind of the core messages that the paper is trying to get across so here the table again with respect to decisiveness and you can see even uh for so for classic chess it's a white has a 50 this is the empirical


score for white under different game conditions so 50.8 percent means most of the time it's a draw so white wins uh with a probability of 50.8 uh most of the time it's a draw and you see even like the most decisive variant torpedo right here is a 54 only um so they they analyze different defenses and how the decisiveness is with respect to different defenses that are not really popular


under classical chess and the results are interesting if you play chess but i would say they're rather they're kind of aha okay if you do not play chess because they consider individual moves and so on what is an interesting part is um this right here where they look at they look at one move that in classical chess so e4 is a very very um popular opening where you move your e


pawn twice for white and nf3 is not a super popular opening and here they compare this in classic chess and in no castling gesso this thing right here is a histogram and the histogram shows you the log probability of opening sequences when you play the individual moves so what does this mean right here if you play e4 then the distribution is something like this which means that


you have some sequences that have no entropy at all which means that once you play e4 and maybe one move more then it's almost it's almost determined what you have to do according to alpha zero you have like no choice except play these few next moves um however if you play nf3 then alpha zero says look this distribution is much more to the right which means that you have a lot more options here


now again this could be because the move is actually less decisive because the move leads to more balanced more interesting situations where you can continue however you know with many choices it could also be because it's simply alpha zero simply doesn't know as well what to do because it leads to more complicated games and you get to give each move one minute to evaluate alpha zero might


just not be as good in those situations because it leads to more complicated situations if it could search for longer maybe this distribution would shift over here just as well again we don't know because you only give this one second or one minute each time for both um and again this goes under the assumption of alpha zero as this perfect player however back to what they


want to say here if you do this in no castling chess you can see that uh this spike right here are all the these berlin defense variants and castling this 0 right here is a big part of that line if you do this in no castling chest you can see that these two moves now the histograms overlap much more which means that and in fact you can see in the in this number of possible moves right


here that they come closer together so not only does the blue shift to the right the orange actually shifts to the left and it basically means that whether you open with e4 or knight f f3 you are going to have about the same complexity of game the same number of moves available to you going from there as you can see right here these lines are the moves available for white and black


under the different rule sets so in e4 here especially as black you do not have many moves available as white a little bit more but also not more um whereas in no castling you do so again small rule change uh big effect on the possible moves that you have can consider and this is the type of this is the type of information that you would want to have when you


design a game and they allude to this also at the end here in their conclusions so the last thing is they also compare the material values of the pieces here in the different rule sets as you might imagine so some pieces become much more or less valuable i find it particularly interesting that if you do something like pawn sideways or then where the pawns are much more


powerful of course all the other pieces drop in value again these results are pretty plausible so i don't want to trash the paper right here because it seems like it seems like the the results are as i say plausible and can give some cool insights so the chess master also gives um gives his opinions on these different strategies that alpha zero comes up with


for the different rules and let's go through the conclusions real quickly so they say assessing the consequences of rule change in the game design process demonstrate on chess where we've trained alpha zero to evaluate nine different variants representing atomic changes to the rules of the game training alpha zero model on these rules changes helps us effectively simulate decades of human


play in a matter of hours and answer the what if question what the play would potentially look like under developed theory in each chess variant we believe that a similar approach could be used for auto balancing game mechanics in other types of games including computer games in cases when a sufficiently performant reinforcement learning system is available and yes this is


i mean this the application here would be for something like this if you design a new game then you want to know what you have some choice with how you can make the rules and you don't want to let humans become really good at each of the rules and then compare you can simply give this to the algorithm and the algorithm will tell you what kind of plays result from each


rule set and then you can choose the one that you find most interesting or most uh maybe commercially viable and what not i actually see this much i see this bigger than just games and this alludes a bit to the salesforce paper on this ai economist i think we can let ai you know get tell us what happens if we change for example uh things like tax policy or any any sort of policy i know


humanity is very complex to model and so on and you're never going to have a perfect simulator which probably makes alpha zero not good but in limited situations like maybe also stock trading rules and so on you could definitely have situations where the rule set is too complicated to solve analytically but you could give it to an rl algorithm and see what happens and whether or not you like


the outcome and whether or not there are any like obvious exploits that uh you did not see so this i find you know pretty it's it's a pretty cool approach and and we should think of this in the future as we build systems that have rules in whatever capacity be this games or policy so the they say okay yada yada yada we showed that there are several chess variants among those


considering the study that are even more decisive than classical chess meaning torpedo chess semia torpedo chess no castling chess and stalemate equals winches we quantified a rising diversity of opening play and the intersection of opening trees between chess variations showing how different the opening theory is for each of the rule changes yeah they again this this diversity


of opening play it really rests on this assumption that alpha zero is a is a good player and any sort of an equally good player in all of these variants right because if it's worse in a variant it might not be as sure about the moves and that would just look like oh you have many possibilities but in fact alpha zero is just worse at it and it doesn't know so they also look at


the intersection of opening trees like if you change a rule how does this change um change the the kind of how does this change the the initial game so a lot of these grandmasters they learn by heart all of these opening trees the initial moves of a game how much would they have to relearn there is a negative correlation between the overall opening diversity


and decisiveness as decisive variants likely require more precise play with fewer plausible choices per move again this is one view right the other view is that there are rule sets that are just make it into a harder game and then alpha zero given the same amount of compute is a worse player and therefore it can't play as well therefore the games are less decisive


and also the opening diversity is higher because it doesn't know if the game could be as decisive it might just be an effect of alpha zero for each of the chess variants we estimated yada yada okay no castling chess being the first variant that we analyzed has already been tried in experimental blitz grand master tournament in chennai as well as a couple of longer grand


master games our assessment suggests that several of the assessed chess variants might be quite appealing to interested players and we hope that this study will prove to be a valuable resource for the wider chess community i yeah i don't know is is the chess community flourishing or going under recently because it seems to me like it once once a game is solved that hard


by computers i mean it's still fun but um yeah i just i just i guess counter strike is also solved by bots real hard uh it's just impressive when humans play or so um yeah i don't know all of this is again if you're into chess look into this paper they have a lot of really interesting results that are not interesting to go into for the general community but i


[Music] this video will present a recent paper from Google AI revisiting self supervised visual representation learning so the headline idea this paper is that the standard architecture designs and convolutional neural network advances that have been working in supervised learning tasks like image net classification don't necessarily translate to these self supervised tasks

Self-Supervised Learning


such as predicting the rotation the permutation of jigsaw puzzles or the exemplar augmentation task so this is the ideas are the neural network architecture designs over fitted to sell to supervised learning tasks such as recognition and detection and should maybe these neural architecture searches be deployed into self supervised learning or into like a pipeline of self

Self-Supervised Learning


supervised learning and then taking the representations into the classification models but then using the hyper parameter neural architecture search and all these heuristic tricks to design this jointly on self supervised tasks forth for the downstream representation learning so self supervised learning is inspired by NLP success and it's just recently it's been tested since like the

Self-Supervised Learning


dawn of alex net but it is definitely gaining traction and so an NLP this is about predicting words from their context so when the sentence predict words from their context words and form would be labelled as positive words and predict one and then tiger an ocean don't appear in the context so they've been labeled as zero or negative so the sole supervised learning has these

Self-Supervised Learning


pretext tasks and so there's other techniques other than context like NLP there's more ideas in visual representation learning and images and computer vision so here the some of the common self supervised visual learning tasks rotation prediction example are classes relative patch location and jigsaw puzzle permutations and many of the studies that have already come out

Self-Supervised Learning


on this use the Aleks net CNN architecture this paper from Google AI is going to use the state-of-the-art ResNet designs like the wide ResNet and then a reversible resume which is a more efficient implementation so rotation task is like this you would take images and then you rotate them either zero degrees 90 degrees 100 to 70 and then the network is basically

Self-Supervised Learning


doing a for class classification task for these different rotations the exemplars used data augmentations to take an image and then change it in like a ton of different ways and then this image corresponds to its own class so it would be like a massive way like maybe like a thousand class and on classification problem so jigsaw puzzle this is an interesting one it's where

Self-Supervised Learning


you take like a patch of the image and then you like you crop the image and then you further refine the crops and then scrambled away in this thing and then try to have you pass each of the crops through the network so each of these each of these square tiles goes into a convolutional Network and then it predicts the where it thinks it might lie in the permutation so I personally

Self-Supervised Learning


don't I'm not really a huge fan of this task I don't really see how it makes a whole lot of sense because how does it know the context really and then relative patch location is the same idea I like this idea more where you just have two patches and you predict how they might relate to one another and then another really interesting study that I'm gonna be making a video on

Self-Supervised Learning


tomorrow so please subscribe if you're interested in this video is a multitask table supervised visual learning where you combine these self supervised learning tasks together and then see what kind of representations are derived from that and they test this and their paper using the ResNet 101 model so representations for image classification what you do is you freeze most of the

Self-Supervised Learning


network and then you take the pre largest layer like the like an intermediate vector representation and then you input that to a logistic regression model and use that as a classifier train with SGD and data augmentation so the vector that they extract feature vector from the sub supervised learning task they vary this from size 2048 4096 6000 144 and 8192 so

Self-Supervised Learning


this is the size of the representation of vector extract from the self supervised learning task so another thing they find is you might ask like how about instead of a logistic regression we do a more complex multi-layer perceptron model so this plot just shows this is the logistic regression in the MLP and they basically perform the same so the logistic regression is has sufficient

Self-Supervised Learning


capacity for this and then another interesting thing is we're in the network do you get the features from so this one thing that's interesting is in the VG g19 network they take the intermediate features from the third block rather than the very end and get better results but with the ResNet architectures they always get better results towards taking it as close to

Self-Supervised Learning


the output as you can get so again this would mean like should you take it from like here or should you take it from down here like towards the output or from the intermediate features so the data sets to the tests our image net which is 1.3 million images in the thousand classes and then the places 205 which is 2.5 million images in 205 classes and these are pretty different

Self-Supervised Learning


data sets and it's used just for a measure of generalization with respect to the data sets so these are the results basically across the rev net ResNet v2 ResNet v1 just showing how amazingly different the results can be for the different architectures so and not only that but inconsistent so even though Greg net kills on rotation it doesn't perform as well on a relative

Self-Supervised Learning


patch location so this is the results in the table so this refers to increasing the widening factor of a resinate so increasing the number of feature maps to each intermediate layer so very interesting trend you see is as they continue to increase the capacity of the model they get better results so the widening factor is highly correlated with success on this task and yeah so as

Self-Supervised Learning


they increase the representation capacity of the model Bay get better results so then also this shows how applying their new ResNet with the winding factor how this compares to the previous papers that have been published on self supervised learning so most interestingly the rotation paper that first came out using an Alex net style architecture achieves 38.7% image net

Self-Supervised Learning


but their model achieves like 20% better and is getting much closer to the fully supervised benchmarks so this is another interesting thing that they present is that the success on this Hope supervised task isn't always correlated with image accuracy like this point right here has like 95 percent on rotation but then only like 20 percent on that image net accuracy so then

Self-Supervised Learning


another very interesting takeaway from the study is that it seems like larger models like increasing the width multiplier and this is the vector that you take from the representation seems like as you increase the width the network and then as you get a bigger feature from the model that you do a better job on image met classification with these vectors so thanks for

Self-Supervised Learning


[Music] hi guys this is Laker from a Judaica the evolution of AI has changed the entire 21st century in terms of technology ai has told in the spotlight and it's advancements are quicker than we predicted with such an exponential growth in AI machine learning is becoming the most training field of the 21st century it is starting to redefine the way we live and it's time we

Supervised vs Unsupervised vs Reinforcement Learning | Data Science Certification Training | Edureka


understood what it is and why it matters in this session we'll be discussing the different types of machine learning and we'll compare them to each other so let me run you through today's agenda we're going to begin the session with an introduction to machine learning next we will discuss the types of machine learning after that we'll compare supervised unsupervised and reinforcement learning


based on a few key parameters we'll finally end the session by discussing a few example problems that can be solved using supervised unsupervised and reinforcement learning algorithms so without any further delay let's get started so guys machine learning is the science of getting computers to act by feeding them data and letting them learn a few tricks on their own without being


explicitly programmed now this sounds awfully a lot like a human child so let's consider a small scenario to understand machine learning now as a child if you had to distinguish between fruits such as cherries apples and oranges you wouldn't even know where to start because you're not familiar with how the fruits look now as we grow up we collect more information and start


developing the capability to distinguish between various fruits the only reason why we are able to make this distinction is because we absorb our surroundings we gathered more data and we learn from our past experiences it's because our brain is capable enough to think and make decisions since we have been feeding it a lot of data and this is exactly how machine learning works it involves


continuously feeding data to a machine so that it can interpret this data understand the useful insides detect patterns and ident my key features to solve problems this is very similar to how our brain works now let's move ahead and take a look at the different types of machine learning so first of all we have supervised learning now guys supervised means to oversee or


direct a certain activity and make sure it's done correctly in this type of learning the machine learns under guidance so at school or teachers guided us and taught us similarly in supervised learning machines learn by feeding them label data and explicitly telling them hey this is the input and this is exactly how the output must look okay so the teacher in this case is the training


data next we have unsupervised learning unsupervised means to act without anyone's supervision or without anybody's direction now here the data is not labeled there is no guide and the machine has to figure out the data set given and it has to find hidden patterns in order to make predictions about the output an example of unsupervised learning is an adult like you and me we


don't need a guide to help us with our daily activities we can figure things out on our own without any supervision finally we have reinforcement learning now guys reinforcement means to establish or encourage a pattern of behavior let's say that you were dropped off at an isolated island what would you do now initially you'd panic and you'd be unsure of what to do where to get


food from how to live and so on but after a while you will have to adapt you must learn how to live in the island adapt to the changing climates learn more to eat and what not to eat so here you're basically following the hit and trial concept because you new to the surrounding and the only way to learn is experience and then learn from your experience this is what reinforcement


learning is it is a learning method wherein an agent which is basically you stuck on the island interacts with its environment which is the island by producing actions and discovers errors or rewards and once the agent gets trained it gets ready to predict the new data presented to it now let's move ahead and look at the differences between supervised answer


and reinforcement learning so let's begin by looking at their definitions now like I mentioned earlier supervised learning is a type of machine learning wherein we teach the machine using label data so an input and your output is label next we have unsupervised learning over here the data provided to the machine is not labeled and the machine has to learn without any supervision so


that's why it should discover hidden patterns and trends in the data finally we have reinforcement learning now the basic concept behind reinforcement learning is that there is an agent now this agent is put in an unknown environment so the agent has to explore the environment by taking actions and transitioning from one state to the other so that he can get maximum


rewards now the next parameter to consider is the type of problems that are solved using supervised unsupervised and reinforcement learning so under supervised learning we have two main categories of problems we have regression problems and we have classification problems now guys there is an important difference between classification and regression basically


classification is about predicting a label or a class whereas regression is about predicting a continuous quantity now let's say that you have to classify your emails into two different routes so here basically we'll be labeling our emails as spam and non-spam mails for this kind of problem where we have to assign our input data into different classes we make use of classification


algorithms on the other hand regression is used to predict a continuous quantity now a continuous variable is a variable that has infinite number of possibilities for example a person's weight so someone could be 180 pounds or they could be 180 point 10 pounds or 180 point 1 1 0 pounds now the number of possibilities for weight are limitless and this is exactly what a continuous


variable is so regression is a predictive analysis used to predict continuous variables here you don't have to label data in two different classes instead you have to predict a final outcome like let's say that you want to predict the price of a stock over a period for such problems you can make use of regression algorithms coming to unsupervised learning this type of


learning can be used to solve association problems and clustering problems association problems basically involve discovering patterns in data finding co-occurrences and so on a classic example of Association rule mining is a relationship between bread and jam so people who tend to buy bread also tend to buy jam over here it's all about finding associations between items


that frequently co-occur or items are similar to each other apart from Association problems unsupervised learning also deals with clustering and anomaly detection problems clustering is used for cases that involve targeted marketing wherein you are given a list of customers and some information about them and what you have to do is you have to cluster these customers based on


their similarity now guys Digital AdWords use a clustering technique to cluster potential buyers into different categories based on their interests and their intent anomaly detection on the other hand is used for tracking unusual activities an example of this is credit card fraud where in various unsupervised algorithms are used to detect suspicious activities then there is reinforcement


learning now this type of learning is comparatively different in reinforcement learning the key difference is that the input itself depends on the actions we take for example in robotics we might start in a situation where the robot does not know anything above the surrounding it is in so after it performs certain actions it finds out more about the world but the world it


sees depends on whether it chooses to move right or whether it shows to move forward or backward in this case the robot is known as the agent and its surrounding is the environment so for each action it takes it can receive a reward or it might receive a punishment now the next parameter is the type of data used to train a machine when it comes to supervised learning it's quite


clear and simple the machine will be provided with a label set of input and output data in the training phase itself so basically you feed the output of your algorithm into the system this means that in supervised learning the machine already knows the output of the algorithm before it starts working on it now an example is classifying a data set into either cats or dogs alright so if


the algorithm is fed an image of a cat the image is labeled as a cat similarly for a dog so guys this is how the model is taught it's told that this is a cat by labeling it after the algorithm is taught it is then tested using a new data set but a point to remember here is that in the training phase for a supervised learning algorithm the beta is labeled alright the input is also


labeled and the output is also labeled in unsupervised learning the machine is only given the input data so here we don't tell the system where to go the system has to understand itself from the input data that we give to it so it does this by finding patterns in the data so if we try to classify images into cats and dogs in unsupervised learning the machine will be fed images of cats and


dogs and at the end it will form two groups one containing cats and the other containing dogs now the only difference here is that it won't add labels to the output okay it will just understand how cats look and cluster them into one group and similarly for dogs coming to reinforcement learning there is no predefined data the input depends on the actions taken by the


agent now these actions are then recorded in the form of matrices so that it can serve as a memory to the agent so basically as the agent explodes the environment it will collect data which was then being used to get the output so guys in reinforcement learning there is no predefined data set given to the machine the agent does all the work from scratch the next parameter to consider


is training in supervised learning the training phase is well defined and very explicit the machine is fed training data where both the input and output is labeled and the only thing the algorithm has to do is map the input to the output so the training data act like a teacher or a guide over here now once the algorithm is well trained it is tested using the new data when it comes to


unsupervised learning the training phase is big because the machine is only given the input and it has to figure out the output on its own so there's no supervisor here or there's no mentor over here in reinforcement learning there is no predefined data and the whole reinforcement learning process itself is a training and testing phase since there is no predefined data given


to the machine it has to learn everything on its own and it starts by exploring and collecting data the next parameter we're going to discuss is the aim of each of these machine learning types the main aim or the end goal of a supervised learning algorithm is to forecast an outcome now obviously that is the basic aim of all these machine learning types but the whole supervised


learning process is built in such a way that it can directly give you a predicted outcome because supervised learning algorithms have a very well-defined training phase unsupervised learning is all about discovering patterns and extracting useful insights now since the algorithm is only fed the input it has to find a way to get to the output by finding trends and


associations in the data coming to reinforcement learning the agent here is a lot like a human child just like how a baby is clueless about the world initially the agent also has no idea about its environment but as it explores the environment it starts learning it learns from the mistakes it makes and it basically learns from its experience now let's look at the approach followed when


it comes to supervised learning it's quite simple like I mentioned earlier all that the algorithm has to do is map the known input to the known output in unsupervised learning the algorithm has to find patterns in data trends in data and keep exploring the data until it reaches the output the approach followed by reinforcement learning is a trial and error method the trial and error method


best explains reinforcement learning because the agent has to try out all possible actions to learn about its environment and to get maximum rewards the next parameter is feedback now in supervised learning there is a direct feedback mechanism since the machine is trained with build input and output for unsupervised learning there is no feedback mechanism because the machine is unaware of the


output during the training phase now in reinforcement learning the feedback is in the form of rewards or punishments from the environment so when an agent takes a suitable action it will get a corresponding reward for that action but if the action is wrong then it gets a punishment so rewards and punishments can be thought with respect to a game now in a game when you win a state you


get extra coins but when you fail you have to go back to the same state and try again now let's look at some of the popular algorithms supervised learning has algorithms like linear regression which is mainly used for regression problems it also has algorithms like support vector machines decision trees and so on and these can also be used for classification problems coming to


unsupervised learning we have algorithms like key means C means for clustering analysis and algorithms like a priori and Association rule mining to deal with Association problems now reinforcement learning is just being explored recently a few algorithms include Q learning and the state action reward state action algorithm next up we have applications so guys supervised learning is widely


used in the business sector for forecasting risks risk analysis predicting sales profit and so on coming to unsupervised learning so guys the recommendations you see when you shop online like for example if you buy a book on Amazon right you get a list of recommendations now these are all done by unsupervised learning algorithms other applications include anomaly


detection credit card fraud detection and so on now reinforcement learning is used in self-driving cars in building games and all of that one famous example is the alphago game I'm sure all if you have heard of that so guys those were the major differences between supervised unsupervised and reinforcement learning so now let me give you a few examples of problems that can be solved using


supervised unsupervised and reinforcement learning algorithms all right so our first use case is to study a bank credit data set and make a decision about whether to approve the loan of an applicant based on his profile so here we are going to be given a bank credit data set now the information that you see over here is for each of the customers so every customer's account balance purpose


credit amount value savings everything is given in the data set and you have to predict whether you can approve the loan of an applicant based on his bank account balance based on his purpose his credit amount and his savings so for this problem you can make use of the supervised learning algorithm known as key and an algorithm or key in your is neighbor algorithm now let's look at our


next use case now here we have to establish a mathematical equation for distance as a function of speed so basically over here you're going to predict the distance that a car can travel based on its speed so guys the best algorithm to use for such a problem is the linear regression algorithm so the linear regression algorithm is basically used to predict continuous


quantities and in this case we have to predict the distance which is a continuous quantity and like I mentioned earlier a linear regression is a type of supervised learning algorithm okay moving on to our next few skills now the problem here is to cluster a set of movies as either good or average based on a social media outreach all right now if you read the problem statement


properly you can see the word cluster alright this clearly means that this is a clustering problem and clustering problems fall under unsupervised learning so here we're going to make use of a algorithm known as k-means algorithm to form two clusters okay one cluster is going to contain popular movies and the other is going to contain non popular movies based on their likes


on social media now moving ahead the our next problem statement is to perform Market Basket analysis by finding association between items bought at the grocery store again over here you can see the keyword association this means that this is an association problem now Association problems fall under the unsupervised learning algorithms and here we can make use of the a priori


algorithm to do this so here what you have to do is basically if and find association between different items so if a person bought bread and butter together it means that there is an association between these two items so in this problem you just going to find the association between different items and you're going to make use of the unsupervised learning algorithm


cause the a priori algorithm so guys this is the last use case and over here the problem statement says that you're going to place an agent in any one of the rooms and basically the rooms are represented as 0 1 2 3 4 & 5 and the goal here is to reach the outside of the building now this is clearly a reinforcement learning problem all right to solve this you can make use of the


cue learning algorithm and your end goal is to reach room number 5 so guys here you can see that there is no data set because the data set is going to be developed by the agent itself so guys over here the agent is responsible for collecting the data all right he's going to explore the environment collect useful information and then he's going to use this information to get to room number 5 so


guys that was it for our use cases and with this we come to the end of today's video I hope all of you enjoyed it if you have any doubts or any queries regarding the session please leave them in the comment section and we'll get back to you at the earliest so guys thank you so much for watching this video have a great day I hope you have enjoyed listening to this video please


supervised learning updates the parameters of a neural network to match predicted class labels with the ground truth labels the construction of these ground truth class vectors is typically done with one hot encoding but other techniques such as label smoothing and knowledge distillation have been developed to overcome the limitations of one hot encoded ground truth class

Meta Pseudo Labels


labels Mehta pseudo labels that uses the meta learning framework to dynamically adapt the target distribution or ground truth class labels throughout training of a student classification network to maximize its resulting validation set accuracy this is done by training the classification model or student network on pseudo labels labeled by a teacher network the teacher network is then

Meta Pseudo Labels


updated to maximize the classification models accuracy on the validation set after it trains and updates itself through back propagation supervised learning on the pseudo labels from the teacher network this involves an interesting gradient through a gradient operation to train the teacher network Mehta pseudo labels achieves 80 6.9% top 1 imagenet accuracy through

Meta Pseudo Labels


semi-supervised learning with additional data and also impressive performances in the limited data settings the authors also introduced a reduced MPL framework to avoid the memory bottleneck of having two high capacity models in memory for the meta learning framework this video will explain meta pseudo labels from researchers at Google ai this video will explain meta pseudo labels from

Meta Pseudo Labels


researchers at Google ai meta sudo labels is a new way to use meta learning to adapt the ground truth class labels during training by using a teacher network to label data and then a student network that learns from those labels a quick overview of the meta pseudo labels algorithm is that a teacher model is trained along with a student model to set the students target distributions

Meta Pseudo Labels


and adapt to the students learning state so typically these target distributions are these one hot encoded vectors where you might have like zero cat one dog and then zero for all of the other classes in the case of say C fart n so the idea here is to have the teacher network produce the way of labeling the data points to say zero point zero three cat zero point seven dog zero point zero

Meta Pseudo Labels


four horse these kind of distributions are going to be assigned by the teacher network rather than heuristic lis encoded with something like one hot encoding label smoothing or even the knowledge distillation pipeline with to temperature tuning so then the idea is to adapt these target distributions to the students learning state so the way this pipeline works is that the

Meta Pseudo Labels


teacher network parameterised by phi is again taking the same training data set and then produce a pseudo label distribution and then the student network is going to try to fit this label distribution that was produced by the teacher network so it's going to do back propagation using the cross entropy loss function between the predictions from the student Network y-prime and

Meta Pseudo Labels


then these pseudo labels Q Phi of X so it's going to back prop this and then update the parameters to theta T plus 1 so now these new parameters that have been updated by training on the pseudo labels from the teacher network are then going to be evaluated to provide a reward signal for the teacher network by taking those parameters and then having them classify a held-out validation set

Meta Pseudo Labels


so the performance on the validation set is the reward signal that goes back through the teacher by taking a gradient through a gradient or do something we'll get into more in the later on in the video the idea is that the teacher model is going to be training changing this distribution of class labels to maximize the performance of the student Network and the held-out validation set these

Meta Pseudo Labels


are some examples of the most common target distributions our ground truth class label vectors Y that are used in machine learning the most common of which is one hot encoded vectors this is how datasets like C fart n are labeled if this is the case of a dog image in the class label corresponding to that image you'd have a one in the position of the dog index and then zero

Meta Pseudo Labels


everywhere else for the other class labels so say one dog zero cat zero truck ship this is how the sea far 10 data set it's labeled so one problem with labeling data sets one hot and code vectors is that the model is going to have these overconfident or over fitted predictions to this kind of a class label distribution you can say it applies any probability density to

Meta Pseudo Labels


another class it's gonna have a penalty from the cross entropy loss for doing so so if it's a C's this dog image and tries to do label it as 0.75 dog and 0.2 cat he's unsure whether it's a cat or a dog it's gonna be penalized for that as if the cat is just as different from a dog as a truck or a ship or a frog or these other C Bartek classes so one solution to the overconfident

Meta Pseudo Labels


predictions are overfitting on the one hand code of vectors is label smoothing so label smoothing is where you apply this uniform weight to all the other class labels in the class suitable vector and then another solution to assigning these target distributions knowledge distillation knowledge distillation in the form of self-training with noisy student currently has the state VR for imagenet

Meta Pseudo Labels


classification and it's also a really powerful technique for model compression such as say distilled Bert or you have these high capacity models then you use the high capacity model to produce a new class label distribution that is better than the one hunter for training the student network and then the student network learned that a combination of this distillation class distribution as

Meta Pseudo Labels


well as the ground truth what hot encoded vectors so these are some examples of different target distributions that have been explored in machine learning and are commonly used to prevent overfitting and then to you know train these models with supervised learning so code from the paper is that from the success of these heuristic tricks it is clear that how to construct

Meta Pseudo Labels


the target distribution plays an important role in the algorithm design and a proper method could lead to a sizeable game motivating this exploration for meta learning the target distributions during training so again we have this problem of what should be this target distribution should we have one hot encode to class level vectors should we smoothing out the labels by

Meta Pseudo Labels


putting uniform weight on the other class labels or should we use this teacher-to-student pipeline and knowledge installation but the solution explored in this paper is to metal learn the pseudo label distribution or the targets that the student networks and we trying to fit during training there are two phases of learning in the meta pseudo labels framework in phase one the

Meta Pseudo Labels


student learns from the teacher the parameters theta of the student classification network are updated by taking the cross entropy loss between the predictions piece of theta of X and then the pseudo label distribution that is produced when you pass these X data points through the teacher Network parameterised by Phi denoted Q sub 5 X to denote this new pseudo label

Meta Pseudo Labels


distribution that comes out of the teacher phase 2 is the teacher learns from the students validation loss this is a much more complex way of structuring this loss this gradient through a gradient meta learning idea of training the teacher network so the teacher network is evaluated on the validation set performance of the student network after it updated as

Meta Pseudo Labels


parameters to theta t plus 1 so the way that this reward is propagated back into the teachers parameters Phi because you have to take the derivative of how much each of these five parameters and their labels and data points impacts the gradient of the student network to change it in the direction of this validation loss it's difficult to completely derive this idea of gradient

Meta Pseudo Labels


through a gradient and in if you're using frameworks like tensorflow PI torch you can utilize automatic differentiation to automatically implement this for you and you don't have to exactly know the math of how say this parameter from x1 input to the hidden state a in the teacher Network gets this lost signal from the student network that is then updated with the gradient of this new label data

Meta Pseudo Labels


set and then moved in the direction of the validation loss but the idea the high-level idea and I think maybe visualizing these two networks even though in practice the teacher network is gonna be a multi-layer perceptron like this but the student network is one of these high capacity classification models like ResNet wide ResNet or efficient that so the idea is you want

Meta Pseudo Labels


to say update this weight from the input to a hidden state in the teacher network and you're gonna try to take the partial derivative with respect to this way in the teacher network with respect to the validation loss on the the student network after at theta t plus 1 on that validation set so this is trained with policy gradients on this reward signal because this isn't like Y prime minus y

Meta Pseudo Labels


there's no ground truth with respect to the validation loss of the student achieves so you're just taking that validation loss reward and treating that as like a reward and like say pac-man or Atari and using policy gradients to update the parameters but basically say like if this parameter contributed a lot to the output and then you get a high reward do more of this like increase the

Meta Pseudo Labels


weight from this connection to do more of it to get more of this reward that's kind of a high-level idea of policy gradients but the idea is that in order to get this derivative to find out like how much of this weight contributed to the validation loss you have to take a gradient through a gradient which is a pretty complex idea that's maybe better explained in the next equation from the

Meta Pseudo Labels


paper hopefully this equation from the paper were further explain the idea of taking a gradient through a gradient to update the parameters v from the teacher network with respect to the parameters theta T plus 1 that are evaluated on the validation set stadia was we're taking a gradient through a gradient so the parameters theta T plus 1 that are responsible for this validation loss

Meta Pseudo Labels


reward signal we're trying to update the teacher network with our updated by taking the parameters theta T and then updating them with a gradient so we want to know how much each parameter infi is responsible for the gradient that updates this so you're taking the derivative with respect to the-- of this validation loss well fee is you know contributes to this validation

Meta Pseudo Labels


loss through the gradient so you have to find out you know how much each of these parameters and the fee network you know as in something like this how much does this parameter the fee network or this parameter contribute to the gradient and then the gradient is what updates the parameters and gives you this new validation loss so you're taking the partial derivative of feet with respect

Meta Pseudo Labels


to this gradient through a gradient idea which is a little complicated but you know really an interesting idea with the meta learning and this medicine labels algorithm the next idea introduced in the meta pseudo labels is to avoid this memory requirement of having two high capacity classification models in memory because say you have an efficient net as the teacher network as well as the

Meta Pseudo Labels


student network now you have to keep both these models in memory especially when you're doing the gradient update for the teacher Network si idea to avoid that is do first train a large teacher network on the label dataset and then use it to produce this new distribution on the unlabeled data and then you use a smaller teacher network so say you originally trained the

Meta Pseudo Labels


teacher network with like an efficient net and then you move it to a multi-layer perceptron because all it's doing now is adjusting the original distribution that was produced by this high capacity model so this high capacity model is already producing a pretty useful target distribution as in knowledge distillation and then the smaller teacher has enough capacity to

Meta Pseudo Labels


be adapting it during training as in the meta learning framework the authors are going to test the performance of meta pseudo labels in the limited data setting as well as the semi supervised learning setting semi-supervised learning is responsible for most of the image net state of the arts like self training with noisy student where they have the labeled image net and they also

Meta Pseudo Labels


leverage this unlabeled jft 300 million data set to get more performance out of the model also the billion scale weekly semi-supervised learning framework from facebook uses the labeled image net data set and the unlabeled Instagram images that are weakly labeled with their hashtags to take advantage of the semi-supervised learning framework which is probably going to be the paradigm

Meta Pseudo Labels


that leads forward since it's so easy to get this unlabeled data compared to label data so they're experimenting with meta pseudo labels on the efficient net architecture for the student network on the full-si 410 image net street view house numbers data set plus extra unlabeled data so in the case of C 410 this is tiny images in image net it's why FCC 100 million in street view house

Meta Pseudo Labels


numbers there's an additional like 500,000 data points that come with the data set to use optionally if you want to test out these kind of algorithms so they achieve 98.6% see of our 10 accuracy and then 6.9% topple and imagenet then here are some other papers to check out if you're interested in semi-supervised learning that have also come out recently and are

Meta Pseudo Labels


really successful in this kind of space these are the results of meta pseudo labels in the semi-supervised learning framework compared with supervised learning and then the self training with noisy student pipeline you see gains in the sea far ten dataset small gains in the street view house members and then big gains on the image net dataset one reason that the authors point towards

Meta Pseudo Labels


these small gains for the street view house numbers is that the extra unlabeled data in street view house numbers is in it's in the distribution so there is this distinction between out of distribution data and in distribution data so the case of image net where you're trying to take in this new data from this Y of cc100 million data you would call that out of distribution data

Meta Pseudo Labels


because it's not like the image net data whereas the street view house numbers that extra data is like really the same exact data that the training set has in terms of this like kind of underlying distribution idea so the idea here is that the meta Studio labels this adaptive adjustment of changing the labels during training is more crucial when the extra unlabeled data is more

Meta Pseudo Labels


out of distribution so if you're dealing with a computer vision problem and you're curious that this algorithm is gonna work for your problem it's interesting to say you know it's the unlabeled a do you have how out of distribution is it how noisy is it and this MPL meta student-level framework is likely to have a bigger gain if this is noisier data compared to in distribution

Meta Pseudo Labels


data the authors also test the meta pseudo labels algorithm in the limited data setting where you have say only 4,000 labeled images in C far 10 1000 in Street View house numbers or 10% of the labeled data in image net in this case you see the performance of meta pseudo labels compared to supervised learning with all the labels Simms CL are fixed match or unsupervised data augmentation

Meta Pseudo Labels


which are all other algorithms that are successful at doing this kind of learning with limited data this plot shows the performance of these models with respect to the different limited data settings changes as you increase the percentage of labelled data points the interesting part about this plot is this top left area where you have the smaller percentage of labeled data and

Meta Pseudo Labels


you see a huge gain of the unsupervised data augmentation plus the menace to toe labels algorithm compared to supervised learning or the Rand augment data augmentation algorithm this table shows the gains of meta pseudo labels with respect to the sea far 10 limited data setting and in the street view house number limited data setting see the performance of different algorithms like

Meta Pseudo Labels


just training with supervised learning on Limited data using that label smoothing way of putting uniform weights on the other class labels then using supervised learning plus meta pseudo labels and then stacking medicine labels with the ran augment and unsupervised data augmentation algorithms these are some of the algorithms that are explored to stack on top of meta pseudo labels

Meta Pseudo Labels


unsupervised data augmentation enforces predictions Y given X to be consistent with the same X data point after it's gone through a data augmentation so in some cases data augmentation might mean like rotating an image translating it or horizontally flipping it in the case of natural language processing it might mean translating the sentence to German and then translating it back to English

Meta Pseudo Labels


which is known as a back translation these are the different ways of augmenting data points and then forcing this cycle consistency to have similar predictions on the data point before and after it's been augmented another these algorithm says stacked on top of meta Steudle labels is Rand augment Rand augment is this automated data augmentation algorithm similar to like

Meta Pseudo Labels


auto augment or population-based augmentation but the idea here is to have this simpler parameterization of the space that actually shows to work better with respect to constructing these automated data augmentation pipelines and the next sort of algorithm to be looking at as well and comparing this with is self-rated with noisy student which is this really popular way

Meta Pseudo Labels


of doing knowledge isolation which is where you take the pseudo labels you apply a lot of noise with respect to training the student model on that teacher target distribution which is the whole idea of meta studio labels is looking at the ways to structure this target distribution to train these neural networks on the authors explore the behavior of the teacher network in

Meta Pseudo Labels


the meta studio label framework so they've learned the Medus to delay bol teacher fits the validation gradient it's not just label Corrections and is not only a regularization or preventing overfitting strategy the authors explore this idea that the teacher encourages the students training gradient to be similar to the students validation gradient on this two moons

Meta Pseudo Labels


data set because it's really difficult for them to do this kind of cosine similarity between validation and training data gradients with these larger data sets like CFR 10 or imagenet so they show the cosine similarity between the training and validation data as gradient with respect to the training progress and showing that the medicine learning framework the teacher is trying

Meta Pseudo Labels


to steer the gradient in the direction of this validation data sets gradient as well the next idea is to explore whether the teacher network is so performing label correction or trying to mimic the behavior of supervised learning with perfect labels so this plot is showing that if it was doing this then these accuracies of the student network should be high as well

Meta Pseudo Labels


and have a similar kind of curve as supervised learning so this is showing that the teacher network isn't just trying to fit the training data it's trying to help with this regularization and preventing overfitting this visualization shows the teacher network isn't just doing preventing of overfitting with respect to how its labeling this data can you see interesting behaviors with respect to

Meta Pseudo Labels


the developments of how it's labeling each of these data points and see if our 10 data set throughout the training you see that it the label with the highest confidence doesn't change it doesn't get steeper between 50 and 75 percent it doesn't do things like flipping labels or dampening distributions in obvious ways that are typically heuristically explored with respect to you know doing

Meta Pseudo Labels


regularization in the class label space another interesting algorithm in the space of meta learning these different components that make up the supervised learning problem is generative teaching networks generative teaching networks have a generator that uses this gradient through a gradient in order to generate this data set that is used to train the student network so it could be

Meta Pseudo Labels


interesting to see if you could stack this meta suit of labeling or having this adaptive labeling with the generated data set as well also definitely a confusing gradient or you could maybe stack the generator and the label are sort of similar to how like alphago zeroes combines the policy and value network into one architecture but it's definitely an interesting kind of

Meta Pseudo Labels


space of emerging algorithms these meta learning algorithms that are generating data generating adaptive labels during the training a lot of different areas where meta learning is being developed and producing these interesting algorithms thanks for watching this explanation of meta pseudo labels a really interesting meta learning algorithm that adapts the target

Meta Pseudo Labels


distribution for the student network as it's learning throughout training to maximize the accuracy on a held out validation set this is a really interesting use of meta learning and this gradient through gradient training to have the teacher-student paradigm where the teacher is taking apart this different component of the supervised learning framework particularly in this

Meta Pseudo Labels


case the target distribution and then the student is learning with the teacher network in this simultaneous I like dual optimization or coevolution framework of training these two models in the meta learning idea this is responsible for really high accuracy on image net with the full data set plus extra Abell data as well as interesting performances on the limited data setting

Meta Pseudo Labels


hi there today we're looking at neural architecture search without training by Joseph Miller Jack Turner amis Torquay and Eliot J Crowley on a high level this paper performs neural architecture search by looking at the correlation matrices of the Jacobian of the of the data when you pass it through the network and it does so at initialization so you pass the data look at the

Neural Architecture Search without Training (Paper Explained)


Jacobian and if it's very correlated then the network is bad and if it's very uncorrelated then the network is good and by simply observing that they can already achieve a very good score on neural architecture search benchmark alright that was a high level and maybe a bit too simplified but that's sort of what's going on ok let's dive in so what's neural architecture search neural


architecture search is the discipline of you are given a data set let's say here we have a data set which could be something like C 410 which is an image data set and you are given a sort of a training procedure let's say Adam or SGD for 100,000 steps or something like this with many batches of size 64 ok and you're given a loss function which the loss function here could be the cross


entropy between the outputs of the network which we'll call L and the label Y and your task is now to find a neural network architecture that conforms to these specifications but gives the lowest possible loss or de sorry the highest possible validation accuracy in this case so this here would be like the Train and then you'd have the test accuracy or the validation accuracy okay


so you could decide well I'm gonna go with you know first like three convolutional layers each one having like a real ooh non-linearity but you could also say well I'm going to build like a skip connection from here to here you could also say that I'm going to down sample by you could have maybe a bigger stride and so on so the kernel size of the convolution you can vary until now


people have done this by hand right in effect we all use like the same 10 to 20 different architectures so if it's an image problem we tend to go for like a res net or a wide ResNet like a vgg style architecture someone has come up with those at some point with each of those discover that it works well and we don't really do much exploration we simply kind of use the same things over


and over and the the truth is that there might be much better architectures that were simply not exploring right there might be much better building plans for networks that we don't know of that might perform a lot better with the same data and the same training so neural architecture searches the process of automatically searching for these better architectures of course that's a


combinatorial problem but the idea is that you know you can actually learn to construct good architectures and by doing so you can you can sort of speed up this process that is manual otherwise and the idea behind it is there some regularity of when an architecture is good there's some like high level of pattern that you as a human maybe cannot really grasp but like a machine can


figure out which architectures are good and which ones aren't so there have been a few inventions in this in this area but they are mostly costly that's what they say here the time and effort involved in hand designing deep neural networks is immense this has prompted development of neural architecture search techniques to automate this design however neural architecture


search algorithms tend to be extremely slow and expensive they need to train vast numbers of candidate networks to inform the search process so what neural architecture search methods do is what they'll have is they'll have something like a controller in the controller itself of course is going to be a neural network so there'll be this thing that will be the controller and the controller will


emit like a building plan so the controller will emit like a building plan for this network right here and then you train the entire thing once through for the entire hundred thousand steps and then you observe the final validation accuracy which might be something like eighty percent and then you know okay this is eighty percent so you feed the eighty percent into your


controller and the controller out puts the next building plan that it thinks will score higher and then you train the entire thing again and you maybe observe a 70% accuracy you again feed that in right and the controller realizes oh I may have done something wrong let me try something else and does it yet if this looks like reinforcement learning to you that's


because this is reinforcement learning so there really is see here the controller would be the agent the percentages here the accuracies would be the reward and the invasions would be basically this thing here this thing would be the actions but sometimes it's the observations and you need to score the different things okay so the problem of course with this is that the


reinforcement learning requires a lot of data it requires a lot of steps to converge because the signal from the reward is just so weak you simply get one number for your action and you don't know what you can change to make it better you simply have to try so you need a lot of steps but this thing here is mighty slow because each each single step in your reinforcement learning


procedure involves training an entire neural network for like this many steps ok so all of this is ginormously slow and resource intensive and that of course blocks a lot of research because you know we started with the plan to automate this part right here but automating it itself is super expensive so they go for a different solution they say this could be


indeed if we could infer at Network sorry if we could infer a networks trained accuracy from its initial state okay it seems a bit out there but let's let's give them benefit of the doubt in this work we examine how the linear maps induced by data points correlate for untrained network architectures in the NASA bench 201 search space and motivate how this can be used to give a measure


of modeling flexibility which is highly indicative of a network strained performance we incorporate this measure into a simple algorithm that allows us to search for powerful networks without any training in a matter of seconds on a single GPU okay and they have the code available right here if you want to go and check that out so let's go in let's go into that the claims are pretty big


and the reasoning behind the claims is the following observation you can already sort of see in this graphic right here we'll we'll go over what it means in one second but what they do is they take different networks in this search space and the search space in this case is given by this benchmark so this benchmark basically has a long list I think of architectures that you could


consider actually so it's a it's a constructive list so they don't actually give you the list but they give you like a a way to construct architectures and they took those architectures and they rank them by how well they score on C 410 so there are very good architectures which are here there are good ones there are mediocre ones and then the bad ones okay and you can see that the histograms


here of whatever they measure they look quite different so the histograms with the good ones they all have kind of spiky around zero and the histograms of the bad ones all sort of look spread out so this is the measure that they're going to propose is they have some sort of number some sort of histogram that they produce and if the histogram is very spiky and close together


around zero then they conclude that this network is good and if the histogram is very spread out like this they conclude that the network is bad now these histograms as you might expect they are computed not from the final trained Network but they are computed from the initial Network so here they show at least you know in this case it seems to be that there is a general correlation


between the trained accuracy and how this histogram looks and we're going to explore what they do so it's essentially it's pretty easy they compute the linear map around each data point so what is that if you imagine a neural network as a nonlinear function which I guess you should because it is so let's imagine it as like a nonlinear function from X to Y what they'll do is


simply they'll look at a given date training data point which could be here right this could be the X and this could be the the Y and in fact let's look at it in lost landscape not even in Y but in L in terms of the loss because we don't need necessarily a single label this could be for unsupervised this could be for anything okay so it Maps a data point to a loss now what we'll do


is we'll simply linearize the function around that point which means we'll just freeze all the nonlinearities in place and that will give us this linear function right here okay we just observe that this linear function can exist it's the tangent to the lost landscape and it's at a particular data point right it's in data space not in in weight space then we look at a different data


point so we look at this data point right here another data point what's the linear function around this one is sort of like whoops T is like that and then around this one is like this okay so this is one function now let's look at a different function right here so L X and we'll look at this function the linear function okay so for some reason this is like this and if we


consider two data points their linearization is very similar now imagine that these two have been produced by the same sort of neural networks it's just the architecture is a little different but they have been produced like they have the same number of parameters in the neural network which neural network would you prefer remember you can in by training the


neural network you can actually shape this loss function you can kind of shape that around so which one would you prefer I personally would prefer the top one because the top one already tells me that hey you know I might have 10 parameters here and this already sort of looks like each of the 10 parameters is doing something so if I then go into my 10 parameters and I you know turn this


knob right here then I might you know up this bump or down this bump or do something with it but the sort of frequencies curvature the randomness of the function the way that it fluctuates tells me that all of the different parameters must have some sort of effect right because it's of quite an expressive function whereas if I have the same number of parameters for a


function like this this sort of tells me well maybe only one of the when we only one of the weights is actually doing something maybe only one of the dimensions is doing something this seems odd right that even though I've initialized it randomly a super regular function like this comes out so maybe all of the all of these parameters down here they don't do anything or it is so


somehow the signal doesn't get through so that's I they don't explicitly say it in these terms but this is how I make sense of this what they're saying is that if you look at the linearizations of the function and you look at the the angle right here so the angle in this case is that and in this case is that and in this case is that so you look at the slope here and


the slope is basically the gradient of these linearized functions and what you want to do is you want to look at the correlation between those of the different data points so here you have three angles one is very short one is very bit longer like this and or no even like this and one is even over ninety degrees like that they are not correlated at all right they're all very


different however the angles here they're all quite the same as you can see so what they propose is the following let's send all the data points or in that case all the data points in a particular mini-batch let's send them through the function and let's calculate their linearizations so the linearization is nothing else than you send them through the network to obtain the F value for


the x value and then you calculate the gradient with respect to the input now you have to get used to this a bit because usually we calculate the gradient with respect to the weight but now we calculate the gradient with respect to the input which if this is a linear function so if you have a live f of X equals WX like a linear function then this gradient del F del X would


just give you the W will give you the slope of the linear function and the same in the neural network when you linearize it alright so we're going to obtain all these linearizations and that gives us the this matrix J right here and what we can do is we can then observe the covariance matrix of J of all these linearizations the covariance matrix simply tells you how to data


points vary with each other and in fact they don't look at the covariance matrix but they look at the correlation matrix which is simply the scaled covariance matrix so one entry in this covariance matrix so you have n data points and this gives you a matrix that's n by n and that particular entry here like the entry IJ would simply state how does the angle of data point I


correlate with the angle of data point J okay that's the that's the covariance matrix and now the hypothesis is if all of these data points are sort of independent like in our very expressive function here then the these correlations they should not be high in fact most data points should be rather uncorrelated however in this case right here if the function is sort of kind of


degenerative or something not very expressive then all of these all of these angles or of these linearizations should be highly correlated and that's what you see in this graph right here this right here now is these correlation histogram of the correlations between local linear maps across all pairs of items in a mini batch of C 410 training data each pod is a histogram for a


single untrained na s bench 201 architecture so remember the expressivity is important because we want to train that function and therefore it's important that every parameter does something and if it's degenerate we can't train it well and that's I find that's the reasoning they they sort of say this but not I might make I might make the wrong sense out of it here but it seems to me like that's


what's actually going on so you can see this is simply these matrix values rolled out and then plot it as a histogram so what does it mean when the histogram is like super spread out like this it means that there are a lot and I think down here our axes yes there are a lot of data points that correlate highly or anti correlate highly with each other okay which means that exactly this


degeneracy happens either too high or too negative high correlation means that they're very much they're kind of the same thing so there is if you have as many parameters as data points that means that one parameter can potentially serve these two data points or these two that are correlated by one or negative one you don't need both parameters and therefore


you have a lot of parameters doing nothing whereas over here with the good networks you can see that this spikes around zero meaning that the data points are not correlated or the linearizations around the data points are not correlated and therefore you can sort of shape the function around each data point however you want which we sort of know that neural networks what they do


is they're so over expressive that they're actually able to shape the functions around the data points without necessarily looking at other data points nearby and that expressivity is what what you want and that expressivity is what this in part measures okay so they make a they have some experiments here where they validate this so for all these architectures in this benchmark


and maybe I should tell you what show you what the benchmark looks like so the benchmark has this particular form this particular form there's this skeleton and in this skeleton there is this block and it's always repeated and you're basically your task is to determine what this block should be so this block has an input node a and an output node D and two intermediate nodes and what you have


to do is basically you have to determine these connections right here so there are six connections and for each one you have the option of putting different things there like you can see you put can put a convolution you can put the identity function which is a skip connection zero eyes I'm I don't maybe that's the zero function so it basically means nothing I'm not so sure


honestly but you could technically put a convolution here and here right or and or different convolutions or things like this so there are these 15,625 possible cells okay so the nurse benchmark contains 15,625 possible architectures that you'll have to search and they take these architectures and they plot now they plot for each architecture the validation accuracy


after training and the training protocol is standardized you don't have to care about that right and the score that they measure at the beginning of training and what you can see is that there is a linear relationship sort of like a sort of from from these experiments what you'll get is like this sort of feeling what they're gonna propose is that you should take that score as a as a measure


and here again also sword of sword sword of there is a there is a clear trend as you can see right here though yeah though this as you can see this sort of spreads out and the most right one is imagenet which is the most difficult one of course so and this is C for 100 which is more difficult than C for 10 so we can see that this sort of relationship at the top it doesn't really hold


anymore if the task gets difficult and this is so what I think is happening this is kind of an interjection of my own opinion what's happening here is that this score that they discover allows them pretty efficiently to see which networks are just degenerate and and cannot be trained like if you try to train them they just perform really poorly okay that it's probably a very


good score for weeding those out and that would mean if you kind of barrier here somewhere right you could just discard a whole lot of this crap or even even here right you could just discard a whole lot of this crap and also now here just you know all of this crap yeah whereas here as you can see some of this score sometimes it's higher then these ones even though they perform


better and again you could probably discard a lot of the crap but it's not as distinctive for the well performing networks because these here are all not the degenerate version right they're not degenerate in the sense that they're they have some fundamental flaw where the function lakhs now expressivity from the very start so you can't train it and then probably other factors come into


play other factors then you can simply determine with this particular score but you know there there is this relationship that's that's you know you can see that and they do some ablations on this here for example or your score is a proxy for a number of parameters and they say no the number of parameters works way worse than this particular score which it always is a cool thing


then how important is a specific mini-batch and initialization and they say look right here we for some architectures we do different mini batch sizes and you can see each of those groups they don't vary too much in how their it influences their score this is I believe this is the same architecture so it's always an architecture that achieves in this case for example wow


that's a not a straight line 77% or so and you can see if you go for different mini batches the score varies only minimally initialization is a bigger variance inducing thing but also here the scores don't vary too much but it is interesting that the different initialization to get you to different score because it would directly support kind of my hypothesis now what's going


on here is that you sort of measure initial degeneracies and you can sort of make up for these initial degeneracies in the architecture sometimes with sort of a different initialization so the different initializations give you differently performing networks we already know this from things like you know a lottery ticket hypothesis and so on that the initialization can


matter to some degree in these types of things now that being said they always train to the same it seems but their their score varies so I might be backwards correct here or not correct but in any case the initialization here matters more but also you can still see this linear relationship and this is particularly interesting this is even the case when you just input white noise so instead of


the data you measure that score by just inputting noise that I guess has some sort of the same magnitude as the data would have but it's just noise and you can still sort of see this linear relationship which is very interesting and that I think also shows some that you what you'll find what you find is a property of the network itself and the fact that it is it is initialized and


built in such a way that it allows you to train it in a very in a sort of a benign manner it has no degeneracies okay so in last experiment they go here and they say we evaluated the score on initialized networks in the PI torch CV library so they go to this library that has a lot of these networks but these networks are not the same as this benchmark this benchmark is specifically


designed to do architecture search now the networks in this library they are all designed to perform really well some are designed to be quite small some are designed to be quite fast and so on but in general they're all of their goal is to perform well and they have been sort of found by humans to perform well so they take now these networks on C 410 and they test them so as you can see


here here is the test accuracy again and here is their score that they give it and they say rip it up put it up now I can't move this anymore hello well okay they say that this linear relationship still sort of holds it doesn't it doesn't hold super super well but you can still sort of if you squint if you squint hard you can see that it sort of goes upward though you really have to


squint hard like what are these things right here and what again what's the case is that if the score is low you will sort of be able to cut off the cut of the worst-performing ones but really at the top here it doesn't seem like there is a particular relation between between these networks and this initial score which sort of strengthens my hypothesis that what this does is just


kind of weed out the bad ones but it's pretty cool because you can weed out the bad ones without any training right it's simply forward prop backward prop there you have it so cool now they come they here is the experiment where they now really do this na s benchmark and they compare with other methods so some of these other methods are designed to do they call it weight sharing which


basically is a technique where you can sort of speed up the speed up the algorithm as compared to non weight sharing and the non weigh cheering that's one of these we have discussed initially that was my initial example with the controller and so on where it takes super long so here you see the method and how long each method takes now the best ones as you can see already


the best ones here are these these methods right here are the best ones they score somewhat like a 93.9 or so on C 410 whereas these weight sharing ones they don't perform too well except this one seems to perform quite well and in this hour's case they perform worse than that but they still perform better than a lot of the weight sharing once so what their point is basically is


that they get a pretty good score which is a ninety one point five on C for ten which is you know it's at least not degenerate it's a it's a good accuracy they score that with simply evaluating ten architectures right and as n goes up as they evaluate more and more architectures they do they do get better but not much so they have a discussion here I'm having trouble moving this all


right so we'll sort of go through the discussion we report results yada yada yada yada as the set up the non weight sharing methods are given a time budget of twelve thousand seconds for our method and the non weight sharing methods are averaged accuracies or averaged over 500 runs for weight sharing methods accuracies are reported over three runs with the exception of G das our method


is able to outperform all the way chairing methods while requiring a fraction of the search time and that you maybe see at the table this is the real I mean this is the real deal here they only use here one point seven seconds compared to the twelve thousand seconds of the other methods and you reach almost the same accuracy now to be said two percent in this particular regime on


C 410 is still a sizable difference and that's the same benchmark right with the same sort of the same training schedule and so on so there's not too much room to tune here you simply have to find a better architecture so these things are still sizably ahead of this and what it appears to me that these methods here that don't perform well they're they're simply crap it seems they're simply I


don't I don't know but they might be trying out something or you know doing something researchy or whatnot but it seems like if you're well able to weed out the bad architectures you might be getting to a score like this and then if you are actually performing a search to find the best one then you might be getting to somewhere like this and you can see this


here throughout so in C for 100 they achieve a better score than these things but a worse score than the non weight sharing method and an image net it gets even the difference is even larger so again what I can see here is that there's is a good method to maybe get you like let's say 90% of the wear of the way you want to go and what's interesting is that here they say we