so CPC was able to somewhat be competitive or match those numbers but not not really that good finally it was also applied in reinforcement learning so in reinforcement learning you can think of accelerating your data efficiency by allowing your model or agent to learn way faster by performing these unsupervised auxilary tasks in parallel along with your reward optimization and the authors try to use contrast of losses as the auxiliary losses and they were able to see some gains on sample efficiency will not


really go to the details here as well so that's basically it for a CPC version 1 or CPC as it is called because the fundamental ideas were put forth in this paper but like like like I said the numbers are not that great yet right like if you look at these numbers the linear classifier with the rest net gets 48.7% whereas a supervised learner with the rezident we do architecture typically gets like something like 76 percent top one so the gap was like really high the like around twenty eight twenty nine percent and and and and so

Lecture 7 Self-Supervised Learning -- UC Berkeley Spring 2020 - CS294-158 Deep Unsupervised Learning


that needs to be addressed so until that's address self supervised is not really worth doing for practical amused classification or practical like this lofty goals we started off with which is hey we just want to learn features from data with our labels is that you can get similar quality of features so this is clearly far away and that was the version that was the goal in in CPC version two to address that gap and this is a work I did during my internship at equine well with Ironman dinner where we basically took CPC


version one and kept on hacking the architecture the the different type of parameters needed to get get all various patches and like add a lot of details in terms of data augmentations and and see how far we could push the number so and what we ended up doing was like like we actually were able to match or sometimes beat supervised learning on various downstream tasks and I want to go through the details here so you've already looked at this where you grid an image into a bunch of matches and you encode every single pass using a really


deep rest net so earlier you saw that arrests at 101 we to the first three snacks were being used in the first CPC version this mist net is much deeper arrests at 161 with 2x wit and your third snack so it's it's having 4,000 features at the end and once you do that you get the embeddings for every single patch and you process that with pixel CNN this Pixar CNN is 2x wider than the pixel CNN use in the original work and now you're trying to predict the bottom future like just like an original work but here we just use one offset we only


predict two rows below and nothing else so like it's like you already saw in the audio experiment doing lots of predictions can hurt you if the amount of information shown art is much lower so we and also doing predictions when the information overlap as much closer will also hurt you so you need to pick the prediction step very carefully depending on your Kratz eyes and so forth so we only predict two rows below and nothing else and so these are your context and latent vectors and you have the same kind of scoring function which


is by linear product and you optimize this with a nonparametric softmax over there negatives and like I said the naked in see how you sample the negatives is really crucial so you can sample negatives by taking negatives from other patches within the same image or you can take patches from other images and we have a version called all neg which is basically taking all the possible negatives you can you can construct which is all the patches in in your whole mini batch which is your mini batch will be a bunch of images and each


image is now a bunch of patches you just take all of them as your negatives in this particular loss so that way you get a lot of negatives this whole stack of optimization is like in general it doesn't matter how you construct the negatives of positives whether they use patches or not but just this whole framework because in general refer to as the info and see a loss like you very basically construct contexts and targets and try to use contrast objectives to optimize for the associations and the implementation is really parallel


because we can just use a pixel CNN with mass convolutions and you do the predictions at every single local position using a one by one composition so the recipe for CPC v2 is this train on unlabeled image net train as long as possible so we trained for 500 bucks and this basically takes you like approximately a week and you augment every single local patch with a lot of species and color augmentations so like I already mentioned in the doors work on relative position prediction making a lot of spatial jitters is really useful


so we take that to the extreme in this work and use all possible augmentations and the effect a number of negatives that you have is number of instances in your mini batch times the number of patches for instance so unlike the earlier work which gridded the image into 7x7 grid of 64 by 64 we actually use much bigger patches and much bigger images like we used to 80 by 280 images and 80 by 80 crops so that that gave us 6 by 6 grid and with an overlap of start of 36 or something is that and so so that way the number of negatives is


approximately 600 so so the fact is like we don't have a lot of negatives but all these negatives are really hard because they're coming from other practices within the same image so it's it's a mix of like instance negatives as for the spacial negatives and it learns both kind of discriminative features so this is basically a diagram that illustrates this whole pipeline where you perform you you you have this feature extractor which is the rest net1 61 running on patches of images and you train the cells provision objective which is CPC


and you do that for like a lot of long time once the mod of the strain there are various ways in which you can evaluate the kind of features that you learn one is to take put a linear classifier as was already done in the past the other is you just take the rest net that you uncovered and instead of freezing those features what if we can actually find unit on a classification task so which is to say that instead of training a linear classifier and all possible available labels what if you're allowed to put a small model on top


finding the entire stack when the situation is you're not going to be given all the labels you're just going to be given a few percentage of the labels so you're allowed to train on all the unlabeled data you have but when you when you're beginning to perform supervised learning you're you're gonna be given label data on different shards you're not gonna be given all of them but we also have benchmarks where you have all of them but in general as imagine the scenario where you can perform classification even with like 1%


of the labels which is like 10 images per class and imagenet and you can also do a transfer learning which is you you're given a completely different data set now you take this rest net throw it on that data set and perform new tasks so which could be something like pascal where you just take that rest that you've got and CPC and you throw it on object detection just like regular computer vision in benchmarking so so that's basically the goal and we will see how all these things work out if you do a lot of engineering so CPC we


do linear classifiers core remember that CPC v1 got 48.7% so just look at CPC v 1 and CP CV to CPC v2 gets 71.5% which is significantly larger than 48.7% and around that time a lot of competitive approaches were published with really good linear classifier scores as well like big buy again which is a largest Caleb I can push it up is 61% I'm Tim was another technique which is also using something very similar to CPC that push it up to 68% and all these different methods just try to go for the same approaches just make your models as


wide as possible or as deep as possible or a combination of both and optimize really long and use a lot of augmentations and careful engineering and CPC was the first method a shape improve and get to all the way up to 70 70 plus and the top rows where the models were different models were using different encoders so it's really hard to see what is helping there so on the bottom you like you can see that you just use the same rest at 50 encoder and then you compare across methods and CPC is better than all the existing methods


this was the story when CPC version 2 was published but it's no longer the case there's also a base line here in this table called momentum contrast which we'll cover as the next topic but note that momentum contrast has also improved a lot from the numbers that are presented on this table so on efficient image recognition which is you take the CPC features and you find unit for supervised learning where you can actually control the amount of label data you have CPC version 2 is able to perform significantly better than just


doing supervised learning so the red line is basically like for that corresponding percentage of label data you just do supervised learning so you just take a rest net you've trained it on whatever labels you have so you can clearly see that that would really well if you have all the labels but as you reduce the amount of label data the rest net performance keeps going down so supervised learning is really really data Hungry's as far as the number of labels your house is concerned whereas if you do unsupervised learning and all


the unlabeled data you have but you only collect label data for that corresponding percentage you can see how much gain it's giving you especially in the low data regime so you basically just need eighty percent fewer labels to match the same amount or accuracy that supervised learning gets so with just ten images per class you're almost close to 80 percent top five accuracy on image classification which is the standard set by alx not and and and you can also see that the supervised state of the art is matched with around like twenty twenty


to thirty percent of the labels that you need so this way sub supervised learning is actually letting you to be very very data efficient for supervised learning and it also means like you need to hire very few data annotators now like instead of collecting 10,000 labels here you're collecting something like 2,000 naples or fewer right so your data annotation is much faster because you already have a great set of features and your and-and-and the most interesting thing is even on even when you have all the labels that is when you have 100% of


the labels your performance from free training and then fine-tuning is better than just doing supervised learning so there is no argument to not use unsupervised learning because even if you have all the labels the performance you get by doing unsupervised learning and then performing supervised learning as a fine tuning step is higher than what do you get by just doing supervised learning and and it's it's uniformly consistent across all the label data regimes so here's like a good graph to understand how CPC version 1


interpolated to version 2 and you can see how each bottom improvement axis on the x-axis added as far as the linear classification accuracy goes and basically Ln mr. layer nom in the clay layer nom helps a lot and you know like bu refers to like bottom-up predictions instead of just using top-down predictions which is basically saying that hey instead of just relating the bottom rows from the top why not do the other way and that also helps a lot augmentation that every single patches which is referred as PA that helps a lot


and so so so various different improvement axes which you can actually refer to the paper like like for instance be using bigger patches of the lot and so on so so you just from 48 49 percent you able to now go close to 72 percent but by just focusing on the engineering details and and doing large-scale optimization really well and that that's really the success story of sub supervised learning just is just do the simple things right and you'll get really good numbers this is a table that shows the whatever graph you saw in


numbers and you can see that on every single data data regime even if you train the deepest possible architecture for supervised based language is arrest in 200 you're able to improve by 1% in the top five or more than that 1.3 percent if it with with cell supervisory training and you can also see that the low data regime your your your top fire accuracies are so good that they are even better than methods that have used very very engineered semi-supervised pipelines which which we'll cover in a future lecture


so using unlabeled data to improve the data efficiency of label data label they like like low rate a regime supervised learning it's not it's not just specific to cell supervised learning like it can also be done using other methods like semi-supervised learning and and and those numbers representing the that the thick methods that just use label propagation pseudo labeling data augmentee unsupervised at augmentation and so forth will not cover that today but it will be covered in a future lecture and those are also very


interesting but they involve more more handcrafted details as to how you go about doing simultaneous loss optimizations where it's in several ways be training is small like just think it moral destroy it once and then use it everywhere so it's much more elegant that way so here is the final Pascal vo C numbers which is also a benchmark that people have cared about in in terms of transfer learning for self supervised learning there's always been this thing that some supervised learning will is only Garant considered to work if if the


features you get from self supervised learning are gonna transfer to something other than classification like like something like object detection on it on a data set where you don't have a lot of labels like pascal and for a long time people believe that you could never be the supervised baseline so if you look at the supervised baseline mean average position with the rest at 152 backbone you get seventy four point seven pascal walk just two thousand the data set with 2007 version and you can look at all these south supervised methods in the


next row including momentum contrast which which was the first method which god a ball supervised and got got got to seventy four point nine and and and and and and they fast now seen and trained on the CPC version two features get seventy six point six as the mean average position which is which is even better so so that goes on to say like well you know well that self supervised learning can actually work even better than supervised learning for downstream tasks so even if you collect a lot of label data you may not actually be able


to get these numbers because these models are actually learning a lot more about the data so now that you looked at the principle of contrast of predictions and contrasted learning and and and and seen the benefits and of it actually working at scale people start like like more people interested in just the image domain started looking at contrasted learning and people asked us question hey this contrast learning is cool but then do we actually need all these patches like like inherently patches is hard to deal with because when you


actually drew it an image into patches you're increasing your bat size by a lot and so even if you reduce your image size it's not it's not particularly good to pre-trained with much smaller images and find you in with larger images and secondly you also want to make sure that you use Bosch norm during your pee training and when you do something I see PC using Bosch norm is much harder because yeah you don't want information to mix in your pixel cnn's so so therefore people want that to have this version of contrast to learning that


just worked at an instance level where the the context is just one version of your image target is another version of the same image and negatives are just any other images so in this case it could be like hey you just take a picture for dog you perform one data augmentation to it which is just apply a grayscale perform another data augmentation to it which is you flip it and you take a particular random crop so and any other image would be a negative for this particular anchor positive there so what does this actually learn


them you're basically trying to learn that hey this particular you know the legs are absent and the other version and so you're trying to learn that there's a dog here depending on the amount of random cropping and data augmentation you use the level of cheating you can afford to identify the two things with the same is lower lower and therefore you're forced to learn good features to make sure that you identify that two different images presented you are actually fundamentally the same thing compared to any other


image and that way you're able to learn really rich features it's very similar to the CPC area done at a patch level except that you're not actually doing any spatial prediction you're all you're trying to do is identify another version of the same image and this is in general referred to the principle of instances combination and to reason people have really taken this far one paper is called Boco or momentum contrast and other paper is called sim clr or simple contrast of learning for representations for vision and we're


just going to look at these two papers though there are like a lot of other papers that have competed with diesel vapors in the recent past but these two papers are like the simplest and cleanest and also the most well functional in terms of state of the art metrics so first let's look at momentum contrast for unsupervised visual representation learning this is a paper by claiming he was also the inventor of rest nets and faster CNN masks are seen and and so forth so the late works as as false you you basically characterized


contrast to learning as a dictionary lookup task where you're saying that okay hey I want to identify if two things are the same so I present one thing treated as a query and whatever I want to pair with is also present and a bunch of keys I have and there are also lots of other keys which could serve as negatives and I want to identify what is the right positive among these bunch of keys so you in Korea query you encode all your keys and you compute the pairwise similarities you know that true target which is because you know the


ground truth for what is real other augmentation of the same image and then you just build this contrast loss and back prop so that does basically the idea of the instance discrimination so where does momentum come in here so the idea of momentum is to make sure that you're not you can use a slowly varying encoder of the same and you basically have an encoder which is used to encode your queries but your keys are going to use a pol yet a historically average version of the same encoder and this gives you lots of


different benefits so one benefit is that it can let you use a lot of negatives without using your current mini packs which is to say that hey if you have a mini batch and if all your negatives are coming from your same mini batch then your number of negatives is limited by the batch size you have so that means you you you require a large batch size to use for your foot for being really efficient because you need a lot of negatives now that you're doing things at the instance level you don't have patches so what if


you use a memory bank where you have a buffer of previous embeddings and then you just use that buffer as your negatives so for that buffer to function well it needs to be reasonably historically average version of your current encoder so that it can contrast well if it's just your current encoder then the previous embedding stored in the buffer or not rather than anymore so that basically so the idea for moko where you basically take an original image oh and you split it into queries in keys which is the two different


augmentations and you encode both of them you encode the targets using your momentum encoder and you construct these query in keys you know the true target and you just optimize with the contrast loss note that this contrast loss in vocal doesn't use the bilinear product it just uses a unit norm vectors of all these in bearings with the temperature softmax and that that version also works pretty well as long as you can pick the right temperature and this is actually the pseudocode for vocal written in rape in a Python style and I think it's best


understood through this so fq and FK our encoder networks with a query and key and your cue you have a dictionary of cues which is your memory bank of negatives and you have a momentum is a temperature parameter so initially you make the key encoder and the query encoder the same to start with and every time you load a mini batch you construct two different augmentations of it the query and the key and you fast forward the query in the key using the you get the embeddings using the query and key encoders and you


who'da stop gradient on the key encoder you're not going to back probe through the momentum encoder and now you're computing the logits of the positives which is you have the same mini batch so you constructed or different augmentations for the same image so that means that the Pirates products of the corresponding batch indices like if you have a batch of 16 images and you created another batch of 16 images every particular index is basically another augmentation of itself so you can actually take the paralyzed or product


for that and get get get the scores for the positives and for negatives you just use your cue or the memory bank of negatives that you have and you just take your query encoder and you just compute the paralyzed our products with all of this memory bank negatives and now you know that you have the positives and the negatives for your influency loss you just concatenate them and then throw across and repeat laws by creating the true labels and users JT to optimize and and finally you have to perform the momentum update for your key encoder so


that your momentum encoder is slowly changing over time so that's the idea in moko and your cue or the memory bank is a first in first out queue so every time you are your plura upload a new batch of negative in bearings you are you also have to like take it take the the least reason keen a bad-size number of negatives out of the buffer and as long as you can do all this without any mistakes this will work and these are the different ways in which you can do instance contrasted learning one is you just do it end to


end don't care about this momentum just use your current mini box as your negatives use the same encoder for the queries in the keys and back prop the gradients to every everything so that is the end of a notion the other version is hey I don't you know you you just say I count referred a really large bat size I want to use a memory bank of negatives and that would stay that would keep changing dynamically but then I can I'm going to use a lot more negatives that way so that's interesting and then you you cannot backprop to the memory bank


but what you can do is you just applaud your most reason negatives every time you pass forward a new mini batch you just collect those and bearings and you just thank you that your memory bank so that has a problem because like I said if it's not just if it's not changing over time dynamically then then it's possible that your turn encoders in bearings can be correlating very easily with the most recent negatives and you can just ignore all the other negatives on your memory bank and so that way you're not actually taking true


advantage of the number of negatives and the final version is the mokele illusion where you basically use a momentum encoder and then you use that for in queueing and D queuing and you use the you don't actually back prop to it and you only back for up to your queries and you have a lot of you you get the best of using you know just a regular end-to-end version but you also make sure that you can afford a large batch size effectively so here's the local plot where the number of negatives basically is increased over time like is


increased a different iteration increase for different runs of the same algorithm like we have three different versions the moco memory bank and n 2n and increasing the number of negatives and enter in the authors only did it up 2024 because increasing it beyond that needs a lot more GPUs a lot more TP of course you know because it's basically your maths at global back size so having a global back size a thousand 24 is the maximum you can afford right now in 8/8 GPU Volta dgx so the authors didn't expand further on that but as we see


very immediately after this the sims ii as our technique is basically the same thing as n 2n technique in moco but the only difference is they were they scale the top to use bigger batch sizes and use a lot more TPU course so local scales gracefully with the number of negatives which is basically the size of your memory bank and you can see that the benefits are there and as you keep increasing number negatives the linear classification accuracy and the y axis keeps going up so that means you're learning better representations so when


mocha was published it was the state of the art for linear classification like CPC version two was not updated yet so and and so the updated results were already presented to you in the previous slide so Mokomoko had really good linear classifier accuracy where when they made the rest net 4x wider they got 68% top one and and and it's a nice plot of how the number of parameters plays a significant role in giving you like better top-1 accuracy of the linear classifier so finally we look at this paper called zoom CLR or Sinclair simple framework


for contrast learning by think chance I mean complet momen Eruzione geoff hinton so the best way to understand since IAM clear now that you already know CPC and moco is it adopts instance discrimination it and it goes for the end-to-end mechanism proposed in moco so if you look at internment plot and figure a here you back propagating through both the query and the keys and you sharing the same encoder for the queries in the keys and your negatives are just coming from your back so and and and that's really this exact same


thing adopted and Sinclair which is you just take X your input your image mini-batch you perform two different augmentations to it get exciting the UNIX j2d you pass them to the same rest that encoder and you get hecha in history and there is another encoder or another MLP head which takes these mean pool rest net embeddings and puts it into a lower dimension for the contrast of loss and then you just operate the same influency loss with unit vectors similar to moco so the new innovation in Sim CLR is this G


functioned the the demo the the there is a transformation G that takes the rest net embedding and puts it into Layden's rather contrasted losses perform so early earlier versions like CPC and moko do not make use of any depth from when you Benny when you take the embeddings of your context to the targets it basically just uses a one by one convolution to reduce the channel dimension and perform the contrast of the last in a lower dimension in sims CLR you are using a few layers of MLP to transform 2048 dimensional vector that you get from


rest net to something much smaller like 128 dimensions to the contrast to optimization so that's basically the difference and that really helped a lot so here is Sim Sim cleaners main algorithm another interesting version another interesting fact and sim clear is that if you're going to use your own batches your positives and negatives you can you can basically flip what is a query and what is a key so if you take a batch you do two different augmentations one of them becomes the query and one that becomes the key but then what which


one is query image from this key is totally arbitrary so why not just do both ways so they actually do both ways and so that is the determine the loss which is L of 2k minus 1 comma 2 k plus L of 2 K comma 2 K minus 1 CD they just flip their order and then you can just use a large large batch gradient descent they use a large optimizer and use bad sizes of 2048 or 4096 on a cloud TPU and and and they're able to perform this optimization really fast and they also train much longer and just like CPC was trained for 500 a


box they train actually 4,000 bucks and they add a little more data augmentations like Gaussian blur that helps a lot and randomized the order of data augmentations so simply errand was published just two weeks ago actually 2 to 2.5 weeks ago it got stained the art performance on imagenet linear classifier so you can see that just with the resident 50 whatever CPC version to God was 63.8 and moco had like 60 point 1 sim clear just took it all the way to 69 percent a huge jump and when they made the resonance wider which is making


the rest at 2x so 4x wider you get more features linear classification now gets all the way up to 76.5% which is as good as a supervisor learner can be at that it's it is using more features it's using more parameters so it's in the same parameter scenario is not as good as supervised but if you just have a confirm for more parameters it's almost as good as supervised and this this kind of result was lacking for a long long time image image classification sub supervised learning was always lagging behind in supervised learning so so


simple clear was the first to ensure that something can exist in this level so here is a more detailed study of how how you know you could ask a question hey what if you made supervisor also wider would supervise increase as much as supervised it turns out to be not true so like a regular supervised resident get 76% but a wider doesn't actually improve a lot more it doesn't prove it gets to 77 78 but not not not more so the gap between self supervised and supervised release is further and number of parameters is more so the hypothesis


of the authors is that supervised models need me you know they cannot particularly benefit from more data augmentations or more parameters where sub supervised models can actually benefit from that because they're trying to do something much harder so when you actually can afford bigger models you would rather want to go for something like self supervised learning instead of supervised learning if you can continue making more improvements so after Sinclair was published the fact that the MLP has helped a lot made the


moko authors consider using MLP heads as well and also doing a little more engineering more hyper parameter sweeps and also train longer and use the same kind of data augmentations that simply are used but and so the moko authors came back came up with reply to Sinclair in some sense and call their model this moko version to say on the left you can see what it basically is simply R which is the end-to-end version of loco on the right you can see the mocha version exactly the appealing thing about moko is it basically can let you use a lot of


negatives with our naxi using large batches because of its memory buffer and so when you add when you add these ideas so MLP had which is the G function aaaghh plus which is the Gaussian blur augmentation used in simpler and another detail that was used in sim clear was the cosine learning rate decay which moko didn't use if you add all these details then mokos accuracy linear classifier just arrests in 50 encoder improves all the way up to 71 point one percent that just which is which is two percent better than simply results and


it also improves the detection results in Pascal and so and so just by training longer and getting all these extra details right the moko authors were able to get very very impressive self supervised results with just using a GPUs and a bath size of 256 so that's that's that is really the state of the art technique right now and here's also the ablation between moko version one vocal version two and simply ER and you can clearly see like mocha version one from sixty point six it went all the way from sixty point six to sixty seven


point five even without training longer like the seven percent improvement just by adding the same details that simply are had which is using MLP heads cosine learning rate DK and you the extra data augmentations and from 67.5 in it can have another 3.5% improvement by by training longer so so at this stage South supervised learning is really at a stage where it the amount of engineering detail you pay attention to and the amount of trickery you add in terms of model optimizations and like using clever techniques like memory


banks and doing very well on large batch training what data augmentations you use what is the learning rate decays or optimizers those are really the most important parts and getting state-of-the-art numbers and it's only a matter of time before like it's gonna match the results of supervised learning on all these benchmarks like seventy one point files still not as good as 76% so that's still some gap to close in but the the rate of progress is because it's just been two or three months since all these papers came out and it's been


rapidly exploring so so that's that's pretty much it for self supervised learning like we haven't really covered language like subsequent learning for language which is actually are arguably the domain where unsupervised ourselves provides free training has really taken off even before all these revision successes and some like bird which is really famous and has 4,000 citations in a year like it it basically is already productionize dan google search already uses it so that those are bigger successes than CPC


Transcriber: asma youssef Reviewer: Denise RQ What do you think is the key to achieve our goals, our success? Some people suggest things like hard work, focus, persistence. But research shows these are all by-products of something else, something much more powerful that we can all develop. It is this very special something that is really critical to success, and is what I am here to discuss with you today. Someone who has achieved great success is Josh Waitzkin, a chess international master and the subject of the movie

The Power of belief -- mindset and success | Eduardo Briceno | TEDxManhattanBeach


"Searching for Bobby Fischer". Nobody has won all the national chess championships that Josh has. But even more impressive, when he turned 21, he took on the challenge of mastering something completely new and very different from chess: martial arts. He realized that he had learned how to grow and succeed, and he could apply that understanding to other domains. And so, he devoted himself relentlessly to tai chi chuan. And after lots of hard work, many failures, and some broken joints, he became a great martial artist, and he won two world championships.


Now he is off to jiu-jitsu. So what does Josh say is the greatest thing ever happened to him? Believe it or not, he says, "Losing my first national chess championship, because it helped me avoid many of the psychological traps." The key trap that Josh avoided was believing that he was special, that he was smarter than other people, and that he didn't have to work hard. He could have thought of himself as a prodigy, but he doesn't think that he has extraordinary intelligence. He says, "The moment we believe that success is determined by an ingrained level of ability,


we will be brittle in the face of adversity." Josh often quotes Stanford Professor Carol Dweck who discovered that some people see intelligence or abilities as fixed what is called a fixed mindset, while other people see them as Josh does, as qualities that can be developed; a growth mindset. More important, Dr. Dweck discovered that these two different mindsets lead to very different behaviors and results. In a study she did with Dr. Lisa Blackwell, several hundreds seventh graders were surveyed to determine


which mindset each student had, and then they were tracked for two years. Results showed that the students with a growth mindset, those who thought they could change their own intelligence increased their grades over time. While those with a fixed mindset did not. You can see the trend, the gap in performance just widens and widens over time. The difference between these two groups: a different perspective on intelligence. Other studies have shown similar effects for our mindset about other abilities like problem solving, playing sports, managing people,


or anything else you'd like, dancing La Macarena. The key to success is not simply effort, or focus, or resilience, but it is the growth mindset that creates them, the mindset itself is critical. Research shows that when we directly try to build grit or persistence, it's not nearly as effective as addressing the mindset that underlies them. How many of us think of ourselves as not math people, or creative, or sociable, or athletic, or conversely, that we are naturals? If we are to fulfill our potential, we have to start thinking differently.


We have to realize we are not chained to our current capabilities. Neuroscience shows the brain is very malleable. And we can change our own ability to think and to perform. In fact, many of the most accomplished people of our era were thought of, by experts, to have no future. People like Charles Darwin, Lucille Ball, Marcel Proust, and many others. But they, along with all great achievers from Mozart to Einstein, built their abilities. But the key insight I would like you to walk away with today is that when we realize that,


when we realize we can change our own abilities, when we have a growth mindset, we bring our game to new levels. So how does a growth mindset do that? It turns out that there are physiological manifestations to mindset. Brain scans show that for people with a fixed mindset, the brain becomes most active when receiving information about how the person performed such as a grade or a score. But for people with a growth mindset, the brain becomes most active when receiving information about what they could do better next time.


In other words, people with a fixed mindset worry the most about how they are judged, while those with a growth mindset focus the most on learning. There are other consequences of mindset: people with a fixed mindset see effort as a bad thing, something that only people with low capabilities need, while those with a growth mindset see effort as what makes us smart, as the way to grow. And when they hit a set back or a failure, people with a fixed mindset tend to conclude that they are incapable. So to protect their ego, they lose interest or withdraw.


We observe that as lack of motivation. But behind it is a fixed mindset, whereas people with a growth mindset understand that set backs are part of growth. So when they hit one, they find a way around it. Like Josh Waitzkin did when he lost in chess or in martial arts. Research clearly shows these effects of mindset. In one study Dr. Dweck did with Dr. Claudia Mueller, they had children do a set of puzzles, and then they praised the kids. To some of the kids, they said, "Wow, that's a really good score, you must be smart at this."


That's fixed mindset praise because it portrays intelligence or abilities as a fixed quality. To other kids they said, "Wow, that's a really good score, you must have tried really hard." That's growth mindset praise because it focuses on the process. Then, they asked the kids, "OK, what kind of puzzle would you like to do next? An easy one or a hard one?" The majority of the kids who received the fixed mindset praise chose to do the easy puzzle. While the great majority of those who received the growth mindset praise


chose to do challenge themselves. Then the researchers gave a hard puzzle to all of the kids because they were interested in seeing what confronting difficulty would do to their performance. Look at what happened when the kids later went back to the set of easier problems that they started with. The kids who received the fixed mindset praise did significantly worse than they had originally, while those who received a growth mindset praise did better. And to top it off, at they very end, kids were asked to report their scores;


and the kids who received the fixed mindset praise lied about their scores over three times more often than those who received the growth mindset praise. They did not have another way to cope with their failure. The difference between these two groups: one short little sentence. How often do we praise kids for being smart or for being great at something? We have been told that this will raise their self-esteem. But instead, it puts them in a fixed mindset. They become afraid of challenges, and they lose confidence when things hit hard.


As Josh Waitzkin says, "It is incredibly important for parents to make their feedback process related as oppose to praising or criticizing talent. If we win because we are winners, then when we lose, it must make us losers." These studies show not only the mechanisms by which mindset affects performance, but they also show something else that is very important: they show that we can change mindsets, and that's important, because most of us have fixed mindsets about something. Another study that showed that we can change mindsets


is one in which Dweck and Blackwell did a workshop with seventh graders to instill a growth mindset in them. As a result of the workshop, the students gained more interest in learning, and they worked harder; and as a result of that, their grades improved. Other studies have shown that when we teach a growth mindset, not only that it improves achievements for students as a whole but it also narrows the achievement gap, because the effects are most pronounced for the students who face negative stereotypes such as minority students, and girls in math.


I have spoken mostly about children, but mindsets affects all of us. In our work places, managers with fixed mindsets don't welcome feedback as much, and they don't mentor employees as much. And employees with growth mindsets about specific skills like negotiations become far better at those skills than people with fixed views. Mindsets can even help us solve big social issues. A recent study showed that when we expose Israelis and Palestinians to the idea that groups can change, they increase their attitudes towards towards one another,


they improve them. and they enhance their willingness to compromise and to work for peace. We also see the effects of mindsets on relationships, sports, health. How is it possible that as a society, we are not asking schools to develop a growth mindset in children? Our myopic efforts to teach them facts, concepts, and even critical critical thinking skills is likely to fail, if we don't also deliberately teach them the essential beliefs that will allow them to succeed not only in school but also beyond. There is a lot that we can do to change mindsets,


but here are three things that any of us can do to instill a growth mindset in ourselves and in those around us. First, recognize that the growth mindset is not only beneficial but it is also supported by science. Neuroscience shows that the brain changes and becomes more capable when we work hard to improve ourselves. Second, learn and teach others about how to develop our abilities. Learn about deliberate practice and what makes for effective effort. When we understand how to develop our abilities, we strengthen our conviction that we are in charge of them.


you're one of the one of the only people who dared boldly to try to formalize our the idea of artificial general intelligence to have a mathematical framework for intelligence just like as we mentioned termed IHC AI X I so let me ask the basic question what is IX e ok so let me first say what it stands for because letter stands for actually that's probably the more basic question but it the first question is usually how how it's pronounced but finally I put it on the website how it's pronounced and you figured it out yeah the name comes from

Marcus Hutter: What is AIXI? | AI Podcast Clips


AI artificial intelligence and the X I is the Greek letter X I which are used for solo manav's distribution for quite stupid reasons which I'm not willing to repeat here in front of camera so it just happened to be more less arbitrary I chose this I but it also has nice other interpretations so their actions and perceptions in this model around an agent has actions and perceptions and overtime so this is a Index IX index I so this action at time I and then followed by reception at time I will go with that I lit out the first part yes


I'm just kidding I have some more interpretations so at some point maybe five years ago or ten years ago I discovered in in Barcelona it wasn't a big church there was in you know it's stone engraved some text and the word I see appeared there very surprised and and and and happy about it and I looked it up so it is Catalan language and it means with some interpretation of dead say that's the right thing to do yeah Eureka oh so it's almost like destined somehow came came to you in a dream so Osceola there's a Chinese word I she


also written our galaxy if you transcribe that opinion then the final one is there is AI crossed with interaction be her status and it's going more to the content now so good old-fashioned AI is more about you know planning known the domestic world and induction is more about often you know iid data and inferring models and essentially what this is a model does is combining these two and I actually also recently I think heard that in Japanese AI means love so so if you can combine excise somehow with that I think we can there might be


some interesting ideas there so I let's then take the next step can you maybe talk at the big level of what is this mathematical framework yeah so it consists essentially of two parts one is the learning and induction and prediction part and the other one is the planning part so let's come first to the learning induction prediction part which essentially explained already before so what we need for any agent to act well is that it can somehow predict what happens I mean if you have no idea what your actions do how can you decide which


acts not good or not so you need to have some model of what your actions affect so what you do is you have some experience you build models like scientists you know of your experience then you hope these models are roughly correct and then you use these models for prediction and the model is sorry to interrupt the model is based on your perception of the world how your actions will affect our world that's not so what is the important part but it is technically important but at this stage we can just think about predicting say stock market


data weather data or IQ sequences one two three four five what comes next yeah so of course our actions affect what we're doing what I come back to that in a second so and I'll keep interrupting so just to draw a line between prediction and planning or what do you mean by prediction in this in this way it's trying to predict the the environment without your long-term action in the environment what is prediction okay if you want to put the actions in now okay then let's put in a now yes so another question ok so the simplest form


of prediction is that you just have data which you passively observe and you want to predict what happens without you know interfering as I said weather forecasting stock market IQ sequences or just anything okay and Salama of theory of induction based on compression so you look for the shortest program which describes your data sequence and then you take this program run it it reproduces your data sequence by definition and then you let it continue running and then it will produce some predictions and you can


rigorously prove that for any prediction task this is essentially the best possible predictor of course if there's a prediction task our task which is unpredictable like you know your fair coin flips yeah I cannot predict the next reckon but Solomon of Tarsus says ok next head is probably 50% it's the best you can do so if something is unpredictable so Lamar will also not magically predict it but if there is some pattern and predictability then Solomonov induction we'll figure that out eventually and not just eventually


but rather quickly and you can have proof convergence rates whatever your data is so there's pure magic in a sense what's the catch well the catch is that is not computable and we come back to that later you cannot just implement it in even this Google resources here and run it and you know predict the stock market and become rich I mean if ray solomonoff already you know tried it at the time but the basic task is you know you're in the environment and you interact with environment to try to learn a model the environment and the


model is in the space as these all these programs and your goal is to get a bunch of programs that are simple and so let's let's go to the actions now but actually good that you asked usually I skip this part also that is also a minor contribution which I did so the action part but they're usually sort of just jump to the decision path so let me explain to the action partner thanks for asking so you have to modify it a little bit by now not just predicting a sequence which just comes to you but you have an observation then you act somehow


and then you want to predict the next observation based on the past observation and your action then you take the next action you don't care about predicting it because you're doing it and then you get the next observation and you want more before you get it you want to predict it again based on your past excellent observation sequence you just condition extra on your actions there's an interesting alternative that you also try to predict your own actions if you want oh in the past or the future what are your future actions


wait let me wrap I think my brain is broke we should maybe discuss that later Biff after I've explained the aixi model that's an interesting variation but there's a really interesting variation and a quick comment I don't know if you want to insert that in here but you're looking at in terms of observations you're looking at the entire the big history a long history of the observations exact that's very important the whole history from birth sort of of the agent and we can come back to that I'm also why this is important here


often you know in RL you have MVPs Markov decision processes which are much more limiting okay so now we can predict conditioned on actions so even if the influence environment but prediction is not all we want to do right we also want to act really in the world and the question is how to choose the actions and we don't want to greedily choose the actions you know just you know what is best in in the next time step and we first I should say you know what is you know how to be measure performance so we measure performance by giving the agent


reward that's the so called reinforcement learning framework so every time step you can give it a positive reward or negative reward or baby no reward it could be a very scarce right like if you play chess just at the end of the game you give +1 for winning or -1 for losing so in the aixi framework that's completely sufficient so occasionally you give a reward signal and you ask the agent to maximize river but not greedily sort of you know the next one next one because that's very bad in the long run if you're greedy so


but over the lifetime of the agent so let's assume the agent lives for M times that was there it dies in sort of hundred years sharp that's just you know the simples model to explain so it looks at the future reward some and ask what is my action sequence or actually more precisely my policy which leads in expectation because of the know the world to the maximum reward some let me give you an analogy in chess for instance we know how to play optimally in theory it's just a minimax strategy I play the move which seems best to me


under the assumption that the opponent plays the move which is best for him so best so worst for me and the assumption that he I play again the best move and then you have this expecting max three to the end of the game and then you back propagate and then you get the best possible move so that is the optimal strategy which for norman already figured out a long time ago for playing adversarial games luckily or maybe unluckily for the theory it becomes harder the world is not always adversarial so it can be if the other


humans in cooperative fear or nature is usually I mean the dead nature is stochastic you know you know things just happen randomly or don't care about you so what you have to take into account is in noise yeah and not necessarily doesn't really so you'll replace the minimum on the opponent's side by an expectation which is general enough to include also adversarial cases so now instead of a minimax trials you have an expected max strategy so far so good so that is well known it's called sequential decision theory but the


question is on which probability distribution do you base that if I have the true probability distribution like say I play backgammon right there's dice and there's certain randomness involved you know I can calculate probabilities and feed it in the expecting max or the signatures eg come up with the optimal decision if I have enough compute but in the before the real world we don't know that you know what is the probability you drive in front of me brakes and I don't know you know so depends on all kinds of things and especially new


situations I don't know so is it this unknown thing about prediction and there's where solomonov comes in so what you do is in sequential decision jury it just replace the true distribution which we don't know by this Universal distribution I didn't expect a talk about it but this is used for universal prediction and plug it into the sequential decision mechanism and then you get the best of both worlds you have a long-term planning agent but it doesn't need to know anything about the world because the Solomon reduction part


learns can you explicitly try to describe the universal distribution and how some of induction plays a role here yeah I'm trying to understand so what it does it so in the simplest case I said take the shortest program describing your data run it have a prediction which would be deterministic yes okay but you should not just take a shortest program but also consider the longer ones but keep it lower a priori probability so in the Bayesian framework you say our priori any distribution which is a model or stochastic program has a certain a


pre or probability which is two - two - and why - ders - length you know I could explain length of this program so longer programs are punished yeah a priori and then you multiplied with the so called likelihood function yeah which is as the name suggests is how likely is this model given the data at hand so if you have a very wrong model it's very unlikely that this model is true so it this very small number so even if the model is simple it gets penalized by that and what you do is then you take just the sum word this is the average


over it and this gives you a probability distribution so it was universal distribution of Solomon of distribution so it's weighed by the simplicity of the program and likelihood yes it's kind of a nice idea yeah so okay and then you said there's you're playing N or M or forgot the letter steps into the future so how difficult is that problem what's involved there okay so here's a cop to mutation problem what do we do yeah so you have a planning problem up to the horizon M and that's exponential time in in the horizon M which is I mean it's


computable but in fact intractable I mean even for chess it's already intractable to do that exactly and you know it could be also discounted kind of framework or so so having a hard horizon you know at number of years it's just for simplicity of discussing the model and also sometimes the mass is simple but there are lots of variations actually quite interesting parameter it's it's there's nothing really problematic about it but it's very interesting so for instance you think no let's let's pin let's let the parameter M tend to infinity right


you want an agent which lives forever all right if you do it novel you have two problems first the mathematics breaks down because you have an infinite revert sum which may give infinity and getting river 0.1 in the time step is infinity and giving you got one every time service affinity so equally good not really what we want another problem is that if you have an infinite life you can be lazy for as long as you want for ten years yeah and then catch up with the same expected reward and you know think about yourself or you know or


maybe you know some friends or so if they knew they lived forever you know why work hard now or you know just enjoy your life you know and then catch up later so that's another problem with infinite horizon and you mentioned yes we can go to discounting but then the standard discounting is so called geometric discounting so $1 today is about worth as much as you know one dollar and five cents tomorrow so if you do this local geometric discounting you have introduced an effective horizon so the agent is now motivated to look ahead


a certain amount of time effectively it's likely moving horizon and for any fixed effective horizon there is a problem to solve which requires larger horizon so if I look ahead you know five time steps I'm a terrible chess player right and I need to look ahead longer if I play go I probably have to look ahead even longer so for every problem no forever horizon there is a problem which this horizon cannot solve yes but I introduced the so-called near harmonic horizon which goes down with one over T rather than


exponential in T which produces an agent which effectively looks into the future proportional to its age so if it's five years old it plans for five years if it's hundred years older than plans for hundred years interesting and a little bit similar to humans - right and my children don't I had very long within we get the doll to a play I had more longer maybe when we get all very old I mean we know that we don't live forever and maybe then how horizon shrinks again so just adjusting the horizon what is there some


mathematical benefit of that of or is it just a nice I mean intuitively empirically we'll probably a good idea to sort of push a horizon back to uh extend the horizon as you experience more of the world but is there some mathematical conclusions here that are beneficial mr. lamagno who talks to the prediction party of extremely strong finite time but no finite data result so you have sown so much data then you lose sown so much so the dt r is really great with the aixi model with the planning part many results are only asymptotic which well


this is what is asymptotic means you can prove for instance that in the long run if the agent you know x long enough then you know it performs optimal or some nice things happens so but you don't know how fast it converges yeah so it may converge fast but we're just not able to prove it because the typical problem or maybe there's a bug in the in the in the model so that is really dead slow yeah so so that is what asymptotic means sort of eventually but we don't know how fast and if I give the agent a fixed horizon M yeah then I cannot prove


our synthetic results right so I mean sort of pivot dies in hundred years then it number uses over cannot say eventually so this is the advantage of the discounting that I can prove our synthetic results so just to clarify so so I okay I made I've built up a model well now in a moment I've have this way of looking several steps ahead how do I pick what action I will take it's like with the playing chess right you do this minimax in this case here do expect the max based on the solo mode of distribution you propagate back and then


while inaction falls out the action which maximizes the future expected reward on the Solano's distribution and then you take the section and then repeat that you get a new observation and you feed it in this extant observation then you repeat and the reward so on yeah so you rewrote - yeah and then maybe you can even predict your own action I love the idea but ok this big framework what is it this is I mean it's kind of a beautiful mathematical framework to think about artificial general intelligence what can


you what does it help you into it about how to build such systems or maybe from another perspective what does it help us to in understanding AGI so when I started in the field I was always interested two things one was you know AGI i'm the name didn't exist 10 2014 area wa strong AI and physics c or everything so i switched back and forth between computer science and physics quite often you said the theory of everything the theory of everything just alike it was a basic that regulators problems before all all of humanity yeah I can explain


if you wanted some later time you know why I'm interested in these two questions Nestle and a small tangent if if if one to be it was one to be solved which one would you if one if you were if an apple fell in your head and there was a brilliant insight and you could arrive at the solution to one would it be a GI or the theory of everything definitely a GI because once the a GI problem solve they can ask the a GI to solve the other problem for me yeah brilliantly okay so so as you were saying about it okay so and the reason


why I didn't settle I mean this thought about you know once you have solved a GI it solves all kinds of other not just as your every problem but all kinds of use more useful problems to humanity it's very appealing to many people and you know and it is thought also but I was quite disappointed with the state of the art of the field of AI there was some theory you know about logical reasoning but I was never convinced that this will fly and then there was this for more holistic approaches with neural networks and I


didn't like these heuristics so and also I didn't have any good idea myself so that's the reason why I toggle back and forth quite some violent even work so four and a half years and a company developing software something completely unrelated but then I had this idea about the ITC model and so what it gives you it gives you a gold standard so I have proven that this is the most intelligent agents which anybody could build built in quotation mark right because it's just mathematical and you need infinite compute yeah but this is the limit and


this is completely specified it's not just a framework and it you know every year tens of frameworks are developed with just have skeletons and then pieces are missing and usually this missing pieces you know turn out to be really really difficult and so this is completely and uniquely defined and we can analyze that mathematically and we've also developed some approximations I can talk about it a little bit later that would dissolve the top-down approach like say for Normans minimax theory that's the theoretical optimal


play of games and now we need to approximate it put heuristics in prune the tree blah blah blah and so on so we can do that also with a hike symbol but for generally I it can also inspire those and most of most researchers go bottom-up right they have the systems that try to make it more general more intelligent it can inspire in which direction to go what do you mean by that so if you have some choice to make right so how should they evaluate my system if I can't do cross validation how should I do my learning if my standard


regularization doesn't work well yeah so the answers always this we have a system which does everything that's actually it's just you know completing the ivory tower completely useless from a practical point of view but you can look at it and see oh yeah maybe you know I can take some aspects and you know instead of Kolmogorov complexity there just take some compressors which has been developed so far and for the planning well we have used it here which is also you know being used in go and it I at least it's inspired me a lot


to have this formal definition and if you look at other fields you know like I always come back to physics because I'm a physics background think about the phenomena of energy that was a long time a mysterious concept and at some point it was completely formalized and that really helped a lot and you can point out a lot of these things which were first mysterious and wake and then they have been rigorously formalized speed and acceleration has been confused tried until it was formally defined here there was a time like this and and people you


know often you know I don't have any background you know still confuse it so and this is a model or the the intelligence definitions which is sort of the dual to it we come back to that later formalizes the notion of intelligence uniquely and rigorously so in in a sense it serves as kind of the light at the end of the tunnel so yeah so I mean there's a million questions I could ask her so maybe the kind of okay let's feel around in the dark a little bit so there's been here a deep mind but in general been a lot of breakthrough


ideas just like we've been saying around reinforcement learning so how do you see the progress in reinforcement learning is different like which subset of IHC does it occupy the current like you said the maybe the Markov assumption is made quite often in reinforce for learning the there's this other assumptions made in order to make the system work what do you see is the difference connection between reinforcement learning and I AXI and so the major difference is that essentially all other approaches they make stronger assumptions so in


reinforcement learning the Markov assumption is that the the next state or next observation only depends on the on the previous observation and not the whole history which makes of course the mathematics much easier rather than dealing with histories of course their profit from it also because then you have algorithms which run on current computers and do something practically useful for generally re all the assumptions which are made by other approaches we know already now they are limiting so for instance usually you need a go


digital assumption in the MTP framework in order to learn it goes this T essentially means that you can recover from your mistakes and that they are not traps in the environment and if you make this assumption then essentially you can you know go back to a previous state go there a couple of times and then learn what what statistics and what the state is like and then in the long run perform well in this state yeah but there are no fundamental problems but in real life we know you know there can be one single


action you know one second of being inattentive while driving a car fast you know you can ruin the rest of my life I can become quadriplegic or whatever so and there's no recovery anymore so the real world is not err gorica i always say you know there are traps and there are situations we are not recover from and very little theory has been developed for this case what about what do you see in there in the context of I see as the role of exploration sort of you mentioned you know in the in the real world and get into trouble and we


make the wrong decisions and really pay for it but exploration is seems to be fundamentally important for learning about this world for gaining new knowledge so is it his exploration baked in another way to ask it what are the parameters of this of IHC it can be controlled yeah I say the good thing is that there are no parameters to control and some other people drag knobs to control and you can do that I mean you can modify axes so that you have some knobs to play with if you want to but the exploration is directly baked in and


that comes from the Bayesian learning and the long-term planning so these together already imply exploration you can nicely and explicitly prove that for a simple problem like so-called bandit problems where you say to give a real good example say you have two medical treatments a and B you don't know the effectiveness you try a a little bit be a little bit but you don't want to harm too many patients so you have to sort of trade-off exploring yeah and at some point you want to explore and you can do the mathematics and


figure out the optimal strategy it took a Bayesian agents they're also non-bayesian agents but it shows that this patient framework by taking a prior over possible worlds doing the Bayesian mixture than the Bayes optimal decision with long term planning that is important automatically implies exploration also to the proper extent not too much exploration and not too little in this very simple settings in the aixi model and was also able to prove that it is a self optimizing theorem or asymptotic optimality theorems or later only asymptotic not


finite time bounds it seems like the long term planning is a really important but the long term part of the planet is really important yes and also I mean maybe a quick tangent how important do you think is removing the Markov assumption and looking at the full history sort of intuitively of course it's important but is it like fundamentally transformative to the entirety of the problem what's your sense of it like because we all we make that assumption quite often it's just throwing away the past now I think it's


absolutely crucial the question is whether there's a way to deal with it in a more holistic and still sufficiently well way so I have to come everything up and fly but you know you have say some you know key event in your life you know a long time ago you know in some city or something you realized you know that's a really dangerous street or whatever right here and you want to remember that forever right in case you come back they're kind of a selective kind of memory so you remember that all the important events in the past but somehow


selecting the importance is see that's very hard yeah and I'm not concerned about you know just storing the whole history just you can calculate you know human life says 30 or 100 years doesn't matter right how much data comes in through the vision system and the auditory system you compress it a little bit in this case La Salette and store it we are soon in the means of just storing it yeah but you still need to the selection for the planning part and the compression for the understanding part the raw storage I'm really not concerned


howdy-ho how's it going so today we are going to try out the DET är the end to end object detection with transformers from facebook AI research and they have a github repo and they pretty much give you everything like the model the pre trained weights and so on so today we're going to check out how easy it is to get started with that so in order to do that they have like a collab but we we won't look at it too much I've glanced at it and we'll basically see how far can we go without looking at it too much and how easy is that so what

[Code] How to use Facebook's DETR object detection algorithm in Python (Full Tutorial)


I've done is I've spun up a collab that I will share at the end and I've imported torch and just loaded the model so you don't have to wait for that to happen so I've loaded that up and now we have it in the cache so now we can basically go ahead and load an image into the model and try to detect objects in the image so first of all this is super easy right you simply load this from torch hub it's kind of like the the tensorflow hub you simply give the name of the model you say I want the pre trained please chugga-boom you now have


a model so if we look at that model this is going to be this entire Det our model right here with all the transformer and ResNet and whatnot okay this is almost a bit too much right here so what we want is an image so let's go find an image where better they find an image than Google so let's find an image of dogs because dogs is one of the classes in this cocoa dataset this one's nice right okay so we want the image address we want to load it in here somehow so so that the URL is let's make this into some sort of like an input thing where


we can paste the URL right here okay there we go so we have this right here and that's the URL all right no that's not the URL at all is it but a beam but a boom what about cool better now we need to load this for that we gonna use the requests library always a pleasure requests requests so the way to load a binary file is you can put the URL here and you can say streamed here I glanced this from the other thing and the raw entry will get you the eventual deeply bytes no oh sorry get darrell streamed stream yeah so this


will get you the sort of the the bytes of the image and then use just say image dot open and of course we need the image from a the pill library the python image library so import image we got that and we can open that image up and with a bit of luck yeah yeah so this model expects I think cocoa dataset is 640 by 480 images but they if you can see right here and we're gonna take a quick glance at their transforming they resize it to 800 so we're gonna we're gonna steal that part right here people last time where some some found it really funny


that I called copy pasting to go Suraj so will from now and we'll call it just Suraj Inge what we also need are the class labels because that's in defined in the cocoa dataset right so these are the class labels let's take those and okay so this T here these are torch vision transforms we're gonna need that so from say so if you don't know torch vision it's kind of an addition to PI torch that just helps you with with images and has a lot of data sets and these transforms they're really helpful because so let's


call this image because you can you know resize but they have much more like random cropping and rotating images and so on pretty much everything you need for pre-training and this here is just the standard image net I believe the image net normalization so these are the means and these are the standard deviations from the image net data set and let's already resize our image actually to this weight hundred and I believe I believe if you rescale the 640 to 800 you get 600 here right fairly sure okay and then let's display it just


because we can okay what it's it's a bit squished but we don't care and let's put that up here so we only need to execute it once nice okay so from now on it should be a breeze so what these transforms do is they resize the image okay we don't need that anymore they make it into a tensor and then they normalize by that so if we run our image through this because our image right now is this is pill image right so our our image is this pill image but if we run it through the transforms then we'll get a tensor so that's pretty cool so the


model as it is a deep learning model that expects batches so we'll unscrew is that in the first dimension and then we get batches so shape let's see we don't have on skis no of course we don't so this is a one image of three channels of 600 by 800 so this is the Y index coordinates I guess are shifted yes in pi torch cool so we'll call this our image tensor now we just need to put it into the model so model we put that in there and since we don't let's actually up here put the model in eval mode I don't know if that's already done


but you know you can never be sure enough that the batch norms aren't so I think it probably doesn't have batch norms okay you're not utilizing the GPU we'll do that we'll do that Thanks so how do we use the GPU we put our model on the GPU model equals model CUDA yes yes yes I think so this is gonna work okay we're gonna come back to this later so we forward our image of course we also need that on the GPU and it's worked did this work this worked nice okay and since this is just for evaluation we should probably go with no


grad right here because we don't need this whole gradient stuff if we do that okay I'm dumb there you go and nothing happens of course because we need to capture the output somehow let's look at that output Wow Wow just wow so the output is a dictionary right because we get back class labels and bounding boxes so let's look at the bread boxes let's look at that tensor that's a tensor very nice let's look at its shape let's not print giant tensors anymore cool so since this was a batch of one we should probably go


the zeroeth and you can see right here there is a hundred bounding boxes and each one has four numbers and if you go with the other thing that's in there the log it's then you'll see that there are also should be a hundred log it's and hello there should be a hundred log it's and each one is of size 92 because there are 92 different classes 92 we'll see about that well one is going to be the nothing class right by the way how many classes do we have we have 91 classes okay cool we can deal with that all right so what


are we gonna do next what we want to do is for each of the for each of the for each of the log it predictions we want to find which classic corresponds to so what we're going to do is we're going to take the Arg max of the last dimension right so you can see here almost all of these things correspond to class 91 and class 91 is not in our classes because our class is only length 91 so that must be the nothing class so what we can technically do is for log its and boxes in let's just zip them together and [Music]


like this okay class is oops class as the law gets Arg max if that's 92 or let's say safe that's larger than the length of our classes we'll just skip it for now okay so that should work somehow and if not then our label should be the class index right here so let's just see what the detector detects right here it detects nothing why does it detect nothing that's isn't seem good what are we doing wrong we zip together the log it's oh yeah of course we still need the zero with entry we are dumb dumb dumb cool so so so so we can delete this and


now finally beautiful dogs - dogs detected excellent so now for each of these dogs we want the bounding box okay so now we somehow need to think of how are we gonna draw this on an image and well let's let's actually make a copy of that image because I don't really trust myself and then at the end of this we're just going to display that image right now actually the reason I make a copy is because in these in this pillow library you can actually draw on these images and we're going to that to draw these bounding boxes so for


that we need an image draw if I remember correctly and I think later we also want some text so we need an image font yes all right so let's draw a bounding box right here where so first of all let's look at that bounding box let's call this box box print box dot shape and break right here what's happening let's not do this right now so this is a boxes of size four now this could be two things it could be X 0 y 0 X 1 Y 1 so the two corner points or the kind of the boundaries or it could be X Y width height now from the paper I know that


they predict the center and the width and the height so I'm gonna go with that and I'm just gonna guess that it's like X Y WH and not some other way around if this is a bad guess then yeah we'll see we can just print out one of these boxes and honestly that looks reason oh by the way we should scale that up yeah so these are normalized coordinates probably between 0 and 1 so we should scale that up so we should probably the x coordinates which is scaled by 800 and the Y by 600 so let's do it so first of all we scale our box by 800 in the X and


here is a Y and the width is the X direction and this is the Y Direction boom okay we should probably get that on CPU will just hack together a bunch of things right here ok so now this isn't the correct so we sold our x and y and WH are going to be this box so now we need to actually draw on the image we're gonna do that so let's first go X 0 X 1 is X minus W 1/2 X plus W half y 0 y 1 is the same for a y with H plus H half Coolio now we need an image draw object so I think draw on this image so whatever you


draw on the draw object will end up on the image so we can use that to draw a bounding box and let's just quickly look it up so pill Python draw rectangle maybe there we go okay so there's this rectangle yeah there's the rectangle function and you can see you put in a shape XY here and width height like this wait for real we wouldn't even have to need to transform it I'm pretty sure you can go X I thought I remember you could do the different thing as well but it's called rectangle okay so let's do that so draw rectangle and we'll go we'll go


X 0 or we'll go X Y width height let's display that down here yeah that looks that looks nothing like we want but it's you know it's a start maybe actually we need the other thing here we need X 0 y 0 X 1 Y 1 mm yes yes doggy okay we still have the break in here now we get both dogs nice nice okay let's do I think Phil yes red and let's go for with five or so five seems like a good width oh god five is a terrible with oh it's not feel I think it's its outline yeah yeah okay okay let's go still go with five cool we got our dogs now we need to


put like some some snappy text labels I think there's actually a pill image draw text I think that exists because I've this font thing yeah exactly so you need the font thing get it font in there and then yeah exactly you could put a text like this okay so you probably need the x and y coordinates of the text so let's do that W dot text and let's just go with x and y right here put it right in the middle and the text is going to be our label of course and we want the fill that's now going to be the color of the text let's


go with white and the font we're going to load some font right here font dot how we're doing this true type true type ah no not cheating let's just go with regular fonts it won't look as fancy but we'll be fine so we're where is our text you see it I don't see it red let's make it red yes there we go okay so it wasn't red enough this should work on it so I did we just I just not see it I'm domina cool so we have two dogs how easy was that actually we wasted the most time with like bounding boxes and stuff absolutely cool right okay so now we can


have some fun with it I'm going to scale this down for a bit because you don't need to see the actual code anymore so much so you can see the image more so we'll go to the images and the first thing I want to do is the dress what does this think of the dress okay so we'll copy that and we'll go into our collab and just paste this right here butter boom but a beam sounds nice and what is wrong the size of a tensor must match the size of a tensor we do something wrong transform image or images this maybe this is like an RGBA image I think


if this is rgba we should just convert it to like an RGB pretty sure you can do something like this right here this should work as an alpha Channel then that will remove it yes now it works okay let's see what the model thinks of this yeah okay apparently there's a car and there's a surfboard and there's a person and there's a person nice see well we didn't figure out whether the dress was blue or white through gold it was just a person now they you could actually like threshold by how sure you are of a given


class but where's the fun in that so let's go further and let's do some Rorschach inkblots because those are always lots and lots of fun so which one should we go for it this one looks like fun okay so we'll put this into here and it's astonishing right it's this cocoa data said it only has these 90 classes like it doesn't have anything anything else so it's a cake it's a cake and this here what is it okay we'll have to go maybe with blue what is it stop sign okay but so you might think it what if what if we want more like what if we


want more predictions so there is a hack right right now the model can always assign math to this not a class thing like right here this class 91 in order for it to say I don't think there's anything there but generally we have a hundred predictions right so you see where this is going so yes let's let's change it but let's change it up a bit and let's go here let's first extract these tensors and boxes okay so we have the boxes and this and log its and boxes okay so we got that what we wanna do is basically we


want to filter the we want to basically just remove the last class before we do the Arg max and thereby we want to force the model to make a prediction now it won't be a very good prediction because of course this is only the second highest class and it's arguable how much that counts but still it will do something so this must be done in the log it's right so well look at the log it's and the log it's our of shape 100 so we have 100 predictions of 92 classes now the first thing we want to do is just remove the last class so let's go


everything here until the last class all right so now we have 91 actually let's make it more generic whatever thing however many classes are okay so we don't have this class anymore so now if we do the softmax over the last thing we can technically we get 91 but now they're normalized so they add up to one so it's kind of a probability distribution next we we want to find the max over this and that that will give us a max output so we don't want to plot all the 100 predictions because that would just be like like squares all over


the place and we'd have no clue what happening so this max output right here this what we're trying to find is we're trying to find a let's say the five best predictions or so the five ones where the model thinks where the model is most confident it's not really good metric but you know so these are the probability values of all of the hundred predictions so what we want is like the top K okay so let's go with five and again we'll get like a top K output let's call that top K and I think it also has like values and indices yes so


now we simply need to filter from the log it's and the boxes where these these top ones are so well filter the log it's [Music] will filter the log it's by that top K indices and we'll also filter thee I am not very gifted today boxes by the way I'm using a collab just because it's nice to kind of play around with a model because if I were to use a file I'd have to restart reload the model over and over again just not as nice so now we have the log it's and the boxes and if we do that right now we get always the top 5 predictions how nice is


that and you can see the top 5 predictions are probably still kkkkkk cake and just to verify that and we can put its shape yeah see this is what I don't like about this stuff yes okay so we just have five predictions of 92 things and we don't want the 92 we've already said so we just want the 91 let's actually could put that here [Music] okay so now we have five by 91 and now to give us the top five are there we go so many takes and many stop sighs that's fine that's cool so the ultimate test right here is going


to be yes the human adversarial example let's check it out so we put in a Jackson Pollock image and we'll see what the model says now we're actually forcing it to make predictions right so it can't escape it will need to do something okay I made another mistake I would need to copy the image address right here like this that's what happens when you're not an idiot you get the actual image so what does the model think of our pretty image okay can't even read that so let's make this into white bird bird bird okay lots


of birds in this image clearly clearly lots of birds in this image let's try another one let's go with this this this one yes yes absolutely love it love it okay so we copy image address and beam Mormons Wow there's a lot of birds in these Pollock images just so many birds okay let's try one last how about this one this one is a bit more human-friendly right put it in here and and and okay we get some detections there's a clock right here there is a what's that how's horses let's print let's print the labels so just so we know what they are


cake horse car horse and clock okay so I see the clock like this here is clearly a clock then this rectangle on the right side must be something let's put this to read as well now that's terrible ah white back to white how about back to white okay clock we got horse right here and house probably and the entire image is again a cake yes okay so as you can see it is a pretty pretty good system but of course it is only these 90 classes but it's for now it's a it's pretty cool and it works pretty well and just the easiness with which


you get which which you can get this stuff elephants in Kruger National Park just the easiness is astonishing you can just load it up kind of have this have a bit of a notebook and with a bit of like a very few lines of code you can put something together that detects these bounding boxes lots of elephants and remember we only have the top five elephants right here so what happens if we go for more where is our top k so here we can let maybe say the top 15 predictions and as always if we want to make the model to


Okay. Hey, everyone, looks like we're on. So as usual, if you have not yet, um, please enter your SUID so that we know you're here in this room. Um, so actually, can you hear me okay at the back? Is it okay? Oh, yes, is the volume okay at the back? All right. No one's responding. Yes, okay. All right. [LAUGHTER] Thank you. Okay. So, um, what I want to do today is, um, share with you two things. You know, we're approaching the end of quarter. Uh, I hope you guys are looking forward to, to the Thanksgiving break, um, next week.

Stanford CS230: Deep Learning | Autumn 2018 | Lecture 8 - Career Advice / Reading Research Papers


Um, actually and I guess we have a lot of home viewers, but those us- those of you that are viewing this from outside California, know that we're all feeling really bad air here in California. So I hope, if you're somebody watching at home you have better air wherever you are. Um, uh, but, uh, what I hope to do today is give you some advice that will set you up for the future, uh, so if even beyond the conclusion of CS230. And in particular, what I want to do today is, um, share with you some advice on how to read research papers, uh, because, you know,


deep learning is evolving fast enough that even though you've learned a lot of foundations of deep learning and learned a lot of tips and tricks and probably know better than many practitioners how to actually get deep, deep learning algorithms to work already. Uh, when you're working on specific applications whether in computer vision or natural language processing or speech recognition or something else, um, for you to be able to efficiently figure out the academic literature on key parts of, uh, the, the deep learning world, will help you keep on developing and, you know,


staying on top of ideas even as they evolve over the next several years or maybe decade. So first thing I wanna do is, uh, give you advice on how, uh, when say, when I'm trying to master a new body of literature, how I go about that and hope that those techniques would be useful to help you be more efficient in how you read research papers. And then a second thing is, in previous offerings of this course, one request from a lot of students was just advice for navigating a career in machine learning. And so in the second half of today,


I want to share some thoughts with you on that. Okay, so it turns out that- so I guess two topics reading research papers, right? Um, and, uh, then second career advice in machine learning. So it turns out that, uh, you know, reading research papers is one of those things that a lot of P- PhD students learn by osmosis, right? Meaning that if you're a PhD student and you see, you know, a few professors or see other PhD students do certain things, then you might try to pick it up by osmosis. But I hope today to accelerate your efficiency


in how you acquire knowledge yourself from the, uh, from the a- academic literature, right? And so let's say that this is the area you want to become good at, let's say you want to build that, um, speech recognition, right? Let's turn this off now. Let's say you want to build that, um, speech recognition system that we talked about, the Robert turn on and the desk lamp. All right. Um, this is what I've read- this is the sequence of steps I recommend you take, uh, which is first: [NOISE] compile lists of papers and the- and by papers,


I mean, both research papers often posted on arXiv, onto the Internet, but also plus Medium posts, um, [NOISE] you know, what maybe some occasional GitHub post although those are rarer. But whatever texts or learning resources you have. And then, um, what I usually do is end up skipping around the list. All right. So if I'm trying to master a new body of knowledge, say you want to learn the most speech recognition systems, this is what it feels like to read a set of papers, which is maybe you initially start off with five papers and if on the horizontal axis,


I plot, you know, 0 percent to 100 percent read/understood, right? The way it feels like reading these papers is often read, you know, ten percent of each paper or try to quickly skim and understand each of these papers. And if based on that you decide that paper number two is a dud, right, other, other, other authors even cite it and say boy they, they sure got it wrong or you read it, and it just doesn't make sense. Then go ahead and forget it. And, uh, as you skip around to different papers, uh, you might decide that paper three is a really seminal one and then


spend a lot of time to go ahead and read and understand the whole thing. And based on that, you might then find a sixth paper from the citations and read that and go back and flesh out your understanding on paper four. And then find a paper seven and go and read that all the way to the conclusion. Um, but this is what it feels like as you, you know, assemble a list of papers and skip around and try to, uh, um, master a body of literature around some topic that you want to learn. And I think, um, some rough guidelines, you know,


if you read 15 to 20 papers I think you have a basic understanding of an- of an area like, maybe good enough to do some work, apply some algorithms. Um, if you read, um, 50 to 100 papers in an area like speech recognition and, and kind of understand a lot of it, then that's probably enough to give you a very good understanding of an area, right? You might, know- I'm, I'm always careful about when I say you're mastering a subject but you read 50 to 100 papers on speech recognition, you have a very good understanding of speech recognition.


Or if you're interested in say domain adaptation, right? By the time you've read 50 or 100 papers, you have a very good understanding of, of a subject like that. But if you read 5 to 20 papers, it's probably enough for you to implement it but maybe not, not sure this is enough for you to do research or be really at the cutting edge but these are maybe some guidelines for the volume of reading you should aspire to if you want to pick up a new area. I'll take one of the subjects in CS230 and go more deeply into it, right?


Um, now [NOISE] how do you read one paper? And, um, I hope most of you brought your laptops. So what I'm gonna do is describe to you how I read one paper, and then after that I'm actually going to ask all of you to, you know, download the paper online and just take, I don't know, uh, uh, take, take a few minutes to read a paper right here in class and see how far you can get understanding of a research paper in just minutes right, right here in class. Okay. So when reading one paper. So the, the, the bad way to read the paper is


to go from the first word until the last word, right? This is a bad way to- when you read a paper like this. Oh, and by the way, actually here, I'll tell you what my real life is like. So, um, I actually pretty much everywhere I go, whenever I backpack this is my actual folder. I don't want to show- this is my actual folder of unread papers. So pretty much everywhere I go, I actually have a paper, you know, a stack of papers is on my personal reading list. This is actually my real life. I didn't bring this to show you.


This is in my backpack all the time. Ah, and I think that- these days on my team at Landing AI and Deeplearning.ai, I personally lead a reading group where I lead a discussion about two papers a week. Uh, but to select two papers, that means I need to read like five or six papers a week to select two, you know, to present and discuss at the Landing AI and Deeplearning.ai meeting group. So this is my real life, right? And how I try to stay on top of the literature and, and I have a- I'm doing a lot. If I can find the time, if I can find the time to read


a couple of papers a week, hopefully all of you can too. Uh, but when I'm reading a paper, uh, this is, this is how I recommend you go about it which is, do- do- don't go for the first word and read until the last word, uh, instead, uh, take multiple passes through the paper [NOISE]. Right? Um, and so, you know, step one is, uh, [NOISE] read the title, [NOISE] the abstract, um, [NOISE] and also the figures. Um, especially in Deep Learning, there are a lot of research papers where sort of the entire paper is summarized in one or two figures in the figure caption.


So, um, so sometimes, just by reading the title, abstract and, you know, the key neural network architecture figure that just describes what the whole papers are, and maybe one or two of the experiments section. You can sometimes get a very good sense of what the whole paper is about without, you know, hardly reading any of the texts in the paper itself, right? Tha- tha- that's the first pass. Um, second pass, I would tend to read more carefully, um, [NOISE] the intro, the conclusions, um, look carefully at all the figures again,