Meredith Whittaker says this is very racist and even if it recognized non-white people it would be very racist this is Coptic it's designed to allow those with power to surveil and control those with less power diverse training sets aren't going to fix it advocating that we should never build these systems and that's a discussion to be had but let me break this to you this isn't


going to help the cops this isn't actually giving you the face of the person that was down pixeled this is simply going to give you the most likely face associated with that down pixel picture given the dataset the algorithm was trained on I don't see this whenever any machine learning algorithm does anything with faces at all people jumping up going like this is cop

[Drama] Yann LeCun against Twitter on Dataset Bias


technology well in line with all the broader impact statement advice can't it also be used to find lost children from very very bad security camera footage and if I already mentioned that this doesn't actually give you back the person on the down stamp old image it will give you back the most likely person given the data set so with that I want to conclude this section please


stop the witch hunting young account made a completely fine tweet here and there's no reason why people should pile on him this hard he doesn't dismiss any of the other problems just because he doesn't mention them and while we all enjoy a good discussion where people disagree genuinely is not helpful to accuse him of things he never said or meant I mean where does this all


lead the result of this is going to be that small labs that don't have the resources to collect their own data sets or check for all the possible biases in their models that are reliant on the data sets that we do have even if they are biased and flawed we'll just be disincentivized from publishing their code or actually doing research at all so this as every other additional


constraint on research is going to help the large corporations with lots of money and maybe that's just my opinion but we should be able to just talk about a problem and the solution to it without always having to make sure that we rabble down all the different things that are and might be wrong according to the Canada and big props to young lecounte here for holding his own 90% of


hi there today we'll look at distributed representations of words and phrases and their compositionality by thomas McAuliffe Ilya sutskever chi-chan Greg Corrado and Jeffrey Dean this is another historical paper it's one of three papers it's the middle one that introduces the original word to Veck algorithm and if you as you might know work to back was extremely influential

[Classic] Word2Vec: Distributed Representations of Words and Phrases and their Compositionality


in NLP since this paper basically until recently where it's sort of gone out of fashion a bit in research with the rise of things like Elmo and Bert but it's still a very very relevant so we'll look at this historical paper today with kind of the hindsight of being a couple years into the future in fact as you see right here this was released in 2013 so it's


seven years later now and we'll look back and we'll see what they said back then about the system this is not going to be like a very you know well enhanced PowerPoint presentation of how we're to Veck works this we're going to look at the paper and read it together if if you if you like you know content like this if you like historical paper readings let me know in the comments


share it out if you do like it and of course subscribe because this these kind of historical papers I enjoy them but you know many people might already know what these things are so yeah ok let's you know go through the paper and pick up their ideas and kind of put them in context they say the recently introduced continues skip graham model is an efficient method for learning


high-quality distributed vector representations that capture a large number of precise syntactic and semantic world relationships so the Skip Graham model was already introduced by mclubbe in an earlier paper that came out I believe not like one or two months prior to this one as I said where to veck is a series of papers I don't think there is a paper called were to vector they here


have released the code along with the with the paper in the code was called - vac so the Skip Graham model was introduced previously but it is replicated right here so this in the Skip Graham model what you're trying to do is you're trying to get a distributed word representation so what does that mean that means that for each word in your language let's take these words


right here for each word in the language you want to come up with a vector that somehow describes that word in a continuous fashion so that with in a the - like me might be mapped to I don't know 0.1 0.9 and 0.3 Learn might be mapped to negative 0.5 and so on so each word gets assigned a vector in the same dimensional space and what the previous paper kind of discovered is that if you


do this correctly then these vectors they have they have some kind of properties and we can already kind of jump ahead because this was already a bit a bit researched in the last paper the semantics of these vectors will be something like this so here they have a two dimensional PCA so these are the first two dimensions of the one thousand dimensional skip gram vector so the


vectors they obtain they can do things like this where they can show that in these spaces for example there appears to be a vector Direction that characterizes the capital of a country so if you take a few countries and their capitals and you average that vector you get a kind of a direction for capital Ness of a city given a country you can see that there is a a pretty clear


relation here now some of these things have later been revised to such that they they are ultimately ended up being not that impressive for example there was always this kind of math with vectors and I don't I believe this is this might not be in this this is in the last paper where they discovered that if you take the vector for King and you subtract the vector for man


and you add the vector for a woman then that would result in the vector for Queen so the way they did it was basically they did this calculation right here and then they searched in the point they ended up they searched for the nearest neighbor in their vocabulary and that turned out to be Queen but in order to make it Queen actually you have to exclude the original word King and


people quickly discovered that if you don't exclude the original word it you know the result of this kind of arithmetic will almost always lead back to the original word and then a lot of these analogy tasks are simply the result of you then discarding that word during the nearest neighbor search and then Queen just happens to be one of the closest words and it's it sort of much


less dependent on which exact calculation you do here so there's been a lot of follow-up work kind of analyzing criticizing these vector maths but definitely we know that these word vectors turned out to be extremely extremely helpful and syntactically and semantically relevant in downstream tasks because they have performed very very well so how does the Skip Graham


model work how does how does it assign vectors to each to each word so first of all it has a dictionary so there is a word an input word and for each word you have a big dictionary and the diction dictionary is basically says that you know - the word - is going to be mapped to this vector point one Dada Dada Dada and so on the word learn is going to be mapped to that vector and then you also


have these output vectors right here and what you're trying to do is you're trying to take a phrase from the data set like this one right here and you take out one word like this word vector right here and you're trying to frame this as a prediction task so you're trying to frame this as in this case four different prediction tasks so you're telling your machine I give you


the word vector and which other words are around the word vector you just tell it that you don't tell it anything else you just say which other words are around the world vector and the correct answers in this case would be to learn word and representations so these you construct for different training examples where you have an x and a y so the X is always vector and the Y is 2


and then the next training sample the X is vector and the Y is learn and and so on ok so this here each training sample is a classification task right and the classification task is and is as you can see no you can't see right here but the classification task is you have the input word and you classify it into one of many many many many many many classes namely there are as many classes as you


have words in the dictionary so each word in the dictionary will have a class associated with it right so an image nets you have like a thousand classes but in these and that's already a lot but in these tasks you're gonna have a hundred thousand classes because there are a hundred thousand words in the English language that you wanna want to treat and there are many more but in


this case they leave away all the words that appear less than five times in their corpus that's still a lot of words so it's like a super duper duper lot of a classification task but ultimately if you do something like this then the originally so the representation that you end up with is going to be very very good at doing these kind of downstream tasks and


that's what they discovered so their skip gram model is nothing else than taking a word in predicting the surrounding words from that word and this is what it means this is the formal statement of the skip gram objective what you want to do is the objective of the skip gram model is to maximize the average log probability this one so for the word we're considering the word t we


want to maximize the log probability of each word W that is in around the word C I'm sorry around the word W in a context window of C that's exactly what we did before we take a word like this model right here and from it we predict all of the words around it in energy in a given window right that's all that's the entire objective and that will give you very good representations and this is


how you would implement that so what you'll have is you'll have these vector representation V that comes from your original dictionary those are the things you learn and then because you have like an 30,000 way classifier you know that a classification layer is nothing else than a linear layer followed by a soft max operation and that linear layer also has parameters these are the the Prime's


okay so first you have the look up in the dictionary for the word vector right here and this is the vector of the classification layer now there are modifications where you can use like the same vectors and so on or you can also make use of these vectors but ultimately you care about these vectors right here and the vectors here are simply the classification layers weights so here


you can see that there is what you're trying to maximize is the inner product between the word that you're considering and the words around that word and you're trying to do a classification task so you need to normalize now this is the normalization constant and it goes over all of your vocabulary so that's what they tackle here they say W is the number of words


in the vocabulary this formulation is impractical because the cost of computing the gradient is proportional to W which is often large and that's 10 to the 5 to 10 to the 7 terms so many 10 like tens of millions of terms in your vocabulary that's just not feasible right so people have been you know sort of trying different ways to get around very very large number of classes and


here it seems that that is really our bottleneck in the previous paper they've already shown that this objective can give you very good word representation but now we need to get around the fact that we have such large vocabularies so the first idea here is hierarchical softmax and this is kind of a tangent i find this paper by the way it's sort of hard to read because it's like a half


engineering paper but yeah so first I introduced this hierarchical softmax which is kind of a a distraction it's kind of a here is what we do here is what we considered first but then didn't end up using really they do compare with it but the flow of text is sort of that you expect this to be part of the final model which it isn't so in the hierarchical softmax what you do instead


of having this giant multi-class classification task right here you take all of these classes right here and you put them in a sort of a tree okay so you take this and you put them into a tree so instead of classifying you know let's say we have a thousand classes instead of classifying a thousand ways we first classify in two ways and then we classify in two ways again from each one


and then we classify in two ways again as you know a thousand is like two to the ten so we need approximately ten layers of this before we are actually arriving at a thousand classes but it also means that we only have to weigh classifications each time so in the hierarchical softmax we build trees like this and then we so we have a word we look up its vector sorry its vector and then we


classify it for each of these nodes so your output isn't going to be a thousand a thousand log probabilities your output is going to be a log probability a binary log probability for each of the nodes right here so you want to know okay here is it in the upper half or the lower half of my classes okay cool it's in the upper half okay here is in the upper half or the lower half and so on


and you learn all to predict all of these junctions right here and that's going to end up you with you having to predict it less now of course you are constrained you impose a very big prior on the class distribution classes are an independently anymore namely if two classes here are in the same subtree that means that they are going to be predicted their predictions are going to


be correlated because the path to them is the same partially so how you arrange the classes here is very important and there has been a lot of work in this but as I said this is rather a distraction right here hierarchical softmax is a way to solve this however they went with a different way right here they went with this approach called negative sampling negative sampling has been it's been


very influential not only in word to vecna --get of sampling is one of the cornerstones of the current trend in self supervised learning in contrastive estimation and so on so this all of this you know it pops up in unlikely ways in other fields and it sort of I'm not gonna say it originated here but definitely it was introduced into the popular deep learning world


right here so they say an alternative to hierarchical softmax is noise contrastive estimation okay so in noise contrastive estimation posits that a good model should be able to differentiate data from noise by means of logistic regression you know that seems very reasonable this is similar to the hinge loss and so on yada yada while NCE can be shown to approximately


maximize the log probability of the softmax the Skip current model is only concerned with learning high-quality vector representations so we are free to simplify in all these contrastive estimation as long as the vector representations retain their quality we define negative sampling by this following objective so this is very interesting they see okay noise contrastive estimation you know it


approximately maximizes the law of probability so the noise contrastive estimation would actually be the correct way to approximate their problem however they say well as long as you know as long as something reasonable comes out we're free to change that up a bit so they go with this negative sampling approach right here and you can see that this is this is almost the same so it's


written a bit differently from the original softmax thing because the original softmax thing was written as a fraction and here it's as a sum but what you're trying to do in the noise contain the negative sampling framework is you trying to maximize the following you're trying to maximize the inner product of the word you're considering and the words around them okay so if


you're trying to still predict the words around you but now instead of having this prediction soft max over all of the classes you only have the soft max over a subset of classes so what you'll do is use sample words from your vocabulary at random and you sample K of them and you're simply trying to now minimize the inner product between those words and your


okay so what is that ultimately lead to it ultimately leads to the following you have a word like this word here- and what you're trying to do is you're not trying that much to predict the word sampling what you're trying to do is you're trying to say that in my space right here I simply want sampling to be closer than any other words that's not in the context window okay so so here is


my word negative and here is my word sampling and I want these two to be close and if I if I sample another word like here this is the word cake if I sorry if I sample that I simply want that to be far away further than than the word sampling okay so this is now a comparative it's not I classify sampling as the highest class it's simply I want to classify the word sampling against


the other classes higher all right so and this is now much much easier so instead of a thousand or ten thousand or a million weigh classification and now maybe have I have a k plus one way classification right pretty easy right I simply sample K other words and as I assumed because I have so many words chances that I actually sample one that's in my context window is very


small right so I simply sample other words and I say well these other words are random they have nothing to do with the current frame that I'm looking at so they should be you know they can be whatever they want but at least they should be farther away than the words that are actually in my in my context and that is negative sampling the process of sampling negatives this


right here and then making sure that the positives which are these here in this case the words in the context are classified with a higher probability than the negatives for a given in right this here is the input word that's it that's negative sampling and of course yeah as I said you recognize this from current things like yourself supervised learning where you wanna have


the same image augmented twice go through the pipeline you know you augment you put a little bit of different noise and then you have a different image and at the end you say these two should be close together well this other one should be far apart it's the exact same thing here except that you have a different way of obtaining the positive and the negative samples in


this case positive samples are everything that's in the context negative samples are just randomly sampled from the data set and that you know works of course that works much much much faster and you can see that this this turns out to give you vectors that are pretty good and you can train with higher vectors sorry with higher dimensional vectors you can train with


bigger vocabularies with this this has turned out to be very very influential as I said now with the rise of Burke and so on work to back is kind of getting forgotten but this was a revolution and distributed vectors so it wasn't the thing really it kind of was a thing before that but it wasn't really a thing that people used what people would do is still they would do n gram models before


that so they would kind of diss diss they would sort of chunk up their sentences into engrams into overlapping engrams and then have a big giant table further where they index their engrams so the word I don't know so the word hello is ID 1 the word hello there is ID 2 and so on so you have a big table for all the engrams and then what we would try to do is you would try to do this


kind of bag of words estimation where you would take a you know whatever engrams appeared in sentence and you would have this big you know classification where you'd associate the engrams with each other and so on so distributed word representations were kind of a revolution at that point especially distributed representation that actually outperformed these old Engram methods so


there are a number of tricks right here that are I think not understood until this day for example the question is how do you sample these negative samples right right here this basically says get K words from your vocabulary at random according to this distribution right here now how are you going to do that basically you have a spectrum of options the one side of the spectrum is going to


be completely uniform okay we sample each word with the same probability and the other side of the spectrum is something like sample this according to their unique gram these are two different things they're they're opposites in this in this fashion so here you say hey some words appear way way way more often than other words shouldn't we prefer them when we sample right shouldn't we if we


have a corpus and shouldn't we sample from the corpus and if in the corpus one word appears 50 times more than the other word then shouldn't we sample that 50 times more as a negative because it's you know so abundant and it should read get a higher classification accuracy whereas on the other hand you could say no no we should simply sample every word in our dictionary uniformly they came up


with something in between which they say both NC and negative sampling have noise distribution as a free parameter we investigated a number of choices and found that the unigram distribution raised to the 3/4 power ie unigram to death recorder outperformed significantly the unigram and uniform distributions for both NC and neg on every task which including language modeling this I think


is a mystery until today and it actually turned out that this exponent right here is magically much better than like the exponent of one or even the exponent of one half like you might be reasonably assumed that the square root you know might be something but the 3/4 I think turned out to be very good and very mystical so what does it what does it mean it means that you have kind of a


balance between words that appear often in words that don't appear often usually in these kind of things you have a power law where we have very few words that appear very often and then you have okay that's the tail shouldn't go up but you have a very long tail of words right and what you want to do is in this case you want to sample these words here more but


they are they appear so much more often than if you simply sample according to their unigram distribution you basically not regard these words right here you'll forget about them and your performance will suffer because they do appear every now and then so what you want to do is you want to push the dose down a little bit and the optimal amount for the little bit turns out to be to raise it


the you raise it to the 3/4 strange but you know turned out to work well the other thing they do is they do the they do a sub sampling of frequent words so again this is a way to kind of push down the often appearing words where they say the most frequent words can easily occur hundreds of millions of times like in the array such words usually provide less information value than the rare


words for example while the Skip current model benefits from observing the co-occurrences of France and Paris it benefits much less from observing the frequent co-occurrences of France and the as nearly every word Co occurs frequently with in a sentence with the so they do another trick here to counter this imbalance between rare and frequent words use a simple subsampling approach


each word in the training set is discarded with probably I computed by that formula right so therefore formula right here and might be asking again why why this formula so this is the sampling probability of a word and it goes with 1 over T T is a temperature parameter and F is the frequency with which the word appears in the corpus so as you can see as the word


appears more in the in the corpus then so this is the frequency as the word appears more then this thing goes down then this thing goes up so it's discarded with this probability so it's discarded with a higher probability if it appears more often where F is frequency if the word T is a throat T is a chosen threshold which shows this subsampling formula because


it aggressively sub-samples words whose frequency is greater than T while preserving the ranking of the frequencies although this subsampling formula was chosen heuristically we found it to work well in practice it accelerates learning and even significantly improves the accuracy of the learn vectors of the rare words as will be shown in the following sections


so again something sort of arbitrary it's even it's more understandable than the 3/4 but still it's sort of arbitrary they experimented around they found this works well and then every everybody ended up you know using that so that's how this kind of stuff happens ok so now we get into the empirical results and the empirical results in this case we're already sort of given in the previous


paper but here they have these the analogical reasoning task where you can see that the negative sampling did outperform the others by quite a bit right here so the negative sampling approaches outperformed the hierarchical softmax and the noise contrastive estimation and in the previous paper they also compared with other baselines and saw that it also outperforms those while being


right time time-efficient so you can see that especially with these subsampling approaches the time here there's 36 minutes for and they again I think they have like a huge corpus that they train on these were to vac code turned out to be really really efficient code and that's why I got so popular as well they did the same thing for phrases right here so for phrases like New York Times


and so on but this was kind of more of a this was more of a side thing the phrase vectors turned out to be you know rather a side thing from the actual code right here so yeah as as I said this paper is very different from other research papers and that it's it's sort of half an engineering paper and all of these papers are they're kinda hard to read because they just kind of state


some things in the order is kind of weird sometimes why they do things is kind of weird sometimes but you can't you know you can't deny that it had the quite the effect on the community and now this it is a very cool paper a very cool series of papers and it's very cool that actually they release the code and they made the code such that it is super duper efficient even like on a single


machine and that was very cool because you know being Google they could have just released code that is very efficient on a distributed data center and they didn't do that so that this is it's sort of not really like today anymore or like today when they release code it's always you need you need like 50 cloud TP use to do it and it's still cool that they release


hi there today we're going to look at end to end object detection with transformers by Nicolas carrion Francisco masa and others at Facebook AI research so on a high level this paper does object detection in images using first a CNN and then a transformer to detect objects and it does so via a bipartite matching training objective and this leaves you basically with an

DETR: End-to-End Object Detection with Transformers (Paper Explained)


architecture that is super super simple compared to the previous architectures that had all kinds of engineering hurdles and thresholds and hyper parameters so really excited for this as always if you like content like this consider leaving a like comment or subscribe let's get into it so let's say you have a picture like this here and you're supposed to detect all the


objects in it and also where they are and what they are this task is called object detection so a good classifier here would say there's a bird right here and so this is a bird and then this here is also a bird right they can be overlapping these bounding boxes so this is you see the first problem that bird why is that green nevermind okay and those are the only two objects so


there's a number of very difficult things here first of all you need to sort of detect the objects you need to know how many there are it's all it's not always the same in each image there can be multiple objects of the same class there can be multiple objects of different classes they can be anywhere of any size that can be overlapping in the background small or


across the entire image they can include each other partially so the problem is a very very difficult problem and previous work has has done a lot of engineering on this like building detectors and then kind of you want to classify every single pixel here and then you you you get like two detection right here that are very close for the same classes they are that must maybe be the same instance


right so there's only one thing here and not things and so on so there there used to be very complicated architectures that solve these problems and this paper here comes up with a super simple architecture and we'll kind of go from the high level to the low to the implementation of each of the parts so what does this paper propose how do we solve a task like this first of all we


put the image and the image here without the labels of course we put it through a convolutional neural network encoder since this is an image task it's you know kind of understandable that we do this mostly because CNN's just works so well for images so this gives us this set of image features and I think this this vector here is not really representative of what's happening so


let's actually take this picture right here and throw it in kind of an angled way and what what we'll do with the CNN is we'll simply sort of scale it down but have it multiple so here it's three channels right it's red green and blue like this three channels but we'll scale it down but we make it more channels so yeah so more channels okay but it's still sort of an image right


here it still has the image form okay so the the CNN basically gives us this thing which which is sort of a higher level representation of the image with many more feature channels but still kind of information where in the image those features are and this is going to be important in a second because now this thing which is this set of image features goes into a transformer encoder


decoder and this is sort of the magic thing here as as a component we'll look into that in this in a second but we'll take it out right here is this set of box predictions so outcomes each of these boxes here is going to be consisting of a tuple and the tuple is going to be the class and the bounding box okay so an example for this could be bird bird at x equals two y equals five okay


that that's an example another example of this could also be there is nothing at x equals seven y equals nine okay so nothing the nothing class is a valid class right here and that's also important but safe to say there is this set of box predictions and then that is basically your output right these things are your output if you have those things you can draw these bounding boxes you


can assign the labels the question is how do you train it now what you're given is a database of images and these images as you see here on the right these images already have by human annotators drawn these bounding boxes in and also labels so this here would be annotated with bird and this here would be annotated with bird but it doesn't have any of these like it doesn't


annotate the nothing classes or and so on so the question is how do you compare the two can you simply say okay if the first one here is the bird and then and the second one is this bird then it's good but then you know that the ordering shouldn't matter you simply simply care whether you have the correct bounding boxes you don't care whether you have put them in the correct order and also


what if your classifier does something like this it outputs those two boxes we see here but it also outputs this here and says bird or like one that is slightly off and says bird and so on so how do you deal with all of these cases so the way that this paper deals with all of these cases is with their bipartite matching loss this thing right here so how does it work let's say your


where can we go let's say your classifier so here is an image I'll have to wait for this to catch up here is an image and we put it through this entire pipeline and we get a set of predictions right and they're going to be class bounding box class bounding box class bounding box now the first thing you need to know is that there are always the same amount of


predictions right there are always this size here is fixed that's large n okay that is sort of that's kind of a maximum of predictions since you can always predict either a class or the nothing class in in this case you could predict anywhere from zero to five objects in the scene right okay and then the second thing is from your from your database you get out an image with its bounding


box annotations right that are made by human laborers let's say these two and you also do class bounding box class bounding box but now you see we only have two two instances so here we just pad with the nothing class so I don't know what the bounding box should be for the nothing class it doesn't really matter nothing no bounding box nothing no bounding box


no bounding box so your ground truth labels if you will are also of size n so you always compare n things here on the left that your classifier output with n things on the right now as we already said the question is how do you deal with you can't simply compare one by one because the the ordering should not be important but also you don't want to encourage your classifier to always kind


of if there is if if the one bird is very prominent right you don't want to encourage your classifier to say do you say that here's a bird here's a bird there's a bird right here hey hey there's a bird there's a bird there's a bird and basically just because the signal for that bird is stronger and basically ignore the other bird what you want to do is you want to encourage some


sort of your classifier to detect if it has already detected an object it shouldn't detect again in a slightly different place so what the way you do this is with this bipartite matching loss so at the time when you compute the loss you go here and you compute what's called a maximum matching now what you have to provide is a loss function so we can there's a loss


function L and L will take two of these things L will take the read the predicted thing of your model and L will take the true under one of the true underlying things and L will compute a number and we'll say how well do these two agree so you can say for example if either of them is the nothing class then I have no loss like I don't care about them that gives you no loss but if the


two if the two classes agree and the two bounding boxes agree then it's very good right and we maybe even gives like some negative loss or give loss zero but if if the bounding boxes agree but the classes don't agree then you say that's bad or the other way around if the classes agree in the bottom or even if everything disagrees it's the worst what what you're basically saying is if if


these two would correspond to each other right if the thing on the left were the prediction for the thing on right which we don't know right it could be that the thing on the right refers to the bird on the right and the thing on the Left refers to the bird on the left so would be natural that the bounding boxes aren't the same but you say if these were corresponding to each other what


what would the loss be how well would they do and now if you compute this bipartite matching what you want I guess it's a it's a minimum matching in this case what you want is you four to find an assignment of things on the left two things on the right a one to one assignment this is an example of a one to one assignment everything on the left is assigned exactly one thing on the


right such that the total loss is minimized right so you're going to say I'm going to align the things on the left with the things on the right such that it's maximally favorable right I give you the maximum benefit of the doubt by aligning these things and what so in the best possible case what's the loss okay hope this is this is somehow clear so this you're trying to find the


assignment from the left to the right that makes that basically is the best case for this output right here where you really say oh okay here you output the output a bird very close to the bird ear in the ground truth label that's this here so I'm going to connect I'm going to connect these two because that's sort of it's it's it gives the model the most benefit of the doubt and


the loss that you have at the end of that matching so this loss here would only then count wherever these connections are that loss is going to be your training loss okay so this solves the problems we had before it is not dependent on the order because if you reorder the things your minimum matching will simplify it will simply swap with it it is it is um if you output the same


bird multiple times only one of these is going to be assigned so if if this here is that bird only one of them only this one maybe is going to be assigned to that one and the other ones can't be assigned to that one are forced to be assigned to a different one let's say this one here and are going to incur a loss so you encourage your model to output let's say diverse bounding boxes


different bounding boxes for things okay so this D solves these problems and it's very clever and there are algorithms to compute these these minimum matchings and they use the Hungarian algorithm which will give you exactly such a matching again this is possible because you have n things on each side and the N is in effect here is a the maximum of objects that


you can detect at once I guess if there is less you can simply pad right here and then the model of course is encouraged to come up with the equal number of no class predictions because if it outputs a prediction when it shouldn't right if it already predicts two things and these are assigned to these two things and then it outputs one more thing it is going to be penalized


because it should output three things with no class but it has output one-to-many with a with a class is going to be penalized okay so the this is a pretty pretty cool thing it again it relies on the fact that you have n on both sides but you can make n so large that basically it covers all of the cases so you can make n like 50 so you can detect up to 50 things in a scene


alright that's the algorithm in a high-level they do show their loss here you see the loss ultimately is going to be so it's going to be over this matching right here that's the minimum bipartite assignment that basically minimizes this total loss over your prediction and label matchings and the loss they come up with here I said you have to give the algorithm a loss is


this one and they kind of go into how they do it I don't think it's super important so the the class algorithm sorry the loss on the class labels I think it's going to be a soft Max or a sorry a cross-entropy loss like in usual classification and the loss on the to say whether to bounding boxes agree is a mixture of the l1 loss that compares to bounding boxes and this iou loss which


is not dependent on the scale of the bounding boxes it kind of computes how much fraction of the two bounding boxes overlap but in any case the lost base they consist of saying how eyeli how much do the labels agree and how much do the bounding boxes agree okay again this is only possible because after that you compute this matching otherwise you would have no clue which boxes to come


which predictions to compare to which other predictions so let's look at this architecture a bit more in detail as we said you have this what they call the backbone which is a convolutional neural network and with that you put in some positional encodings now I already said the you should look at the these features right here as just smaller feature versions of the image but they


still have some image nature then they are flattened so once they are put in the transformer encoder because the transformer is naturally a sequence processing unit okay so it takes in just a sequence of vectors right here and since an image is not a sequence what you'll do is if you have your image features and we said we have a bunch of channels let's say we


have four channels and their height and width and see you're going to unroll and flatten that into one sequence so this is height times width you basically unroll across these axes right here into this axis and it's channels I so basically you have a sequence here of of C dimensional feature vectors that you then put into your encoder okay so your encoder will now transform this sequence


into an equally long sequence yet again of features and the good thing about a transformer because why do you use a transformer the good thing about the transformer is that in such a sequence and I've done videos on transformers it you can basic mainly look at the video attention is all you need if you want to under than this more fully this thing can basically have a tension so it has


attention layers it can attend from each position to each position in a one-shot manner so as it transforms this representation up the transformer layers at each step it can basically aggregate information from everywhere in the sequence to anywhere else and therefore it's very it's very powerful if you have a sequence and you need sort of global connections across the sequence this is


very good for a language processing because in a sentence let's look at this sentence the input images are matched together all right applying blah blah blah blah blah blah blah blah blah blah and then there is they write the word they and you need you need to know that they refers to the input images okay and but you see this is very very far away in the sentence so


you need a model that makes use of long range dependencies and they make the case that in such a task right here you also need the long range dependencies because these bounding boxes as you see right here there can be quite large so if you have an image you need that this part here communicates with these and this and this and this part basically anywhere in the bounding box and these


bounding boxes can be quite large so the transformer architecture actually makes sense here now I want to go a bit later into why I think it actually makes even more sense for a bounding box detection but right now I just want to keep going through this through this architecture right here so if my computer here decides to come back yes we can go on so what will get out is yet another so in


here we put this thing we put down here we put into the transformer encoder and we get an equally sized equally shaped sequence out of the transformer encoder you see that this thing here goes as a side input into this transformer decoder so the transformer encoder here is just a bit more of a feature mapping technically just for the architecture you could think of just putting this


into here but of course it's gonna go better with the transformer encoder the transformer decoder now does something similar but you see it has the encoder as a side input this is very much like this is not like Burt Burt is like a only encoder transformer whereas this is much like the original attention is all you need transformer that has an encoder and then the decoder as a side input


basically as conditioning information has the encoder output what does the decoder do again since it's a transformer it's going to take a sequence and output a sequence the sequence it takes is right here is what they call object queries and this also is different from the attention is all you need papers and they don't do it autoregressive lee they just do it one


shot what does it mean it means that you start with a sequence here of four things and this is these are the this is this big n and you out you output the sequence of a sequence of four things and it's important to see what they're going to end up so these things are then directly going through a classifier that now outputs the so these things here are these class label bounding box outputs


okay so each of these things is going to after transformation end up being one of these bounding boxes either defining an object or saying that there isn't an object somewhere okay you see here this bounding box refers to this bird this bounding box refers to this bird so each of these things is going to to be one bounding box and the what they call object queries is the question of course


is what do you input here right actually I want to transform this image information that comes from the left here I want to transform that into the bounding boxes what do I input here and the answer is you just input at the start you just input n random vectors because what's that gonna give you is basically n outputs you want and outputs because you want n of these


bounding box classifications so you need n things and if I input n things into a transformer it's going to give me n things as an output and then in each step I can simply condition on the information that comes in the images and it it'll give me right then I can incorporate that information it's a very deep learning way of thinking about it actually that you just need the


information somewhere in there and I need n things now they go more into detail into this transformer architecture help help in the helpful fashion in the appendix and will go there quickly so this I think here makes more sense so the image features come in here right and you see this is just a transformer stack an encoder stack of multi-head self attention and feed-forward in instants


wise or like token wise feed-forward network and then that information is taken and is given as conditioning information over here now in here as I said you input these object queries which at the beginning are just n random vectors and what you're going to do you Argos are going to feature and code them and then you combine it with this image information so ultimately if you think


of this one of these things one of these things is going to be a vector right and then that vector is going to be transformed and then it will have as it is transformed it will have the opportunity to basically look at features that come from here now the arrow is in the wrong direction so you have already taken the image and you've transformed it into a feature


representation which is also a vector right you have the features of the image right here now as you transform this vector this object query queue you have the opportunity to look at the image features right and that's how do you get the image information in there so the image features will come in here transform that through attention so this is an attention mechanism on the image and


then what you will output is a bounding box and a little class label it's really hard to explain I would guess you need to understand really what attention mechanisms are and of course the crucial part of of course is what what's this what do you input at the beginning and these object queries aren't actually random as I said they are learned so what you're going to do is you're going


to learn independent of the input image you're going to learn n different object queries and these object queries now it's very it's very interesting because these object queries are sort of going to be different it's like you have different people that can ask the input image different questions right and this they have so there n is 100 but they show 20 of these object queries that


they learn and so did they have visualization of all bounding box predictions on all images so it's it's sort of like you have n different people at your disposal and you train these n different people to kind of ask different questions of the input image ok you say this person up here will always ask irrespective of what the input image is will always ask sort of


hey input image what's what's on your bottom left right that's I'm really interested what's on your bottom left and sometimes I'm a bit interested in what's here but I'm mainly interested what's on the bottom left of the image whereas this person right here sorry this person right here is more interested in what's in the center that the different colors here is


refer to different sizes of bounding boxes so this person is also interested so the person on the top-left is interested mainly in I think small bounding boxes that are on the bottom left and the person here is mostly interested in what I'm really interested what's in the center what's large in the center I want give me large things that are in the center right and then this


person right here is really interested on stuff that's on the right side of the image so you see in order to get different sort of a difference in bounding box predictions you train n different people to ask different questions of the of the input image and this asking of questions is exactly what an attention mechanism is so this person right here let's let's take this this


person and I'm saying person these are vectors these are learned object queries but this person first they will simply ask the question what's on what's on the right side and then the image features right I'm getting poor drawing the image features it will have an attention mechanism to this part of the image features and then it will get back some signal right and then it will transform


that with its own signal up and then it will ask maybe again okay now that I know more because you see that person is interested in multiple things it's interested in those things and those things so at first it will focus on these things but then it says oh now I'm now I know more right there is there I know I see there is actually something on the right side so in the higher


layers it can then go back and ask the image more questions by sending these cue vectors of the attention mechanism and it will get back the V vectors from the image features that correspond to these cue things so up and up the layers this person can ask more refined questions about what that particular person is interested in okay and since you have the different people here that


ask different questions you basically learn the people in a way such that across the data set they all together they cover every possible image pretty well again these people what they're interested in initially is not dependent on the picture you simply learn this in a global manner all right this is the best way I have of describing it you basically learn n people that are each


one is interested in different things different classes and different regions in the image and each one of these people is going to output their best guess of what is where based on what they're interested in so that person might say I'm you know I'm the person that's interested kind of in the left side of things so I am going to output that there is a bird right here now


these people if this is a transformer right and everything can attend to everything they can actually communicate with each other as they incorporate information from the image so in each layer they can do both they can incorporate information from the image and they can communicate with each other and then in the next layer that can do it again and again and again and thereby


they can sort of they can sort of say well you already got the left side I will take the right side you already got the bird class I will take the elephant class and so on so you see here how the the architecture of the transformer actually is also very conducive to doing this bounding box prediction in that these different things can sort of attend to each other and therefore


communicate with each other all right I hope that sort of makes sense now before we get into the experiments I want to list a third reason of why the transformer especially the encoders might actually also make a giant amount of sense here since you on the image into height and width and you have to imagine what does the transformer do the transformer as we said here has this notion of a tension


where from any point in the sequence it can gather information from any other point in the sequence and this that's usually one of the downsides of the Transformers is done via a quadratic attention mechanism so if I just list one feature channel go over here if I just list one feature Channel right here this is height times width of the image right this is this is the entire image


unrolled in one vector height times width and here I unroll it again height times width then this this matrix that I can build right here which is called the attention matrix right here it will tell me which parts of the sequence attends to which other parts okay so if you have an image that has the let's say the number three and you really want to figure out whether or not this is a


three then the bow up here must communicate with the bow down here right they need to share information you say oh there's a bow here there's a bow here and there is a spiky thing here that must be a three so you want something this is rather at the beginning of the sequence you want this to attend first of all it will attend itself so you get fairly high values along the diagonal


maybe 1010 1011 1112 and I saw this all eg skated a hundred million nine nine but it also like this this part here at the beginning of the sequence let's say it's here because this is unrolled right needs to attend to the end so this needs to attend to the end which we will put an 11 here and the other way around doesn't always need to be symmetrical by the way


okay but in any case this is going to be a H times W squared matrix because everything can attend to everything and that's the attention mechanism why do I think this is so good for bounding boxes because let's let's imagine you actually have a matrix that is like this okay height times width times height times width every single point in here actually defines a


bounding box because this point this point right here in this dimension corresponds to one location in the image and on this axis it corresponds to another location now in the attention matrix simply means these two points need to communicate but if you have two pixels you actually have defined a bounding box right here right you you were actually you're defining a bounding


box and the fact that this is happening in the exact same matrices could mean that the Transformers are uniquely well the Transformers across sequences of these height times with unrolled images are uniquely well conducive to these bounding box prediction tasks I'm actually a bit astounded because when I first just read the title this immediately popped to my mind I'm like


oh yes of course and they're going to predict the bounding boxes by simply training so what you would do what I thought this was gonna be as out you output an actual matrix like this and then you simply each point you can you can simply classify right so you can classify here whether whether or not like at in this direction there is a bird right and then if you have two


points like this for example you and you also classify whether in this direction there is a bird right and this naturally defines a bounding box or you could like take this matrix and actually just classify individual points in this matrix to be the bounding boxes because they already define bounding boxes so I just I think these these quadratic things are are uniquely I mean someone


must have thought of this or if not like the YouTube channel it would be funny first paper ever to actually have to cite the YouTube channel but again yeah so transformers seem to be a good idea for these kinds of things so how do they do of course they do well they are on par where with these other much much much more complex architectures these faster our CNN models they are apparently much


more complex but they are on par with this they do however train forever I think they train for like six days on eight GPUs is not that much if you compare to like language models on hundreds of TP use but still okay I don't want to go into the numbers of experiments but what is really cool is that they can now visualize this sort of attention and you can see right here


that if they look at a particular point in the image and visualize the attention it will actually attend to the instance itself so it will like these are usually the problems for these detection algorithms when things overlap and are partially occluded but you can see right here that the attention is on the part of the image that makes the instance in the back and the attention here is on


the part of this and it doesn't sort of overlap into the others so that is one thing that's pretty impressive about these architectures the other thing they show is for example it can generalize to many many instances so here it has never seen 24 giraffes in one image but yet it can absolutely do that and giraffe giraffe to rupture after of and the one of the coolest images I find are these


here where you can see right here again attention visualisation and you see that even within the bounding box of the front elephant here you see that the attention on this foot of the back elephant is is is assigned to this blue bounding box so this is the blue basically the blue bounding box person that is attending to that back foot that means they they these things really sort of


understand or they learn these things like occlusion and you know just hard I have a hard time describing it but you can see it visually here right like how it clearly learns that these are two instances that are sort of occluding each other but this this this instance can actually appear within the bounding box of the other instance and the same goes for the zebra here that are


partially occluding each other and you can see that the attention is correctly like even here this back foot of this zebra is correctly labeled so all in all that is pretty cool and they take it a step further and they say well with this architecture we can actually pretty easily do pixel wise classification so this is this cocoa stuff and things data set where I don't know which one is the


stuff and which one is the things I think things is the objects and stuff is like sky and mountains and so on and so this is a classification task where you actually have to label every single pixel so what they do is they simply input this through their detector and they detect the instances they take the attention maps of the instances and then they scale it up this right here is just


a CNN sort of in Reverse that scales up the image because they have scaled it down as we said they scale it up again and then they scan simply classify each pixel where each of these you remember we had these different people here that it that cared about different things in the image each of these people will classify their respective pixels the pixels they feel responsible for and


then you simply merge all of these people's predictions together into this prediction and again this gives pretty pretty impressive results I am I mean this is this is fun this looks like it sort of works I haven't they do quantitative analysis of course but I'm just impressed by the examples right here alright that was sort of it I really enjoyed reading this papers the


simplicity is pretty cool they do have not only do they have code in the paper to show how ridiculously easy it is to get this to run this is all you need in pi torch but they do actually have code and as I understand they also have pre trained models so they have this model Zoo right here where they give you the pre trained models so you can play with it and you can even load it from torch


let me introduce you to mrs. mani she came to the emergency room she searched his fifty two-years-old she came to the emergency room with a foot sore doctors investigated of with saw and she ended up staying there in the hospital for 22 days here's what happened when she came to the emergency room for a foot sore they inspected her they saw no real reason for medical concern but they

Better Medicine Through Machine Learning | Suchi Saria | TEDxBoston


wanted to monitor in case her foot sore was infected so they put her in the general ward on day three she starts developing symptoms of what looks like mild pneumonia they give her the usual treatment of antibiotics and all's good but then her condition starts to worsen on day six she develops what's called tachycardia that means in medical speak her heart rhythm has accelerated


dramatically she then has trouble breathing on day seven she experiences septic shock that means her body is in crisis incidentally mortality in shock is one in two now it's only at this point that the doctors get really concerned and they transfer her to the intensive care unit I see use other units where the most critically ill patients get cared for they here they give her every possible


treatment to stabilize her but her condition only worsens first her kidneys start to fail then her lungs fail and on day 22 she dies mrs. mani did receive the right set of treatments the problem is she received them only too late what mrs. mani experienced was an infection that turned into sepsis let me tell you a little bit about what substance is sepsis occurs when infection releases


chemicals in your blood to tackle the infection so your body releases chemicals to fight the infection now this chemical can trigger an egg of inflammatory response when this in when this inflammation triggers this negative inflammatory response what it can then do is cause a cascade of changes leading your organs to fail leading to depth sepsis is the 11th leading cause of death more than breast


cancer and prostate cancer combined turns out substance is preventable if treated early okay so then what's the catch doctors find it very hard to recognize sepsis in fact a Harvard study shows with 93 leading academic experts that when they were given several cases of patients with and without sepsis they couldn't agree two years ago my nephew he was admitted to the best Hospital in


India and he died of sepsis my family was devastated I'm a machine learning expert and what I do is study ways in which we can use large messy datasets to enable intelligent decision-making so natural question for me was could machine learning of health could machine learning of help mrs. Manny and my nephew so this led to a massive effort with my colleagues at Hopkins to design


what we call the targeted real-time early warning system or trues based on machine learning I'll give you a sneak peek into what chooses and how we're using it to tackle sepsis let me take a step back and tell you a little bit about what machine learning is and what's AI artificial intelligence is a field of study very design very teach computers how to learn okay just like


you teach your kids machine learning is one way of doing this by designing code or programs that teach computers stuff over time by by interacting with the environment or watching okay so I'm going to show you a video of some robots learning how to walk I find it funny how it shudders so you're probably now are thinking this is hopeless well so the question is how can


we teach robots or machines how to walk intuitively you can think of it as designing a game the goal of the game is for the computer or the robot to learn how to walk for as long as possible without following ok so to do this first we have to design write down the goal in a language the computer understands for this we'll use math okay so now you're wondering well how do we write the goal


of walking without falling as long as possible in math well that's often hard for different tasks but you can think of it as writing down a formula and what this formula does is it scores so in the case of walking it'll score every move the robot makes if the move it makes helps the robot walk it gets a high score if the move that the robot speaking makes the robot unstable it


gets a low score and now the robots goal is to experiment with the sequence of moves in order to be able to maximize its score so how does it know which moves to try right well there are two strategies for doing it first it expected learns by interacting with the environment okay so here the robot will just make a guess it guesses it makes a move if the move gets a high score


that's positive feedback and the robot builds on it okay the second strategy is by watching other robots in other words the robot finds data from past robots that are similar to this robot it watches what moves that robot did when it was in very similar positions and now it emulates or replicates those moves okay so those are the two strategies so I'm going to show you a video of robot


learning how to walk using the strategy I just described okay so in the beginning it's going to look hopeless but I promise you it gets better and just to be clear this is not so this is the skeleton off the robot and so this is not a human animator going there and just moving or animating this video this is really the robot the algorithm choosing which moves to make by moving


the joints of the skeleton that you're seeing and you can see it's already getting better now suddenly the robots a will blue walk and run for a lot longer than it was doing right so essentially the basic principle is as follows you figure out a game that the computer can play you write it down using a language it understands and then we train it to optimise the score right this is how we


teach cars how to drive computers how to play the game of go an Alexa to understand say your preference of coconut water so let's go back in our case stir the problem of sepsis so the goal here is to identify sepsis as quickly as possible right and for this truths learns by watching in other words using data from past patients this avoids the need for tools to have to


experiment on new patients right so to do that what are the pieces truth needs to do so one big change that has happened in medicine that's interesting to note is in the past five years the introduction of electronic health records in EHRs every single measurement every single lab test that is ever done when you walk into the clinic or you're in the hospital gets collected trues


analyzes this data from thousands of patients to identify subtle signs and symptoms that appear in patients with sepsis than those without okay but that's not alone what truths also needs to do is to figure out how to think about every signal in the context of every other signal let me give you an example let's look at the example of creatinine in SOL creatinine is a waste molecule


okay and your kidneys filter it out okay but here's the catch so when your body is septic it affects your kidneys it deteriorates your kidneys ability to filter out creatinine so creatinine level rises but there are many other things that can affect your kidneys ability to filter out creatinine for example if you have chronic kidney disease you're very likely to have high


creatinine levels so now what truth has to do is to figure out is your creatinine high because of sepsis or because of chronic kidney disease or the numerous other factors that need to high creatinine levels but that's not enough it needs to do this for every single signal that exists in the electronic health record and chooses think about every signal in the context of every


other signal to identify signs and symptoms that occur more often in patients with sepsis than those without let's return to mrs. Manny research by Kumar and colleagues have shown that for every hour treatment is delayed mortality goes up by seven to eight percent so timing is critical we went and took mrs. Manny's data and we ran throughs on it and here's what we found


truth would have detected mrs. Manny's sepsis 12 hours before doctors currently did as my clinical colleagues would say that is the difference between life and death last year we showed using data from 16,000 patients that throughs on average would have detected on most patients on average 24 more than 24 hours prior to the shock onset that's not 24 hours in two-thirds of these


patients their sepsis was detected prior to any organ dysfunction whatsoever and to put this result in context that's 60% increase in performance over state of the art so what truth is really doing is doctors a much longer window to come in and intervene in order to prevent organ dysfunction and mortality this year the independently validated truths in data from Howard County


General Hospital in Maryland and now we're working to do real-time integration in order to make something like truths available to every doctor at Hopkins I'm also really excited because after we've published our papers several other health systems are now already implementing the published version of truths in order to be able to develop it in their own environment so I'm going to


highlight like a few perhaps three salient characteristics that I think makes a strategy like truths very powerful ok first truths runs 24/7 what it does is it gives doctors a second pair of reliable eyes right - it's hard to scale-up doctors it's easier I think much easier to scale up computers and what truth is really doing is allowing us to get expertise from the best


doctors everywhere here's the third one which i think is very interesting in many cases like we see in substance we might not need new measurements the signs and symptoms were already in your data and what truth is really doing is discovering these signs and symptoms to learn something that we couldn't see by eye finally there's been a lot of buzz about big data and I want to make a


little subtle point about a technical problem that I think truces solving that is very interesting it truths would be able to learn learn much faster if it had a lot of data on you or it could get more data by experimenting on you but we don't want that right so what truths really has to do is leverage your limited data to figure out what's right for you right so in other words what


truths really has to solve ism is a challenging small data problem in other words it has limited data on you and has to figure out what is the right treatment for you and for that it has to let it leverages vast amounts of data from other patients and figures out what information to borrow in order to make these assessments reliably and precisely so I also want to tell you a


little bit about how the strategy is not unique to sepsis so very broadly if you think about it in many diseases essentially where you have profile of symptoms and the response to treatments varies a great deal across individuals you can use the strategy like truth in order to target treatment so you're wondering like for example if you consider cancer diabetes multiple


sclerosis Parkinson's lupus so there are many such diseases on which a strategy like Truths is amenable in fact in our own lab with experts in rheumatic diseases or immune diseases in particular we're looking at how in scleroderma for instance we can use strategies similar to tools to avoid giving strong immunosuppressants to patients who don't need them other


colleagues this is William Pelham Susan Murphy and their team they're studying kids with ADHD and looking at how using similar data-driven strategies they can identify when kids can be benefit from behavioral therapy and we can avoid the need for giving them psychostimulants altogether so the strategy is very very powerful so I was speaking about substance so let's go back to substance


again so I said it was sepsis Awareness Month and the CDC has declared substance to be a medical emergency rightfully so remember 750,000 people annually are affected by sepsis a patient's family recently asked me what will it take to bring this to a hospital near us I think that can be done in fact it can even be done within a year but we don't want to stop there we want it to be possible to


bring strategy like truth - hospitals everywhere and so the question is to do that what will it take right so I think the three key things we need your help for one we need super smart engineers to be working in healthcare we need your help in building and scaling up such technologies don't go to wall street healthcare needs you right we need policymakers to create


incentives to open up electronic medical records as an expert at a leading health institution it's taken me more than a year because the EMR is so closed in order to be able to figure out how to implement rules against the EMR it really should be easier than this three we need a healthcare system that's based on quality our current healthcare system is incentivized to optimize volume


rather than quality right now you can choose which restaurants to go to based on the quality of food should you be able to choose the hospitals you go to based on quality of care part of the problem is that quality data at the moment is not very visible to consumers and we really need to make a bigger effort to make this quality a visible so that you can choose based on quality so


to summarize sepsis is one preventable killer in many pressing medical problems like we saw in sepsis the answers for knowing whom to treat when to treat and what to treat with might only be in your data sometimes I wonder if we had done this work to use earlier if I could have prevented my nephew Nick induced depth I can't wait for this to be the way medicine is practiced thank you


all right so I'll talk these two parts to my talk about three parts so one is about a little bit of kind of state of the art of supervisor on you in France Maroni and and then the second part about supervised running and that's the title really of the talk and with an introduction to something I call energy based running which is sort of a way of sort of a general framework or paradigm

Self-Supervised Learning


if you want to approach to approach learning in general should I use this all right much better oh this is just for recording I guess okay so we all know what supervised running is about I'm told you all know what supervised learning is about and this is the situation where you train a machine by telling it where the correct answer is for a bunch of training samples and this

Self-Supervised Learning


works really well if you have lots of data it works for image recognition translation natural language processing speech recognition you're all kinds of applications but those are applications where the economics are such that it's worth actually labeling a lot of data by hand and and of course you know with that context machine learning is basically comes down to finding a good

Self-Supervised Learning


form for a parameterize function preferably differentiable at least almost everywhere in such a way that by using gradient descent type algorithm you can tune the parameters to optimize the performance of the system right so everything is differentiable or almost differentiable you can optimize using gradient or sub gradient and and it all works we all know about that and there

Self-Supervised Learning


is you know guarantees of generalization if the capacity of the machine is limited is of course another form of running which I'm not going to talk about very much called reinforcement learning and if respect running has seen a lot of success over the last several years this success are almost all can be restricted to gains or virtual environments there are also applications

Self-Supervised Learning


of reinforcement running in situations where you can collect lots of data really quickly to have like a really fast adaptation so if you are you know you want to control you want to show people content and you want to kind of figure out you know well where you want to rank the content there is no differentiable objective function because you don't know what people are gonna are gonna do

Self-Supervised Learning


so you can use whether they click on a piece of content or not as kind of a reinforcement and then optimize the the policy as to what you show to people to maximize the to maximize that but these are you know on these situations where we get lots and lots of feedbacks and otherwise it works for games because you can get wishes to look to to play games really quickly and so you so they can

Self-Supervised Learning


play millions of games per you can run it in parallel on lots of computers and so you can train which is to carry games to play go to play Starcraft dot you know whatever what kind of games and it's only due to the fact that you can run those games faster than we all-time on many machines if the machine had to run at the same speed as as we do which means real time play the games in real

Self-Supervised Learning


time it basically wouldn't be practical it would take about 80 hours for the best current algorithms to learn to play a single Atari game - level of performance that a human can reach in about 15 minutes so the efficiency the sample efficiency of this type of enforcement running is horrible compared to humans at least for go so you've probably heard of alphago alphago 0 from from deep mind

Self-Supervised Learning


whether the details are not fully released and there is no open source code there is a system called alpha pango which was released by Facebook and this one you can just download and run or trade it yourself it's used by a lot of different people who are interested in this this one required about 20 million South play games running on 2000 GT use for 2 weeks to reach superhuman

Self-Supervised Learning


performance so this is not cheap in terms of computation as you can tell if you were to to buy this on a on a you know cloud computing server it would cost you a couple million bucks and and it you know it's more games that you know a single person can play in a lifetime probably more than all your humanity is played in a in a number of years there's a very interesting paper

Self-Supervised Learning


recently but it mind by our vineyards group alpha star which plays a single map on StarCraft with a single type of player and the training for this to the equivalent of two hundred years of real-time real-time play which is definitely more than any single Starcraft player has been able to do there's no paper on this as far as I can tell yet so they all use you know deep

Self-Supervised Learning


architectures they all use conventional Nets actually with the combination of other things transformers and start quite a few particular but as you can tell in terms of sampling efficiency it's very bad and it's a huge problem because what that means is that we can't really use me for my training for other than in simulation to Train real-world systems like a car to drive itself or

Self-Supervised Learning


robot that grabs objects unless you have a room full of robots you know training all day so if you were to use we first went running at the moment to train a car to drive itself it will have to you know drive itself with millions of hours and cause tons and tons of accidents and and and it's just not practical right so people do it in simulation it kind of works simulators are not very accurate

Self-Supervised Learning


so there is a problem of sort of transferring from simulation to the real world it's got to be to work on this but it's a big mystery there which I'll come back to in the second half of the talk and the mystery is is how is it that humans can learn to drive a car in about 20 hours of training without causing any accident and the sort of preview of the answer to

Self-Supervised Learning


this is that we have internal predictive models of the world that allows us to predict that if we drive near a cliff and we turn the wheel to the right the car is going to run off the cliff because the gravity is going to fall and nothing good is going to come out of it we don't need to actually try it to predict this and so perhaps the answer is for eventually machines to learn to

Self-Supervised Learning


to learn to have those predictive models of the of the world that will allow them to predict the consequences of your actions before before they occur and plan plan ahead and to some extent we can say that the essence of intelligence is the ability to predict but for now let's stick with supervised running so I'm sure so who isn't who doesn't know what the condition that is don't be shy okay

Self-Supervised Learning


that's great I can skip a lot of stuff okay so a combination that of course is an architecture that is designed to you know recognize images but in fact it's designed to recognize array data where the property is that there is strong local correlations in the in the features and and some sort of translation invariance of the statistics of the signal so it's true for images is

Self-Supervised Learning


true for audio signals it's true for basically anything that comes to you in the form of an array where the locality in the array has some meaning and of course you know the first applications of this were on character recognition but we quickly realized that we could recognize multiple objects with those things not just single objects by kind of scanning if you want or doing the

Self-Supervised Learning


equivalent of scanning a commercial net on the only big image which of course you don't have to do it stupidly because it's all the layers are convolution you don't actually need to explicitly recompute the converse on that every location you just make each layer bigger and then you you know make every layer convolution people rename these fully conventional Nets afterwards but it's

Self-Supervised Learning


just complex on nets and when you apply this to natural images you can you know you can train systems like this to detect objects and natural images you can apply them locally you can apply accomplish on that locally to an image to have it label every pixel in the image with for example the category of the object it belongs to and the conversation that has some sort of you know each output of the

Self-Supervised Learning


network has some sort of window of influence on the input which in this case is actually quite large so to decide of the category of a single pixel the the network here looks at a wide contextual window around this pixel and and then gives you an output for that particular pixel and and this is done sort of conditionally so it's very cheap so the system that was built about ten

Self-Supervised Learning


years ago and it could run at about thirty frames per second on a on the specialized Hardware actually an FPGA and of course with what you probably all know is that around 2012 2013 those networks started you know beating other methods for object recognition for by a large margin largely due to the fact that the data sets we can became bigger those systems are pretty hungry in terms

Self-Supervised Learning


of data more than whatever methods people were using before and so the appearance of things like data sets like image net of you know sort of made it possible to sort of really exploit the capacity in those in those networks and then the second thing was the availability of GPUs which allowed to run those systems really quickly but you all know this and what you all know also

Self-Supervised Learning


is that there's been an inflation in a number of layers using in those networks over the over the years where you know some of the workhorse of image recognition nowadays is you know some sort of backbone convolutional net similar to resonate for example so ResNet is a composition on that where every pair of layers I mean a block or residual block I'm sure again many of

Self-Supervised Learning


you have heard of this but you basically have pairs of layers completion non-linearity completion sometimes you have subsampling pooling as well but this one this one doesn't and then you have some sort of connection that can skips pairs of layers and so essentially what you can think of the function of one of those blocks as basically computing the identity function and those those those

Self-Supervised Learning


layers compute the deviation of the function of that layer from the identity so that sounds kind of a waste to just have a layer that computes the identity function and it is in fact many of the layers in those systems don't do much you can kind of get rid of them actually after after training but what it does is that it makes the system sort of fault tolerant if you want so if the learning

Self-Supervised Learning


algorithm somehow kind of gets into a situation where some layers die which can happen it's not catastrophic because you always have the information going through the bypass connection and so that you know a pair of layers just you know kind of checks itself out of the network so it's not it's not used but it doesn't kill the entire entire effort so that's one of the advantages of ResNet

Self-Supervised Learning


you can you can think of the the long succession of layers as sort of progressively we finding the the answer and cleaning up the the the output or the representation between variations of this where you have you know skipping connections that skip multiple layers etc that's called dense dense net so I'm sure a lot of people will talk you know talk to you about progressing purpura

Self-Supervised Learning


vision over the next few years and there's been a huge amount of progress over the last few years with things like Maersk our CN n which is sort of a two pass image recognition system that can pick out every instance of every object in an image and with you know really good performance so there's sort of a first first few layers that kind of identify regions of interest and then

Self-Supervised Learning


you kind of apply a second no neural net to the conditional net to the the regions of interest that you've identified by the first one there's also kind of one pass systems that my colleague at in Menlo Park at Facebook use the you know call rich internet or future pyramid network and you can think of this as let's say you want to produce a dense map of everything that's in the image so for

Self-Supervised Learning


every pixel in the input you want to give a category of an instance or or category whether it's an object or kind of a background region if you want so you have you know a bunch of layers of a commercial net where the spatial resolution goes down as you go up because of subsampling and then you have sort of a similarly architected network that can it goes the other way from low

Self-Supervised Learning


resolution to high resolution you have kind of skipping connections that go from one map in the in the sort of you know abstraction pyramid if you want to the corresponing that in the output you know the part of the network that produces the output and you can train this end to end with sort of weekly supervised architectures you can plug classifiers taking inputs from sort of

Self-Supervised Learning


various levels in the in the network and this works amazingly well so this is result from a scarf CNN actually not from the retina net or future I mean network but it's the results are quite similar you can get every instance of every object outlined together with a box so the colors are actually produced by the network and correspond to two categories and you know it's pretty it's

Self-Supervised Learning


pretty amazing how well this works you need data these are results from this sort of single-pass sheeter pyramid network again the colors indicate so the individual objects but this system actually labels not just the object but also the background regions so it's sort of they call this panel optic vision so these this type of architectures we have sort of a commercial net with decreasing

Self-Supervised Learning


resolution followed by another one with sort of increasing resolution which some people called accomplished on that this is a paper from I guess 2012 or so or 11 by my colleague Bob Fergus on this idea geek on visual net and this architecture is used quite a lot in image segmentation particularly for applications in medical image analysis and some people call this kind of

Self-Supervised Learning


architecture a unit because of the of the shape when you represent it this way so it's exactly you know very very much the same idea I showed before except now the the layers of the kind of feed-forward part of the network I kind of drawn on this side and then the the sort of resolution increasing half is drawn on that side with skipping connections going directly so it looks

Self-Supervised Learning


like a u and these sort of variations of this so this is work from my colleagues that at NYU who are working on with medical image analysis these are 3d MRI scans and so the cognate here is three-dimensional the completions take place in three dimensions over the three spatial dimensions and and every voxel now is labeled as you know one of a number of categories so you can do

Self-Supervised Learning


things like segment hip bones and things like this for you know preparing for hip replacement surgery and stuff like that and and it works really it works much better if you use 3d rather than 2d because you get the consistency of all the slices so as you see at the top here is there are artifacts of the recognition if you use kind of 2d segmentation perhaps with a little bit

Self-Supervised Learning


of cleanup and if you really you you sort of get with your of those artifacts the same team with a different subset of people has applied this to things like mammograms so this is 2d data but you have multiple images from sort of multiple with use angles of view and here's a surprising thing so this is the kind of application that you some of you may not have heard of which

Self-Supervised Learning


is the application of comp nets in physics this is an example in astrophysics so this is a paper from I think was in PNAS published in PNAS a few months ago from the Flatiron Institute which is a private research institute in in New York and what they did here was use a accomplished on a to accelerate the solution of partial differential equations solvers so what

Self-Supervised Learning


they were interested in these are cosmologists and they're interested in you know what are the initial conditions of you know baby universe that will cause the kind of universe verbs or in today what you have to do for that is basically simulate the entire universe at its birth you know the expansion the first expansion phase of the universe you can do this because you could do

Self-Supervised Learning


this in principle because you know you have the density of matter and dark matter ordinary matter dark matter photons whatever at every location and you can solve a partial differential equation which basically is just you know physics at every location and I'm gonna compute the evolution of the universe this way the problem of with this is that she only do this at the

Self-Supervised Learning


scale of the universe given the size of the the grid the grid that you have to use to solve this equation it will take too long and so what they did here was they used one of those solvers known PDE solvers to kind of solve those equations on kind of small domains small four-dimensional domains right because it's three dimensions of space and one dimensional time and they train

Self-Supervised Learning


a convolutional net to produce the same result but that convolutional net has a bigger grid right so a PDE solver basically takes one value when vauxhall okay four dimensional voxel if you want or 3-dimensional Vauxhall and then looks at the neighbors and then passes it to some function that computes the new value for the next time step let's say of the the center or the central grid

Self-Supervised Learning


cell so it's a completion like operation except that you know it's maybe nonlinear so so what they did was train accomplish on that with a few layers and they it you use this it uses this unit architecture so we can take a fairly large context into account not just the neighboring cells but sort of a bigger neighborhood rather big grid cells and it's trained to produce the result that

Self-Supervised Learning


the PD solver would would produce and they can easily generate data by running the PD solver but they run it on kind of small 3d domains and then once they have this commercial net they can run it on the the big scale of you know universai universe size scale if you want and and what they get is those kind of match here which are the displacement maps of densities and there's also different

Self-Supervised Learning


methods and the colors indicate errors blue is low error and you know red is high error and those are also various ways of doing this and this is kind of their proposed method and compared with what the PDE solver we do for a relatively small domain so that's kind of an interesting thing which is to use neural nets or D planning in general as a phenomenological model of something

Self-Supervised Learning


that we might possibly know the underlying physics but it's computationally too expensive people are doing this also for predicting the properties of materials for solving problems in molecular dynamics so for example confirmation of protein where the two proteins are going to stick to each other you know things like that which is of course super important for

Self-Supervised Learning


ticks like drug design I was at Harvard a couple weeks ago and I talked to people who are trying to use neural nets to predict the property of certain solids so if you take graphene which is a two-dimensional mesh of carbon atoms and you take two and it's cool you know just a single atom thick layer you take two layers of graphene and the one on top you twist it

Self-Supervised Learning


just a little bit compared to the the one at the bottom there's a particular angle at which this material becomes superconductor and nobody has any idea why and so there's some idea of using neural Nets to kind of build phenomenological models of all those properties so that perhaps we could predict other properties there's interesting work along those lines also

Self-Supervised Learning


by Pascale flora who is actually originally a vision guy at a pea ferry and what he's doing what he's been doing was to predict the aerodynamic or hydrodynamic properties of a solid using a by training accomplish on that feel accomplished on that basically so you feed the shape of the the solid to the to the system and again using fluid dynamics computation to generate data

Self-Supervised Learning


you train it to produce the properties of that shape for example its drag or lift if you are interested in designing air foils for for you know blades of propellers or airplanes or hydrofoils or whatever and then what you have now is on your net that predicts those properties and so because it's on your own that is differentiable so now you can optimize the shape by doing gradient

Self-Supervised Learning


descent in input space you can optimize the shape so as to get the properties you want on the output which you you know you can't really do it with a regular computational fluid dynamics piece of code so it's really interesting he actually has a startup that works on this yes huh well I guess you have to really know the underlying physics to be able to make that generalization so I mean you can

Self-Supervised Learning


test on relatively small spatial domain because you can run the PD solver so you know how accurate your new ComNet is the question is when you extend the size is it still accurate okay and there is a leap of faith there's no question now to your comment that this has nothing to do with Commerce on that no it does it has very very much to do with Congress on it

Self-Supervised Learning


because all of those piggies are local operations that basically look like convolutions that are essentially the same you know press some nonlinear thing because you know if you do navier-stokes equation for free dynamics you have to do some projection afterwards that's nonlinear but you know but it's a local operation and it's the same operation you do everywhere in the image so or if

Self-Supervised Learning


we're in the in the volume so it is accomplish on that it's it's directly it's probably one of the most appropriate use of kinetically imagine so of course it's been quite a bit of progress in in things like you know start driving cars it's working progress these are actually videos that are quite a few years old I think about five years five years old from this one is from

Self-Supervised Learning


mobile I which is now Intel and NVIDIA and there's a huge amount of work on surviving cars as you know a lot of engineering goes into this but all the perception system use you know some sort of cognitive process either images from cameras or from various other types of sensors like lidar and other things like this okay so all this is great it's all supervised and reinforcement and one big

Self-Supervised Learning


question that we can ask ourselves is is this going to take us to the possibility of building you know truly intelligent machines machines were that you know I'm not talking about human level intelligence but maybe intelligence of the house cat or something like that so a house cat has more common sense than any AI systems that we build that we can build today and the answer is no

Self-Supervised Learning


we need we need you know significant can a conceptual progress if you really if you really want to make machines that are more intelligent that we have today so we can do all the stuff we have on the on the left assuming that we put enough efforts in them engineering efforts like you know separate cars you know semi autonomous cars better medical image analysis systems you know

Self-Supervised Learning