supervised of them sometimes there's like a giant gap right here for the same model it seems so all of these numbers um I'm not sure you should put too much weight on the fact that this is now outperforming the other methods I would not put like unless this is like SuperDuper replicated very often I would not put a lot of weight on the fact that it is better what I would put a lot of weight on is the fact that it works at all and and achieves you know good performance and there is more they make they have like experiments right here that show


that their method they be a B y ou L is much more resistant to like changes in hyper parameters so here you can see that it falls off much later when you reduce the batch size which makes sense right because seem clear is one of these methods that uses negative samples and for negative samples it uses the other samples in the mini batch now if you have less samples in the mini batch that means you have a less representative distribution of your entire data set as negative samples and therefore if you increase as decrease the mini batch then

BYOL: Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (Paper Explained)


this drops off and also they show that for example their method is much more robust to the removal of a couple of these image augmentations so all of this I find actually pretty cool but the the actual numbers here first I'm not super duper interested that they get like two or one points more in something but they do perform like a lot of experiments and that it shows that you can apply the method to different things it's not only like in one setting so that's pretty cool it works at least at the you can say it works at least as well as other


methods and it is a lot easier because you don't have this negative sample things now the last quarrel I have with the paper and where is it where is it somewhere they say that we release the code that they release the pseudocode they don't release the code they release the pseudocode in the appendix so I mean there are reasons why you sometimes want to release pseudocode and that's if Liane algorithm is so high level and so simple in its high level T and so modular to be fleshed out that you can't like it makes more sense but here it's


like pseudocode in Jax and and come on is it really that competitively advantageous to retain your code this is it's just not reproducible with this you know that they have like 50 billion hacks in their code and yeah so deep mind has this history of just not releasing like publishing behind paywalls and and just giving pseudocode that has lots of mistakes in them like the new zero pseudocode you can't even like run it in its basic form if you fill in the things it's it's a it's a bit annoying in any way the method


itself seems promising for representation learning as I said especially because it's pretty simple it still heavily relies on these augmentation methods so and that's what they say right here nevertheless py will remind remains dependent on existing sets of augmentations that are specific to vision applications to generalize beyond to other modalities it is necessary to obtain similarly suitable augmentations for each of them designing such augmentations may require significant effort and expertise there for automating the search for these


augmentations would be an important next step to generalize to other modalities and I'm not sure if you can do this automating the search for these augmentations I guess you can do it if you have like a supervised data set and then you can search and then you can use those augmentations for the unsupervised but it seems a bit bootstrap eeeh no pun intended right here I think the the power of the of these representations again comes from the fact that we have these augmentations carefully constructed so oh yes


the last thing broader impact statement just read this like try to estimate the perplexity of this broader impact statement let's go the presented research should be categorized as research in the field of unsupervised learning this work may inspire new algorithms theoretical and experimental investigation the algorithm presented here can be used for many different vision applications and a particular use may have both positive or negative impacts which is known as the dual use problem besides as vision datasets could be biased the


representation learned by Buell could be susceptible to replicate these biases like come up so people who advocated for making everyone do this is this what you wanted is this like is this a satisfactory result for you and if you have this as a reviewer is this okay or not I mean let's just cross out some words here blank that's black like field let's just put field or machine learning why not machine learning machine learning this work inspire new algorithms yes the algorithm presented here can be used for many different


machine learning applications and the particular use may have both men - yes besides as datasets could be biased representation learned by this paper could be susceptible to replicate these biases well there is a copy pasting that you can apparently put into any and all papers that you write from that one and hey deepmind's doing it so you know there you go okay may be a bit cynical but I'm I like I told you this would happen I told you and you know okay so that was it for my comments right here they do have like a giant ton of


hi there today we'll look at self training with noisy student improves image net classification by chidze sie mintan luang eduard hovi and kwok v li so this paper takes an imagenet classifier that's been trained on the imagenet dataset and uses that classifier as a teacher model to label a whole bunch of unlabeled images and then it trains a student model that is larger than the original teacher model on those teacher labeled images and that turns out to improve the classification on the imagenet validation set now that

Self-training with Noisy Student improves ImageNet classification (Paper Explained)


there is a couple of things that make this all work and today we're going to explore how this paper does it and what they say is important if you enjoy content like this as always don't hesitate to share it out or tell your friends about it and if you're not subscribed yet then do so um i would appreciate that and you'll get more content so win-win so this this paper is about semi-supervised learning in um in effect so it's at the intersection actually of semi-supervised learning knowledge distillation and transfer


learning so what do we mean by semi-supervised learning usually in supervised learning you'll have some sort of data set and the data set will contain let's say it's an image net it's image data set so the data set will contain images this is an image with like some sort of cat on it and it will contain the labels according to that so cat now in semi-supervised learning you you assume that so this is supervised learning in semi-supervised learning you assume that only part of your data set has the labels so like only this part down here has the


labels and the upper part does not have the labels so that's semi-supervised learning it's often the case when it's very expensive to get labels so you can only get labels for a couple of images in your data set but very often in semi-supervised learning you still assume it's the same data set there is a slightly different setup here that's called transfer learning so in transfer learning what you'll have is you'll have your data set that has the labels but it's very small so you'll notice i've drawn it smaller


that means you have very little that is also the case when it's very expensive to get labels but also it's expensive to get the data itself this is often the case like say in medical data where not only is it expensive to get labels for like a ct scan it's actually expensive to get the ct scan so what the goal in transfer learning is is to say well i do i do have only this small data set but i do have this giant other data set over here now can't i it's not the same it's maybe they're not ct so these are ct scans maybe these are


x-rays right they're fairly similar similar technology um if you slice the ct it will give you sort of an x-ray can i you know train my model pre-train my model on x-ray data and then fine-tune it on the ct data so that's called uh transfer learning usually now this can be done with or without labels so it can be that for the x-ray data set you do have the labels or you don't have the labels there are techniques for all of those now what we're going to look at today is kind of this situation right here it's the transfer learning situation


where you do not have the labels for this x-ray data set but other than in this x-ray example what we're going to look at is the small data set is going to be our imagenet database so our original picture with label database so you'll see immediately the difference here is that in the transfer learning setting we usually assume that the data set we want to train on is fairly small here you know imagenet is already sizeable but what we have is we have a much larger database of unlabeled images that we can just get


from the internet so we can scrape the internet for any kind of pictures and that will be our unlabeled data set and what we'll try to do is somehow incorporate this unlabeled data set here into the training process to get better on the imagenet data set okay so this is the the problem statement is you have the imagenet dataset and you have a second much larger data set of unlabeled images and you somehow want to make use of them so i hope you see how this is sort of connected to the others it's essentially sort of a


transfer semi-supervised learning setting but with the exception that usually in transfer learning you assume that the the labeled data set is like super small which is not the case here and that's going to result in us being able to apply a different technique so this different technique is called the noisy student now usually what you might do in a transfer learning setting is you might want to start with that big data set right because that's the data set that's sizeable enough to allow you to train a really big model on it and then you


fine tune and you you sort of hope that the information transfers over here on the other hand what we want to do is we start with the imagenet data set so first we train this in a supervised learning fashion into our model now this model is going to be called the teacher model we know how to do this we know to train imagenet models right so we can train this into a teacher model that has a reasonable accuracy on the imagenet data set step two we're going to take that big data set over here and use the teacher model to label


the unlabeled images so for each image for each image coming in here the teacher so maybe this is again another cat the teacher will say that's a cat okay so that gives you the big data set where now you have images along with labels just the labels aren't true labels they're generated by the teacher and then in the third step you train this big data set you train on this big data set and that's what you call your student model and then the student model in this paper will see how can we make it such that the student


is then better at the original imagenet task than the teacher ever was which seems counterintuitive at first because all of the information that the student is trained from is basically what the teacher already knows right all the labels here come from the teacher therefore the student shouldn't be able to outperform the teacher but in this case the student will be able to outperform the teacher and their argument here is that this is mainly due to the fact that you use noise in this training procedure so when you train the student what


you'll do is you'll use noise and one of the types of noise is that you severely augment this data right here in order to train the student now we've known for a long time that data augmentation for example in the frameworks of self-supervised learning and so on can have a very large benefit to training and here the fact that we incorporate this at extra data and we use noise and augmentations on it is going to result in a student that can sort of learn more about the data than than the teacher did know okay this this is basically it and


as you can see this is kind of their main final result where they say on imagenet our top one accuracy sort of increases right here and uh even on these kind of subsets of imagenet or these are sort of corrupted sets of imagenet they make even more substantial improvements as you can see here now we'll go into what these corrupted subsets are but you know just for now these here are very difficult variants of imagenet they can be severely corrupted or or distorted and so on and you can see that the model improves severely over


the previous state of the art which basically means that this model is more robust and that's a direct consequence of the noise now one last thing i should say is that the student here is also larger than the teacher so that's also one thing that makes the student better so what you will make is the student model is larger than the teacher model as a model as the architecture so in combination with the noise right here with the noise in combination that means the student model is probably able to capture more of the variance of the data it's


larger it has more parameters it can learn more about the data together with the noise it can probably be a more robust and that's what makes it generalized better and we'll also see as we see here it's more robust to these transformations and it's also going to be more robust to adversarial perturbations so the the technique again is uh illustrated here as as we said it's pretty simple first so step one step one train the teacher model with label data as you would step two you infer the pseudo labels on unlabeled data step three


you make a student you make sorry with step three over here train an equal or larger student model with combined data and noise injected so they don't they use the original label data here and the pseudo-labeled data right here in order to train the student but still this the student doesn't have more information more label information than the teacher had it simply has this teacher labeled teacher labeled unlabeled data also to train on now the crucial part here is well first of all that the student can be larger and second of all that there


can be noise and the noise comes in three different forms so first of all you use data augmentation which we've already seen this is sort of like random cropping or mild rotations color jitter whatever they use a rand augment here which is a specific technique to apply these augmentations um they use dropout which is a fairly old technique where you in the student model that you train you randomly drop out connections which makes it more robust and more generalizing and then you also use stochastic depth now stochastic


depth is a technique when you train a model what you'll do during training instead of always passing your data forward through the layers like this you use some sort of a drop out but with entire layers so what you'll do is you'll pass your data forward and then randomly you'll skip a layer and then pass it forward again now these these might seem weird first because uh yeah it might seem weird but in if you know that most models especially computer vision models nowadays are residual networks which means that


their layers look like so you have the input you have some computation and then you have the output and then there is already a residual connection that basically adds the original signal together to the result of the computation so all you do in this stochastic layer dropout or this stochastic depth right here is you basically disable use you disable this connection right here and all the signal has to flow through here if you read the residual the resnet original resnet paper they make it pretty clear why the residual connection is a good


idea basically they say these computations here they if you have a very deep network each layer only has to basically um do very a little bit of computation that that can be bypassed uh fairly efficiently for a lot of data points so it's not that hurtful to bypass a layer and in this case they actually use it to just bypass some of these small computations and inject some more robustness into the student model so with these three strategies to bring noise into the training process one is on the data and two is on the


student model itself they train the student model and then fourth and this is what we didn't have before fourth or maybe we put four here make the student a new teacher so now you can iterate you can use the student model that you just trained to again label the unlabeled data and then you can use another student model again under the influence of noise to train from that student model and so on and you can go on and they do up to like three iterations of this where they always take the new the student as the new teacher and then


use a new student model to train from that teacher and they get better and better as they do this of course there's like a diminishing returns but it's pretty impressive that this even works right the new students in fact aren't even larger than the old students it's just that the students are larger than the original teacher model in most of these cases so here's the algorithm written down you'll require labeled images right here and unlabeled images which are the ones with the tilde so first you learn the teacher


model which minimizes the cross entropy on labeled images this we already know this right this is the label this is the image according to the label and you train the teacher model which is this thing here and you can see here noised so already in the teacher training process you want to introduce this noise you want to introduce these data augmentations these are as i said these are standard techniques to make models more robust and therefore more generalizable yeah we know from these from these self-supervised papers that these


augmentations are very powerful and the way you design them basically if you one of these augmentations is a random crop which means if you have an image you randomly crop out like part of that image and then that's your training sample and not the entire thing so by doing this you basically teaching the model to ignore the exact location and scale of things on an image and you can do this because you as a human know that you know i can zoom in i can zoom out into something and it won't change what's on the picture and so


that's you use these augmentations to kind of heuristically tell the model what it should be invariant to and that is that is a very powerful technique uh to regularize basically to to robustify these deep deep methods and this is used the same here so already in the teacher model we train with these noise and then step two use a normal i.e not noise teacher model to generate soft or hard pseudo labels for the clean i.e not distorted unlabeled images and this is important they stress this here that when you when you label the


unlabeled images you want to use the model that is without the noise and you do it on the not distorted unlabeled images so when you infer the labels it's very important that you have clean accurate labels without any sort of noise in them so label noise is not something that they have found to help in this case so not label noise on the teacher that is so you can see right here on the unlabeled images we'll use that teacher model without the noise um to infer the labels now they say these can be hard model hard labels or soft labels


so what does that mean if we generate hard pseudo labels that means that the y here is simply going to be either 0 or 1 or 2 or 3 and so on so just the index of the class whichever class is most likely that's going to be our label this is exactly how the supervised data sets come right so this is what you'll think first when you see that however soft pseudo labels means that the y will be a distribution so instead of being of class 0 it will be sort of let's say 90 percent of class zero but also five percent class one and five


percent class two right so you'll output the distribution um instead of the just the label and they have found that the soft pseudo labels work slightly slightly better than the hard pseudo labels okay thanks so they use the soft pseudo labels here because they work slightly better but you can do it with hard or soft labels the important thing is that you use the teacher to generate as accurate as possible labels for your unlabeled data then third we've already seen this learn an equal or larger student model which


minimizes the cross entropy loss on labeled images and unlabeled images with noise added to the student model so as you can see labeled images and unlabeled images so we're in this semi semi supervised learning setting right now you take in both together with noise and noise here is in bold which means they stress it again this is important so you can see that the loss is composed of two different things these are the true images of your original model and you use that and this means you noise the student model


and that noise can be on the data or in the model itself and here also the unlabeled images that you have labeled with the teacher you do the exact same thing so you train on both of these data sets and step four is if you want to do iterative training use the student as a teacher and go back to step two now they have uh some more tricks when they do this iterative training they are also up the batch size during the iterative training and so on so they do a lot of things to make the student learn something more something better than the


teacher and i think this the whole paper it doesn't it doesn't state it explicitly but i think the whole paper everything they do here is to kind of force or allow the student to become better than the teacher by by giving more noise by making the student larger by making the batch size for the student larger and so on so you you want to sort of inject as much invariance as you can and that will make the student learn more so they say here noising student when the student is deliberately noised in its it is trained to be consistent


to the teacher that is not noised when it generates the pseudo labels in our experiments we use two types of noise input noise and model noise all right first data augmentation is an important noising method in noisy student training because it forces the student to ensure prediction consistency across augmented versions of an image specifically in our method the teacher produces high quality pseudo labels by reading in clean images while the student is required to produce to reproduce those labels with augmented images as an input


second when dropout and stochastic depth function are used as noise the teacher behaves like an ensemble at inference time when it generates pseudo labels whereas the student behaves like a single model in other words the student is forced to mimic a more powerful ensemble model we present an ablation study so this it's a bit weird what they say here um don't be confused you use the dropout and the stochastic depth on the student model and they they say here if you do this the teacher behaves like an ensemble at inference time


whereas the student behaves like a single model and yeah it's it's a bit of a weird formulation but it's it's true like the teacher the teacher will produce these same uh the label for different pathways through the student if you use dropout and kind of stochastic depth and therefore the student is kind of required to approximate each time each forward pass has a different forward pass through the layers through the connections with dropout and it's forced to approximate that teacher label with all of these


um different things so you see that you you put in a lot a lot of techniques so they have even other techniques um there is one additional trick and it's not and it's not one actually they have so many tricks and if you look at their experimental setup that it's crazy like they describe exactly we reduce the learning rate like this and the batch size like this and so on so to get state of the art on imagenet it's not enough to just have a good idea of a new thing to do what you you have to have the good idea and then


execute it almost like really well um because you have to regard all of these additional tricks that people have figured out over the years in any case they say it works better with an additional trick data filtering and balancing specifically we filter images that the teacher model has low confidence on since they are usually out of domain images so that goes to a point where if you see we have this imagenet label data set right and we have the larger data set now the larger dataset simply contains images and there is no guarantee


that the images are actually of the classes that we have in the imagenet data set right here we have a thousand classes here there's no guarantee that these images fit into any of those classes yet we still ask the teacher model to put them in some of these classes now you can filter out part of those images um if you can look at the teacher model and you look at its confidence so when it outputs a distribution if if there's just two labels let's say if it outputs a distribution like this that's wildly different than if it


outputs a distribution like this both are class 1 labels but one is much more confident than the other so what you want to do is you want to filter out these low confidence labels because you know the model isn't really sure but it has to assign a class but that's usually an indication that it is an out of domain image so if they filter this it works better and then also to ensure that the distribution of the unlabeled images match that of the training set we also need to balance the number of unlabeled images for each class as all


classes in imagenet have a similar number of labeled images for this purpose we duplicate images in classes where there are not enough images for classes where we have too many images we take the images with the highest confidence okay so this is just another technique this has basically nothing to do with their core idea but this is just another thing uh where they say okay we can treat this big uh thing that we scrape from the internet you know we can somehow filter and balance it smartly and that will work


even better all right so let's go into the experiments of course there um so what they do i think where is the graphic what they do is they take an image net sorry they take an efficient net right here and they trade they first train an efficient net um a smaller efficient net as we said for to be the teacher and then they train a larger efficient net for the student the best model in our experiments is a result of three iterations of putting back the student as a new teacher we first train an efficient at b7 on


imagenet as the teacher model so you can see in the table right here what the b7 achieves the efficient net b7 here you can see it has 66 million parameters which is fairly small compared to these other kind of previous state-of-the-art methods on imagenet right so they first train this and that will achieve something like an 85 percent accuracy now if you just train a larger model this efficient net l2 right here that has you can see 480 million parameters so a lot of more mainly parameters but you just train it


on the same data set on imagenet you will get a 0.5 improvement and you can see that here with noisy student training with the exact same model so it has the same amount of parameters you'll actually get an 88.4 so i like a more than a three percent improvement and that's with the same model just with this different training procedure and inputting these 300 million unlabeled images that you have laying around but the all the information about all the label information comes from the imagenet dataset and comes from this efficientnetb7 teacher


model so that's basically you can it's a testament that out of this out of this 85 you can make this 88 uh just by smartly using the information that the model that this model has learned about the data and transferring it to new data so they train an efficient net b7 that's the small model as a teach model then by using the b7 model as the teacher we trained an efficient net l2 model with the unlabeled batch size set to 14 times the labeled batch size and they stress that it's important that you up the batch size


that's another thing that makes the student learn more than the teacher then we trained a new efficient net so by the way these this 14 times it's also it can be done because now you have more data right so you can also up the batch size then we trained a new efficient net l2 model with the efficient net l2 model as the teacher lastly we iterated again and used an unlabeled batch size of 28 times the label batch size the detailed result of the three iterations and so on okay so you can see that it's a fairly complicated procedure but you can


gain and gain and gain by simply up upping the um by simply upping the or iterating on this procedure and i think they have it somewhere here yes so as you can see if iteration one you train the efficient net l2 you start it with the the b7 you train the efficient at a2 with a batch size 14 times larger and you gain significantly right this gains about two percent over the original efficient net then you iterate again with the same batch size and you get uh like as a 5.5 improvement and you iterate again with


an even larger batch size and you get a 0.3 improvement so there's diminishing returns but still you can see that you know the more with the introduction of noise with the introduction of the larger model with the introduction of the larger batch size these are all things that help the student basically become better than the teacher all right so they do a bunch of other experiments so their main comparison is right here where they say look if we if even if we train the same model with this noisy student training we can make


you know pretty large gains over the model over the same model where we do not train it with this noisy student training so this really seems to help you know due to the noise due to the additional data they do a lot of ablation studies so that's pretty interesting and they also do these studies on this special imagenet data set for example imagenet c you can see that there are quite a bit of distortions right here i don't even see if you can see it on this video but this is a swing so the swing right here is like something like this


but you almost can't see it and you see that the bold on the left is always the prediction of their model while the thing on the right is the prediction of the original model so this model they claim is significantly more robust to these kinds of perturbations and they do an analysis of this where they show yes in fact it is [Music] so i think we've already seen this at the beginning that the noisy student is significantly uh more robust to these perturbations and they also test this to adversarial perturbations


so right here you can see that the original model drops pretty quickly as you increase the epsilon the epsilon is kind of the strength of the adversarial perturbation and the noisy the original model drops very quickly to you know fairly low accuracy while as the [Music] noisy student training uh drops much much less quickly now this um is another testament to the fact that what you do i think what's happening is you have your data space right and you have your data points in it now when you do the like normal data


augmentation what you'll do is you not only force the model to predict those points correctly but you'll sort of make a bit of a cloud around them and you force the model to predict that cloud correctly now if you introduce more data and you do even more noise what you do is you'll make these clouds kind of larger and that means the model is more robust to any sort of perturbations in these clouds right and and that means it's probably also going to be more robust to adversarial perturbations so that's sort of how you can think of


this uh this introduction of noise uh to make it more generalizable how does this generalize better so if you think of this data point right here if i'm looking to generalize that means you know i have this iid data set so probably my test data is going to be related to the training data so i might get a data point that's fairly close to that data point and generalizing means i classify it correctly now if this cloud is very small like it is here my decision boundary could be like here right and even though the


tres test data set is fairly close to the original training data point it's it won't it will be classified uh incorrectly however if my original cloud during training is larger you can see if i train a model it can maybe put the decision boundary here and then my test data point will be included in on that same side so that's kind of the idea behind generalizing better of course that's a vast simplification and also to say that this here is an fgsm attack so this is kind of the weakest attack in the adversarial perturbation spectrum


they do say under a stronger attack pgd which is a fairly strong attack with 10 iterations at epsilon equals 16. noisy student training improves efficient net l2's accuracy from 1.1 percent to 4.4 percent and not this um like you know 1.1 percent really means the model is almost like dead this is lower this is like random performance and 4.4 is still a bit above random performance but um yeah you could probably you could probably get there by simply using any sort of noise in that case but still you can see that


it is more robust to especially to natural distortions and therefore it generalizes better as i said they do quite a bit of drop sorry not drop out ablation studies to figure out where exactly um the performance comes from and the answer is it pretty much comes from all the things that they've described so here you can see um the in the effect of that extra data set and you can see pretty much with that extra data set all the all the situations improve here you can see what do you what is happening when you do not


augment the student uh when you do not date to augment you can immediately see that the accuracy drops and then when you do not augment and also don't use these model noises then the performance drops again and lastly when you use the teacher but you noise the teacher you can see also here the performance is dropping from the original um quite a bit so all of these things kind of contribute and they do much more ablations and they have listed their findings here so using a large teacher model with better performance leads to better results so


you know as the original teacher you should use as good as possible a teacher model you can find second a large amount of unlabeled data is necessary for better performance okay so if you want to do this you better get a large large amount of extra data because that's one thing that makes the student perform better soft pseudo labels work better than hard pseudo labels for out of domain data in certain cases fourth a large student model is important to enable the student to learn a more more powerful model okay so because usually this um


knowledge distillation is what it this this is usually called knowledge distillation if you use a teacher model to train a student model and it is often used when the student model is smaller than the teacher because you want to kind of become more efficient you from so the teacher is large you make the student small and you usually sacrifice some accuracy and here they say if you want to gain some accuracy you need a large student model it can't be like a small one number five data balancing is useful for small models


number six joint training on label data and unlabeled data outperforms the pipeline that first pre-trains with unlabeled data and then fine-tunes on labeled data so this is in contrast to like what people have done before in the self-supervised learning and so on where it's always kind of pre-training then fine-tuning or in the in the transfer learning setting seven using a large ratio between unlabeled batch size and label batch size enables models to train longer on non-labeled data to it to achieve a higher accuracy okay


we've already seen that they have used that and number eight training the student from scratch is sometimes better than initializing the student with the teacher and the student initialized with the teachers still requires a large number of training epochs to perform well this is fairly interesting because it kind of alludes to the fact that the minima in weight space if so if this is of course the case if the student model is the same as the teacher model so in like iteration two or three or what not um it means that you know in weight


space if we look at you know you might want to start the student here and the minimum is right here and you might want to think that if i learn the same thing then the minima are fairly close together right so the the teacher's minima might be here and the student minima might be fairly close so it might be beneficial if i if i start not over here but actually start at the teacher's minimum but this doesn't always seem to be the case and that is a fairly interesting observation because it kind of means that we're talking about


different minima here we're talking about the student model learning different things and that's what we've discussed already the student model kind of learns to be robust and that's probably a minimum that's fairly far away in weight space at least in in a sort of energy landscape weight space uh might be the case that it needs to actually overcome kind of a a hill here even though the minimum might be close there's lots of research in like how minima are distributed in these weight spaces which i don't want to go into right here


but it is a fairly interesting observation that it's not always helpful to initialize the teacher sorry the student at the teacher's optimum okay so this was the paper and you know this is this is the type of research where i do appreciate kind of the these large labs taking it on because they have the resources to do all of these ablations all of these different models cross them with these giant data sets and so on which i guess university labs just would not have and this is a fairly um thorough paper really investigating which parts of the


pipeline you know do something and which ones don't and usually i i'm fairly critical of pipelines that have like 50 billion tricks um because you never know where the improvement exactly is coming from but you can sort of mitigate that criticism by doing all of these kind of ablations on the different parts and really showing look this is important but this is also important but this is also important but this is also important so yeah that was my two cents to this paper i hope you enjoyed this and i'll see you


thank you very much Michael very much for the invitation it's a great pleasure to be here and we're in some sense more of a user of many of the deep learning techniques which have been developed here in this community and I just wanted to highlight a few examples of how we can use deep learning and medical imaging and more specifically talk about image reconstruction super-resolution and segmentation but probably in the same spirit as in as a last talk just as a sort of introduction there is a lot of excitement in in medical imaging in this

Daniel Rueckert: "Deep learning in medical imaging"


in this area but there is also a lot of hype so if you look at at several magazine covers over the last last year actually there's you two amount of excitement for me is if you really read the headlines there's also quite a lot of over-excitement and in some sense the press has picked up on this and quite often taken a number of things out of context so if you look at for example one of the the comments which geoff hinton made in 2017 said well you should rather stop training radiologists and if you read the whole interview he said many


more things with many caveats but of course this one single headline is more exciting than actually quoting really what he said in detail but a very famous radiologist said something more sensible probably is just a question where the AI will really replace radiologist probably the answer's no but actually a radiologist who do AI will replace a radiologist who don't and I think that's very it's very good a comment now the other thing is you probably all you're familiar with conferences such as nips and you've seen it and many many more


people going to these conferences so some of our radiology colleagues go to a conference called RNA and if you think nips is big and very unmanageable our SNA has around 45 to 50,000 attendees in Chicago is apparently the only city in the world big enough to host this conference but one of the things which really happened this year or last year 2017 which is very interesting is normally if you go to these conferences you have a strand on x-ray stand on CT on molecular imaging it's basically all the different


imaging modality and this year for the first time an equal strand is machine learning which really shows you how much machine learning is changing medicine in particular probably medical imaging there are huge number of opportunities many of which we sort of copied effectively from the vision area but another another really important aspect is there's a huge amount of data which is now publicly available in medical imaging and I just want to highlight one particular example because I've been involved in in this in the UK is


something called the UK biobank for those of you who don't know what UK biobank is actually it's a population study of nearly 500,000 people but a hundred thousand of these subjects will be imaged and actually they have already done 1/5 so they've already done over 20,000 of these subjects and so they're acquiring very high-resolution imaging data from from this subject and one of the things which is interesting is they're not only acquiring images of the whole body which of course is very useful with MRI but they're also


acquiring a dedicated brain imaging so not only structural brain imaging functional brain imaging and diffusion brain imaging and they're also quite dynamic images of the heart for example are allowing you to look at cardiovascular function and for example study the interaction between the brain and the heart which is something which is not very well understood at the moment and more importantly you also have available lifestyle information how many cross wants to eat how many cups of coffee you drink genetics and Clint and


links to clinical records now why is this exciting is because actually that data set is available everybody in the world to use so if you wanted to use that data set you have to fill out an application make sure that you don't try to be anonymized data but otherwise you can download the data and and use it for for research purposes so that's it for example really something which is a game changer I think in medicine so if you look at what are the opportunities more specifically really sort of there is a sort of pyramid where


the value of what we do increases the sort of lowest level you can use machine learning to help you with image reconstruction for example you can automatically plan your scans so if you go if you line in mo a scanner for example there's quite a lot of fiddling by an operator by not lo radiologist but typically a radiographer who sort of plans the imaging you can of course use image enhancement super resolution is something which I'll show you in a moment you can do the conventional semantic image interpretation for example find organs


Sigma and these organs you can quantify biomarkers for example measure a tumor volume and then sort of if you come to the more higher level of the more we're actually probably highest added value is sort of computer-aided interpretation and diagnosis there are probably very few applications at the moment really being used and probably the only one I can really think of where actually machine learning has had an impact at those high level features is at the moment in for example mammography screenings so there are some systems


which effectively act as a second reader to a radiologist but really we haven't really cracked the top of this pyramid but there's quite a lot of work going on at the bottom I also want to show you a few challenges where I think really we we need your help in developing new methods sort of at the moment most of the techniques we use are supervised techniques so that means our training data is absolutely critical for what we do if you look at our colleagues envision what they typically do is well you go crowdsource your your labels or your


annotations and if you say that to radiologists they don't really react very well to this so they they obviously saying well she'd really asked some experts to help you with we're doing that but in the UK at least for example we already trained far too few radiologists so if you ask them to generate training data ie label data it's really very challenging and quite difficult to do and more importantly and this is something which is quite I think not always perfectly understood is if I ask an observer to tell me whether


there's a car in a photo or not then the answer is pretty unambiguous either there is a car or there isn't in in medical imaging our training data is often imperfect ie if you ask three different radiologists that will give you hopefully not three different answers but they might give you more than one answer so you really need to train in a scenario where your data might not be perfectly labeled and of course if that's then not perfectly labeled how do you actually validate your algorithm against something where


there is really no gold standard the other thing is quite often is we've had very great success are reporting fantastic algorithms in a scientific paper and then when you deploy them in a clinical scenario they don't really work that well and if you actually look at scanners they're really only three big scanner manufacturers and when they produce scanners they're not only very in the color as you see here but they really produce slightly different images to a human observer that doesn't really matter but actually turns out to most of


our models we train with machine learning it actually matters and quite often we don't have access to data from all different sites or from all different vendors and if you every time you then go to your radiologist colleague and tell them well just annotate a few more data set and then I can do some retraining they don't really react very well to this at least in my experience okay so I want to show you three different applications for where we at the moment use deep learning and we're actually has had significant impact really


transformed quite a lot of the things we do and the first one is really on image reconstruction and and this is really probably something which a number of you will have worked on it's actually quite well understood problem using it as an inverse problem but I want to show you a particular application here and that is using a modality which some of you may know is called magnetic resonance imaging it's a great modality because it's it's safe it can do a lot of things can show you many different properties of the body but it's relatively slow


that means it's good for measuring or acquiring images of the brain but not so good if you want to for example do cardiac imaging which you see here so if you look at these cardiac images which you see on this particular slide this looks like it's one single heartbeat but unfortunately mr imaging is not fast enough to acquire this so what you typically do is you measure with an ECG in which actor in which state the heart is and then I'm actually taking bits of data from different heartbeats and I'm assuming that if my ECG signal shows me


I'm in the same hearts state as as previously then I can take these measurements and combine them to form an image so this particular image here is typically probably an average of half I had 10 different heart beats so it's not a single heartbeat be much nicer if we can acquire this faster because actually to acquire the state of the patient has to hold their breath and that's especially if you have heart disease then of course it gets a bit more tricky so 10 heart beats is right around 8 to 10 seconds that's how long you have to


hold your breath and you really would like to do this fast enough you don't have to hold your breath at all okay so just a very primitive explanation of how we take our measurements in MRI imaging and really the physics is not so important but it's just want to highlight why this is so slow what we typically do is we traverse our measurement space in which we take measurements which is called the K space and we have to traverse it sequentially for a number of reasons which I don't really want to go into too much detail


but once I've made my measurements in k space then actually image reconstruction is trivial I just applied the Fourier transform and I get back my image so that is a relatively slow process because I have to acquire these measurements in k space sequentially and of course if I want to for example create dynamic images of the heart I have to keep on doing the same thing over and over again but actually the heart is only changing a bit between these different acquisitions so there's an enormous amount of spatial temporal


redundancy in the data now what is the easiest solution to accelerate your imaging process well instead of acquiring all of the measurements I need i just acquire a subset of the measurements which i need so for example if i acquire as you see here in the bottom only 25% of the data of course i'm twenty and four times faster which is great for the patient but for the radiologist this image is not really acceptable because it looks actually much degraded because i have a lot of aliasing artifacts in the data so this


problem has been well studied there are many techniques which can try to help you recover that that information which is in the top image you want to effectively denoise or D alias this image here at the bottom and the most commonly used techniques are effectively compressed sensing techniques so these compressed sensing techniques have been around for a while and MRI imaging they've been around for ten years very successfully used but more recently some some machine learning techniques have really significantly outperformed


these techniques and really what is the key difference between the compressed sensing techniques you see in the in the top and those here at the bottom so the ones in the top they effectively use generic priors sparsity low ranked they're not really data-driven priors whereas the techniques in the bottom they effectively try to learn the price from the data and try to improve our acquisition in that bad sense so just want to quickly show you the problem formulation we have we have effectively our case based measurements so all our


measurements are because we're making frequency measurements or complex values so the images we typically see are just the magnitude of those complex numbers but our measurements are complex valued and the image we want to recover is also complex valued and in fact we our measurements are related by to the image through a undersampled for you including matrix so this effectively under samples your measurement space and applies a Fourier transform and of course our acquisition noise and the under samples for your operation if you want to write


it down differently you can just write it down as a Fourier operation and then effectively a mask which defines how you're under sampled case space and that's that's quite important okay so in what you're trying to then solve mathematically is typically an unconstrained optimization problem consisting of your regularization term if you would use compressed sensing you'd probably use here an l1 or LZ row prior a norm or and the data fidelity term and that data fidelity term is quite crucial and it's actually something which is not so easy to bring


into into the equation if you use a machine learning so you know if we try to for example use a CNN because in some sense we we're trying to formulate this as a Dean problems with taking as an input one image and we're trying to produce a denounced image then we effectively have two different terms first which we're going to use effectively tries to make our denies image which our CNN estimates close to two the image we have but at the second time we also need to make sure that the image which we reconstruct


is actually close to the case based measurements we have obtained and this leads us to when we develop our CNN to have a layer which is probably not very common in other applications which we call a data consistency layer so this forces our data our fidelity and what this effectively does is this equation you see here is we have some part of missing key space if we make an estimate of this we simply keep that estimate of of K space and we have some part where we have measured K space and that measured bit of K space we're going to


average together with with our estimated K space depending on how noisy this is so if you have a completely noise free case you would assume that lambda s goes to infinity and you would only keep your original measurements in k space of course you have some measurement noise you might actually really average those two together and so here in this particular equation s CNN is effectively the the foria version of the image which I've reconstructed and this here's our zero field K space so this is what what we would normally have so having this as


a layer will force you to have a consistent image reconstruction with what you have measured and it turns out that because we want to train this end to end we need to be able to do the forward and the backward classes in order to also propagate our gradients back now the forty operation is a linear operation so actually turns out that the forward and the backward pass you can write down quite easily in solution and really in the backward pass you this is a Jacobian of the data consistency layer and if you decided


your lambda is trainable because you don't know how much measurement noise you have you can also write down the derivative with respect to lambda so this is quite nice because I can basically propagate my gradients easily through this data consistency there so here's what you actually end up then with is you have an input image a complex valued input image you have your case based measurements and then you have a number of denoising layers which try to effectively remove aliasing from the image and then after these the


noising layers you have your data consistency layer which then forces your measurement to be consistent with K space and effectively that links to your k space measurements so and then you can sort of in analog to a sort of iterative optimization you can on you can cascade these networks to an arbitrary depth we typically use five different of these cascades so here's an example what you end up with when you see on the left hand side is the image as acquired with six fold under sampling so if it's six fold on the sampling that means the


acquisition is now six times faster I only have 1/6 of the data and you can see that the image which are reconstruct here is virtually useless this year is a technique which is based on dictionary learning and sort of more compressed sensing technique this is our CNN and this is the fully sampled image so of course I I've been sort of assimilating this under sampling here and what you see is that you can virtually see no difference between the fully sampled image and the reconstruction using CNN's the compressed sensing here one is also


quite good but it turns out it's much slower and in terms of psnr it's several it's probably 10 percent are worse and the CNN you can push this even even to higher undersampling right so here's an example of 11 fold on the sampling which is really quite aggressive and you can still recover the the image very well so this is actually a really quite nice result now probably one of the biggest advantages is actually here so of course the the CNN is better in terms of peers and all that's that's that's nice but really one of the most important


advantages is a speed so he used a compressed sensing technique with dictionary learning it takes around six hours to reconstruct an entire image sequence which in some clinical scenarios is acceptable but for example if you want to use your images for navigation because you want to for example perform a biopsy or something else then actually this is far too slow and the CNN is of course very very much faster so this actually now and now you can really do this probably in in 100 milliseconds so this is fast enough to


do it on the scanner for image guided surgery and that's a that's a very big advantage in these techniques good I want to now go from image reconstruction and talk a bit about a two related Table Topics image segmentation and super resolution and you'll see in a moment why why I've sort of lumped them together because they share a number of problems which we have in medical imaging so okay so if I if I adopt a sort of standard approach for image segmentation I'm not going to show you anything in anything new here we for


example use a sort of variant of an FCN for medical image segmentation if I pair that with a large enough data set for training so this is actually quite nice that I said which is also publicly available where a group in in Oxford have annotated 5000 subjects they have taken 5,000 subjects and annotated over 90,000 images in this data set you can use this for training and actually turns out that if you use this for training to fix don't try to segment the heart here you probably can't see this very clearly you


can actually do a very good job in all these images which are part of this UK biobank so even in in slices which are sort of for example here at the apex of the heart where the heart ends which are typically very difficult because the heart is moving in and out of the plane this really works very very well you don't really have any any problems with that if you then try to compare how good does a machine do to to a human then actually the automated measurements for clinically important parameters are pretty much within the variability of


what different humans do so we're not performing better than a human does so there is no superhuman performance but actually we're doing as well as a human does and I think that's quite natural because seems to me quite hard to get superhuman performance from from training data which actually in some sense is quite flawed so you can do this quite easily so there is no real challenge here we can just a talk what what many of you guys have done in envision what is the challenge is that actually the imaging data has a lot of


artifacts which you need to understand in order to really make best use of the data so one of the things is we typically do is when we acquire the heart we acquire one slice we image for we ask the patient to hold a breast for 10 seconds we acquire then the second slice do the same thing again third slice do all of this again and our slices are quite thick which means they have an isotropic resolution so the high resolution inflame but low resolution out of planes so if I if I take that data stack it together and show you that


as a sort of reformatting you see these ugly staircases which comes from the fact that probably every slice is is around one centimeter thick and in in planar probably have around one philometor solution there's the second problem you have which comes from the fact that the patient has to hold their breath for every slice and they might hold their breath in a different position between slices which means that actually one of the slices are shifted correct if you basically treat this as one volume you end up with with problems and for


example if you take this volume here you can quite clearly see well in two of those slices of patient has who held their breath in the wrong location so now I can do segmentation with this data set in 3d or super resolution and this is what you end up when you do for example super resolution which is actually quite nice this super resolution here takes this dataset produces this and has this fantastic thing here where there's effectively a hole in the heart of course the patient doesn't have a hole in the heart because


it would probably not survive with that but of course the super resolution can only do whatever the data give if you give it and similarly in the segmentation here you can also see this disconnected our regions so what we really would like to do is take that data same data and produce either a super resolution you see here or a segmentation like you see here and for that we have to incorporate anatomical knowledge if if you ask a clinician to look at this data they will look at the only slice by slice but in their head


they will build up a 3d representation of what they're looking at and they are completely immune to the fact that you have motion between these different slices so one of the challenges we we came across is really that these standard these lost functions which we normally use for segmentation or super resolution are not really very good in those situations so we thought about what can we do differently because we really want to put a low resolution input data into our network and end up with a high resolution segmentation


segmentation or super resolution so this network not only does acts perform semantic segmentation but also increases the resolution of the data so we decided let's let's try out something which looked very cool called TL Network which has been proposed in in graphics and really has effectively sort of two components the one component is a sort of authoring code and encoder which forces you into a latent space with variables age and then a decoder and that network effectively is trained with segmentation so not with it with will


effectively will label maps and then a second branch which is the predictor network which for example takes an intensity image and predicts this latent representation from from this from this intensity image and we can sort of train this this network in an enjoy fashion now when you look at this you might say well hang on what's this what's this useful for well for example if we train our network for doing segmentation one of the things we can do is we put our low resolution image in we obtain a segmentation using our segmentation


Network so this is our segmentation but we want an anatomically plausible segmentation so we encode that segmentation into our latent space and for our ground truth labels we can also encode it in our latent space and we have then a sort of loss function on that latent space which forces you to be similar in in the anatomical representation of what you're looking for and then you can couple this with your standard cross entropy loss okay so you now have two loss functions one which is a normal cross entropy and one


loss in this latent space which forces your shape to look similar to what you had seen during training and if you do that you actually get a very nice result so instead of now getting these really weird shapes where you get biologically implausible shapes you can constrain your data to be very close to what is the ground truth if you acquire high resolution images by work by the way these high resolution images you can only acquire if you hold your breath for 40 seconds so this requires really sort of very dedicated


volunteers to be able to do that and if you do this with super resolution you can now do the same thing correct with super resolution the only difference here is that actually my super resolution network produces as output an intensity image and then I'm predicting from that intensity image my latent space representation and and I have here the same for the ground truth so now here I do the same thing my latent space representation though comes from these predictor networks which which go from intensity space to latent space rather


than from segmentation space to labor space and here's an example of of what that does if you use a sort of low resolution image this is your standard super resolution approach out of the box this is an anatomically constrained super resolution and this is what you would end up with if you he look at the higher resolution ground truth and here's sort of a movie showing exactly the same thing in the dynamic image sequence so this really works very well and it's actually quite powerful I think really interesting I mentioned to you


before that one of the things which we quite often face is we have trained our models for example using UK biobank which is great because it's so much data available you then deploy it in the clinic and it doesn't really work that well and it's mostly due to really differences in not only the hardware but also the knobs which people turn when they acquire the images so MRI is great because it's so it's effectively a programmable device but it also means you can produce very different looking images and I already mentioned that to


you so one of the things which we have sort of played around with is can we use adversarial training to try to make sure that we learn feature representations which are invariant to the data but also which don't require us to have annotations for the test data because that's actually very expensive to do in in medical imaging so I guess I this problem I think I don't really need to explain to you in very much detail but you basically end up with training your machine on on the source domain trying to separate one set of labels


from another set of labels so this might be scanner a where you have training data then when you go to scanner B you actually see that you have this domain shift where your distribution of features which the network has changed I learned change because the images look slightly different and what we're trying to do here is basically try to find a way in which we can use adversarial learning to effectively help us that these images like these samples get misclassified and we instead would like to learn a classifier which is more


generalizable and this has been really sort of it was very nice paper on this how you can do this with with neural networks which was sort of published a two year or three years ago not two years so it's an ancient paper in terms of machine learning but really works quite well where what you try to do is you try to train a classifier where you try to learn a domain classifier which can tell you whether your data comes from domain a or domain B and you try to minimize the Icarus II of this domain classifier because if that domain


classifier does a good job then obviously you haven't learned features which are very domain in their head and the nice thing is for this you only need labels whether your data comes from scanner a and scanner B I don't need annotations for scanner B so here's the approach which we have used we use a neural network which has has a fancy name called deep medic which was one of the nicer the first ones who could actually do properly 3d convolutions and now of course you can do this quite easily sort of has two different pathways I don't


really need to explain it in too much detail one high resolution one low resolution pathway is really designed to spot brain tumors and has been actually quite successful in in this so here's an example of what it would actually output in 3d so you can produce your fancy 3d renderings for this so we take this network and if the if you train this network it will not be a domain invariant what we what we instead do is we add to the normal segmentation pathway we add this the discrimination pathway where you basically take the


features add a low-level features the mid-level features and the high-level features you put them into your adversarial branch and you try to learn this or you try to prevent this from being a good domain discriminator and then by minimizing the accuracy of that domain our discriminator you tend to learn features which are more generalizable so effectively you have two different terms and your cost function one is sort of your normal cross entropy loss and the other one is your how well you can discriminate the


domain but the important thing is the top this this loss here I can only evaluate for samples from scanner a because that's my training set whereas this domain discriminator i can actually evaluate for samples from scanner a and b because i only need to know whether they come from scanner a and b and that's quite easy to do and it turns out that this actually does does really quite well so here's an example what happens when you don't do this domain adaptation so here actually instead of using data from different scanner we


assume that at test time we don't have one of sequences available so we have to use another imaging sequence and if you don't do then the main invariant features you see you end up with horrible results in your in your segmentation where you're supposed to to spot a brain tumor which you can probably see here quite nicely and here when you have done the domain adaptation even if you switch one sequence to another you actually do quite well good so the last two minutes I just want to talk about some challenges which we have


practically faced and I think many of you might face in in other scenarios is there are many good networks out there for example a unit which in medical imaging quite a lot of people use FCN they have a lot of meta parameters a lot of different architectures you can choose that we influence the behavior and then really it's at least at live to us it quite often looks like it's very hard to make predictions which model will work really well for a given task so one option is to use I guess something you all very familiar with


this sort of non solve all of these different models and try to be as insensitive as possible and unbiased as possible so here's for example in example when I have a flare image with a brain tumor so you see the core of the tumor in red and the edema in yellow then if I use the same network but I switch here cross-entropy for i/o you as a sort of a loss function I get very different behavior and for example this this intersection over Union effectively forces you to make a very hard segmentation and that gives you overly


high confidence values for this so it might not be a very good thing because these things you miss classify with very high confidence okay so the approach which were sort of used as trying to basically approximate the probability distribution which we're really interested in by this model where you have these different meta parameters in there and usually just pick one meta parameter and then run with it so what we really wanted to do is sort of marginalize out over these meta parameters and try to find a more robust


way of doing this and so we use different network architectures for example a diplomatic FCN a unit approach but also we tried out different architectures what we different training loss functions different sampling strategies there are many knobs you can you can turn on and at the mechanic conference which I guess is sort of the the the cbpo for handicap for those who work in medical imaging is sort of they run challenges and this type of approach really was quite successful in this for one the first price out of 50 competitor


us and and really it's it's quite simple because you didn't really need to spend huge amount of time engineering the particular approach okay so I just want to sort of summarize in computer deep learning you've seen a number of really nice papers and literature showing really great progress in this but there's also quite a lot of hype and quite a lot of discussion about whether we actually asking the right or whether we're posing the right problems to machine learning in medical imaging and really to make this truly intelligent we


have to move beyond images so we have to even a radiologist somebody told me actually if you are if you want to study medicine and you really hate patients the one thing you should do is you should become a radiologist correct because you normally never have to interact with a patient but even the radiologist will look at an on imaging information and so really there is a lot of data available this is really a challenging validation like in the previous speaker I think really if unless you really work together in teams


with the with a clinicians engineers you can't really solve this problems or you might actually end up solving the wrong problem and one thing which is I think what exciting is something which which we're trying to do in the future in and currently it's sort of at the moment you have these three separate blocks your quiet your data you reconstruct your data you are now you then somebody define tells you what I wanted to measure and then you do the analysis and you pop out some of some results but really if all of this I can


formulate with deep learning then one of the things I am actually very excited about is that I can do end-to-end optimization so if I know what clinical measurements I want to make I can optimize the acquisition the reconstruction and the analysis for exactly that purpose and I think that's a very powerful paradigm especially because these scanners are effectively programmable there are piece of programmable hardware and you can optimize what they do and of course you can couple it with what Big Data and an multimodal data so I just want to finish


morning everybody and welcome to day two of hike on our first two speakers are Angela liking and Lady moody say and they are from a major telco and there are data scientists and are going to be talking about anomaly detection using also encoders hello okay that's good morning everyone today we'll be sharing a talk about anomaly detection using what encoders I'm melody moody say hi I'm Angela and we are both data scientists obviously at a telco and this is also our first time speaking at PyCon yeah so just to take you through some of

Anomaly Detection using Autoencoders


the content we'll be sharing with you today we'll give you an introduction of auto-encoders how the algorithm actually works a brief history about them thereafter we will give you some architectures different type of popular architectures of autoencoders which may be useful for you for your use cases also particular use cases that are out there where auto-encoders are good to solve we'll also be sharing some Python packages popular Python packages that you could use we'll take you through at repeater notebook we will introduce the

Anomaly Detection using Autoencoders


notion of fraud anomalies and how to actually implement that then right after we'll have a quick sense visualization to show you how assets data scientists interpret the results as well as for business stakeholders then lastly we'll be sharing key takeaways from our experience from implementing this type of problem so how many of you are aware of neural networks I'm sure most of us were there yesterday at Alex's talk so I'm sure you are familiar with convolutional neural it works I mean he went into quite a lot

Anomaly Detection using Autoencoders


of details feed-forward neural networks recurrent neural networks and all these type are solving for particular problems like computer vision machine translations and so forth Auto encoders are a part of the family of neural networks so yeah so as milady mentioned before right Auto encoders are a type of neural network whose goal is to determine an output based on a similar input so as you can see right the goal of the input data is to be compressed so that it's in a lower dimensional space such that when the decoder comes along it takes that

Anomaly Detection using Autoencoders


learn representation of that data the pattern such that it's able to replicate this learned image of this mushroom so now just to get a bit more in depth in terms of the algorithm of the auto-encoders so it's split up into an encoder and a decoder the encoder is simply just a function of your input and your decoder is a function of your hidden layers now as you can see overall your algorithm is represented by the G f of X is equal to R now you want R to be as close as possible to your input layer so you want that data to be very very

Anomaly Detection using Autoencoders


close to each other so and that's exactly why the objective of an auto encoder is to minimize the loss function now what the loss function means is that you want to reduce and minimize the error between your input and your output the way that these neural networks are trained they are trained through back propagation and what that means is that is the recursive process such that it's able to minimize the error between your input and your output and also something just really interesting but I think maybe you might find interesting what I

Anomaly Detection using Autoencoders


do so autoencoders have been around for decades now people such as the laocoon and Hinton have used it are you all familiar with them I mean ok cool now let's move on to uses of auto-encoders right so the first being dimensionality reduction that means that you take your data you condense it into a lower dimensional space the reason for doing that is so that your data itself can be more easily represented visually and this will really assist before you applied into a neural network the next example would be denoising of data you can see that

Anomaly Detection using Autoencoders


initially these images are very hazy fuzzy it you know you can't really see what's going on right now but through the power of auto-encoders what happens is that the noise is removed and it's a more crisper image you can see so now a third example is anomaly detection now what anomaly detection is it's basically a technique for identifying patterns within data so patterns that do not follow the norm so for example in autoencoders for example we have this idea of reconstruction errors so if an observation right it's passed in and it

Anomaly Detection using Autoencoders


doesn't seem very similar to its input like there's a drastic change there the difference then that would be considered as an outlier hence it would be anomalous so you would see these red images these red dots that's an outlier and lastly we get a view of feature extraction so auto-encoders give you a view of which features in your dataset are useful or not so to take you through some of the different architectures that are out there of auto-encoders a very popular one is the restricted Boltzmann machine and this is actually produced

Anomaly Detection using Autoencoders


this particular papers produced by our beloved Hinton and a restricted Boltzmann machine is basically a two layer or two encoder so how it works is that it has a visible layer and a hidden layer the visible layer is where our input would come in our variable inputs it would use a combination of that to get into the hidden layer then basically what it's learning is the difference between the hidden layer and a visible layer it uses metric called KL divergence to measure that difference between the two this particular paper by

Anomaly Detection using Autoencoders


Hinton I would encourage that you go forward and read it if you want to get into auto-encoders it's basically how he used auto-encoders restricted Boltzmann machine and auto-encoders for dimensionality reduction and he actually compares this with PCA and the results that he gets is that with autoencoders he's able to reduce dimensions of nonlinear type of data so the results that he gets the patterns that he under lives were much better than he got for PCA within the field of autoencoders there's two different types of popular

Anomaly Detection using Autoencoders


architectures there's under complete and over complete so what angela has just described to you is an under complete architecture so remember what she said is that we're trying to find the underlying pattern within our input but to do that what we need to do is to ensure that the neurons within our hidden layer are less than the neurons within our input layer to ensure that whatever our reconstruction of our output is it's not a direct copy of the input then it didn't learn the underlying pattern it needs to be a

Anomaly Detection using Autoencoders


pattern that's how we ensure that so that is under complete so that's usually for most use cases that's how we implement an an autoencoder then we have the over completes architecture and there's three different types of them so this parse this denoising and contractor so how many of you are familiar with regularization in neural networks okay a few of you so within your own networks um what we usually do is that within these different if if we find that our neural network is overfitting what we sometimes can't do one technique to listen that is

Anomaly Detection using Autoencoders


to put in a regularizer which means it's to penalize the variables within our weights right but with an autoencoder the spa's auto encoder uses a regularizer but it regularizes the activation functions before getting into the hidden layer as well as here that is to say that you could have an architecture of any type however some of the activation functions are not being initialized that means they not all of the inputs would have been necessarily used so if you experiencing if you build your auto encoder and you

Anomaly Detection using Autoencoders


like oh my gosh still not finding that underlying pattern there's a lot of noise in my data this is a good thing a good technique that you could use another problem that occurs when implementing an auto encoder is that you get the exact copy like it's so annoying but what you can't do if you have such a problem is used the denoising so what do you know is ink does is that you add noise to your input layer and then you use the same under complete architecture of an auto encoder so it is quite a lot if your reconstruction layer is exactly

Anomaly Detection using Autoencoders


like your input the contractive is similar to denoising the problem with adding noise to an input is that you really don't know how much noise you should admin so what contractive does is that in your activation functions it finds the derivative of each activation function so it reduces what they as the inputs so what it's it but that in tells is that it's more robust to noise so the more noise you have in your input because of those derivatives it's easier to learn that particular inherent pattern so we have many Python libraries

Anomaly Detection using Autoencoders


available to us if you are interested in building your very own auto-encoders so first being Kerris which is basically just an abstraction level that sits on top of tensorflow then we have pi torch and then we all know if I could learn I'm sure so the very well notes I could learn and then we have h2o but for this purposes for today we'll be showcasing h2o alright so now we've reached the stage of the jupiter notebook but before we begin there I just want to ask you all a question so who have you have experienced any

Anomaly Detection using Autoencoders


fraudulent acts in your life just array the pans cool so that seems like quite a few of you right now imagine like within industry as well they must probably be experienced vast amounts of fortune and activities that happen to them on a daily basis for instance we could look at within the banking sector were all very familiar with the tappan go system right so now imagine if a card is being tapped 200 times on the same day isn't that a huge red flag like someone's clearly taking your money unless you really like chopping a lot

Anomaly Detection using Autoencoders


LMO sorry and then telecoms we get for engine cases like SIM swap fraud or delivery jessa4 so for instance right it's your customer information however the product that's being delivered to you is not sent to your address but it's sent to an address that's who knows 200 kilometers away from where you stay once again yet another red flag right and then in the retail space you can get fortunate acts like stocktaking or online purchases yeah so earn an example of an actual fraud case that has happened was what's called the Japan ATM

Anomaly Detection using Autoencoders


scam this affected the standard bank that we know though it happened within Japan so what these fraudsters did it's like for real Ocean's eleven but these fraudsters that it was around a hundred people according to the article it's a suspected out around a hundred people went to various of ATMs within Japan and started taking out cash one of the banks that were affected within South Africa was Standard Bank and Stennett Bank lost 295 million rent from this particular activity they did this under three hours that's unlike the solutions so we would

Anomaly Detection using Autoencoders


if I was the CEO of Standard Bank I'll definitely be like okay you fool me once I definitely wouldn't want again criminals are stealing from me the exact same way rights so we find such an emergent type of fraud that occurs and a business get scared so what we do is that we would ensure to reduce that we'd have either supervised learning model or rules so that in let's say the first month we get such a big spike of fraud in the next month that we want to reduce that so we would compact that but definitely those guys who stole that

Anomaly Detection using Autoencoders


money I'm sure they have a new creative way of of stealing from a different type of company all standard bank and so what she wants to do within an organization is to try combats that emergent type of fraud so you can have usual fraud cases but also new types of fraud and if we have an actual algorithm that does work well let's say it's 70% accurate maybe 50% of that money could have been saved cool so just like my lady was mentioning you have like the whole idea of emerging fraud and then like a rule based fraud

Anomaly Detection using Autoencoders


so banks if they really know the kind of fraud that they that are happening right now there's just these will based systems that'll combat it so just once you explain the concept behind anomalies first fraud so as you can see in this Venn diagram right something that is anomalous does not necessarily mean that is fraudulent but something that's fortunate may mean that it's anomalous cool so now I just want to ask you guys as well have a look at this table what stands out to you what is the anomaly yay you get a chocolate cool so

Anomaly Detection using Autoencoders


fantastic but now but now if we think about this right imagine in a real-life situation when we we're not only just looking at six rows now we're looking at ten million rows and we want to cater for real-time situations in real time are we able to identify the anomalies in the data set and not only what we just have password change occurrence as a variable we'll have millions more so cool that's where anomaly detection using water encoders can play a role so now we move into the Cagle data set apologies for spelling we are data scientist not

Anomaly Detection using Autoencoders


English teachers cool right so the keidel data set is just the data set which I'm sure you're all quite familiar with it's called the credit card data set and it's based on transactions of customers so as we begin you can see that this data set is highly imbalanced as you can see there are very few fraudulent cases which make up 0.17% of the data set now for machine learning algorithm to learn such a thing it makes it really difficult so cool but we'll explain how to combat that later on and then we read in our normal import so

Anomaly Detection using Autoencoders


because we were speaking to h2o we'll be using the h2o deep learning estimator library we read in our normal packages we then begin by initiating your spa context and your h2o context followed by now this is where the fun begins right we read in our data set using spark we transform that spark data set into h2o because remember we're working with h2o models right now you can't pass in a spark data frame into an h2o library so hence you need to convert it then over here we defined our features list now because this is an online data set it's

Anomaly Detection using Autoencoders


anonymized but in real case situations these features could represent things like maybe the number of times you've was drawn from an ATM is your card linked to the app how like how often are you in overdraft you know just like those kind of features then you take your data set and you split it into a chain test and then remember before we were showing or showing you how the data itself was highly imbalanced so in order to combat that right you train the model on what looks like normal what is considered normal you chain that so that

Anomaly Detection using Autoencoders


the model learns so that when it's given unseen data and it picks up patterns that don't follow what it learned then it will flag that as an anomaly cool so now we begin with defining our h2o deep learning estimator we pass it through a variety of parameters I'll just go through a few so the one being the model ID which is purely just the name of your auto encoder so when you do save it for reuse of later on you can reference it an activation function of ten and a few hidden layers then you chain your model over here now you savor your model cool

Anomaly Detection using Autoencoders


so now that we saved our model we want to reload it now that we've reloaded the model this is where the fun begins this is where we actually identify anomalous behavior so we apply it to a testing set and we produce these reconstruction errors now if you remember these reconstruction errors are how different is the output from the input so as you can see equal this is the overall reconstruction error but now what if you are interested in identifying the reconstruction errors per feature we can view that over here

Anomaly Detection using Autoencoders


so this over here will show you the reconstruction error per feature so it's just to show you and give you a sense of which feature contributed more to a particular observation for a customer yeah and then if you are interested you know after this presentation you can go home and you can build your own auto-encoders you can visit the Kegel comm website and you can get this data set and so just a recap right of reconstruction errors in terms of like a real-life situation with this image right the input data would be your

Anomaly Detection using Autoencoders


pixels and then the output would be the reconstruction errors without the noise cool so I hope that clarifies reconstruction errors all right so now we're going to get into showing you the trick sense dashboard that we've built that we as data scientists and business stakeholders may be interested in might be a bit cheeky with tricking and holding the mic so as you can see within this dashboard we chose what you see in over here is what's down then red is the normal pattern that the algorithm caught and what we have above in blue is the

Anomaly Detection using Autoencoders


anomalies so how we picked the anomalies is was we we picked a threshold of 0.01 and we picked that threshold just based on what we saw from the particular diagram so what the amounts of anomalies we caught is a hundred and five anomalies right so let's say I want to check as a data scientist that okay how much was actual fraud and how much did my prediction was one as you can see or if I close predictions as you can see that I my anomaly detection model picked up 69 fraudulent cases out of 89 if I want to check as

Anomaly Detection using Autoencoders


the data scientists that how many fraud cases that my predictor not get so out of all the fraudulent cases it didn't pick up 20 fraudulent cases the anomaly detection model so it's not a bad model it's pretty it's pretty neat for a fraud project well as you can see here we have 0.16 percent of fraudulent cases and more there's more anomalies that we found which goes to the results that we have what you see here is actually the reconstruction errors that Angela described for each and every variable so from fraud analysts point of view that

Anomaly Detection using Autoencoders


first one might be number of possible changes our example our initial example so the fraud analyst would see that okay these are the variables that are most impactful for different types of fraud if the if the fraud analyst wants to see for a particular customer customer one seven zero five they experience fraud we picked up that fraud they would see that these are the variables that actually impacts at that particular customer what we added to the dataset Angela added we added places we just added that randomly

Anomaly Detection using Autoencoders


but usually with a project you want to know maybe a particular area that is experiencing fraud and by how much and see what variables are impacting there so that you're able to contact the customer and help them accordingly okay so just to share with you some key takeaways that we have from building this models at scale the first thing is the interpretability so we showed you how to interpret for the particular for this particular model and what does happen that if you have quite a lot of features it can be difficult for a fraud

Anomaly Detection using Autoencoders


analyst or whoever the business stakeholder is to be able to interpret why is this particular case fraud so that's a common problem that we have another problem is that and maybe this is a general machine learning problem is that if there isn't an underlying pattern in your data then the or think ah don't do anything for it so if that's the case you could think about building more features that may assist you to get a particular pattern then when it comes to maintainability when we build it at scale a big reason why we chose h2o

Anomaly Detection using Autoencoders


because we build models at scale with that so if you want to build it you can use the Cagle data set but we also chose it because of that then just the difference with k-means and an autoencoder using an anomaly detection problem so we have used k-means before and you'd find the distance between the chester centroid and the observation does show anomalies but maintaining that code so when you have to retrain your model with new data your cluster sense has change your clusters change inevitably what you are trying to find

Anomaly Detection using Autoencoders


in anomalies to change but with an autoencoder it is much more consistent then with a threshold this goes to capacity so you saw we detected 105 anomalies when you are working at scale with much more data it might be 10,000 anomalies now sending 10,000 anomalies to an actual business they call that to work through might be a bit difficult so they might not have the capacity to do that so picking a threshold usually you have to work with the business area to understand what threshold is been suited for them and then lastly a

Anomaly Detection using Autoencoders


feedback loop we would want overtime to know that what we pick up as anomalies was actual fraudulent behavior and sometimes getting that feedback loop is difficulty so just to really like sum it up right so concerned parent if all your friends jumped off a bridge would you follow them machine learning algorithm yes so basically all in all just want to sum it up to what we spoke about today just because a model may say that something is anomalous at the end of the day you also need to check does it make sense to the use case now that make

Anomaly Detection using Autoencoders


sense to the stakeholders I don't just listen to the machine learning algorithm there's still more to it you still need to bring in like you have the human side and to understand that it makes sense with business say yeah I mean thank you so much for listening thanks ladies presentation was great with regards to the Reconstruction era have you experienced any cases where you've got like really high variances in the range of your reconstruction errors and then like if you have what approaches have you taken to like scale those or have

Anomaly Detection using Autoencoders


you worked with them just as you know so let's say you've got a reconstruction error on 0.05 on one observation and then you got on another observation a reconstruction error of I don't know like a hundred right so in that case like have you experienced that and if you have like have you dealt with any sort of like normalization of step or standardization reconstruction era the one that is higher to us we looking for that it's noise it's the anomaly that we're trying to find but so yeah we haven't we haven't dealt with that yeah even with

Anomaly Detection using Autoencoders


the following is a conversation with jurgen schmidhuber he's the co-director of a CSA a lab and a co-creator of long short term memory networks LS TMS are used in billions of devices today for speech recognition translation and much more over 30 years he has proposed a lot of interesting out-of-the-box ideas a meta learning adversarial networks computer vision and even a formal theory of quote creativity curiosity and fun this conversation is part of the MIT course and artificial general intelligence and the artificial

Juergen Schmidhuber: Godel Machines, Meta-Learning, and LSTMs | Lex Fridman Podcast #11


intelligence podcast if you enjoy it subscribe on youtube itunes or simply connect with me on twitter at Lex Friedman spelled Fri D and now here's my conversation with jurgen schmidhuber early on you dreamed of AI systems that self-improve recursively when was that dream born when I was a baby no it's not true I mean it was a teenager and what was the catalyst for that birth what was the thing that first inspired you when I was a boy I'm I was thinking about what to do in my life and then I thought the most exciting thing