[Rant] coronavirus

14 and they're going to transmit that to two three four five people after these


[Rant] coronavirus

measures everyone's going to be longing for social contacts and large groups and


[Rant] coronavirus

we might gradually loosen the restrictions but still a new outbreak is


[Rant] coronavirus

inevitable it seems so what you'll have again is a spike and then the country


[Rant] coronavirus

might enact measures again and so on but I believe the world we're going to live


[Rant] coronavirus

in if we really lock down people if we really enforce these measures is a world


[Rant] coronavirus

of multiple repeated seasonal peaks of this disease and that means we are in


[Rant] coronavirus

for the long term I don't I don't want to say ever that we shouldn't do that


[Rant] coronavirus

because it of course effectively reduces the number of deaths which should be our


[Rant] coronavirus

ultimate goal here but just know that flattening the curve once like these


[Rant] coronavirus

graphics here is a bit misleading I believe we need to be thinking about a


[Rant] coronavirus

long-term plan here and since we're going long term and with long term I


[Rant] coronavirus

mean months I mean multiple years with long term the problem here is the people


[Rant] coronavirus

and I want to elaborate on that so the largest problem are the people people


[Rant] coronavirus

aren't just machines that you can command around people are individuals


[Rant] coronavirus

they have their own ideas they have their own goals that they want to


[Rant] coronavirus

fulfill right at some point I want to go on a vacation this is an island with a


[Rant] coronavirus

tree so let's talk about lockdown lockdown it appears to be a thing that


[Rant] coronavirus

is necessary in some parts if you ask some people again I don't want to give


[Rant] coronavirus

advice on this I just want to give some thoughts so what do you get with lockdown


[Rant] coronavirus

with lockdown you get OMG it's happening and so that's day one day three you get


[Rant] coronavirus

funny YouTube videos everyone that is in lockdown will do be like oh I'm stuck at


[Rant] coronavirus

home it's so boring already forgetting that other people have major issues with


[Rant] coronavirus

being locked down a lot of people sitting on top of each other is going to


[Rant] coronavirus

create a lot of problems and eventually more and more people are going to long


[Rant] coronavirus

for this to end and I'm not saying that you know that is that response to a


[Rant] coronavirus

virus should be fun but what I'm saying is that people are going to break this


[Rant] coronavirus

it is inevitable first some are going to break then more you have a very delicate


[Rant] coronavirus

balance going on here right now there's a lot of support a lot of people are


[Rant] coronavirus

on the side of locking things down in a lockdown a lot of people are


[Rant] coronavirus

conscientious staying home avoiding social contact as much as possible but


[Rant] coronavirus

some are going to be the first ones to go over there some are going to break


[Rant] coronavirus

some are going to find excuses not to keep to it and the problem is the harder


[Rant] coronavirus

the measures are the harder you are down here the stronger the pole is going to


[Rant] coronavirus

be for people to go on this other side and I guarantee you the people on social


[Rant] coronavirus

media that are shaming others the most that are yelling out the loudest for


[Rant] coronavirus

others to not break the lockdown either they have an extremely comfortable


[Rant] coronavirus

living at their own homes which is an extreme privilege or they are the worst


[Rant] coronavirus

ones to break it to themselves to find every excuse they can why they are


[Rant] coronavirus

exempt from it and people are going to see this more and more people are going


[Rant] coronavirus

to be over here and with more people over here look they have the sunshine


[Rant] coronavirus

they're out and about they're doing their things more like normal the people


[Rant] coronavirus

over here they're going to see this and more and more people will be hey why am


[Rant] coronavirus

I keeping to this why am I not over there why can't these people do that and


[Rant] coronavirus

they'll go and at some point the scale is going to tip and any lockdown barring


[Rant] coronavirus

martial law and the threat of being shot if you go outside will be ineffective


[Rant] coronavirus

and at that point wherever you are the cases are going to spike and it will be


[Rant] coronavirus

even worse than when you did nothing or as bad so I believe that is very


[Rant] coronavirus

delicate balance that you have to strike here total lockdown people aren't going


[Rant] coronavirus

to take this for a long time and you need to think about a long time here I


[Rant] coronavirus

don't know what the answer is I don't know where exactly the scale of just


[Rant] coronavirus

keep apart to stay home whatever it takes is I just think that too harsh


[Rant] coronavirus

measures can also be counterproductive I'm very fortunate to live in Switzerland


[Rant] coronavirus

most of our neighbors have instituted total lockdowns and the Swiss


[Rant] coronavirus

government has recently decided not to do so at this time with I believe much


[Rant] coronavirus

of the same reasoning as I'm just laying out we need to think about this long term


[Rant] coronavirus

and people are not going to keep to a lockdown long term and it'll be worse if


[Rant] coronavirus

they don't now I believe the best response to something like this is a


[Rant] coronavirus

distributed one I believe the best response is to go to people and their


[Rant] coronavirus

networks people usually care about people around them enough so that they


[Rant] coronavirus

will take responsibility into the hand I believe you should and I believe you


[Rant] coronavirus

should give the people the responsibility as much responsibility as


[Rant] coronavirus

you can and I believe the network of people each one arranging themselves in


[Rant] coronavirus

the most pro-social way can be the best response better than any government


[Rant] coronavirus

could do governments can do things such as prohibit large gatherings you know


[Rant] coronavirus

sometimes if you don't do that even the individual people can't do anything


[Rant] coronavirus

against that but to actually believe in your citizens and believe in the


[Rant] coronavirus

fundamental goodness of humans and the fundamental care for other humans is a


[Rant] coronavirus

strong suit here on the other hand you see other governments I have read that a


[Rant] coronavirus

city in Norway is thinking about employing a monitoring systems where


[Rant] coronavirus

they track everyone's phone and if more than a certain amount of people are in


[Rant] coronavirus

the same place they will basically send everyone a text message saying you


[Rant] coronavirus

should disperse while this is an effective measure and I believe can


[Rant] coronavirus

definitely help and it is something that you need to be very careful about as we


[Rant] coronavirus

saw with 9-11 as soon as governments get power they rarely let it go as Edward


[Rant] coronavirus

Snowden finally demonstrated if you enact something like this you must


[Rant] coronavirus

definitely make sure that there is a time limit on it any government measure


[Rant] coronavirus

right now be that spending to help the economy which is certainly a good thing


[Rant] coronavirus

be this measures to increase social distancing to prohibit public gatherings


[Rant] coronavirus

I support this but it must be time-limited otherwise governments


[Rant] coronavirus

aren't gonna let this go finally I would like to come to a more global scale of


[Rant] coronavirus

long-term thinking countries and other countries as you go on you need to think


[Rant] coronavirus

about your economy our economies were growing a fairly good pace until this


[Rant] coronavirus

hit and now they're plunging at any point they're going to be opportunists


[Rant] coronavirus

they're going to be personal opportunists hoarding toilet paper and


[Rant] coronavirus

hand sanitizer and trying to sell them for marked up prices and they're going


[Rant] coronavirus

to be country opportunists and everything's falling down if you're the


[Rant] coronavirus

country that locks things down now your economy is going to fall eventually


[Rant] coronavirus

though you'll have to get back the countries that get back sooner will be


[Rant] coronavirus

in an upswing sooner basically the question is where is the ideal point


[Rant] coronavirus

here to leave the to not react anymore to let people do their thing to get back


[Rant] coronavirus

on track I don't know where that is but I believe you're going to see a Cold War


[Rant] coronavirus

like situation in the world where countries are going to accuse other


[Rant] coronavirus

countries of not doing enough or doing too much of not playing fairly of


[Rant] coronavirus

helping to spread the virus and I believe that it will be the case for the


[Rant] coronavirus

years to come because what happens over the long time of course right now you


[Rant] coronavirus

can afford to not fix that pipe under your house that's broken you can afford


[Rant] coronavirus

to not clean the to not get the person to clean the chimney you can afford to


[Rant] coronavirus

not get dental work done I don't even know how to draw a tooth let's say this


[Rant] coronavirus

is a tooth it probably has some peaks here over the long term though all of


[Rant] coronavirus

these things are going to break and we need to get back to normal and the


[Rant] coronavirus

longer a state keeps up these measures the worse it's going to get finally we


[Rant] coronavirus

need to talk about risk people people at risk tend to be older tend to be ones


[Rant] coronavirus

with health issues think about this if you're an old person having health


[Rant] coronavirus

issues you're looking at a long term once you realize this is not going to be


[Rant] coronavirus

over in a few weeks what do you do you're old and the next year or so in


[Rant] coronavirus

lockdown mode it's going to be hard for you and for everyone but a year if


[Rant] coronavirus

you're that old and sick is probably more quality life you have left then


[Rant] coronavirus

after it so you need to be thinking either I'm going to survive this because


[Rant] coronavirus

I bunker in my house don't get the virus but what is it worth because my other


[Rant] coronavirus

diseases will get me afterwards otherwise I could be spending the


[Rant] coronavirus

quality time I have with my family with my children with my grandchildren could


[Rant] coronavirus

be spending it with my friends and if I die I die it is not an easy question but


[Rant] coronavirus

I'm absolutely sure there are people right now who are asking themselves this


[Rant] coronavirus

if you're a government and you're thinking about mandatory lockdowns I I


[Rant] coronavirus

do see that this is in order to save people in order to not have people


[Rant] coronavirus

walking around that spread the virus to vulnerable populations but you need to


[Rant] coronavirus

be thinking about the people you're trying to help some of them would


[Rant] coronavirus

actually be on this side I don't know what the best response is to everything


[Rant] coronavirus

here I think we're just going to see and I don't want to give advice this is just


[Rant] coronavirus

some of the things I think I wish everyone the absolute healthiest season


[Rant] coronavirus

they can have right now take care please think about others please do not make


[Rant] coronavirus

the problem worse yourself you're part of a network and you can be a powerful


[Rant] coronavirus

force for good during this time think about long term if you're asking your


[Rant] coronavirus

government to do things think about what's the best situation and how we


[Rant] coronavirus


Adversarial Examples Are Not Bugs, They Are Features

Hi there, today we're looking at Adversarial Examples are not bugs, they are features by


Andrew Ilias et al.


So this paper is pretty interesting, it has a catchy title, and we'll try to kind of dissect


what it says.


So first of all, in the abstract they say Adversarial Examples have attracted significant


attention but the reasons for their existence and pervasiveness remain unclear.


So if you don't know what an Adversarial Example is, an Adversarial Example is basically the


following.


Say you have an image classifier, right, classifier, boom, neural network image here, and the image


is of a, let's say a cat, right, this is my best attempt at a cat, bang, cat.


And you feed it through the classifier and the classifier says cat.


Now if you perturb this image, if you derive an image from it, and you perturb it just


very slightly, very subtly, so you introduce some pixels here, there, here, there, right,


you change some pixels in a very targeted way, and you feed that new image through here,


then the classifier will say dog or something really, you can make it say anything like


airplane or I don't know, sky or whatever you want.


So these are called Adversarial Examples.


And it's true their existence and the reasons for their existence and pervasiveness remain


unclear.


They say we demonstrate that Adversarial Examples can be directly attributed to the presence


of non robust features.


So they're basically, their paper is about these non robust features and they define


later what they mean exactly.


But here they say features derived from patterns in the data distribution that are highly predictive,


yet brittle and incomprehensible to humans.


And this is this pretty neat.


So the fundamental idea, as I understand it, and I'm going to, you know, take this away


right here, that if you have images, let's say here of cats, and I'm gonna, I'm gonna


draw another one over here.


If you have an image, say of cats, there is multiple features in this image.


And the feature is something that the classifier can pick up on and kind of learn to, this


is a horrible cat, learn to learn to classify images from.


So features that we humans generally use are a cat has ear, ear, eyes, whiskers, right,


and the general relationship to each other of these things.


This is what constitutes a cat.


And that's how we classify.


But also, they say there are other features that are also very indicative, right?


If you think what differentiates a cat from a dog, and the dog here, let's pick fluffy


ears.


Also eyes, yeah, not going to go further with the dog too much.


The what differentiates cat from a dog and we of course, we would say, well, the head


shape is different, and the ears are different than the relationship to them to each other


are different.


It could also be and this is a simplistic right now, right?


But it's also that cats, for example, have different fur than dogs.


And yeah, being overly simplistic here, but bear with me.


So let's say in our hypothetical world, cats have fur that goes like this left to right,


right?


Every hair is basically vertical, sorry, horizontal.


If you look at it like that, and dog fur, on the other hand is always like this, right?


Is is vertical, right?


Top to bottom.


And so the the classifier might just as well pick up on the fur direction in order to classify


images, right?


Since all cats have that type of fur and all dogs have that other type of fur, that classifier


might just as well pick up on that, right?


And to us humans, we don't really pay attention to these things because they're minute, right?


You don't look at the directions of the individual hairs to classify in an animal to cat or dog.


You you much rather go for these kind of large features like where the ears, how do they


look and so on.


But a classifier there's actually you can make an argument that the classifier would


more likely pick up on the fur direction, right?


In order to in order to classify since we're using convolutional neural networks and they're


generally neighborhood pixel neighborhood operators, it can much easier pick up on these


patterns than it can on the general relationship of the of the large features.


So if a classifier now learns that cats always have fur like this, and dogs always have fur


like that, what we could do is we can go over here to the dog and change its fur, right?


In the image change its fur to this direction.


Now to us humans that would still very much look like a dog because the fur direction


is almost imperceptible.


But to the classifier that has only learned, hey, a cat always has this type of fur and


the dog always has that type of fur, that new image would totally look like a cat.


Right?


So this paper argues exactly that this paper argues that in the data set, there are features


and these are real features like this, this actually could be the case that cats fur is


all always like that and dogs fur is always like this.


It could be the case and the classifier could pick up on this, right?


And then the adversarial examples, the reason why they exist is because the classifier has


picked up on these imperceptible features.


And so by changing the features, we can change the classifier's decision and without changing


the image in a large scale.


So they say that they make this hypothesis and they kind of, they say, okay, we established


a widespread existence in standard data sets.


So they kind of give supporting evidence for their hypothesis.


And then they say, finally, we present a simple setting, which is a theoretical setting where


we can rigorously tie the phenomena we observe to a misalignment between the human specified


notion of robustness and the inherent geometry of the data.


So it's kind of different pieces of the, of this paper and we're going to look at them


in succession.


So the introduction, we largely skip except that their main claim here is specifically


we claim that adversarial vulnerability is a direct result of our models sensitivity


to well generalizing features in the data.


So that's the core point I think is well generalizing features, which is what we mentioned.


These are features that actually describe the data well, but features that are kind


of imperceptibly small to humans or don't fit our notion of robustness.


All right, so they go on and they define more clearly what they mean.


Here whenever we talk of a feature, right, remember we had the, our classifier here,


then we input an image and the image is called X, right?


And that classifier usually if we look at it closer consists of multiple layers of interconnected


neurons, whatever.


And the last layer will be an output layer into different classes.


Right?


And so the features, when they say a feature, what they mean specifically is the last here,


the last represent representation before it goes into the classifier.


So the way you would classify then, and here they just establish a two class setting.


The way you would establish that is you have feature one, feature two, feature three, and


you have a weight vector w1 for each feature w2, w3, you make the inner product and that


will give you a y hat basically.


If that is high, you say it's class one.


If that is low, you say it's class minus one.


So the classes here are plus one and minus one, just to make things simple.


So but you see the features are basically what comes out after these layers, what is


then used to make a linear classification.


This last thing is basically just a logistic regression.


So you can think of the features as the output of the neural network, but before it goes


into the classifier.


So a feature, basically, since then it's linearly classified, if the feature is high, it will


give a signal for one class.


And if a feature is low, it will give a signal for the other class, depending on of course,


if this w is negative or positive.


Alright, so they say we call a feature row useful.


And if this thing holds here, and what is this thing?


This thing means so the expectation over the dates, so generally in the data set, this


must hold y times the feature.


So y is the class and remember, it's plus or minus one.


And the feature, as we've seen is some some number, y times a feature must be higher than


some some number.


So what does it mean when a product is high, it means either both are high or both are


low.


So they're correlated, that's what that means.


So basically, this is says a feature f is useful, if whenever it at an example x is


of class one, if it's class, class one, or let's if it if y is one plus one, then f is


high.


And whenever y is minus one, then f is low, which means it's high in the negative direction.


Right?


So if this is our, this is intuitive, right?


If a feature is useful, it means it should say one thing in samples of class one, and


it should say another thing in samples of class two, then I can actually use the feature


to make a decision when it's, you know, very correlated with the class.


So that, you know, that makes perfect sense.


So that's kind of when is a feature useful if it correlates with the class label?


Yes.


Cool.


But the usefulness, simply any feature basically that a classifier will extract will be useful.


That's an assumption we can make otherwise the classifier wouldn't extract it.


So the neural network here, that's an assumption will only extract useful features.


Right, because the non useful features, there would simply be no reason for it to extract


them, because they don't contribute to solving the task because they're not correlated with


an output class.


Right.


So next, they define robust, robustly useful features.


So in addition to being useful, they're now also robust.


What does it mean?


Again, we want a correlation of y and the feature to be higher than some constant.


But not only the feature of the image x, but the feature of the image x that has been perturbed


by a small perturbation.


So and we take the the infimum here over a class of perturbations.


Of course, this class of perturbations is exactly the adversarial perturbations.


Basically what this means is it says that however we try to perturb x, right, and the


infinite mirror means that the minimum correlation, however, we try to make the feature not correlated


with y, however much we try, we can't get it lower than some some gamma, some number,


right, we can't we can't get it down.


So whatever we try to make the feature bad for the classifier, basically, we can't.


If this holds for a feature, if this is the case, then we call that feature a robust feature,


right?


Then that feature is robustly useful.


If it correlates, no matter how hard we try to make it not correlate.


And of course, a non robust features, so a useful non robust feature is a feature which


is useful, you see here is useful, but is not gamma robust feature for any gamma.


So it is a feature that is useful like the cat fur, right?


So this year, an example of this would be that the cat's eyes and ear position, right?


We can't just make a small perturbation for the image and make the ears be somewhere completely


else.


That's just that would require a large perturbation of the image.


So the position of the ears and eyes are pretty robust features.


But here the cat's fur, no matter how, no matter how small we we make this this gamma,


we can always kind of change the fur to make the feature not robust, to make the feature


not useful, right?


If we can change the cat fur into a dog fur and the dog fur into a cat fur, then the feature


will become not useful anymore.


Because we can, you know, we can we can change that arbitrarily for any image and then the


classifier will have no clue.


It can't be like, well, this fur could be of any of any class, right?


So the feature is not useful anymore.


So this is a non robust feature.


The technique you can say any feature that is useful but not robust is a non robust feature.


Alright, so this is kind of the definition of what robust and non robust features are.


Yeah, remember, maybe remember robust features like position of the ears and their shape.


And non robust features would be which direction are the individual hairs in the fur going,


right?


In in our world where cat fur is going different ways than dog fur.


So they now go into experimental evidence for their for their hypothesis.


And here you have to understand they do two experiments, which give pretty good indication


that their hypothesis is actually correct.


And what you have to understand before this is, is two things.


First of all, here you basically you just have to assume that they already they have


some procedure where they can do the following where they can take an image of the training


data set, and they can decompose it into its robust and non robust features, right?


Don't, I mean, don't ask yet how they do this, but they can decompose it into these two parts,


So that's assumption one, they have a procedure that can actually do that.


And then number two is what they what they do here is basically the general theme of


these experiments is they they have a training data set, right?


This is the original training is they create a derived version of it.


So let's put a tick here.


This is a derived version of the data set.


And they train a regular neural network with that.


So what you can do with a neural network if you train one, all right, what you usually


do is you feed images, x, you feed images in, it gives you some output y hat and you


say, well, but I know y is the true label.


So I feed an image of a cat that the network says airplane, you say, well, but this should


be a cat.


So please make this y more to be more to be.


Please make this y hat more be like y and then you have a loss function here you say


this is wrong, please correct this, you back propagate and all the network in here will


update to make that a bit more likely.


That's how you train usually a neural network.


Now what you can do is if you want to become robust adversarial examples, you can do what


is called adversarial training, which means that you have the same network here.


But of each of the training data points, you create a derived version, an adversarial example


to that to this x, you feed the adversarial examples through the network together with


the original examples, then this will give you some y hat two.


And then you say, but this should also be equal to y.


Basically, you train the classifier also on adversarial examples, right?


Since the hypothesis is if you train on an image data set, then you can teach the classifier


about that data set, right?


Like you do with the regular data set, say, well, okay, I can now just train on adversarial


examples.


And my classifier will be able to better classify these correctly, right?


This usually works.


It's called adversarial training.


And it's been a kind of a standard method to make your classifier robust.


They don't do that here.


They don't do this, they simply want to say, okay, we now have, we have a regular training


procedure, right, like this, except for what we change is here, the training data set,


we change this to in one case, for example, only robust images.


So we've changed all the x to be only robust.


And we do the regular training procedure.


And then we evaluate that resulting classifier here, this thing, we evaluate that, how does


that behave?


So it's kind of a new approach where you modify the original data set.


So what did they do?


First of all, they decompose this training data set into a version that is only robust


features, right?


We assume we have such a procedure, we then train a regular neural network on that, right?


We train a regular neural network on this, on this data set.


And what we get is two things.


First of all, good standard accuracy.


What does good standard accuracy mean?


It means that we, we can test it on what's called the unmodified test set.


So the test set, the original test set of the data set, the test set belonging to this


training data set, we can test it on that.


And it works just fine, right?


So that basically means that the robust features are predictive of the, of the kind of, they


generalize well.


It means that if I train a classifier only on robust features, that can actually classify


well to, to the, to the test set, right?


So that means that's standard accuracy.


Standard accuracy is how well do I classify the test set.


Just a non-modified test set.


So they also obtain good robust accuracy, which means that what is robust accuracy?


Robust accuracy means your accuracy on adversarial examples of the test set.


And usually classifiers are vulnerable to this.


So classifiers usually obtain good standard accuracy, but bad robust accuracy.


But if I only train my classifier on what they call robust features, then I all of a


sudden retain good standard accuracy, but I also get good robust accuracy.


Which means that it gives pretty good support to their hypothesis that the adversarial examples


are abusing the fact that the classifiers learn the non-robust features.


Which means if I don't have any non-robust features, it means my classifier can't learn


any non-robust features, which in turn means my classifier isn't vulnerable to adversarial


attacks because they would abuse the fact that the classifier has learned about the


non-robust features.


So that's pretty good evidence for their hypothesis.


Second thing they do is they now create this on this modified data set where they only


have non-robust features.


Right, so the only thing they have is non-robust features.


Again, they train a standard neural network, they train just a regular neural network on


that.


And they also get good standard accuracy.


So this means that also the non-robust features as we seen like the cat's fur direction, can


lead to you generalize well to the test set since in the test set also the cats will have


that property.


But you get bad robust accuracy.


And this gives further support to their hypothesis if you train a classifier on only non-robust


features.


They are features because they generalize well, but they are very vulnerable because


they're non-robust.


Right?


So the classifier that has learned about non-robust features is vulnerable.


They didn't do a third experiment, which I find pretty cool, where they take they take


the training image.


And of course, it's an unmodified training image.


So it's robust features will basically say this is a dog.


It's non-robust features will also say this is a dog because it you know, it's a it's


a training image of a dog.


And what they then do is they derive from this dog an adversarial example towards the


cat class.


So what does it mean in their hypothesis if their hypothesis is correct, it now means


that the robust features still say it's a dog.


We can also see this here, right?


The the kind of big shape of the image still is a dog to us humans.


But the non-robust features will say it's a cat, right?


This hinges on their hypothesis that adversarial examples actually abuse the non-robust features,


right?


They create an adversarial example.


So if their hypothesis is correct, the non-robust features now say that's a cat.


So they derive a an entire data set where they change every image to another image,


and they also change the labels accordingly.


And then they train again, a regular neural network on this.


And they look what happens on the unmodified test set.


So the unmodified test set will so imagine if you're the you're this classifier, and


what you get is an image X, and it has robust features that say dog and has non-robust features


say cat and its label, you're asked to predict cat, right?


And then we see the next image and the next image X2, the non-robust features, maybe it's


derived from some other class, it will say plane.


But the robust, the non-robust features again say cat, right?


And you're asked to predict cat.


So basically, the constructed data set where the non-robust features always agree with


with the label, but the robust features, they don't.


So naturally, what you can expect is the classifier will learn to disregard the robust features


because they're no longer useful, right?


But it will actually own it will learn to view these features.


It's different from before before we only had these features.


Now we these features are still in there, right?


But they're not informative.


So the classifier will naturally learn to pick up on the non-robust features and classify


and classify according to them so much that if we now test on the test set, and we feed


in an actual cat, right?


It of course, it's robust features will say cat and it's non-robust features will say


cat and the classifier is able to accurately predict this is a cat even though the all


the images of cats it has seen during training were actually of basically of non-cats of


here a dog.


But so this is pretty cool and shows that kind of these these features that these non-robust


features that adversarial examples abuse since they're created by adversarial examples.


They they are actually predictive and generalize to the test set.


So that's pretty pretty good evidence for their hypothesis so far.


Now the kind of final remaining question is how do they create?


What is the procedure where they can create a robust and then basically non robust version


of the data set?


And here is kind of where we get into the into the sort of what I find.


Yeah, so here you see basically examples of so this is an original image of a ship in


the CIFAR 10 data set, I believe.


And this is a robust sample.


So these are only robust features of the ship.


And this is a ship made with only non robust features you see is actually a moose.


But the non robust features have been changed to ship.


All right.


So the way they construct a robust version of the data set, they have a formal definition,


but the way they do it is as follows.


So and then they say, okay, here is where we where we get into the details.


They say, imagine we have a classifier.


Right?


The classifier outputs features and here we call them here they call them G, which is


the representative it can be larger than features, it can be a bigger class.


But in essence, G is the features, which then goes into the into the classifier and into


the labels and so on.


So the neural network outputs the features inputs some x.


Now what if what if I have another x?


Let's say x prime, and I just initialize this with random noise.


And if I feed this and I get G prime here, and I try to make the two as close as possible


by changing x.


So I'm going to change my x here.


Basically I'm going to change my image such that the outputs the features here match each


other as close as possible.


What does it mean?


And I do this via back propagation, right?


I match these and I back propagate to x, I can do that with gradient descent.


What happens is that my image x will basically pick up will match the image my x prime will


match the x in all the ways that are relevant for the features.


Basically I will transfer all of the features from x to x prime, but nothing else right


since I start with random.


Now what if my classifier and that's what they do?


What if the classifier is a robust classifier?


So remember, we talked about we can actually robustify a classifier by doing adversarial


training.


What if I have a classifier like such that is robust?


If I input an x and it outputs me a feature representation of x, if the classifier is


robust, that representation will only contain robust features.


And then if I have a second image x, or and I started from random noise, and I match the


representation of x and by by changing x, or basically I will transfer all of the robust


features from x, but nothing else right?


Given that I start from random noise here, this means random noise has no features.


Let me that's the assumption random noise has no features since it's random noise.


And if I transfer only the robust features, basically what I've done is I've I've have


now an image that I know has no non robust features, and only robust features of x.


So that's how they derive a how they derive a robustified version of x.


Second, how do they derive a non robust version?


And that's even even easier if I have a classifier, right, a regular classifier, and I want a


non robust version of x, have x input output, g output some label.


What I do is I simply derive an adversarial example of x, like we did before adversarial


example in here, out here.


And that gives me some x y two, which is different from y, right?


If I have a adversarial example, then basically, I've transferred I've transferred the non


robust features that lead to class y two, I've transferred the non robust features here,


while still maintaining the robust features from here.


So if this is too abstract, imagine here x is an image of a dog, right?


Dog.


And I derive from it an adversarial image that now says airplane, right?


So the robust features will still be of a dog will still be of the original image, but


the non robust features will be of the airplane class.


Alright, so that's how I derive a non robust, non robust version that has features of kind


of one robust features of one class, but non robust features of the other class.


That's what you see up here with the moose, right?


The moose clearly has been started from the image of the moose and then has been has received


non robust features from the ship class.


And that's just your classic adversarial example procedure.


So that's the that's the kind of procedure.


And so what's kind of my criticism here, if you look at the first part, the first part


where they say, well, in order to determine what the robust features are, we actually


need a classifier that's already robust.


So we've seen before we have a we have a data set, sorry, let's go up here.


They say, aha, here we have a data set, right?


And we can disentangle this and then it will which color have we not used, we have a data


set, we only we robustify the data set to a robust data set, we train a standard neural


network.


And that gives us good robust accuracy, why, which is really cool, because we don't do


anything special during training, and we still get good robust accuracy.


But in order to do this procedure here, this one, you actually have to have a robust classifier,


right, you have to have this already robustified classifier, which you have obtained by adversarially


training, the robust classifier, basically, what you're doing now is you take this adversarial


training procedure, which the point here is that you don't do anything different during


training, right.


But here you take the adversarial training procedure, and via training the robust classifier


and via changing this data set here, you basically get good robust accuracy, which to me, is


just a reflection that you've obtained the data set using this robust classifier in the


first place.


I mean, yeah, of course, their method gives a hint that I can actually, this is actually


due to things in the data set themselves, right.


But they're, I mean, that's really important, because it surely means that it's not a point


of, let's say, the classifier itself, but it's a point of the data set, which also say,


it also explains why these adversarial examples transfer between classifiers, if I have two


classifiers that are different, but classify the same thing, they're vulnerable to the


same adversarial example, which basically means it must be some property of the data


set that these things learn.


But to then say, we have a procedure to extract the robust features, and if we only train


on the robust features, we become robust, right, as here.


But you obtain the robust features by using a robustified classifier, which you have adversarially


trained.


To me, that's kind of backdooring adversarial training into this whole procedure.


And yeah, so that's kind of my first criticism.


My second criticism is the fact that, I mean, it's an interesting take on this, but this


whole notion, this whole seeing of these features are robust, these features are non-robust,


is basically just reframing the problem of adversarial examples in terms of features.


It says nothing why these features are there, except simply postulating that they're there.


It says nothing why they're there.


It says nothing about why the classifiers pick up on them or how they do it or how this


is to be mitigated without first having a robustly trained network to extract the robust


features.


So it basically just shoves the problem into the realm of the dataset rather than from


the realm of the classifiers.


So it's very much widely, or not, things are very much widely not known about these samples.


It's just a reframing of the problem, I feel.


And it's cool experiments.


I mean, it does show a lot of things about these adversarial examples, but it's certainly


not an explanation, I find.


At least that's my opinion.


All right.


So down here then, they show that they make a kind of simplified version of this, a theoretical


setting where they can analyze this.


And they basically say, okay, this is generally what happens at the fundamental level.


At the fundamental level, you have classes, and let's say the classes are distributed


like this, right?


These are the examples in the dataset and they're distributed like that, right?


You have mean and you have some covariance.


These are Gaussians here.


So they're distributed like that.


If I have two classes like this, such as here, right, and they're distributed like that,


and I create like the separator, the linear classifier, the linear classifier will classify


like this, it will be like super, this is the best linear classifier, right?


We can calculate this accurately.


But what do I say when I say, okay, I want an adversarial example?


Adversarial examples means that I can shift my examples by a little bit, but achieve a


big change in output.


And since this distance here, right, so if I have a sample here, I need to go a long


way to the boundary to achieve another output.


But if I go into another direction, right, if I go down here, I only need to go a very


short way.


And since adversarial examples, as they're specified, they say, okay, we want to go a


short way and the short way is characterized by going a short way in any direction, right?


This is a terrible circle.


In any direction, we want to go a short way.


That's an adversarial example.


And you see that if I have this any direction property, there's actually directions where


this classification boundary is very, very close.


And so that's what they say, this is a fundamental misalignment between the geometry of the data,


which is like this, and the geometry of how we specify adversarial examples, which is,


you know, kind of equal in each direction, which leads to that.


And they say, okay, what if I now robust parameters?


So what if I adversarially train my network to be robust?


It basically means that I expand my data, because I add adversarial examples, right,


of this circle here, I actually add adversarial examples.


So my, my class, my data distribution will actually more like this.


And my separating hyperplane will change here.


And the geometry of the adversarial examples will be much more aligned with my separating


hyperplane.


So this is kind of a toy example of where they say this is fundamentally what's going


on, there's a misalignment between the geometry of the adversarial examples and the inherent


geometry of the data.


So that's kind of the theoretical analysis they do.


And with that, I finish here.


And I hope this was clear enough.


DeepMind's AlphaFold 2 Explained! AI Breakthrough in Protein Folding! What we know (& what we don't)

It will change everything. DeepMind solves 50 year old grand challenge. The game has changed.


DeepMind's latest AI breakthrough achieves historic new milestone, helps solve how diseases invade


cells, improve protein folding prediction. AI breakthrough it also wipes your butt automatically.


It is the newest DeepMind big publication. Actually, it's not a publication yet. But so


what happened and I'm sure you've heard this is that every year there is this competition of


protein folding prediction. So proteins are the structures that fold in a given way. And we'll go


into that in a bit. But basically every year there is this competition. And the results of this year's


competition came out. And they looked something like this. Namely, every entry here you see is a


team participating in that competition of protein folding prediction. And there is one team which is


DeepMind's system alpha fold two, which completely dominates all the others to the point where the


problem is now considered to be solved. Now solved in this case, simply means that you're past a


certain number in this in this test set. And if you're past that certain number, your predictions


are useful enough so that other scientists can basically take them and base work on them. So


that's what it means for this protein folding problem to be solved. Now we don't have much


information on alpha fold two yet other than it's really good. And like a blog post and a bunch of


advertisement videos by DeepMind, they are writing a paper on it. But today I want to go into this


blog post, maybe parse out what we can gather from that blog post. And I also want to go actually


through the alpha fold one paper. So as you can see, the performance here increased drastically


with alpha fold two, but you know, guesses are high that the system is going to be somewhat


similar to alpha fold one of which we do have a paper. So today we'll go into alpha fold one,


we'll go into some speculations of alpha fold two, I can already give you my speculation,


it's transformers, it's attention, that all of a sudden made this big jump together with probably


a few other improvements to the alpha fold one system. Basically, transformers continuing to


dominate the entire field. So where do we start? It's probably best, by the way, if this is not a


great meme template, I don't know what is just saying, just saying. Yeah, so let's actually start


with the problem itself. I realize if you're here, you're probably a machine learning person, might


not know too much about protein folding. So these things here are computer representations of


proteins. They don't really look that way, but sort of similar. A protein essentially is a chain


of amino acids. So an amino acid, where do we have this right here? Amino acids are these what


they're called basic building blocks of life, since the proteins, proteins are what make the


cell do things. So protein are sort of the workers in the cell, they are used as signaling molecules,


receptors, they are parts of your muscles, that actually the parts that move are proteins. So they


they are all the work doers. Whenever something needs to work in a cell, do mechanical or work,


proteins are involved. And amino acids are the building blocks of proteins. So each amino acid


has an has a given a certain common structure. And there are 21 of them. So all the proteins in


the world are simply made out of chains of these 21 amino acids. And these chains they are formed


in. So there's always this sort of body that can link up to other bodies of amino acids. It's very


similar, if you maybe know how DNA is structured, is a very similar concept, except in DNA, there


are four different bases. Here, there are 21 amino acids. And each amino acid is a little bit


different in each amino acid has like a tail that hangs off. So the tail can be no look like this,


or it can look like this, like as with a side chain, are there is there one where it's like,


maybe a cyclic one, I'm not sure maybe it can look out here, or it can have sort of no tail at all. I


think that's the case for glycine. So the important part is depending on these on this tail, the


properties, the chemical properties of the amino acids are different. And then what happens next is


really interesting. Once this amino acid chain is built in a in this. So this is the central dogma


of modern biology is that you have DNA. And DNA is translated to RNA, sorry. And then it's translated


to. So it's read off copied to RNA, which is sort of a DNA clone. And then the RNA is translated


into the amino acid chain. And there's always three, three pieces of DNA mapped to one amino


acid. This is very much it's like a compiler. Notably, the interesting part is that these


steps right here, this compilation steps are done by proteins. So there are proteins that do these


things. So nature in a very real sense is its own compiler. So this here you can see as like the


binary. And this here is like the source code. But what happens once you build this chain of amino


acid, and you set it out into the cell because of these different properties of these side chains,


they're also called residues, this chain begins to fold. And so this is, if you know a bit of


chemistry, you might know that these are these are sort of atoms that are linked with covalent


bonds in this case. And it can be that part of this chain is rather like electrically negatively


charged. And here part of this chain might be like electrically positively charged in a given place


over a given other place. And it also depends on the surrounding medium, of course. And that means


that in this case, for example, these two things will attract. And so if you release this amino


acid chain, what you're going to get is sort of a bend, where now the the chain sort of bends and


these two this chain right here, this tail goes like here, this tail goes like here, I'm sorry,


if there is no, if there is no, if there is no, I don't even know what to call it,


pyrene rings or something like this. There isn't an amino acid with that, I apologize. But the


point is that these two things attract and sort of form this shape. And this shape is very important.


We know that proteins and proteins consist of, it can be hundreds, thousands, tens of thousands of


these amino acids in a chain, the proteins function is, interestingly, largely determined by its


structure by its 3d structure, not necessarily by the actual amino acid. So technically, you can


substitute amino acids for each other. So this amino acid here can be could be substituted for


another amino acid that maybe isn't the same, but is has the same properties of its side chain, such


that if the structure is still the same, the protein would perform the same function. So that


that is, is very special property of proteins, namely their 3d structure, largely determines their


function. So for example, in this step here, when you read off the RNA to the DNA, as you know, the


RNA is sorry, the DNA is like this double strand of connected base pairs. And in order to replicate


the DNA or to read it off, there is a there more or let's call it there's also this step of DNA


replication, right where you copy the DNA in mitosis. In order to do that, you need to split off


the two strands, you need to split it up, because you want to get like a protein needs to get here


to actually read it off. For that there is a protein, a specific protein that will insert right


here to split up the DNA, which is called a helicase. And that really is very important how


that protein is shaped. So the shape needs to be actually such that it kind of removes these bonds


from each other. So the shape is very, very important for a protein. And conceivably, you


could build a helicase from many, many different amino acid sequences, as long as it has the same


shape. Now, I think something like something like fundamental like a helicase is probably


conserved in the evolutionary tree. But I hope you get the point, the shape is super duper


important. Now, the shape isn't just arbitrary, there are so the amino acid chain is called the


primary structure. And then the first thing that happens is that two very distinct kind of sub


shapes appear. So often repeating shapes, these things I think are called alpha helix, helices,


or helix, this is a helix. And this here is I don't know what's in English, it's probably called


a strand or something like this. These are like long sheets, like, I think they're called beta


strands. And these things form these are often repeated sequences. And then the third, the


tertiary structure is when the whole thing starts to kind of fold on itself and so on and give itself


the the final structure. So this is part I guess, of the RNA polymerase, which is the molecule that


reads DNA and outputs RNA. And there are many, many, many proteins. Now, since the shape is so


important, it is vital that we know of it, right. And technically, technically, this is what why


this problem is 50 years old, I guess, they say it's a 50 year old problem. I think that's due to


the fact that 50 years ago, a Nobel laureate said the following, since a protein is fully determined


by its amino acid chain, and since the you know, I mean, acid chain determines the structure that


is going to go because of these kind of chemical properties, it should be possible to read in the


amino acid sequence or read in the DNA sequence, we know what amino acid sequence results, and


output the shape of a protein. However, this is an extremely complicated problem, it turned out to be


because they're very subtle interactions, they're not always the same, it depends, right, like


somewhere out here, there could be some amino acid with like some weird chain that, you know,


everything folds on itself all the time. So at some point, these get in contact, and they change


is kind of the local properties here. So this is a very, very difficult problem to solve. And


people have have sort of tried to do this. And now apparently, deep mind the first system that


does this to such a satisfaction that it's beneficial. Alright, now I lost my train of thought. Yeah, so


the shape prediction, what happened so far is what you'd have to do is you'd have to sort of


do this, determine this experimentally. So you'd have to take these proteins and crystallize them,


and then like shoot x rays at them, and then infer the structure, you can you can do that from


crystallized proteins, because I think it's due to crystals are like very regular accumulations


of proteins. So if you look at a snowflake, that is, if we knew nothing about the water molecule,


that it's like H2O, if we knew nothing of that, we could just look at a snowflake and determine


this structure, this this these specific angles here from the snowflake. We would just look at


the snowflakes. And if someone tells us, look, that's all the same material, that's all water,


we could infer what the water molecule looks like, just by analyzing snowflakes, because they're


crystals. And the pretty much the same here is you build you make crystals out of these materials,


you shoot x rays at them, and then you sort of reason over the patterns that come out.


This is very, very difficult, very expensive. And so to solve this problem computationally,


is super important. I will get to this graphic in a minute. This is sort of the only thing we know


about alpha fold two is this graphic right now, because they have not yet released the the paper


or any descriptions of the model, as I said, but what we'll do is we'll go into alpha fold one. So


this is alpha fold one. And alpha fold one was participating in the same competition two years


ago and was already dominant there, but not yet dominant to the point of having quote unquote,


solved the problem just better than other systems. So this is the basic structure of alpha fold one.


So what do you what do you have right here? Let's let's give us ourselves an overview. So the


overview is the following. There are two different stages to this algorithm. Stage one is over here,


and stage two is over here. Maybe it's easiest to start with stage two. So the output of stage one


is this thing right here, a distance and torsion distribution prediction. So this this this matrix


here that's kind of tilted on its side, I believe there are more down here, right? Okay. So what you


do right here is you you take an amino acid sequence and you line it up right here, you line


it up. This is the amino acid sequence is a bit harder if there's like a split. But let's just say


a protein is actually there can't be a split. Sorry, that's in the amino acids. I'm dumb. So


a protein is sing a single chain of these amino acids. There can be multiple sort of parts to a


bigger protein conglomerate. But there is this chain, you line it up here and here. So now we're


building sort of a pairwise matrix between the sequence and itself. Okay. And this pairwise


matrix is going to be a distance matrix. So what we are going to do is we're going to input some


features about this sequence of amino acids, right, that's what we get as an input. And


we're going to predict for any pair, right, so we have the sequence. And we're going to predict for


any pair, how far are they apart? So of course, here, the answer is always kind of zero, they're


zero apart. But you might say, you know, these two are five apart. And these two here are seven


apart. But these two here are only one apart. So it's reasonable, you know, that the final structure


the these two are close together. We don't worry about close together right now, we just worry


about for each two will predict how far they are apart. Okay, so this is you can view this as you


know, a machine learning problem, right, you have an input, see, you have a sequence, and you simply


want to predict the distance matrix. So here you can see that in fact, you can see the top and


bottom one is the predicted and one is the real, I don't even remember which one's which, you can see


that this system does a pretty good job at that there are minute differences. If you really go


look like down here, you can see a bit of a difference. Over here, there is a bit of a


difference. But in general, this system does a pretty good job. So this is the output of stage


one is this matrix, it's a bunch of other, it's like also the torsion angles and so on. But the


main thing is you predict the distances between those two. That's what you take as a input to


stage two. So what stage two does is stage two builds a model of this molecule. And the model


is sort of a differentiable geometrical model. So they say they where is it this I don't get these


nature papers like they're split into two parts, but then they are they largely say the same


things. I am absolutely confused by them. So we're going to jump around a fair bit. They say we


parameterize protein structures by the backbone torsion angles of all residues and build a


differentiable model of protein geometry to compute the coordinates for all residues. And


thus the interresidue distances. So what they do is essentially, they build a computer model


of these amino acids. And these are parameterized by the torsion angles. Now the torsion angle is


simply the angle between any two of them. So this would be like a torsion angle of 180 degrees.


And then if it folds like this, it will be torsion angle of 90 degrees and so on. And you need two


torsion angles because you're in 3d. But essentially, the torsion angles determine the


structure of the protein. So it's one way of parameterizing it. So they built a differentiable


model, a differentiable model of protein geometry. Okay, now the important thing is that they


don't do any learning with this differentiable model. The purpose of this differentiable model


is such that what you can do now, if you have a differentiable model, you can run gradient descent.


So imagine they pretty much lay it out right here. So they have the x, x is x is the output of your


differentiable geometry, right of your torsion angles, let's just call it this Greek letter phi,


psi, whatever. If x is the output, and now x goes into your loss function, so x goes into your loss


function, and the loss function simply compares x to the predicted x, okay, so the loss function


will take in x, and it will compare it to the x that you predicted from from this thing here. Okay,


so we start off with a flat chain, maybe, actually, I think we start off with some initialization,


because they also predict the torsion angles directly, right here, they're predicted torsion


angles direction, and that's what we initialize from. But let's just say we initialize from the


flat chain. And then, because this is differentiable, we do so your your L, your L is x minus x prime,


okay. And what we do is we derive the loss with respect to the angle to the torsion angle. So what


and we can do this since this is differentiable. So now we know how do we need to change the angle,


which is this thing right here, in order to make the loss smaller, right? And maybe it says you


need actually you need to turn it down, right? Make the angle smaller. And we do that, okay, cool. Now


it's only 90 degrees. And then we do it again, and again, and again. And you can see that by


changing all the angles, such that this loss is smaller, we end up through steps, step, step, step,


step, step, we, we in our computer model, we sort of replicate this process that happens in nature,


where what we feed in is how far any two amino acids should be apart. And by running gradient


descent, just gradient descent on the torsion angles, we figure out what do the angles need to


be in order to make this happen? Okay, so first, we predict all the distances, and then we figure


out how do we need to set the angles such that these distances are fulfilled? These are not true


distances, these are predicted distances, right? So everything depends on how well we can predict


these distances. But once we have them, we can sort of replicate in our computers the process as


it happens in nature, except in nature, the the whole folding is dependent on these all these


chemical interactions and so on. And now we do none of this, we simply look see how do we need to


fold in order to make these distances in our computer model, like these like the distance


between this and this, and this and this, any two distances may agree with the distances that we


have predicted right here. And you can see that over time, this as you run gradient descent,


this goes up, this this TM score was up the root mean square distance goes down between


then you of course can compare it if you have a test set with stuff that people have already


figured out, you can analyze these metrics and see that indeed, you do get the correct


folding. It's also pretty interesting that so here in blue and red, I believe you have Yeah,


exactly. So the the helix in blue, and the strands in red. So in this case, you from if you have this


folded structure, partially folded structure, you can already see that these sort of substructures


emerge like this is a helix, right, as you can see, and then you sort of made this may be a


strand and so on. There are ways to heuristically classify that. And you can see that if you look at


the database, right, you can see that this here is the strand, these are helices, and this is a


strand and these are here, this is a strand and so on. And you can see that the model here is what


the model thinks at the beginning, it doesn't get many things correct, though it does some,


but then over time, it sort of refines its guesses until at the end, it's pretty much, you know,


equal to what the to what the database to what the true sample is. And here is simply the


distribution of, I guess, confidence about these things, and the the torsion angles right here. So


it, as you can see, this two step process is the key here to do that. Now, AlphaFold2 conceivably


probably changes this a little bit. But again, we're not sure. The step one right here is a deep


learning system. So step two is simply a gradient descent procedure that you run at inference time,


right? This at training, you can you can just do step one. So step one is, is the machine learning


bit. So the goal is to output this distance, this distance tensor right here. And there are more


things than distances, as we said, there are torsion angles, and so on. But ultimately, you


want to output this distance matrix. And how do they do it, you can already see it's a deep neural


network. So you want to build a input data point, let's say, of l by l, which is sequence length by


sequence length. So you want to collect some features, you don't know the distances yet,


right? But you can collect some features that are either either pairwise features between these two


things, right? So here, maybe this is I don't know, leucine, and this is what's a different


amino acid, glycine. And in here, you want to put features, maybe it can be features for that


position, right? Maybe leucine here is at the 100th position in the in this particular protein,


and this is at the 90th position. So we want to put in some features that of that that you can


derive from a data set, you can put in correlation statistics in general between these two amino


acids, you can even put in just single features. So you have these tiled l by one features, which


is just features for the sequence itself, not pairwise features. But what you do is you simply


replicate them along along any given dimension right here, you always put the same features,


this is very common in convnets. And you can even do a scalar feature. So there are some scalar


features. And what you would do is you would simply fill an entire plane with that scalar


feature, all the same number, it's just easier to do it like this, because it fits into the


convolutional architecture well. So you want to provide all kinds of features and the features


they provide are, you know, plentiful, and a lot of them do introduce some domain tools,


domain expertise, and so on. But once they have that, they simply take that sort of image with


many, many channels, and they predict this image if you want. So it's just an image to image


translation problem. And they do this via a convolutional neural network. As you can see,


there are 220 residual convolutional blocks. Now, I assume that most of the viewers of this video


are familiar what convolutional neural networks are, if not, I'm deeply sorry, but we'll not go


into that. But you can see they sort of they tile this tensor right here, and they tile it differently


from from from instance to instance. So they tile it, they in the training procedure, they always


tile it differently. That's a form of data augmentation. But ultimately, you slide over


this image with this 64 by 64 ConvNet, and you produce the image on the right, you can see an


inherent weakness of these approaches, namely, that this thing can only ever look at 64


amino acids at a time. So now that can, that can be the same if you're on the diagonal of this,


let's say, let's say this is not 64 by 64, but three by three, right? If you're on the diagonal,


you would only consider three amino acids and their interactions with each other, right, any


to any interactions with each other. If you're off the diagonal, what you would consider is maybe


these three amino acids and these three amino acids, and you would only consider you consider


features for maybe for those three, but interactions only in between, like the these not


interactions actually within the same amino acids. So you're the thing that you can look at any point


in time is going to be very limited, right? And these so these distances that you get out here,


they necessarily cannot directly depend on, let's say this amino acid right here, you always have


this limited view of your protein, that sort of local now, people argue that that's actually


enough, if you look at maybe the green connections right here, in order to establish them, what's


most important is the vicinity of these of this amino acid and the immediate vicinity of this


amino acid. And of course, the interaction between those two vicinities, but it is quite conceivable


that this green thing down here being so close will actually sort of push the two apart and


sort of do this interaction, which, in my understanding would not be covered by a system


like this. And that's where alpha fold two, I believe is is one point where it makes the big


gains that it does. Now the features that go in here, as I said, they are, they're quite plentiful.


One of the more interesting features is this MSA these multiple sequence alignment, and I believe


they're they're up right here. Yeah, sequences. So here they introduce them in recent years,


the accuracy of structural predictions has improved through the use of evolutionary


covariation data that are found in sets of related sequences. sequences that are similar to the


target sequence are found by searching large data sets of protein sequences derived from DNA


sequencing and aligned to the target sequence to generate a multiple sequence alignment.


Correlated changes in the positions of two amino acid residues across the sequences of MSA can be


used to infer which residues might be in contact. So what what this I've searched out one of the


papers right here and this is from a paper called improved contact prediction proteins using


pseudo likelihoods to infer POTS models. The entire basis here is that here is your chain of


amino acid that you're considering. And this is you, this is the human and they actually have one


like a very similar graphic in their blog post, but we'll draw this ourselves. I'll just kind of


sort of copy it. And what you do is you go and look into your database, right? This this is the


amino acid sequence and each amino acid can actually be abbreviated by a single letter since


there are 21. And luckily, the holy alphabet creators have given us what 26. So that fits. So


each of these can be done by like s, y, c, m, d, and so on. Can be then you go look into your


database and your database is of sort of all of life. And you go look for similar sequences. And


there are tools that you can very quickly see through databases and get out similar sequences


to yours and that those are sequences that are overlapping in amino acid sequence, right? So


you could find in the fish, this is an alpha, this is not a fish. In the fish, there is a similar


sequence right here in the iron, like this is okay. In the whatever this is, this might be a


horsey. No, this is not a horse. Let's make an alligator out of this. So in the alligator raw


does the alligator have there might be a sequence and so you get the point my drawing skills are to


to be criticized in another video. So you search for all of these similar sequences just by by


amino acid sequence. And from the correlations, you can derive something for example, I've already


told you that sometimes you can substitute an amino acid in the sort of function of the protein


isn't really affected. And this may be what you can see right here. So in the human, this is maybe


a D, but or sorry, maybe this here, it's a C. But in the in the let's call this an M. In the fish,


it's a C too. But you know, in the alligator, it's a P and in the cockroach, it's K and so on.


You can see that maybe if the alignment is good, right, this is sort of from the same protein or


from a protein that does maybe the same thing in these life forms because life is continuous.


Often these things are preserved or slightly modified. So here, there are variations that


happen in life, right mutations, variations. And so we can safely maybe assume that you know, a K,


whether there's a K or a P or a C in this particular point, it doesn't really matter,


the shape doesn't seem to be too affected. Okay, that's so that's step one. And now,


so this might be this this protein, this amino acid right here, you see, whether it's this chain,


or whether it's this chain, maybe doesn't really matter for the function of the protein. However,


if you look at two proteins that are in contact, what needs to happen? So if my protein here has


this chain, and the other protein has has sort of is in contact, that means there is like a chemical


interaction between the two, okay. So now if a mutation happens, if a mutation happens, and the


protein is still functioning the same way, but the mutation happened, let's say, it's now this right


here, that must mean the shape is still the same sort of, and that must mean that probably, if one


of them changed, the other one probably changed, sort of analogously at the same time, because


structure is preserved function is preserved. So structure is preserved. And since structures


determined by chemical interactions, one of the parts changed, that means probably the other part


has changed as well. So maybe now this is sort of this chain right here. So what you would expect


to see in the statistics is that if one changes, the other one changes accordingly. So there can


be variations, right, there can be mutations. But if the mutation happens in one of them,


a corresponding mutation should happen in the other one as well. Otherwise, the protein would


be nonfunctional and the organism would sort of die. Not always, but you know, this is kind of a


statistics game. And this is what you see here, like the fish has an S like the human and an H


right here, but the alligator has an F and a W right here. And then in the cockroach, you see


the S and the H again, and so on. And here down here, you see the F and the W again. And this


is an indication that these, the correlation here is an indication that these two things might be in


contact with each other. Now, there have been systems, for example, in this paper right here,


that directly go from these statistics to contact predictions, and so on. Alpha fold simply takes


in this stuff as features. So this right here, all of this, there can be I think they derive


488 features from this. So this goes down here. I think they say it again, as I said, this is


confused, like here, article stops references, article starts again, thanks. And they like say


almost the same things. It's just a little bit more detailed, it's not longer. So here, they


derive 484 features from these multiple sequence alignment for each residue pair, right. So in our


big tensor right here, right here, each dot each thing right here already now has 400. So


each one of these already has 484 features. And then some more, right, this is already this is


from the MSA, but then more features. So they incorporate lots of features right here.


Where are we at here, incorporate lots of features. In addition, we provide the network


with features that explicitly represent gaps and deletions. They also represent scalar features,


and so on. So here you can see they have scalar features, sequence length features, amino acid


type profiles, HH blitz profiles, these are all sort of these comp bio tools, these genetic tools.


And so on. You also have sequence length features. These are these 484 features and so on.


So these are all akin, there are some positional, one of these acts as positional encodings,


so on. So lots of features, input convolutional network output, the distance matrix. And that's


that, right. So there you have the inputs, the distance matrix from the distance matrix,


you can run gradient descent to get the protein structure at inference time. And they make some


pretty cool points. Not only do they compare the distance matrices, but they here is the not only


the single prediction for the distance, but they of course, output a probability distribution,


they've been all of these distances, they output a probability distribution. And you can see that


the black line in these histograms. So this is, this is for a particular thing. This is for this,


this red line, this red row right here, it's the extraction. So it's for one of the amino acid,


the distribution of probabilities of distance bins with each of the other ones. So this is number 29.


And we look at the distance between number 29 and 123, and so on. The black line represent,


the represents, I think, eight angstroms, which is generally considered the barrier for being in


contact or not being in contact. And here, it's colored in blue, if not in contact and in green,


if in contact, and the red bar represents the true distance. And you can see this is pretty accurate.


So whenever the network predicts blue, usually the red line is on the right of the black line.


And if the network predicts, no, sorry, this green and blue is the ground truth. So whenever


it's blue, the network's distribution is usually shifted towards the right. And whenever it's


green, the network's distribution is shifted towards the left. There are some failure cases,


as you can see right here, the network predicts a higher distance than the, than the, the truth,


right. You can also see what's pretty interesting is that the most accurate predictions sort of the


highest confidence, the smallest variation in distribution are around here, which is exactly


around. So 29 would be in the middle right here. And that's where we find the most accurate


predictions, of course, since local local distances are much more easier. And then as you go farther


away, you get less sure. And this is a cool thing. So here you can see model prediction versus true


distance fits fairly well. But you can also see that here they plot the standard deviation of


their prediction. And you can see that the the means are very close, but the higher the sort


of standard deviation, the less sure the model is. So there seems to be a there seems to be like a


built in confidence metric, right. So you can see the distance error it makes here are bigger. And


also its standard deviation is bigger at the same time, which means that you can sort of look at the


standard deviation of this distribution right here. And that is an estimate for how sure how


confident the model is in its prediction. And apparently, that's something that in alpha fold


to the the model relies upon very, very crucially. So here you these are just the on the bottom, you


see one of these residual blocks here, more distance matrices, they do a lot of analysis in


this article, which is pretty cool. So you can go into it fairly far. They also have look at what


the network pays attention to. And it makes a lot of sense, like it pays attention to kind of these


these helices, and then these interactions between the helices and the parts where it's


close in close contact with, and so on. But now we want to go into alpha fold to alpha fold to


now the what we have isn't much we have this graphic right here, which is also in the article,


it's probably better we go to the blog post to the blog post is like a fluff piece, saying we,


they are going to publish a paper. But of course, they don't have it yet, because we've just gotten


the results. Yeah, they have they have these these cool these videos were like, ah, so good.


As I said, I've, like, there's so many Twitter threads with, I'm not usually up for the hype,


but this is the best thing and so on. And everyone's everyone's hyping. And I thought,


is it really up to me to be the grumpy one here? But then I couldn't find anything to be grumpy


about. So this is what we what we get. Let's see, it's it's deep mind. I expect them to


not fully maybe release the code, maybe they will. But in alpha fold one, they've released


like half the code, which is already pretty cool. So there are open source implementations


based on that. So, again, nothing to be grumpy about. Alright, so what can we what can we say,


they say, a folded, a folded protein can be thought of as a spatial graph. And then


this is kind of a new word they introduced. But ultimately, it's simply this distance matrix that


we've seen before is a representation of that spatial graph, right? It's simply a graph of nodes


and the edges say whether or not they're in contact or respectively how far they are apart,


where the residues are nodes and edges connect the residues in close proximity. This graph is


important for understanding the physical interactions within proteins as well as their


evolutionary history. For the latest version of alpha fold used at CAS 14, that's this challenge,


we created an attention based neural network system trained end to end that attempts to


interpret the structure of this graph while reasoning over the implicit graph that it's


building. I look this, it sounds like this, this is fluff, maybe, I don't know, but this here,


attention based, okay, so I'm going to guess for sure that they've replaced this convnet


with an with a transformer style with an attention, attention layer or multiple attention


layers. They say it uses evolutionary evolutionarily related sequences, multiple sequence alignment and


the representation of amino acid residue pairs to refine this graph. This is this is what we've


already seen. So use these other sequences plus like a lot of stats that you can gather from the


data sets on amino acid pairs in order to develop this, this graph and the graph is distance,


the distance matrix, or other things we'll see in just a second. They say by iterating this process,


system develops strong predictions of the underlying physical structure of the protein


and is able to determine highly accurate structures in a matter of days. Additionally,


alpha fall can predict which parts of each predicted protein structure are reliable in


the future using an internal confidence measure. Again, this is something that we've already sort


of seen in alpha fold one that there is sort of an internal confidence measure. And the part here


is they say by iterating this process, which could mean that it's no longer just this two stage


approach, but it could be an actually fully cycling approach that sort of goes back to the


neural network to refine the structure that it's building with the gradient descent procedure.


It's entirely possible. So this is the graphic of alpha fold two, you can see at the very beginning,


you have protein sequence. And at first, you have this embed and outer embed and outer sum,


which I'm going to guess this is just kind of features for pairs or individual amino acids.


This this is correlation statistics from your data set, it can be, you know, chemical properties,


whatever it just a bunch of features that you can attach to each of these amino acids in the


sequence, right. The other path here is this genetic search and embed. So this is what we've


already seen with the MSA embedding, I saw I told you they have the same graphic. So there's human,


there's fishy, there's rabbit, and you simply search for sequences in your database, it could


even be from other humans, right, that are similar. And from that from those, you can also derive


features. So here is where I'm a bit confused. You can see they build up this again, this square


matrix right here. I mean, this, it already screamed attention before, right. So I'm going


to guess they no longer limit themselves to the maybe, maybe to the 64 by 64. Maybe they do


something bigger. Maybe they use local attention, who knows, but I'm going to guess they use


attention to. And these, this here is simply given by an attention layer of some sort to go into the


next to just this is basically, I would guess this is a big transformer right here. The interesting


part is that it appears to interact much like much like the original transformer, maybe encoder


decoder. Here, they pass information around. So this top thing isn't amino acid sequence to


amino acid sequence like to itself, but it appears to be a matrix that you build up between the amino


acid sequence and these sequences you built. So I would guess that they are no longer, let's say,


happy with simply inputting the features of these algorithms that go over these other sequences.


But now they also want to sort of put these features through through steps of transformations.


So again, I would guess this is an attention layer. And how can we interpret this matrix? As


you can see, this matrix relates individual amino acids in the sequence to other species. So I would


guess that this square here represents something like how important is this particular location in


the chain, which is a purple thingy in the human? How important is that in the in the in the chicken?


Or how related is that to the chicken at that particular position? Or as a whole? I don't know,


probably DeepMind doesn't know, like they probably just ship these features in here, right? And then


they just ship it through transformers, they pass information around, I don't know whether it's just


in this direction. And then in this direction, or whether there's like an arrow right here,


conceivably, but in any case, it seems like they've replaced what was a ConvNet. So no longer


friends with ConvNet new best friend is transformer. And then at the end, you see what


they get out is these pairwise distances again. Now, it's also not really clear, because I would


expect maybe an arrow going like this, if they again, use these pairwise distances to predict


the structure, I don't know, okay. Or if that's just a side output, I would guess they still


actually use the pairwise distances. And the confidence score, again, you can it might be


something very similar that we've saw again, being the sort of standard deviation on the predicted


distances, but they could also refine that. And then the last thing is, I don't know if this


iterative process is simply referring to there being multiple layers of this attention and


passing around. So the passing around will simply be like, you stack the representations on top of


each other. I don't know if this is the iterative procedure, or if there is actually like the


structure module actually sort of builds the structure and then goes back. And then you


consult the neural network again, and then you build some more of the structure, and so on. I


can't tell right now, it's quite conceivable that they they do like that the search here is not only


gradient descent, but is actually informed by the neural network. So you sort of go back and


refine though I don't know, there doesn't seem to be any features in the neural networks that would


represent that would represent whatever you could read from a partially built 3d model. So, you


know, the boring guess is that the part two is very is is a lot of the same, but there could also


be substantial improvements in that part. Alright, I hope this was this was sort of a good overview.


So, as I said, the paper isn't out yet. If you want to cite this, I guess you can you can refer


to the blog post and here they say, until we've published a paper on this work, please cite high


accuracy protein structure prediction using deep learning by these people. I just want to say that


I just want to highlight shout out to to Anna, who was educated right here. She was an intern. So in


a way, I'm actually saying that this is my discovery and I take full responsibility for it.


You're welcome world. Shout out to Anna. Very nice job. Good work. Good work to all of these people.


And yeah, I hope that was enough. If I got something horribly wrong, please tell me in the


SupSup: Supermasks in Superposition (Paper Explained)

Hi there, today we'll look at super masks in superposition by Mitchell Wirtzman, Vivek


Ramanujan et al.


So on a high level, this paper tackles the problem of sequentially learning many, many


tasks without catastrophic forgetting by leveraging these things called super masks.


A super mask is basically a binary mask that you unlace over a randomly initialized neural


network to make the mask network perform better than a random initialization.


They will train these masks for each of the tasks that they consider.


And then at inference time, they can recover the task that the data is from, and therefore


kind of do this lifelong multitask learning better than the baselines that they compare


against.


In fact, they can do better without knowing the task than the baselines can with knowing


the task.


So that's pretty, pretty cool.


This is a pretty dense paper in terms of content.


And we won't go over everything in the paper, but we'll go over the ideas and what kind


of what I think makes them work.


So yeah, stick around if you want to know that.


Also consider sharing this video out, tell your friends about it and subscribe if you


haven't, it helps.


So yeah, cool.


So let's dive in.


We present the super masks in superposition model capable of sequentially learning 1000s


of tasks without catastrophic forgetting.


So the term catastrophic forgetting comes from the world of this kind of sequential


multitask learning, where you have a model, let's call say this is your model, the black


box, and you let it learn on a tasks.


Let's say this is an image recognition task.


So you have a data set, and you let it run on this data set, you learn the data set,


it's set, maybe it's c far 10, right?


So this is c far 10.


Cool.


And now the model can do c far 10 pretty well, then you also want to learn a different task,


you want to learn MNIST.


Okay, so you have MNIST, and you want to learn MNIST, and you want to learn that one.


So your hope is that your final model can do both.


So you'll take this one, and you simply train it on MNIST as well.


And then, you know, we know there's this kind of fine tuning, pre training, and so on.


So your hope would be that at the end, it can do both.


But then you want another one, you want ImageNet.


Okay, now ImageNet is a pretty big data set.


So you take your model and you also train it on ImageNet.


And with time, the model is always going to be very good at the task you just learned.


But it is going to forget the tasks that you learned previously.


This is the catastrophic forgetting problem.


You might ask, why don't I just train on all the tasks equally, like at the same time?


And that's a valid question, you can do that.


But this in the task description here, it's necessary that we learn the task one after


another, because, you know, maybe we get this data in this year, and then it's pretty big


data, we can't just form a four to retrain on all the data all the time, we want to kind


of continuously integrate our knowledge, this is very important in the fields of like lifelong


learning, where you want to kind of the hope is you can build a system that continuously


integrates experience, but doesn't forget the old experience.


Okay, and the experience might come from new data sets and so on.


But you don't want to forget the old ones.


So catastrophic forgetting is one of the main problems in these types of research in this


field of research of lifelong learning.


And this paper is going to tackle this, how it's sort of.


So if you think of what could you do right here, what you could do is you could simply


not use the same model, right?


You can simply train the different models for each task and just keep them around, right?


And at, you know, test time, you need some way of deciding.


So there are two different scenarios in at test time.


So you learn all of these models.


And then at test time, there's an image.


And it could be that I tell you that this image, by the way, that's an MNIST image,


right?


So you just grab this model, and you apply it.


Very cool.


Or it could be that I don't tell you what image it is, like, you have no clue, then


you need a way to decide where it comes from.


But once you do decide where it comes from, it's again, pretty easy.


Once you think, I think this is an MNIST, the thing, you can apply this one.


So you could technically do that, but it's very unhelpful, because these models, they


can be large, right?


First of all, they can be large.


So that means it costs you to store those.


And second of all, there might actually be some overlap, like C410 and ImageNet are both


natural images.


So they might benefit from each other's feature in some way.


Now what we're going to do here is we're sort of going to do this separate models approach.


Namely, we're going to use these, we're going to build these super masks.


So super masks are the second thing that we're going to combine here.


Our approach uses a randomly initialized fixed base network and for each task, find a sub


network, a super mask that achieves good performance.


So what's a super mask?


A super mask comes from these kind of papers about lottery ticket hypothesis.


And one of these papers discovered basically, or conjectured and then showed in evidence


that if you have a network that is randomly initialized, just like this is your neural


network, the gray thing, and there is a way to mask it, which means masking basically


means that you either activate or inactivate connections.


So you have your network and you simply multiply it by a binary mask that for each connection


is a one or a zero.


So the one, so here is like 00000.


This is a one, this is a 000.


This is a one.


So the network isn't going to be zeros and ones, but it's going to be multiplied.


Each connection is going to be multiplied by a zero or a one, which means wherever there's


a one, whatever weight that connection had, that will be the value of the weight of the


connection.


If it is a zero, whatever weight that connection had, it will be pinned to zero.


So there will be no signal flowing.


So this paper established that if you take a randomly initialized neural network, there


is a way to mask it and you can find those masks where if you mask it in a particular


way, the network will already perform better than random on a given task.


So there is a way to solve MNIST by using a randomly initialized neural network and


then simply masking it cleverly.


And then the mask network will have a good accuracy on MNIST.


And they found that and I have made a video about that and the sort of intuition behind


the super masks is, this is just my intuition, is that, you know, MNIST, this is what I'm


guessing MNIST is a relatively easy task.


In fact, most of the tasks they're considering in these papers are relatively easy.


And if you have a randomly initialized neural network, basically what you have around is


a bunch of weight, right?


So if, if I have my two layers right here, and then each connection here is a number


like point two, five, this is, you know, seven, this is negative three, and so on.


Now they're going to consider, they here are going to consider weights that are initialized


in a very special way.


But ultimately, you just have a bunch of random weights lying around.


And if the task is super easy, let's say, and the the neural network is sufficiently


over parameterized, there might be many, many ways of achieving your goal.


So rather than being able to adjust the weights, like you would do when you train the neural


network, you would actually change those numbers, you get away with simply selecting the combination


of weights that will, you know, give you a good performance.


So in its kind of, it's sort of a mix of dropout and vector quantization.


So in vector quantization, you also you get away with quantizing the vectors to given


precision.


And here, the task is easy enough such that by simple over parameterization and selecting


of the weights that you have around mixing them correctly by simply so you can't mix


arbitrarily, but you can mix with zero or one, you get good enough.


Okay, so this is sort of my hypothesis, my hypothesis would be that the harder task,


the the harder it gets to find super masks that perform well.


That's what I think is going on.


But nevertheless, to say for the tasks they're considering here, you can find these super


masks, and there is a way to do that by using gradient descent, even though the super masks


are discrete.


So what we're going to do is we're going to use the same randomly initialized neural network


for each of the tasks, right.


So this is like c410.


This is MNIST, this is ImageNet, we're going to use the same gray network, but we're going


to find an individual mask for each of those networks for each of those tasks on top of


the same network.


And they're all going to perform relatively well according to the super mask conjecture.


Now again, this is not surprising.


And the fact that we always use the same randomly initialized network, you know, isn't really


it's not really necessary that we always use the same.


But in this case, they say, okay, we always use the same.


And then we only need to store the mask for each task.


The mask is much simpler than the weights, because, you know, a 32 bit floating point


number is 32 bits, while a masking bit is only one bit.


So we save basically a factor of 32 in our models.


But essentially, essentially, right, it's not the case that we are training the same


model and continue learning.


It's much more akin to training a sink a training one model per task, and then inferring the


task, and just that we do it in a much more crude way.


So it's more like learning a compressed model per task, I find it's a better way to look


at it, then, then continuous learning.


In any case, you learn the super masks.


And then here is the the the hard bit of the easy bit is if I tell you which tasks the


inference data point the test data point comes from, you have a pretty easy time classifying


it, you simply select the mask accordingly, you run forward pass, and that's it.


If I don't tell you where the test data point comes from, that's the hard part.


Now, they need a way to decide where the data point comes from.


And the the idea that they have right here, they have sort of multiple ideas.


But the main idea, the first idea is that if you have trained these individual models