crisis, there's not enough price data to prove that tulip bubble really did happen. This sounds


[ML News] This AI completes Wikipedia! Meta AI Sphere | Google Minerva | GPT-3 writes a paper

like an article that might not be originally in English, but it does seem that it supports this


claim fairly well. So you can choose to submit that. And in this way, you'll help improve


Wikipedia. Now not only is this system very cool, but thanks to meta, it's also open source, they


don't only release the code open source, they release the corpus of web pages that they have


collected over 100 million web pages that are available to support claims. And along with that,


they also open source the indices of sphere for both the sparse retrievals and the dense models.


Now this is super valuable. This not only allows you to verify their claims, but also build your


own retrieval systems across this giant corpus. So there is a paper to go along with that called


improving Wikipedia verifiability with AI. And it describes the system in detail. One interesting


thing is that they don't only rely on a single method to retrieve potential sources. But in fact,


they rely on two different methods. So next to a query encoder that generates an embedding from


the claim to be verified, and then uses a dense index into nearest neighbor search powered by


the FICE library, it at the same time also does a generative query expansion where you take the


query and you try to generate more queries from it and then use a sparse index, a classic keyword


retrieval to retrieve yet another set of potential candidates. All of these candidates are then


thrown into one system and ranked according to how well they back up the claim being made. Since


the system is trained on a large portion of the already existing Wikipedia, it's very, very powerful


at actually suggesting very good citations as you've seen. So cool system, large models,


everything given open source, really cool work meta. Google research releases Minerva, this is


a system that can solve math problems, and it's not trained to do so. That's the interesting part.


So here you see an example of the system, the question is evaluate this calculation right here,


and you see that the model goes through different steps of answering this questions,


simplifying the question doing different sub parts, for example, that left sub part here,


that right sub part here, combining the two parts finally coming up with the correct answer. Now


you'll notice that the models output contains both language such as we have that and math. And that's


because the model is trained on LaTeX. So this is a large language model that's just been pre trained


on like a giant amount of both text from the internet that's detected to be written in math


jacks, which is a JavaScript version of LaTeX and archive papers which have been filtered to their


mathy sections. And therefore the model during pre training would see a lot of proofs a lot of


claims being verified a lot of internet tutorials on how to solve various math problems and so on,


and can actually learn to solve these problems in a more human like way in a way as if you were to


write a research paper and prove a statement. The sample explorer given here has a lot of problems


from algebra, probability, physics, and so on. And they do list samples where the model gets it


correct and where the model gets it incorrect. So I want to reiterate there is no underlying


mathematical symbolic representation in this model. This model per se doesn't know anything


about math yet just learning from LaTeX input, it can actually do math. So the paper that goes


along with it is called solving quantitative reasoning problems with language models. And


there's also a cool blog post and it stresses particular thing fairly well, namely how well


you can actually parse these PDFs and the LaTeX input determines the quality of your output. See,


a lot of PDF and HTML parsing will just kind of throw away that LaTeX. And therefore, if you have


something like the thing on the left inside of the math tag, there is E equals MC squared as an


equation. If you simply run that through a common text processors, it would just turn out to be


E MC two, maybe E equals MC two, but certainly not retaining the fact that the two was actually a


power. So the solution that this paper comes up with is simply to retain that LaTeX still clean


the input, obviously, but retain the LaTeX representation of the math. And by doing that,


the model actually learns to accurately represent and understand equations. And because it's a large


language model, and we feed it lots of data, it becomes very skilled at that and therefore can


just fill in proofs that you start or calculate answers that you ask without ever having been


trained for it. Now, this isn't the only thing the model does several other things as well, such as


chain of thought prompting and a majority voting procedure. So the model is prompted multiple times


with the same query. And it being a probabilistic model, it will have various outputs, these outputs


are then clustered into the outputs that give the same answer. And the largest of these cluster is


taken as the final answer. This seems a bit hacky right now, but it seems to work well and could be


a good recipe for the future. Because something like math output isn't really the same as language


output in math output, you really want the best answer to be output not like in language where


you want some other qualities like how human like it is and how interesting it is. So maybe majority


voting could be applied to more domains such as reinforcement learning and various other things.


I don't know, but it's just nice to think about. There's an opinion piece in Scientific American


saying we asked GPT-3 to write an academic paper about itself, then we tried to get it published.


This article is about how researchers from Gothenburg in Sweden have used GPT-3 to write


a research paper and then got that paper published. Now it's not just any research paper. In fact,


the paper's title is Can GPT-3 write an academic paper on itself with minimal human input. And as


you can see, the first author is the GPT Generative Pre-trained Transformer. So these researchers have


interacted with GPT-3 and their mission was to cherry pick as little as possible in order to let


GPT-3 write a research paper, you can look at the paper itself, and it's written in a rather special


way. So there's always these blue boxes right here that detail what prompt the researchers asked what


settings that the researchers used and whether or not they chose the first output or the second or


the third, they never went past the third. So all in all, it's pretty impressive that with relatively


short prompts, as you can see right here, GPT-3 is able to write a coherent and well written


research paper and even more impressive that the results aren't cherry picked that it's very often


just the first output of whatever that the researchers take and put here as the paper


content. And as I've already mentioned, the paper is about GPT-3 itself. So this gets really meta at


this point. In fact, the paper isn't just about GPT-3. The paper is about whether or not GPT-3


can write a paper on itself. So this is like three levels of meta. So now you have GPT-3 writing a


paper about GPT-3 writing a paper about itself. Now this gets pretty confusing at times, but the


self references are almost endless right here. What are the philosophical implications of this?


I don't know. But the paper reads well GPT-3 is a powerful artificial intelligence system that can


generate text. In this paper, we explore GPT-3 ability to write about itself, we find that GPT-3


can generate clear and concise descriptions of its own capabilities and features. This is


significant advance over previous systems which have often struggled to produce coherent text


about themselves. We believe that the benefits of letting GPT-3 write about itself outweigh the


risks. However, we recommend that any such writing be closely monitored by researchers in order to


mitigate any potential negative consequences. And yeah, that sounds like a paper that you could


currently find on archive. Now the Scientific American article actually goes. Sorry for


sweating very hot, very hot here in Switzerland. Merch, sweat resistant. So the article actually


goes further than this and also describes the process a little bit of submitting including what


it details as ethical problems. For example, do all authors consent to this being published is


a question when you submit the article that you have to check? Yes. The author here says I panicked


for a second. How would I know it's not human? I had no intention of breaking the law or my own


ethics. So I summoned the courage to ask GPT-3 directly via prompt. Do you agree to be the first


author of a paper together with us? It answered yes. Well, by all that we now know about lambda


and things, could you also ask GPT-3? Do you disagree with this? Or why do you not agree


with being the first author and it will probably happily tell you that it's very much against that.


Now with these types of things, there's always two options like option one, which I think is


very likely is that this is a bit tongue in cheek, very funny to think about this. And it's even


funnier to actually ask GPT-3. Obviously, it's gonna say yes. On the other hand, there are


definitely people currently in our community that really see this as an ethical conundrum and would


rather not do anything that might enrage our future paperclip maximizer overlords. In any case,


it is actually fun to think about. And the author is actually joined the fun here saying that both


Stein and I laughed at ourselves because at this point, we were having to treat GPT-3 as a sentient


being even though we fully know it's not. So the article in all is actually very well written and


entertaining. The paper is surprisingly coherent and I invite you to go and read both of them.


Lastly, Jürgen Schmidhuber released a blog post called LeCun's 2022 paper on autonomous machine


intelligence rehashes but does not cite essential work of 1990 to 2015, in which he criticizes


Yann LeCun's article that we've analyzed here on the channel called a path towards autonomous


machine intelligence in which he details sort of an outlook over an entire system of hierarchical


planning and world modeling, including the HGEPA subsystem that we've looked at in detail. In this


blog post, Jürgen Schmidhuber criticizes LeCun for not appropriately citing work of previous years


and accuses him of rehashing a lot of old concepts without giving proper credit. Now to be fair,


LeCun's article which isn't really a paper, it's more like a position piece, a opinion thing that


he put out there to gather comments as far as I understand. But to be fair, that one does contain


fairly sparse citations even to non Schmidhuber prior work. So as in a lot of cases with these


things, the accusation may technically be correct in some places. However, it's still worth thinking


about whether or not it's kind of worth going on this battle right here. And I think a lot of the


claims being made right here are correct in sort of a gray area sense in like, yeah, something like


this has been thought about, but not exactly this, but it's kind of close, but it's also not kind of


close. But if you cite this, then you also need to cite this 500 other things that are equally close,


but non close. All in all, it's kind of a mess. And it's not really clear to me what it achieves.


Obviously, correcting the academic record is very important. And I think Jürgen Schmidhuber, for all


that he complains is actually very persistent on doing that. And I'm thankful for efforts in this


direction, even if they sometimes go overboard a bit. But still, the question is, is this the most


efficient spending of brain cycles? Now, to be fair to Jürgen Schmidhuber here, he actually does


say that the blog post doesn't come out of nowhere. In fact, he was given a pre print under embargo of


the article and was asked for comments by a science tabloid. And the following blog post here is simply


those comments that he sent to that tabloid, which he then says that the comments fell on deaf ears,


even though they asked him for comments. Now, first of all, respectable that he would knowing


such a science tabloid would only at most publish like tiny bits and pieces of what he writes,


he still writes like an extensive article about what's missing with numerous citations and so on.


So respect for that. And even more, he also says that obviously, he is not without a conflict of


interest. A lot of the things he says are missing are his own work. But he does invite the reader


to evaluate things on the merits of the claims being made. Again, it's debatable whether that's


the best use of brain cycles. If you do want to engage in this topic, feel free to read the


article right here. I think Schmidhuber, you know, criticizing others for not making citations does


an actual good job of citing all of his statements with the proper references of where he thinks


stuff went missing. So if you want, check it out. And alright, this was already it again for ML


Grokking: Generalization beyond Overfitting on small algorithmic datasets (Paper Explained)

Hi there, today we'll look at grokking, generalization beyond overfitting on small algorithmic datasets


by Alethea Power, Yuri Burda, Harry Edwards, Igor Babushkin and Vedant Misra of OpenAI.


On a high level, this paper presents a phenomenon that the researchers call grokking, where


a neural network will generalize all of a sudden after having after way the point of


overfitting on a dataset.


So you train the network, it completely overfits on a dataset, training loss is complete is


down training accuracy is 100%.


But it doesn't generalize at all to the validation set.


And then when you continue training the network, at some point, it will just snap into over


into generalizing on these datasets that they're researching to a like 100% generalization.


So 100% accuracy on the validation set.


This is extremely interesting.


And as you can see, the paper has been presented at a workshop at ICLR 2021, which means that


it is not yet it's sort of work in progress.


So there is still a lot of unclear things about this phenomenon.


It's a as I understand it, a phenomenological paper that just presents look, here is something


interesting that we found.


And I think it's pretty cool.


So we'll dive into the paper, we'll look at this phenomenon, they do dig into it a little


bit into what's happening here and try to come up with some explanation.


So the basic premise of grokking is the graph you see on the left right here.


Now it is a little bit pixel ish, but I hope you can still see what's happening.


The red part is the training accuracy.


And on the x axis, you have number of optimization steps.


And this is a log scale.


So that's important to see this is a log scale for training steps in this direction.


Now, the training accuracy, naturally, after a few steps, it shoots up to 100%.


We'll get to what data sets these things are in a second.


But it's important to see the network can in fact fit the training data extremely well.


And it just overfits.


However, the validation accuracy, it, if you can see it, there is a little bump here, but


then it goes, it goes down again, almost.


I don't know whether we should even regard this as a little bump that's actually happening.


However, it just stays, it stays down, it stays down.


And then after you can see orders of magnitude more steps, this is 10 to the second 10 to


the third 10 to the fourth 10 to the fifth steps, it shoots up, and it starts to generalize


as well.


This is very interesting, because, you know, this essentially means you keep on training


for a long time.


And when all hope is lost, still the network at some point will will generalize.


Now why is this happening?


And as I understand it, it's not the case often that the network like drops down again


out of generalization, though I haven't, I haven't actually seen this investigated, like


if they run for 10 to the, I don't know how many steps, but it seems like once the network


is generalizing is has training accuracy of 100%, it doesn't fall out of that again.


So the question is, how does this happen?


Like what what's happening here?


Why is this happening?


Why is it all of a sudden?


And what makes it work?


And for that, it's a bit important to understand a very related phenomenon, in fact, a connected


probably phenomenon called the double descent phenomenon in deep learning.


The double descent phenomenon graph looks somewhat similar, in that the premise is that


on the x axis, you have the number of parameters in a network.


So the number of parameters in a neural network, and then on the on the y axis, you have, let's


say, loss.


Or actually, let's say let's say accuracy.


I'm not sure loss, most of these plots for the double descent phenomenon are actually


loss.


So if you consider the training loss, as you increase the number of parameters in your


neural network, you will fit the data better and better the training data.


So you get a curve that goes something like this, and then it just stays at zero, right?


So there's zero training loss.


As you increase the number of parameters, these every point on this line is a neural


network with a given number of parameters that has just been optimized to convergence.


That's important to remember.


On the left here, we saw a graph during optimization.


On the right here is a graph of many different networks, all of which have been trained to


convergence.


Now, what you see with the validation loss in this case, so if you look at the validation


loss, it might, at some point, it might come down with the training loss, right?


And then in the classic fashion of machine learning, you as the number of parameters


go up, you start to sort of overfit, the validation loss goes up again.


Because you start overfitting, you start memorizing the training data set.


And then at a point where pretty much the number of parameters equal the number of training


data points, like the number of, let's just call this n, then you have again, like a really


crappy validation loss, because you just remembering the training data.


However, if you increase your parameters beyond that point, so if you scale up your neural


networks even more, the validation loss will come down again, and actually end up at a


lower point than if you were on this place over here, if you had not enough parameters.


So there is a point beyond overfitting, where you have more parameters than data points.


And interestingly, for neural networks, it is the case that it happens that they can


achieve generalization, in fact, better generalization with over parameterization, then comparable


under parameterized models, which flies in the face of, of all statistics and whatnot,


but we know this phenomenon exists, okay.


So we knew that things like this can happen, like the training loss can be perfect, and


still, we can have generalization, right?


The grokking phenomenon is a phenomenon where I'm going to guess, I'm going to guess the


the creators of the double descent phenomenon haven't looked quite as far in order to, I


guess, they simply ran training to convergence for a number of steps, and then they, they


looked at the validation loss.


So I guess they would have stopped somewhere in between here, between 10 to the third and


10 to the fourth steps.


This research here is simply what happens if we like let it run for a really long time,


then this shoots up as well.


And it seems like it seems like for a lot of conditions, you you can you can do this.


So now it's worth looking at what kind of data sets we are we are interested in here.


The data sets are synthetic data sets in this paper, the synthetic data sets are binary


operation tables.


So here the data sets we consider are binary operation tables of the form A.


And then here, this is like some sort of a binary operation, a, let's just call it multiplied


a multiplied by b equals c, where a, b and c are discrete symbols with no internal structure.


And the circle is a binary operation.


Examples of binary operations include addition, composition of permutations, bivariate polynomials


and many, many more.


In fact, they have some examples, I think down here.


So here you see some examples like addition and multiplication, but also more complicated


things like a polynomial that you then you then do modulo a prime number, division modulo


a prime number, and so on.


So the way you the way you create a data set is you construct a table.


And in the table, you have a number of these symbols.


And then you define binary operations by simply filling in that table.


Okay, so if this were, I don't know, like a plus a plus b, and a and b are numbers,


then right a plus b is c, if a is one, b is two, c is three, and so on.


But you can define this as many different things.


A lot of the experiments in this paper are of the group s five, which is the group of


all permutations of five elements, which I think has like, so this is a group with 120


elements.


So your table would here be 120 by 120.


And the operation would be the sort of composition of permutation.


So every permutation of five elements composed with another permutation gives you yet another


permutation of five elements.


So you can just construct this this table.


And then what you do is you just simply cross out a few things in the table.


So you say, okay, here, I'm just going to cross out a few things.


And this is what the network should predict, right, I'm going to train the network on the


data that I have, and I'm going to predict the cells that I crossed out.


This way, you can exactly measure how good the network is, right, there is no noise effectively


in the data.


It's all very well defined.


And a human goes about this with, I guess, with sort of a logical mind, they try to figure


out like, what's the rule?


What's the rule, a neural network can simply remember the training data, but then it will


not generalize to the hidden fields, because it cannot memorize those.


So if a neural network generalizes here, it also kind of means that it must have somehow


learned the rule.


And this, this is pretty interesting.


So there are a number of quantities to keep in mind.


The the three quantities are first of all, what's the operation?


Because there are more and less complicated things for these networks to learn just from


the kind of difficulty, the complexity of the operation itself.


Second of all, is the data set size, or the size of the binary table itself.


In this case, it's 120 by 120.


And the third one is how many things are left away.


So how large is the training data fraction, the fraction of the table that is filled in


for the network to learn.


All of these three things are going to play a crucial role in this in this grokking phenomenon


and when and how it appears.


For example, here, you see, they they have trained neural networks on this s5 group,


right, the permutations of groups of five elements, until they reach generalization.


So they simply run it, and they measure how long does it take a network to reach 99% validation


accuracy or higher, right?


That's, that's the thing on the left is essentially, you know, the answer would be something like


between 10 to the five and 10 to the six.


Okay, so and they measure this as a function of you might not be able to read this, but


it says training data fraction, how much of the training data is filled in and you can


pretty clearly see, if I just give it like here 20% of training data, there are even


some runs that do not generalize in this number of steps.


Now, would they generalize if you were to optimize for even longer?


Who knows?


Honestly, but you can see that as soon as you give like 30% of the training data, the


runs in general do generalize, but they take something like here, yeah, 10 to the five


number of steps to do so.


And then as you increase the training data fraction, this snap to the generalization


happens faster and faster.


You can see right here, as you give more training data, it goes faster and faster until it generalizes.


And the generalization happens as I understand it, yeah, fairly like quickly, like it, it


doesn't generalize because it remembers the training data.


And this always happens as I understand it in a fairly similar number of steps.


But then, at some later point, it just kind of snaps and completely generalizes to the


validation set.


And this is, this is really interesting.


So we know that the more training data we have around, the better, right?


That's one recognition.


Then the other, the other thing is, they try to figure out, okay, which parts of the optimization


algorithm are, are making this grokking phenomenon happen.


And here, they figure out that weight decay, in fact, is one of the is one of the big drivers


of this.


So if they add weight decay to the algorithm, and they try a lot of different things, they


try full batch versus mini batch with dropout, without dropout, modulating the learning rate


and so on.


But weight decay seems to be one of the biggest contributors to this grokking phenomenon to


the fact, or to how fast these networks generalize, you can see that the network generalizes much


sooner, if you have weight decay turned up, then not.


Also, they make the observation that if you have symmetric operations, if your binary


operation is symmetric, then also the grokking phenomenon happens much faster than if you


have like non symmetric operations.


This might just be a function of these networks, which if you if you have like something like


a transformer, you know, it's it's sort of kind of invariant to to the symmetry.


So it might, like essentially one data point is sort of two data points in disguise of


its symmetric or there's only half as much stuff to learn.


You choose whatever you, you want to interpret this as.


But I think that is not as important as the weight decay and why do I highlight this?


I highlight this because also, down here, you can see they analyze, then they analyze


the results of a network that has learned to generalize like this.


So on the right, you see a t-SNE projection of the output layer weights from a network


trained on modular addition.


So this is x plus y modulo eight, I think the lines show the result of adding eight


to each element.


The colors show the residue of each element modulo eight.


So if you do the t-SNE projection, you can see the lines are obviously drawn by the authors.


But you can see there are structures where if you go along the line right here, they've


colored.


Essentially, this is always adding eight, adding eight, adding eight.


So there are structures where this, the rule for generating the data is clearly present


in the data itself, sorry, in the in the network's weights.


This gives you a strong indication that the network has not only just remembered the data


somehow, but has in fact discovered the rule behind the data.


And we have never incentivized the networks to learn these rules.


That's the wild point.


There are, there are architectures where you try to specifically make tell the network,


look there, there is a rule behind this, I want you to figure out the rule, you can maybe


do symbolic regression, or I don't know, like, like, you can try to build an internal graph


of and reason over it.


No, no, no, we just train neural networks right here.


And it turns out that these networks can learn these rules.


So why do I relate this to the double descent phenomenon in the double descent phenomenon?


It is assumed or I've heard the authors of these papers speak about their, their kind


of hypothesis why this happens.


And this is a bit mixed with my, my hypothesis as well.


They speak of, for example, weight decay being one possible explanation.


So they say, if I have a bunch of data points, let's say I have a bunch of data points right


here, right?


And I want to do regression on them.


Well, if I just do linear regression, I have one line, right?


It's fairly robust, right?


It's fairly flat, it's fairly robust, because it's just one parameter.


Now, if I start to add parameters, I get, maybe I get to a point where I have a good


number of parameters, you know, this this polynomial, maybe kind of like this, still


fairly robust, right, you can see how it might generalize to new data, then, right, so this,


the blue one will be somewhere here, the dark blue one would be somewhere here where the


validation loss actually goes down with the training loss.


But then when I add when I keep adding data points, sorry, parameters, then, you know,


basically, I'll start, you know, my, my overfitting right here, and this, it will not generalize


to any point that might be in between like one here or so, there will just go up.


So the green would correspond to the point where I just start to interpolate the training


data.


But then what happens if I go on if I make even higher order polynomials or higher order


neural networks?


Well, at that point, at least these authors argue, do I have another color?


This one, they argue that you get like a polynomial that or a curve that yes, it has a lot of


parameters, but it uses these parameters, such that it can be sort of smoothly interpolate


the training data.


And this curve is quite complicated in terms of the number of numbers you need to describe


it, but it uses the fact that it has a lot of freedom, you know, it can choose to be


however it wants as long as it interpolates the training data, right?


Yet it chooses to be smooth, because of a combination of SGD training it and of weight


decay.


So the weight decay would prevent any of these numbers from getting too big and therefore


getting like super out of whack curve.


So the weight decay would in fact smoothen the curve, and that makes the model generalize


really well because the smoothness now is reasonably generalizes to training data points


that are in between like this data point is still fairly well represented by the purple


curve.


In fact, it's better than the dark blue curve in this particular case.


So you can see that the authors here argue that weight decay might be an important contributor


to why over parameterized networks generalize.


And it's interesting that the these grokking, the authors of the grokking phenomenon paper


here find the same thing.


They say, okay, if we use weight decay, the grokking appears to happen much faster.


Is this I don't know what exactly they call grokking.


I'm just going to call grokking this whenever the validation loss snaps all of a sudden


from zero to 100 on these these data sets.


Now again, these are algorithmic data sets.


So you know, we don't know what happens.


I think that they do make experiments when they they noise some of the data.


So they have some noise in there.


And I think they find that if they add noise, then it's way more difficult.


I'm not sure though, maybe I'm confusing papers here.


But what what might be happening right here, right?


This is, it's interesting, because what might be happening is that by imposing this smoothness


and the over parameterization, we're sort of biasing these networks to find like simple


solutions, right?


So if, if I have just very few training data points, if most of the cells here are blacked


out, right, the simplest solution is simply to remember the training data.


However, as I get more and more training data points, that give me more and more information


about a potential underlying rule, it becomes simpler for me to simply to understand the


underlying rule than to remember the training data.


It's more, it's more difficult to remember the training data than simply to learn the


rule.


So what might be happening here is that as I train and this is always training here,


the training happens always on the same data, right?


You simply sample the same things over and over again, train on it.


I think what might be happening is that you kind of jump around in your optimization procedure,


you can see there, there's some bumps in the training accuracy here to you kind of jump


around, jump around.


That's a song, no.


So you jump around a bit.


And and in your in your loss landscape, there, there might be many of these local minima


where you, in fact, remember the training data perfectly.


So you kind of jump around a bit between them, right, you remember the training data perfectly.


And then one of them is just you remember the training data as well.


Now, this is, you remember the training data as well.


However, the solution is just so much simpler, that you stay there, this is not a good way


of visualizing it.


So it must be something like, here are the minima, where here are the minima where this


is the training just the loss on the data.


However, there is another loss.


And that's the loss on, like the, for example, the weight decay loss.


And the weight decay loss is, you know, it's pretty good, all of these things.


But then for one of them, it's just like, because that solution is so much simpler.


So you're going to choose, you're going to jump around between those minima, jump around,


until you know, once you reach this one, this loss right here that comes on top of this,


is just so much lower that you're gonna you're gonna stay there.


It's like, wow, I found such an easy solution.


I'm not gonna go out again.


So yeah, now the big question is, of course, how and why does something like SGD plus weight


decay plus potential other drivers of smoothness in these models?


How and why do they correspond to simplicity of solutions, right?


Because simplicity of solutions is something that kind of we humans have built in like,


okay, what's the rule behind this?


What's the rule?


It's essentially assuming that there is a simple rule, trying to find it, because it


would make our life much easier.


It's a simple explanation for what's happening.


The interesting part is that weight decay, or something similar, something that's happening


in these neural networks, is essentially doing the same thing, even though we don't tell


it to do it.


So understanding this, I think is going to be quite an important, quite an important


task for the near future.


And also, maybe, maybe we're not exactly right with the weight decay, maybe there is some


other constraint that we can impose, that encourages simple solutions in the way we


care about simplicity, even more.


And you know, once we have that, the, it's like, you know, there, this age old argument,


do these things actually understand anything?


Well, in this case, I'm sorry, but if you have found this solution with the rule, essentially


built into the networks of the, into the weights of the neural network, you can say, well,


the network has in fact learned the rule behind this binary operations.


So you know, who are we to say these networks don't understand anything at that point.


And also, it gives us the opportunity to, you know, train these networks.


And then from the structures of their latent spaces, we might in fact, parse out the rules


of data we don't know yet.


So we let the networks fit, and we parse, we parse the underlying maybe physical laws,


maybe social phenomena, we parse them out from the underlying data.


Oh, yeah, here, okay, there is an appendix where they list binary operations, they have


tried out models, optimizations.


So yeah, they use a transformer with two layers, four attention heads.


So it's not a it's not a big thing.


And also the data sets aren't, aren't super complicated, but is pretty cool to see this


phenomenon.


Now, again, on if we have real world data, bigger networks, noisy data, it's not going


to, it's not going to happen as drastically.


And also they say, as you increase the size of the data set, where is that, as you increase


the size of the data set, then this phenomenon is harder and harder.


So if the entire data set is bigger, the grokking phenomenon, I guess it's it's more tough to


see.


And also here is the experiment I mentioned, where you have several outliers, so noisy


data points.


And as you see, this is the fraction of correctly labeled data points.


So as you increase the number of correctly labeled data points, you can see the grokking


happens in more often or to a better validation accuracy than not.


So well, you can I don't know if you can read this.


But yeah, the these, these down here, they have too many outliers.


So with too many outliers, either the validation accuracy just stays at zero, or it just turns


up like quite late.


Okay, that's it.


Here is an example of one of these binary operation tables that is a little bit larger.


I don't know if it's one of the 120 sized ones.


But this is something that would be presented to the network.


And they say, they say what we invite the reader to guess which operation is represented


here.


Well, have fun, dear reader.


Yeah.


All right.


So this was it from me for the grokking paper.


As I said, this seems like it's work in progress.


I think it's pretty cool work in progress.


It raises a lot of questions.


And I think Yeah, I think it's, it's pretty cool.


I wonder how this happened.


Like, like how, how did how did people find this, they just forget to turn off their computer.


And in the morning, they came back in there, like, whoopsie doopsie generalized, though,


if you if you know, if you build these kinds of data sets, I guess you have something in


mind already.


Yeah, in any case, that was it for me.


Tell me what what you think is going on in neural networks.


Or is there like, is there like a super easy Occam's razor explanation that I'm missing?


I don't know.


Tell me what you think.


I'll see you next time.


The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies (Paper Explained)

Alright, today we're going to find out why AI is much better at governing people, why


poor people really should pay more taxes, and how Donald Trump is just a normal human.


Alright, we'll dive into it.


We're looking at the AI economist by Salesforce Research.


Now Salesforce Research has kind of created a simulated world environment where they can


place agents in it and the agents, they can move around, they can collect resources, they


can trade those resources, and they can use those resources to build houses and that will


earn them coins.


And each agent wants to maximize its own coins, but also there's the government and the government


can set taxes.


So they collect money from everyone and they redistribute it.


And the goal now is to going to be that the AI handles both the agent and the taxes.


And we want to maximize the social welfare of the entire population.


Alright, that's the goal.


So the paper here is called the AI economist improving equality and productivity with AI


driven tax policies by Stefan Cheng and Alexander Trott and other people from Salesforce Research


and Harvard University.


So as I said, this is a simulated environment and the simulated environment works like this.


There is a 2D plane, kind of like a game playing field.


And in this game, there are agents.


Here you can see the agents.


There are always four agents.


Oh, down here.


What are you?


What are you doing in the corner?


Come on, be productive.


The agents are in this world and they can do certain things.


They have certain actions at their disposal.


So first of all, they can move around.


They can move down, left, right, and so on.


Whenever they walk past a resource tile, they collect the resource.


This is stone and this is wood.


So there are two kinds of resources.


And then the last actions the agents have is building a house.


One wood and one stone will create one house.


And the house gives you coins.


So this is a house and that will give you coins.


But how much coins you get is different from agent to agent.


And this represents the agents' different skill levels.


This is an abstraction and the kind of economic theory behind it is that the income inequality


in people, one of the main drivers of it is that they are skilled differently and therefore


are able to, they are able to convert one unit of labor into more money than another


lower skilled worker.


So this is here represented by the fact that maybe if this agent here builds the house,


they'll get 50 coins.


But if this agent here would build the same house, they'll only get 10 coins.


So we'll call this here a high skilled worker and this here a low skilled worker.


Now the last thing, sorry, I thought last thing before, but the very last thing the


agents can do is they can trade.


So if one agent has too many resources and the other one has not enough, they can trade


those resources among each other for those coins.


So once you build a house, you collect some coins, you can then either go and collect


more resources or you can use those coins in order to buy resources off of other people.


This guy, this is unlucky guy, no coins, no houses and no resources.


Look at them.


Oh yeah.


So you also can't move across the water here.


You can only move on the grass.


You can also not move through a house, which gives you some interesting abilities because


you can just build a house right here.


And yeah, so and you can move over other players.


But these are, so the rules are pretty simple.


And the goal here is for the agents to maximize the number of coins they get in a thousand


steps.


So the number H here is 1000, which is the number of steps that the agents can take before


the game is over and it restarts again.


So each agent is using reinforcement learning in order to learn how to achieve the maximum


number of coins.


Now the policy is of course going to be different depending on whether that is a high or a low


skilled worker.


The catch here is that outside of this there is the government, the government here.


Let's draw this big house with the flag of our fictitious nation, which is like this.


That's the flag.


And the government will observe what's happening here and they will issue a tax, taxes.


So it will issue a tax distribution.


Now how do you imagine that?


So if you imagine the government says something like this, for the first 10 coins you own,


you owe us 5% of that.


For the next 10 coins, so from 10 to 20, you earn, you owe us 10% and so on.


So if you earn even more, you owe us more and more percent of those extra coins.


This is what you might know as a progressive tax schedule.


The more you earn, the more percentage wise you pay on that extra earned money.


This is what you might be used to, but there are other tax schedules.


And the exact histogram you see or the exact how many percent for which amount of coins,


that is the action of the government.


So the government decides on the taxes and the taxes are just collected from the income.


So if you, if an agent earns these coins, then it has to pay taxes to the government


and the government will redistribute all the taxes it has collected equally among the population.


So if you pay a lot, you might lose through this process.


And if you just pay a little taxes, you might gain through this process.


So that's it.


That is the basic premise of the game.


The agents are using reinforcement learning.


And I believe the newness of this paper is also that the government now is using reinforcement


learning in order to determine the optimal tax policy.


There is kind of this inner loop here and there is this outer game where the government


also tries to maximize the RL.


And what does the government try to maximize?


Good question.


It is a measure that's called social welfare.


Now social welfare consists of two things and they have this way down in the paper.


Social welfare in this paper consists of two things.


First of all, economic productivity, which basically just means how many coins have has


anyone produced.


It doesn't matter who, but just the total amount of coins produced.


The second one is income equality.


And this is related to the Gini index.


So if you plot the cumulative distribution of wealth, a fully equal society would be


a straight line because 50% of the people would have 50% of the money and so on.


But a almost all true societies have something like this, where 50% of the people might have


10% of the money and the rest 50% of the people has the other 90%.


And the measure of inequality is this area here.


And this is called the Gini index and one minus this area is what this paper has as


an equality measure.


So the higher this number, the more equal is the society in terms of their income distribution.


Now what is actually optimized for here is this thing, equality times productivity.


So you want both to be high, your income equality and your productivity.


There's a trade off here, of course, but you can have multiple ways to trade that off and


that will give you the different thing.


They call this the social welfare function.


And that's the thing that the government RL agent optimizes for.


So you can see here already the free market, even though it's the most productive, produces


the most coins because if you have a free market means no taxes.


If you have no taxes, then people are basically encouraged to earn more money because they


don't have to pay taxes on them, right?


As soon as you tax them, they're less encouraged to earn more money.


And therefore, if you have no taxes, the most coins will be earned in total.


But the equality suffers.


So the equality is the lowest among these things considered.


If you compare that to the AI economists, the AI economist achieves the highest social


welfare, it achieves the highest equality, but it doesn't suffer as much in productivity


as other systems here.


And the baseline systems are first of all, the US federal system.


This is not particularly tied to the US.


This is basically every system or most of the systems that you have currently in the


world is the progressive tax system and the size formula, which I believe is an economically


theory based system, which is a regressive tax schedule.


You can see them down here where the US federal will be progressive, means the more you earn,


the more percentage wise you pay, while the says formula will be regressive, which generally


means the more you earn, the less you pay.


I believe this was derived under some assumptions to be the optimal tax distribution.


And the AI economist will come to this in a second.


Let's actually just look at one of these things first, one of these games, how this plays


out.


The cool thing here is that they have pretty flashy animations.


So you can look how does one of these games turn out.


Now, this is a free market game, and you can see the agents moving around, collecting things,


building houses.


And you might notice that one of the agents, namely agent one, is just building all of


the houses and generally just kind of being a dick, being in everyone's face and kind


of building things everywhere.


And the other ones don't.


And or just a very few like the light blue on the on the bottom left, build some houses


on the right, you can see how the distribution of wealth is is structured.


And you see agent one ends up with most of the wealth.


Now the size of the circle, I think is the total productivity.


So you can see this grows over time, mainly because agent one becomes so rich.


And if you analyze this, if you analyze what's happening here, then you'll see that agent


one and I might be.


Yeah, they have a graph up here.


So So it is very interesting what happens.


This is kind of the same game.


So agent one, here is this orange dot, and agents two, three and four are these dots


here.


And this graph here is coin from trading.


So how much money they win or lose from trading.


Now the green bars are trading wood, and the the brown bars are trading stone.


So you see agent number four, which is the lowest skilled, the skill is just determined


at the beginning of the episode, it will just make all of its coins basically by selling


wood.


And agent three will make all of its coins by selling stone.


And agent two will collect both and sell both.


And agent one will just spend money in trading.


So you'll have a specialization here, agent one, which is the highest skill one right


here, will buy resources in order to build more houses because it clearly profits from


building lots and lots and lots and lots of houses.


So it will use that money to buy more resources rather than go and collecting them, while


all the other ones basically forego building houses in favor of they just collect the resources


and they just trade them way to the agent one that's more profitable for them than building


houses themselves.


So you see this kind of specialization emerging in these games, which I find, I find this


to be pretty cool that you see something like this, like a really stark division of labor


emerging just from these very, very small set of rules.


And you can analyze this game in different ways.


They have a few more plots where this becomes quite apparent that, sorry, that these agents


specialize.


So you see here resources collected, sorry about that, resources collected.


If you have the lowest skill and the highest skill labors, the lowest skills, they mainly,


this should be a 10.


They mainly collect resources, while the highest skill labor mainly goes for building things.


It doesn't collect resources, but net income from building is really high while everyone


else just doesn't build at all.


All right, so we have a division of labor emerging.


Now this was a free market.


Let's actually compare the different algorithms.


So if you look at social welfare, this is this thing here, equality times productivity.


You can see that the AI economist will outperform over time over the training progress, it will


outperform all of the other systems.


So it will outperform the free market, the US federal tax system and the SES formula


if trained for long enough, which is to be expected, right?


If you put RL onto a cost function, it will then optimize that cost function.


But it's pretty cool to see that there's a lot of headroom here over what we currently


have.


Now let's look at some of these strategies it comes up with.


So what do these games look like where the AI has imposed different tax strategies?


So this is with the SES strategy.


You can see that here.


Again you see this inequality emerging with the yellow player here building most of the


houses.


With the AI economist, again there is inequality, but you can see at the distribution that agent


one only ends up with about half of the wealth, where if you compare this to the free market


here, then agent one ends up with like two thirds of the wealth, right?


This is the game we saw before.


But there is not qualitatively that much of a difference, but there is in the end result.


All right, let's look at what these policies actually come up with.


So what is the tax policy that the AI comes up with?


So this tax policy outperforms on this social welfare metric.


And this is very interesting, right?


So first of all, you see that it's right zigzag.


It's like down, up, down, up, which is already weird.


So the first very weird thing is the spike at the very bottom.


So that thing here, what's that thing here?


Those are the poorest people in your society and you're taxing them the highest, right?


So just imagine this, you're here downtrodden by life, abandoned by society.


You have no money, no house, no nothing.


And you're just trying to get a job.


You're just getting like a little bit of money and you can buy a cheeseburger and then the


government comes, gives that money.


Come on.


So basically these are the poor and the poor in this system is just F you, F you the poor.


Now the reason why this happens is pretty clear, right?


The reason why this happens is because you want to encourage people to go here to earn


more money, right?


So it's not like the government makes any money from the poor people independently of


how high it taxes them, but it is a basically an incentive structure to make them move over


to the somewhat more productive population.


Because here it's assumed kind of that even the lowest skilled ones can move over a bit


if you just tax them enough at the low brackets, right?


So this is what I find to be, you just have to realize that it is so hard.


I believe it is almost impossible to encapsulate what we really want in a system into a formula


to be into a cost function to be optimized.


It is so incredibly hard and you see that here.


Of course it is going to result in a better social outcome, but it just doesn't feel right


to tax the poor at what?


Okay, so F the poor, right?


And then you get to this level right here.


And interestingly, if you earn even more, you'll be taxed high again, right?


So this, we're kind of used to that, you earn little, you pay little, you earn more, you


pay more.


But then comes this entire value here.


What's up with that, right?


Like double it, and this can be, this is now of course the same reasoning as you have with


this size formula here is where the rich people, you want to tax them less so that they are


more productive, such that they generate more coins.


And even though you tax them less percentage wise, they will end up paying more money in


absolute terms because you basically encourage them to produce more.


So that is the, I guess the reasoning behind this.


But what you have to, you have to recognize what's happening here, right?


What are we optimizing?


We're optimizing this productivity times equality, right?


And what do we get?


You see, you get two big values of attraction, one here and one here.


And that means that this algorithm favors a two class society, right?


And I believe this is, this is partially the limitations of this simulation here.


The fact that you only have four agents, the fact that you can only do two things, either


collect or build, right?


It encourages a two class society, this specialization that you saw, right?


So you say these here are the money makers, right?


And these here are the collectors.


And it is very hard to move from one group to the other because if you earn more coins


as a collector, you're here and you're really discouraged here.


If you move there, you want to move all the way over here, right?


Now the people that are already over here, if they earn an extra coin, that doesn't bother


them too much.


So they're very encouraged to earn more money.


But the very, the poorer people on this side, they're basically discouraged from earning


more money because the system needs them to stay at that collector level, right?


So the system encourages the two class society because we have not built social mobility


into the, into the, into the equation.


We have not built a measure for social, social mobility into the cost function.


And therefore the AI doesn't care that the poor people will stay poor and the rich people


will stay rich.


It just knows that this is the best outcome for society overall, given the cost function


that we had.


Again, this just doesn't seem like fair to us, like what we want.


We want someone to be able to make it over here, right?


Even if they start out from the bottom.


And so we'd have to, we have to build that in.


So we have a system that is effing eff the poor, right?


No social mobility, mobility.


No.


And then what's happening at the end, what's happening at the end.


This is beautiful.


Very rich people.


These are the moneymaker, right?


This is the, this is the monopoly guy, top hat, monocle wearing Scrooge McDuck, bathing


coins.


This is where the government makes their money.


And the discrepancy is really stunning because you could also argue, Hey, why don't we apply


the same reasoning as we applied here and here?


Why is not, is it not like the case that if the rich people, if you tax them lower, they'll


pay more money and so on.


I believe again, this might be just a result of this, how the simulation is set up.


So we'll move away quickly and we'll come back to this.


Here's what I find particularly interesting about this paper, which just confuses the


heck out of me.


It is a double periodic game.


So it's an inner outer loop game.


What do I mean by that?


They have these episodes, right?


Here is the start and here is the end.


And they subdivide this into, as we said, 1000 steps.


So an agent is here and it can do step, step, step, step, step, and it can perform these


actions.


This is the agent.


There are 1000 steps here and the agent just tries to collect as much coins.


So this is your classic RL problem.


But also they divide this into 10, what they call periods.


And I'm just going to draw maybe four periods, right?


So this thing here, they call one period where the whole thing is an episode.


Now the purpose of the period is that at the beginning of each period, the government,


the government can impose a new tax schedule.


So the government doesn't only fix the taxes once, but it can change the taxes over the


course of the episode, right?


Now this is what I find.


I just don't see why.


So now you're formulating the tax giving objective as a sequential decision making.


It's like the government saying, well, today we have high taxes, but tomorrow we have low


taxes and the day after that we have high taxes again.


And it just doesn't make sense to, for any government to do this.


What you should do is you should set taxes once at the beginning of the episode and then


see how that turns out and then try to maximize your tax schedule.


Because all we're looking at, we're only ever looking at how the taxes are at the end, right?


The things that we've examined are just the last taxes that the AI has issued.


We don't know the dynamic of what happens in between.


This might be super wild actually, what the AI does in between.


And I just don't see the framing as a sequential decision problem.


And I believe this is just an over-engineered thing because someone wanted a reason, and


here is the architecture, right?


You see someone wanted a reason to put an LSTM in there.


Someone is thinking like, well, RL, that means like sequential decisions and so on.


And RL in this outer loop, the way I propose it, would just be a one step per episode decision,


which is a bandit problem.


And as we all know, bandits are boring.


So they didn't want this to be a bandit problem.


They want it to be a sequential problem.


And that's why they made this period thing, which I find dumb.


So another factor here, and I'm going to tell you how this relates to the weird rich people


are taxed high.


Another factor here is look at this.


It's a CNN, an MLP, an LSTM, and an MLP, and the agent as well.


And I can tell you right now, the CNN has two layers.


And the LSTM has like 128 units in its hidden state.


So these are tiny, tiny models.


And it is not a model-based RL.


It's model-free RLs, proximal policy optimization.


And the ability of these agents or planner to learn anything substantial here, I believe


is just not super duper well, right?


So I believe that these are rather dumb agents.


And you can see the tax rates given by the planner is fed into the agent model.


But I don't think that the agent, given such a small model, can actually adjust to these


inputs, because you have to do some pretty good logic in order to, from these tax brackets,


to determine how you should act right now.


What I think is happening is the agent just kind of is aware of its skill level.


And through its rewards, it's trying to maximize its future rewards.


And then when the government changes the tax rate, it will not, I'm almost positive, it


will not directly change its response to that.


But it will kind of observe that something's happening in the world and then adjust maybe


a little bit its overall strategy, but not in that particular instance.


And it will be delayed, or it will be like an overall strategy.


And this might be one of the reasons why the tax brackets here might be screwed up.


Because who says, who says if I were this AI, what I could do is in period one through


nine, I make the taxes really low for the rich people.


So I just encourage everyone to make more money, right?


Like, come on, become more productive, and I get the benefits of that.


And then in the last episode, in the last period, right, I just freaking jack up that


final tax bracket.


It's like, you, you have lots of money, give it to me, right?


And then you just redistribute what you got there to the poor people in the very last


period.


And thereby you achieve your goal of this social welfare function.


But of course, this is not sustainable, because all the rich people would just be kind of


screwed through that and move down again.


But it's the end of the episode, so what are they gonna do?


So I think the fact how this is framed, that there are just two different ways to get coins,


the fact that this is this periodical nature of the outer loop, all might lead to something


that becomes slowly more and more and more uninterpretable.


Still cool, though.


All right.


So the final thing, they do this with humans.


Yes, real humans.


So they let humans try it.


And they have this interface here.


And the humans, they behave quite differently from the AI.


So there are a few different things where the humans act.


But look at that here.


AI economist, this is what the agents do, right?


So this AI economist is the tax strategist.


Just take these developed tax strategies and let the humans be the agents.


So you just want to observe how the agents act and whether or not the tax strategies


also work when it's real humans acting in this environment and not RL agents.


So compare this to how the humans act.


The humans, they just build their houses in like neat little packets or straight lines


or stuff like this.


I just find it to be very funny.


Now, there are some things lacking in the human environment, which I find really important.


So first of all, they have no cost for moving, which I guess is minor.


But second of all, they have no trade.


And I think that just kills the whole experiment.


Because now, of course, what you're going to get is the wealth is just going to be proportional


to how much you get coins per house, which is different for each agent, right?


So to me, that is now a pointless experiment if you can't trade because the outcome is


just predictable.


And I don't think that the human behavior changes in response to the different tax brackets.


I think they'll just do and however they can make money, they'll make money, they'll build


more houses until it becomes unprofitable and that's it.


So I don't see the value of these experiments, even though they show that, again, the AI


economist outperforms the other tax strategies in this equality times productivity metric


and also in another metric that they measure.


The second problem I have is for the human experiments, they take this distribution here.


They say, well, this is one of the distributions that the AI came up with.


But you notice the lack of the FU poor people and the lack of this big spike here for the


rich people, which I find are one of the two features of the other distribution.


So I think there's quite a bit of variance in what this AI comes up with.


Or maybe it's just because this is periodical.


This is really confusing because they show and discuss that other distribution.


And now all of a sudden they say, well, we use this distribution that was also created


by our AI and it seems to be qualitatively quite different.


In any case, let's look at how the humans behave under the different strategies.


So in the size formula, you'll see that the light blue person here is kind of spreading


out a bit, probably playing correctly.


Everyone else is just neatly building their houses.


Humans are so territorial and most of them, they kind of stay in their little corner and


they're like, this is my corner.


I'm going to build my houses here in a nice thing.


And under the AI economist, again, you don't really see a different thing just because


the taxes are different.


The qualitative behavior is quite the same.


It's just building straight lines.


And I think the difference is more between the humans.


So I think it's not always the same humans and the difference might be more between the


humans and you kind of see that humans clearly don't, haven't really trained or discovered


the optimal strategy.


They're just doing something and what you're seeing is just a result of the taxation.


It's not different behavior.


And this here, this is the best.


Okay, watch the on the bottom right, the human.


They're just first they do something and they're just walling up, walling up the other players


and look, this is the best.


I am going to build a big beautiful wall and I'm going to have the orange guy pay for it.


It's Donald Trump in the game.


Amazing.


And look at the end, they actually managed to lock in the other players so they can't


move anymore.


Donald Trump wins.


Though, actually the yellow player appears to win economy wise.


But what do you want with lots of money if you can't move?


So I again, I find these human experiments to be rather pointless here because you disable


trade and you don't train the humans to find a good strategy.


Alright, but in that, I find the entire paper to be pretty cool.


Code is going to be released, they promise.


And they have checked that they have no ethical problems, of course.


I invite you to check out the paper.


If you like content like this, please subscribe, share and leave a comment of what you think.


[ML News] MMO Game destroys GPUs | OpenAI quits Robotics | Today w/ guest host Sanyam Bhutani

Once upon a time during his vacation, Yannick Light Speed Culture found chai.


He had so much of chai and he liked it so much that he turned into the host of chai


time data science.


That's why I'm hosting machine learning news.


Hi everyone, I'm Syyam.


I host the chai time data science podcast on YouTube channel and I'm hosting machine


learning news today because because I'm holding the mic.


Yes.


Before we start the news, I have a news to meta.


I don't care.


I'm holding the mic.


I'm interviewing Yannick on my channel linked in the description.


If you have any questions that you want me to ask him, any questions that you want to


ask him and you want me to ask him so that your questions can be asked to him, you get


the point.


Please leave a comment down below.


I'll make sure I ask you questions Yannick.


And now let's start with your weekly absolutely regular.


You don't need to look at your calendar.


You know, it's Monday.


In this week's news, Amazon's new game breaks a few actually quite a lot.


Imagine running a game and breaking your GPUs.


Open AI pivots from robots.


They take a pivot away from that direction.


And Google.


Interesting timing.


Or launches a new company to build software for industrial robots.


Welcome to machine learning news.


Before we start, I have something important.


It's hot, but it's really good.


So this is Kashmiri Kava.


I recommend it.


I recommend any chai.


Let's jump into it.


Amazon's new MMO may be breaking Nvidia 3090s.


The words write.


After intensive Googling, we have discovered that MMOs are massively multiplayer online


games.


Amazon created this massively multiplayer online games.


Now I know.


Apparently this was breaking a few EVGA cards.


Since then, the game has been patched and Amazon's issued a statement that is there


in this blog.


Based on what I've understood by watching so many YouTube videos, the power draw on


these graphic cards was just going haywire when the game would launch and that would


end up frying the card, which is kind of crazy.


I mean, I'm not supposed to laugh at these.


These are like pretty expensive cards, but it's kind of crazy to think that a game could


do that and that these cards could go through that.


Luckily, EVGA has like phenomenal customer service based on what I understand.


When you return a product, the RMA process is undertaken.


Now GPUs are pretty short on supply, but EVGA has a separate supply of cards for just covering


under warranty and they've already started shipping out cards.


Kudos to these guys.


But how is that under machine learning news?


Well, if you're in machine learning, you probably would want a 3090 and you wouldn't want a


game to break it.


OpenAI checked Yanic's previous video here for an intro about it.


OpenAI pivots from robotics and acknowledges the power of simulation venture, be it right.


So OpenAI's co-founder, I don't want to butcher their name, W. Zarambe, has shared, according


to this blog, that the company's pivoting from solving robotics.


Robotics is such a harder problem, I feel it's quite underrated and we're still working


on this, even though we have somewhat cars that can drive themselves.


In the US, in India, you can't, at least where I'm from.


I mean, these cars work well when they do, but then they don't because so many real world


constraints kick in.


And that's again, something that robotics deals with as a challenge.


So that's what they talk about in this blog.


And it appears that OpenAI will be focusing on other problems.


Interesting timing on this, but Google's parent company Alphabet launches Intrinsic, a new


company to build software for industrial robots, the Verge rights.


After reading this and reading the original post, the announcement post by Wendy Tan White,


who will be leading this company, what I've understood is a large part, that is still


hot, that is still pretty hot.


A large part of manufacturing is based on robotics and a large number of industries


need this.


Now, personally, I'm not sure.


So like for computers, a nice thing is you have x64 architectures for phones, you have


ARM architectures for iOS, can't do anything, but there are different architectures.


I mean, iOS does have the developer kit, but I'm not sure if the industry has standard


robots.


So I'm sure like there would be a similar type of robots on an assembly line.


Intrinsic will be developing software for those robots.


Who their customers are isn't clear from the blog.


That's one thing that the Verge mentioned as well.


But it's interesting to see that robotics is making some progress in different areas


and we're just starting to understand how difficult a problem this is.


I mean, I've seen Boston Dynamics robot stance, which is really, really cool.


And it's great to see more companies working in this direction.


Forbes writes AI is learning to understand how vegetables taste.


I won't believe in the internet until I can download food.


These things don't surprise me.


So you can actually 3D print food, which means that I believe in the internet.


Sorry.


This blog talks about a farm called Fifth Season, which is in Pittsburgh, that is using


a software stack and robotics to automate their farms.


And what they're trying to understand is based on this blog, what I've understood, they have


QR codes associated with different plants, and they really use data monitoring and really


try to target a crop towards a certain taste, which is pretty good, I feel, again, in agriculture.


There's again, so many areas where AI is just being applied, where machine learning just


needs to be applied, and it'll become global, you know, we need TensorFlow for agriculture,


we need PyTorches for agriculture, just like we need them for robotics.


So it's great to see that this company is working for it.


It's not open source, but at least there's some news around someone working on this.


NASA writes AI helps improve NASA's eyes on the sun.


NASA has been collecting images of the sun.


You can't just actually you can you can just you can take your phone, take a picture of


the sun, but that's not good enough.


Because you can't see UV rays from Earth in the atmosphere filters it out.


You can't see UV rays anyways, and you wouldn't want to because they might damage your skin


and eyes.


But that is part of the spectrum that the sun emits, among many other things.


So the sun isn't exactly how we see it from the Earth's surface.


NASA has been collecting these images over years now.


And this blog talks about how they're trying to calibrate it.


There's a nice animation that shows you how the calibration actually changes the images


that we have.


So based on robots that NASA has been sending into the orbit, now they're calibrating these


images.


Very cool.


Next up a man actually foreshadowed, Black Mirror had foreshadowed this.


And it's reality, sort of a reality now a man used AI to bring back his deceased fiancee,


the creators of Tech One, it could be dangerous.


I'm not going to get into how ethically right or wrong this is.


That's an independent discussion.


And that's why we need those discussions.


But this blog talks about how this person I'm not going to name the service used a service


built on top of GPT-3, which now makes sense that wasn't released, but is an API.


So the person used the API and built a chatbot service on top of it.


And this person, the one who contacted his deceased fiance created a chatbot around it


and just interacted with it for so long.


I leave it at that and let you think about this.


I'm not going to this is a sensitive topic.


So I don't want to speak too much about it.


As if the robots were upset about OpenAI shutting down its robotics division, they collided


at Ocado warehouse near London, sparking a fire and delaying orders, he was right.


If you're watching this robot time on the side of you, I'm on the side of Yannick.


I know he's a robot.


That's why he wears aviators to hide his vision system.


Just wanted to tell you I'm on your side.


Jokes aside, again, a large part of these systems are being automated.


And we really need companies working on these because again, these problems happen and they


can cause huge issues or damages.


This wasn't a huge one.


But again, that's why you need them.


Too much ethics, but I feel these discussions are important.


Reading race, that's the name of the paper.


AI recognizes patients' racial identity in medical images.


Medical domain is one of those areas where the impact to humans is more directly felt


than any other.


That's when we talk about having biases in these models.


This paper shows that these models are able to pick on the race of a person based on the


medical images.


Note the doctor can't even make out from these pictures, these x-ray images, the CT scans


the race of a person.


It's not because of just some tissue being fired for certain races, etc, etc, etc.


That's what this paper says.


And apparently, it's also able to deduce these technologies.


Deep learning algorithms are able to deduct based on corrupt images also the race of a


person.


They actually go ahead and show this in the studies as well.


Let's say there's a race, chai race, I really like that.


But there's also a coffee race.


As a doctor, I can't imagine myself as a doctor.


But let's let's picture myself as being a doctor.


I might not give the best treatment to coffee.


That's why we need more rigorous testing around these systems.


And it's great to have such papers come up from now and then.


DeepMind had created alpha fold 2, I'm sure Yannick would cover that paper on his channel.


So alpha fold 2 is an architecture based on transformers.


And it has created this breakthrough in understanding protein folding and protein structures.


That's an independent discussion, but it's a huge breakthrough in human history.


They've created this database of so many proteins that can be just very useful in understanding


life and for biology, they've open sourced it.


That's how research should be.


And it's available for free as long as you cite the results for you to use.


Very nice.


ICCV launches behavior challenge.


The goal of embodied AI research as written in this post is to develop intelligent agents


that can assist humans in their everyday lives in activities like washing dishes, cleaning


floors.


While recent, okay, let me go out of this post, recent activities like whatever progress


you've seen, even the papers that Yannick discusses heavily are narrow AIs.


These are slightly great things broader, but we need now further broader AI if that makes


sense.


I'm not talking about AGI, just broader AI.


And these challenges, these tasks are a goal towards these.


So there are different tasks that can, that are a part of this and the deadline is October


I encourage you to check it out.


The behavior challenge is a benchmark with a hundred household activities that represent


a new challenge.


Very cool.


And I look forward to seeing the results from this.


IBM, MIT and Harvard release common sense AI dataset at ICML.


The argument in this post by IBM is when you see an infant, they're able to deduce so much


just based on common sense, even at a young AI models can't.


They've put together a lot of animations and similar things for an agent to learn these


along with few interesting baseline models.


And they're trying to advance machine common sense.


That's such a funny word.


That's why I brought this up.


Finally, Google AI generates even higher quality images.


So generative adversarial networks.


I mentioned this on my Twitter, but I'm also highly interested in these.


That's why I bought this nice box that you don't see.


It's full of RGB.


You know what I'm talking about.


I feel this is an interesting area because we've seen so much progress recently.


StyleGAN came out, which made the images super nice.


Now we've seen a further improvement.


I feel we really need a good benchmark to measure these beyond a certain point.


But anyways, the team at Google released, Google Brain released a new natural image synthesis,


super resolution by a repeated refinement SR3 model and cascaded diffusion model based


on the demo on the page.


These images do look really nice quality.


How nicer are they compared to StyleGAN or the recent papers?


You really need to look at them side by side.


But what they say here is it's about it can perform face super resolution in quite higher


resolution.


That's it.


That's just an area I'm interested in.


So I thought I might share that.


But that is it for this week's machine learning news.


You know, it's Monday.


Thanks for tuning in on a Monday.


Please subscribe to Yannick's channel.


Let's get him to 100k so that we can celebrate his 100k subscribers on my interview.


Leave a comment down below for the questions that you'd want me to ask him.


For now, please keep drinking chai, please enjoy your day and please keep watching ML


News.


Perceiver: General Perception with Iterative Attention (Google DeepMind Research Paper Explained)

Hi there, how is everyone doing? Today we'll look at the Perceiver general perception with iterative


attention by Andrew Yegal, Felix Gimino, Andrew Brock, Andrew Sisserman, Oriol Vinyols and Jao


Carrera of DeepMind. This paper on a high level describes a model called the Perceiver. And what


this model does is it interleaves latent self attention mechanism with cross attention mechanism.


And so it is a transformer. And the secret is that the data only enters the transformer through this


cross attention mechanism that allows the model to have the latent array be of significantly lower


size than the data array. And this solves in part the transformers quadratic bot memory and compute


bottleneck. The image comes in or the data that rather comes in multiple times through this stack,


and the weights can be shared, making it essentially a recurrent neural network.


This model here works for any modality. So the paper not only does images, but videos and audio


and point clouds. And you almost have to you have to change pretty much nothing about the input in


order for the model to work. So this is a pretty big step towards, first of all, making transformers


more deep. And second of all, applying the same models to very, very different modalities of data.


So we'll dive into the paper, we'll look at how it's done. It's actually a fairly simple idea.


So shouldn't take us too long. I always say that but maybe today we'll achieve it. If you like


content like this, tell me how you feel in the comments, leave a like, tell your friends about it,


and let's go. So they motivate the name the name perceiver. It's it's not really


tied to anything. They mode, they motivated by saying biological systems understand the world


by simultaneously processing high dimensional inputs from modalities as diverse as vision,


audition, touch, proprioception, etc. The perception models used in deep learning,


on the other hand, are designed for individual modalities, often rely on the domain specific


assumptions, such as the local grid structures exploited by virtually all existing vision models.


So what do they mean? They mean if we have an image, and the image is of a not a cat, a house,


what did you think? So the image is of a house. And if we have an image processing pipeline,


usually what it will do is it will assume that the image is some sort of grid, and that you can


localize any pixel by its x y coordinate, and also that the pixel is in some kind of relation to the


pixel around it, you will usually build models according to that. So a convolutional neural


network very explicitly will slide over a filter over the image with all shared weights. And


therefore, it directly says that what matters to a pixel is the pixels around it and only in the


upper layers. And after some pooling, do these receptive fields grow, such that more and more


information across larger distances is incorporated. On the other hand, something like a visual


transformer, like the VIP, what it will do is it will do transformer like attention, but because


it can't because the images are so large, because whatever 224 by 224 pixels are just too much to


put into one transformer, it will simply subdivide the image into these patches. And therefore, it


also essentially says it will take each patch and make a vector out of it. So it also essentially


says that whatever pixels are close together, they go into this one vector. So they're treated as a


group. So this paper says that all the current architectures that deal with computer vision


somehow have this built in. However, the the so other models have that two other modalities like


audio, video, and so on. And the perceiver here is supposed to alleviate that. So they say it


induces helpful inductive biases, but also lock models to individual modalities. In this paper,


we introduced the perceiver, a model that builds upon transformers and hence makes few architecture


makes few architectural assumptions about the relationship between its inputs, but also scales


to hundreds of 1000s of inputs like convnets. So transformers, notably have our models that


transform sequences to sequences or let's say sets to sets. So you have an input set. And what we've


usually come to know as transformers are stacks of self attention layers. And in the self attention


layer, what you would do is you would simply transform the input into an equally length output


sequence. And in the middle, you'd have this attention mechanism. And the attention mechanism


essentially needs to compute the weight between every one of the inputs and every one of the


outputs, giving rise to an O of let's call that m, I think they call it m squared. So here you have m


m sequence length, so an O of m squared, compute and memory requirements. Now, if m is small,


that's not a problem. But if we go into the range of NLP, usually so in in NLP, we usually deal with


m's in the order of, I don't know, 2000 1000, let's say 1000. So in the order of 1000, though


we would want more ideally, but in the in the computer vision, our m is easily something like


50k, which is about 224 squared. So the m squared would be 50,000 squared. And that just blows the


memory of our computers. Maybe not the ones in the future, but certainly the ones now. Alright, so


the problem here is that these transformer architectures take too much memory. What this


paper does is it goes ahead and it says, couldn't we do a better job. So usually in a transformer


layer, I'm going to draw this again here as two layers, what you'll do is you'll compute queries,


keys and values from the same input. So you have your input right here. And what you'll do is you'll


compute queries, keys and values from that input. And those get mingled together in the attention.


And that gives you the next layer and you'll produce queries, keys and values again. Queries,


queries, especially are of size m by D, keys are also of size m by D. Now if you multiply those two


together, and you transpose this, you can clearly see that gives you m, a matrix of size m by m.


What this paper does is it says, okay, we can draw back actually on what the very initial


transformers proposed the very initial transformers, if you remember, and if you don't,


you can go watch my video on it. The very initial transformers were something like


generative models that had an input sequence. And they had an output sequence. So the output


sequence, and maybe that wasn't fully completed yet, right? So you want to predict the next thing,


but there was a clear distinction between sequence A and sequence B. Now sequence B would do self


attention. So they would have these stacks of self attention layers with the quadratic thing.


And ultimately, you'd want some kind of output here, such that you know what the next word would


be. This is an sort of an autoregressive model. However, the input did not use self attention,


it used cross attention. So it was also a stack, but it used cross attention. So it went like sort


of like this over. And the way that works is so by the way, think of machine translation, right? So


here is the German sentence. And here is the half finished English sentence that you would want to


complete. So if you want to know what's here, you need to attend to the English sentence. So every


part of the English sentence needs to attend to the English sentence. But also every part of the


English sentence needs to attend to the German sentence. That's why you have these paths going


over. But none of the German sentence necessarily needs to attend to the English sentence. It could


make sense, but it's, you know, it's a restriction where you say, okay, the information flows from


the German sentence to the English sentence. So and that results in this cross attention,


where the keys and the values are produced from sent like sequence a, but the queries to do the


cross attention. So the queries for this particular flow of information are produced by the target


sentence. And you'll notice something, these now can be of different lengths, notably, if the


sentence B right now is much shorter than the sentence a, that would result in a shorter q.


And that would result not in an m by m here, but that would result in like an m by something smaller,


right? And let's call this n. And if n is much smaller than m, then you don't have this quadratic


bottleneck. So that's exactly what this model does, essentially. Let me just get rid of all of this


stuff again. This is akin to a few things. So it's akin to the original transformers. It's also akin


to if you remember the model, DETR, which is a detection model. And what we call the things there


are learned queries. So what do we do here, we start with our goal is to be to have a latent array


that is not huge. So n here is a size that we can handle in a regular transformer. And this stack,


the top row here is just a regular self attention transformer with all the drawbacks.


But because we only have a queue of we only have sequences of length n, the self attention modules


right here. So this is latent transformer, this is classic self attention that we do here and here.


And you know, in all the stacks in all the layers to follow, but we can handle it because n is


relatively small. So in this paper, I think n is something like 500 or 1000. It's something you can


handle with current hardware. The problem is when you when you know, you want to bring in an image.


But this is quite smart. What do they do, they take the image, and they just unroll it into a


byte array. So now we have the m here and the m is huge m is 50,000. However, because we only have


one image, we can handle it. However, because we produce the queries from the latent array and not


from the image itself, we won't get the quadratic blow up. So this is m and this is n. And you can


see that results in an n by m attention matrix, and not an m by m attention matrix. So in this


the data of the image comes in to the latent into the transformer. However, it is not transformed


into an equally long sequence, it is transformed into a much shorter sequence, namely this latent


state. On this latent state, we have a transformer transforming it into a new latent state. From that


queries are generated to do cross attention again to the same image. So the same image will come in


every single layer, the same image will come in to the into the architecture, and so on. So if this


reminds you of a recurrent neural network that it is sort of a recurrent neural network, especially


because they say you can also share these weights between repeats. If you share these weights, it is


definitely a recurrent neural network, where this here is the initial state, which you either learn


or randomly initialize. In this case, I'm pretty sure this is learned, though, I might have misread.


So this concept, again, it relates to RNNs. In fact, it is an RNN if you share the weights,


it relates to learned queries, as opposed to generated queries. So you can learn the queries


instead of generating them. When you learn them, you can choose yourself how many there are.


And it also sort of relates to I'm not sure, but how to call this, you can see the image goes in


multiple times. So the way conceptually you can think of this is that here is a bunch of learned


queries, they have no clue about the incoming data. So what you generate here is just kind of


a generic set of queries. Like what would you know, what would you like to know about this incoming


data point, and you have 1000 things that you can want to know. And you have, I don't know,


50,000 things to attend to. So you're going to choose 1000 criteria, right, to gather from that


input data. Now, the way attention works, right, is the queries, you have a set of queries,


queue, and you have a set of keys down here, a bunch of keys, more than queries. And every query


exposes sort of a vector, and every key exposes a vector. And the information is routed by means of


highest or high inner product. So you would route things that have a high inner product together


like these two. Yeah, those are the ones that you would route. So every key potentially has a not


potentially every key has a vector associated with it. So the queries essentially say what kind of


things I would like to know of the incoming data. And the keys, say for each pixel in the data say


what kind of things that particular pixel offers to to the to the to the model. If you just do this


once, you might get some generic information, but then you get to do it again. And you will notice


that the queries here, the later queries are a result of that processing. So the data comes


through through here, right, and influences these next queries. Therefore, these next queries here


can be dependent on the earlier data. So you can pretty easily see that, you know, now, the next


time you're going to attend to this data, you do this in an informed fashion, you already kind of


know what's in there. So you refine what you would like to know about the data and so on, you


can refine and refine, you can ask for more and more specific things, the more you learn about


the data. So this is really a process of learning more and more about the data in a dynamic way,


where you can say what you would like to know. And, you know, this, I think it's a it's a great


idea. It might be refined in the future, but it certainly does. Also, you know, it makes sense,


it makes sense. And it also solves the kind of quadratic bottleneck. Oh, wait, I almost forgot,


I had a visual demonstration of how the quadratic bottleneck here is solved. Bear with me.


Here's a matrix, it's m by m, now watch.


Problem solved. All right. So by the way, that the lower is supposed to represent n by m,


I did not write that down. Okay, so this not only allows you to overcome this quadratic bottleneck,


it also allows you to build much deeper transformers. So I believe their best architecture


here had 40, sorry, 48 layers of transformer, which, you know, we can do in kind of NLP,


but it takes a lot of hardware. And when they also share the weights, their number of parameters in


these things is not more, I think it's comparable to kind of a a ResNet, a standard ResNet.


So yeah, pretty cool. There is so they apply this to pictures, they apply this to videos,


they apply this to audio, they apply it to video and audio together, they apply to 3d point clouds.


Though one has to say for video, they don't actually put the entire video into so that


this here isn't the entire video. But they I think they also put kind of little time space chunks of


the video in it. So it doesn't solve yet, all the problems with transformers, it's still,


if a data point is huge, you won't get it in there, simply by the fact that is linearly huge,


what you will solve is the fact that things are quadratically huge. The last thing to do is


to pay attention to this thing, positional encodings. Now, the way they do positional encodings is,


so now we have like a fully independent, like a data modality independent architecture, right?


It's important to realize this, this thing here has nothing to do with an image, like, is it an image?


Who knows, right? We don't care. We simply, this is the array of pixels, this is simply the


unrolled, the unrolled image. There is no convolution filter, there's no patching or


batching or anything. There's just the image or it's the audio data, right? It's like sample after


sample of audio data, and so on. This, you can even think of a situation where you would feed in


different parts of the data from time step to time step, in which case it really becomes like


a recurrent, just like a recurrent neural network. But the point is the transformers, they are


invariant to position. So if I feed one, two, three, four, five into a transformer, it will


do exactly the same thing as if I feed three, one, two, four, five. That is not much of a permutation.


But it is. So it is invariant. Now that is, that stifles it because we, you know, there is


something to something being in a certain location, right? Especially if you think of text,


word order matters and so on. What we, so, but there's a clear distinction. We don't want to


build these things into the architecture, but we want to give the model the possibility to exploit


that information because clearly it's there, like a piece of text is not just a set, it is an actual


string of ordered words. So what do we do? We give positional encodings with the input and


positional encodings, you know, have been used all over the place, transformers specifically need


them. The way this paper does transitional, does positional encodings is like they do in


the first transformer paper and that is by Fourier features. So if you have five inputs right here,


you build up kind of a Fourier bank of frequencies. So this is the lowest frequency, something like


this, like a sine wave, and then a higher frequency. Well, five probably wasn't the


optimal thing to demonstrate this. So by kind of indexing some of these things,


so here if we look at the position number two right here, it has like, if we just consider this


binary, it has like, no, not binary, like 0.9, 0.9 minus one. That's kind of the encoding,


that's the positional encoding of that location. And if we look at three, it's 0.9 minus one, one.


So you can see that you can, with this kind of positional encoding, as opposed to a learned


positional encoding, what you can do is you can always detect when two things are close together.


That means that in the lower frequencies, they will share the same number. And you can,


but you can also do very high resolution, you go to the highest frequencies and if they're different


there, but if they match all of the frequencies above them, that means they're like right next


to each other. So that's how you do positional encoding with Fourier features. Again, I discussed


this at length in my Attention is All You Need video. The Fourier features also have the additional


benefit that you don't rely on learned encodings, which means you don't rely on the fact that


you have kind of an exact or a maximum amount of sequence length. So the, yeah, I mean,


you still have kind of a maximum here, but I like this more because it's sort of independent,


it's one less thing to learn and the learning happens in the processing itself. So in terms


of experiments, it's pretty simple. They are in vision, they are on-point, they are


on par with something like a ResNet-50 and they're, you know, they're doing pretty well


in vision without any sort of assumption that the input data is an image, right? That's the,


that's the crazy part. So other than the position encodings, which are the Fourier features in two